What if you were to wake one morning to the news that someone had just published a way into your supervisory control and data acquisition (SCADA) system? What if a researcher had posted 34 vulnerabilities in four popular SCADA systems to the Internet, and this list contained some of yours? Would you be prepared to defend your control network against intruders?
If your company is like most, probably not, according to Bryan Singer, principal consultant for Kenexis Security Corp. headquartered in Columbus, Ohio. Most manufacturing facilities are using conventional cyber-security protection that has depended upon keeping up with the latest threats and blocking them. Unfortunately, last summer’s Stuxnet infection and more recent events in March have proven that this strategy is no longer secure. Manufactures need to replace it with a defense-in-depth strategy that erects multiple lines of defense that not only block known threats but also actively search for intruders.
Mounting an active, multilayered defense is more important now than ever before because security vulnerabilities are no longer dribbling out one or two at a time through controlled channels. As far as Singer and other security consultants are concerned, the waking-to-a-nightmare scenario already occurred this year in late March. One Monday, Italian researcher Luigi Auriemma published a list of 34 SCADA vulnerabilities in the BugTraq e-mail list. To make matters worse, a few days before, a Moscow-based security firm named Gleg published 11 unpatched exploits of SCADA vulnerabilities in its Agora SCADA+ exploit package.
Consequently, a large number of vulnerabilities and exploits had been circulating for a while before vendors could even begin to generate patches, says Singer. After searching the Internet for exposed systems on the morning that the news broke, “I found dozens of them within a very short amount of time,” he reports. “These control systems are directly accessible across the Internet, and we have active exploit code available for some.”
Even when such vulnerabilities and exploits come to the attention of automation vendors and security firms, fortifying control networks with threat signatures and patches takes time. Not only do patches take time to develop, but they also require testing to ensure that they are safe to implement. In a study conducted by AstraZeneca, the fastest that the pharmaceutical company could safely deploy patches to all systems throughout its plants was 31 days, reports Eric Byres, chief technology officer at Byres Security Inc., developer of specialized firewalls for industrial control systems in Lantzville, British Columbia, Canada.
Deploy diverse defenses
Because repairing vulnerabilities takes so much time, the current thinking on protecting industrial control systems involves multiple layers of protection. “This strategy accounts for the probability that there will be vulnerabilities in your cyber armor,” explains John Cusimano, director of security services and managing director of exida.com LLC, a safety-consulting firm based in Sellersville, Pa. “Through multiple layers and dissimilar technology, the strategy provides for other mechanisms to prevent threats from reaching their targets.” The idea is to construct their networks so that critical systems are many layers removed from most threats.
Singer at Kenexis thinks that the best way to implement a defense-in-depth strategy is to follow the ISA99 Manufacturing and Control Systems Security standard being developed by the International Society of Automation (ISA) in Research Triangle Park, N.C. “There is a lot of good guidance [in the standard] for understanding and describing your environment from a security perspective,” says Singer, who is co-chair of the committee.
The zone-and-conduit model outlined in the standard breaks a network into zones that can be secured independently of one another. Data, then, flow among the zones by means of carefully controlled conduits and are inspected whenever they cross boundaries. Hence, once inside a network, no one has free access everywhere within it.
Singer reports that, of the standard’s 14 documents, two have already been published and the others are pending. “We’ve also released two technical reports,” he adds. As the various parts of ISA99 are approved and published, the International Electrotechnical Commission in Geneva has been internationalizing them as IEC 62443.
As with the ISA99 model, IEC 62443 cannot be used as a mere checklist, warns Cusimano at exida. “Suppliers’ reference architectures need to be adjusted for ‘real’ applications, and data collection must be performed very carefully on live control systems,” he says.
One of Cusimano’s customers, a South African petrochemical company, learned these lessons the hard way when a worm shut down two OPC (a communications standard) servers in December 2009. The operators had to run the plant partially blind for eight hours while the engineering staff rebuilt the servers. Luckily, the plant was able to recover without losing production, but the incident made management a believer in segmenting the networks and embarking upon a multilayered defense program.
The program includes instituting a mechanism for installing all software patches as they become available, and deploying anti-malware software according to supplier recommendations. Meanwhile, technicians are busy removing any unnecessary file shares and purging the control network of all unnecessary applications, such as Microsoft Outlook and Internet Explorer. They are also locking cables into used ports, disabling or blocking access to unused USB and Ethernet ports, and looking for monitoring technology that actively searches for intruders.
Monitor network traffic
The various kinds of monitoring technologies available today exploit an important advantage that control environments possess. “Compared to an IT (information technology) system, they are extremely stable day after day,” explains Byres. “Users don’t often install new applications, and there is not a lot of unexpected traffic. The systems that I commissioned 20 years ago probably have similar traffic patterns today that they did back then.” This high degree of stability makes it easy to monitor and control traffic.
It also makes it practical to adopt the more aggressive of the two basic security strategies that exist. This stricter principle—that which is not expressly permitted is denied—offers much tighter security than the other, more popular principle used by anti-malware software—that which is not expressly prohibited is allowed.
There is a tradeoff, however. “Although access is controlled more tightly, the network is not as flexible,” notes Philip Cox, director of security and compliance at SystemExperts Corp. in Sudbury, Mass. “If somebody installs an application, it’s not going to work.” New applications and upgrades require someone to reconfigure security settings on operating systems, firewalls, and other tools to allow them to run.
Because reconfiguration is neither difficult nor necessary very often on control networks, a technique known as “whitelisting” is gaining a measure of popularity. Many automation vendors are developing technology that compares network traffic to a list of approved applications and files and permits only those that are on the whitelist to operate. “Unlike blacklisting technologies, whitelisting takes the approach of locking down all known good traffic and blocking everything that is not trusted,” says Markus Braendle, group head of cyber security at ABB Inc. in Raleigh, N.C.
Braendle, however, urges asset owners to continue using anti-malware software in coordination with whitelisting. “Having whitelisting in place will allow them to use anti-malware technology in a less resource-intensive and intrusive way,” he says. Yet, it maintains another important line of defense.
Another set of monitoring and prevention technologies being developed for control-system security is content inspection. Borrowing concepts from e-mail inspection developed for the IT space, developers of firewalls for control systems are designing tools for inspecting data attempting to cross boundaries and checkpoints in control networks. “You allow it to pass only if the data are formatted according to the specifications, and if it’s for a specific function such as reading data, and if it’s is going to a specific device, and so on,” says Byres, a developer of such products. “You tighten the criteria down so that a worm like Stuxnet won’t be able to write to the PLCs (programmable logic controllers) it wants to.”
Check your logs
Another way to exploit the stability of automated processes in securing control networks is to implement a technique known as security information monitoring (SIM). In this technique, SIM software examines the various logs existing throughout the system for abnormal activity. “Theoretically, you could do it by hand,” notes Cox of SystemExperts. “Practically, though, you need some type of automated mechanism for sifting through the traffic on any sizeable network. The intelligence built into the correlation engines has gotten much better over the last few years.”
The biggest obstacle to effective use of monitoring tools is resources. Besides having the necessary physical infrastructure and tools installed and configured correctly, any monitoring scheme needs people dedicated to it to make sure that the process is working, to tune the rules as circumstances evolve, and to interpret the data being collected.
As a case in point, Cox recounts the experience of an additive manufacturer that asked his firm to take charge of the enVision security monitoring system that it had installed earlier from Bedford, Mass.-based RSA, the security division of EMC Corp. Although the software was running and the company was paying the requisite maintenance fees, “nobody was doing anything with it,” explains Cox. The company had a lot of the other IT hygiene in place, but no one was really investigating the anomalies and alerts that the monitoring system was generating, a fact uncovered by an ISO 27000 security audit.
Another problem was that the company had not been reaping the full benefits of its investment. To erect several active lines of defense within the network, Cox has been incrementally adding checkpoints that look for unusual activity. For example, he configured the system to look for login failures in the active directories of a set of database servers that he had found to contain critical information. “In an automated environment, systems are programmed to login and talk to other systems,” he notes. “There shouldn’t be any failures.”
Failures, then, indicate a problem that needs investigating. Sometimes, they catch something as simple as a file that accidentally got corrupted. But other times, it can be a burglar alarm, says Cox. He reports that, since he began configuring and overseeing the monitoring system for this additive manufacturer, the software has alerted the company to at least three important events that required the attention of someone responsible for monitoring the network.
In fact, despite the weaknesses found in the software, lax security was the most important contributor to the success of the Stuxnet last year according to Ernie Rakaczky, program manager of control system security at Invensys Operations Management of Plano, Texas. “Plant networks are not usually compromised in one day in a targeted attack,” he says. “In these attacks, somebody has usually gathered information in many ways.” Someone monitoring the network can see the abnormalities generated by the initial probing and mount a defense against it.
For this reason, Rakaczky maintains that good network security requires the attention of someone capable of noticing and investigating these abnormalities. “Ninety percent of being successful in cyber security is how well you manage what you have in place, rather than how much you have in place,” he says.
Another reason for giving someone responsibility—whether an employee or a contractor—is that adding monitoring layers to any defense takes time. Cox at SystemExperts has been working with the additive manufacturer for more than a year now, and he estimates that he is only 20 percent into the project.
Rugged VPNs
If these and other monitoring technologies are going to add a reliable layer of protection to a defense-in-depth security scheme, the hardware must be reliable. For many installations, that means that the hardware must be rugged enough to withstand the industrial environments. This was crucial for United Water, a utility based in Harrington Park, N.J. The company manages more than 300 remote field sites across 23 states using a variety of communications technologies, such as modems, leased lines, dry pairs, and licensed radio.
Hence, the engineering staff went looking for industrial-grade hardware that would support the firewalls and encrypted virtual private networks (VPNs) protecting the various zones in its SCADA networks. “In the past, we had mixed results using office-network-grade products that were expensive, required special skills to configure, and failed frequently,” notes Keith Kolkebeck, systems engineering project manager.
His search yielded fruit, however, when he tested a dozen mGuard industrial-network security appliances early last year from Phoenix Contact USA in Middletown, Pa. Created and developed by Phoenix subsidiary Innominate Security Technologies in Berlin, Germany, the small, industrial-rated modules contribute to the layers of protection needed in the infrastructure. They offer LAN-level security through firewalls and integrity monitoring, as well as secure remote connectivity through VPNs and firewall control of the VPN traffic, according to Torsten Roessel, Innominate’s director of business development.
Stateful firewalls—which are firewalls that keep track of the state of network connections such as TCP streams and UDP communication travelling across them—apply rules configured from templates on a central server. These rules can restrict the type and duration of access to authorized individuals, who may login and authenticate themselves from various locations. “Integrity monitoring can protect file systems against unexpected modifications of executable code, by Stuxnet-derived malware for instance, by sending alerts to administrators,” says Frank Dickman, a Chicago-based engineering consultant who was involved in the project.
The appliances perform their work invisibly and transparently for two reasons. First, their bidirectional communications create no perceptible latency on 100-Mb/s Ethernet networks. Second, they connect to the equipment that they are protecting and assume their media-access-control (MAC) and Internet-protocol (IP) addresses. Consequently, installation requires no changes to the network configuration, which has the added benefit of allowing a technician without special IT training to install one in about 10 minutes, according to Dickman.
As Dickman and others have discovered, a number of tools are available for implementing a defense-in-depth strategy. Companies simply have to awaken to the need, and be determined to add layers of protection.
February 2011, Related Feature – Cyber Attacks Victimize Oil and Gas Companies
To read the feature article, visit http://www.automationworld.com/news-8338
