Plant Security: Who's Trying to Hack into Your Automation System?

How safe is your plant from cyber attack? Engineers at the Coors Brewing Co., in Golden, Colo., thought the company’s bottling system was decently secured until someone with access logged in and inadvertently changed a timer for a maintenance device on a filler. “It was supposed to squirt grease into the bearing every 20 minutes and it was changed to once every eight hours,” explains Matt Meyer, plant engineer at Coors. The bearing soon froze. The line that fills 1,200 bottles per minute ground to a halt. The damage? “It was enough to create a $100,000 loss,” says Meyer. With well-intentioned engineers monkeying around in the automation system, who needs terrorists or disgruntled employees?

When Coors personnel clamped down on plant security, the culprit they were trying to circumvent was a well-meaning electrician. “We realized we needed to create different levels of security so we could determine who has access to PLCs (programmable logic controllers) and who can change PLCs,” says Meyer. “We found that electricians were tweaking PLCs for efficiency, but then the changed PLC became a big problem.” Rockwell Software, a Milwaukee business of Rockwell Automation, helped Coors put security into a manufacturing system that was designed primarily for efficiency and reliability.

Easy Internet access to control systems makes life simpler for plant personnel. Managers can monitor plant activity from home now. But the technology also allows those managers to change plant settings from home. Most manufacturers use security to disallow remote configuring—yet it’s technically possible. And in some instances, bad guys—or well-intentioned good guys—have figured out that plant networks are pretty easy to access. The plant is fully networked and the network systems are now based on easy-to-operate open standards.

Unintended holes

The big challenge to manufacturers comes from changes in the nature of their automation systems. Over the past few years, these systems have become information networks, and those networks are now made up of open systems. That’s great news from an operations viewpoint. It’s also good news for anyone who may want to crack into the system. “Manufacturers have networked their equipment significantly to better run their plants,” says Mitchell Ashley, chief technology officer and vice president of engineering at StillSecure, a company in Louisville, Colo., that specializes in plant network security. “Many of these manufacturers have based their networks on readily available software such as Microsoft Windows, and they don’t think in terms of securing the operation.”

The trend toward open standards has created unintended holes in manufacturing networks. Connections to the outside world are not new; connections using Windows are. “People have been connecting manufacturing processes to the outside world for a while, but they were custom systems—expensive and divorced from open standards,” says Rich Ryan, president of Rockwell Software, a division of Milwaukee’s Rockwell Automation Inc. “Now, controls have IP (Internet Protocol) addresses and use the same software we use in our offices. You can bring it all up on a browser from home.”

William Moore, security analyst at ARC Advisory Group, in Dedham, Mass., agrees that the biggest recent change in system vulnerability comes from the move to open software. “The change from proprietary systems to the commercial off-the-shelf technology is a lot of the reason security issues have come up.”

The upside of open systems is that they offer wide interoperability among all locations. This benefit is usually viewed as being greater than the downside brought by increased security risk. “People want open systems because they communicate so well. They don’t want to go back to closed systems,” says Rashesh Mody, chief technology officer at Wonderware, a Lake Forest, Calif., division of Invensys Systems Inc.

Assessing the risk

Those controlling plant operations now have to include security as part of their risk management world. Risks on the plant floor used to center on order delivery metrics and physical safety. Now, plant managers have to determine how easily someone could maliciously (or inadvertently) bring the line to a crashing halt. “Somebody at the manufacturer needs to do risk assessment. That’s the bottleneck,” says Wonderware’s Mody. “Once you figure out the risk, you can do the mitigation. You also know how to keep it updated and you know how quickly you can react.”

In the chemical industry, companies are scrambling to adopt cyber security measures in order to avoid government regulation. “These companies don’t want to hear someone say, ‘I’m from the government and I want to help,’ ” explains ARC’s Moore. “If an enlightened management can institute proper security procedures, they will do a better job than regulations can.” He notes, however, that not all companies are run by enlightened management. He points to chemical giants such as Dow and DuPont as example of companies that are on top of security. He notes that smaller chemical companies tend to fall behind on security.

As part of risk analysis, StillSecure examines the system from an end user’s point of view. “We do risk analysis from an end user’s computer, since many attacks—such as the MyDoom worm—leverage end user computers as part of the attack,” says Ashley. “So we do a vulnerability assessment on each end-user unit. This end-point security is designed to know who is connected to the network.”

In some cases, concerns about security have put a damper on the adoption of wireless connectivity because manufacturers fear wireless is easier to crack. “People are backing off from thinking about mobile or wireless interfaces for controlling machines,” says Jim Taylor, manager for industrial automation at Venture Development Corp., a Natick, Mass., research firm. “They don’t want people accessing control when they’re not near the machine.”

As companies add security, they’re bringing it on both as retrofit and as planned security for new systems. Plant closures, according to Taylor, are typically not needed. “I have not heard of any case where they shut down a plant to apply security technology.”

So whose access into your plant’s operations should cause you concern? Terrorists? Disgruntled employees? The senior executive who wanted to peek into production numbers from home but accidentally change control settings?

“There are two behaviors to watch for. One is malicious—the disgruntled employee. The other is inadvertent,” says Rockwell’s Ryan. “You have to create a security culture to deal with both of those possibilities.” Using Coors as an example, Ryan emphasizes the importance of restricting access even from well-meaning personnel. “You have to decide who gets access and what they get access to.”

Though it may seem U.S. enemies are a likely source of attack, most analysts do not view terrorists as the greatest danger to manufacturing systems. “Terrorists are the least of the security concerns,” says ARC’s Moore. “Since most manufacturing systems are based on open Microsoft operating systems, and since we need to be connected to the plant floor, the real danger is people doing dumb things that impact manufacturing operations.”

Bleeding networks

For pharmaceutical manufacturers, Information Technology (IT) security takes on the additional complexity of Food and Drug Administration (FDA) oversight. The security system has to meet FDA approval or be shut down at the potential cost of millions of dollars per day. Some pharmaceutical manufacturers meet FDA standards by using biometrics as part of system security. This can include face, voice or fingerprint recognition to create an additional layer of security, and it also makes accessing the system less cumbersome for employees. “In some systems, people are entering their passwords 300 times a day,” says Tom Doggett, director of marketing at Saflink Corp., a security software company in Bellevue, Wash. “Fingerprint access increases the system’s efficiency.”

One of the biggest hurdles facing manufacturers is simply the notion that plant operations have become networks that bleed into the outside world and thus need to be protected. The human resources and finance departments have long been accustomed to protecting information. But for plant executives, the notion of creating a secure IT world is very new.

For years, manufacturers didn’t sweat the vulnerability of their manufacturing networks, because the outside world was simply not aware the systems existed. “In the past, when we asked manufacturers how they protected their plant systems, the answer was typically, ‘No one could figure out how to break into the system,’ ” says Dick Mackey, principal of SystemExperts, a security company in Sudbury, Mass. “Their protection was called ‘security by obscurity.’ ” Mackey believes that those relatively safe days have passed.

Some don’t believe there’s a culture gap at all among those running automation systems. Everyone knows the potential damage that can be caused by an Internet worm, so they’re aware that any company network is vulnerable if not deliberately secured. “Manufacturers are susceptible. They’ve seen their plants attacked,” says StillSecure’s Ashley. “It’s easy to explain to senior management because most companies have experienced a worm and they know the pain.”

Addressing cyber security thoroughly may require a cross functional team that includes IT personnel as well as experts in plant systems. “You need cross-departmental groups that understand both the manufacturing process and the IT on the floor,” says ARC’s Moore. “You look at whether the risk is high or low, then you look at what you need to do to reduce those risks.”

So whether the potential for cyber attacks comes from terrorists, angry employees or hapless engineers, manufacturers are beginning to realize they’re at risk. The plant is connected by an IT network and that network can be accessed from the outside. As with most IT advancements, large manufacturers tend to be ahead of the game on security, with small to mid-size manufacturers most at risk.

See the sidebar that goes with this story: The Plant Floor Culture Gap