Industrial Cybersecurity Lessons from the Colonial Pipeline Breach

As cyber-attacks on industry continue to increase, there’s plenty to learn about avoiding and mitigating the effects of a breach. Experts from cybersecurity providers Verve Industrial and Keyfactor offer their insights.

Not long ago, most cyber-attacks on industry happened largely behind the scenes. The companies whose systems were breached rarely went public about the event and if information about these events was ever discussed publicly, it was usually years after the event and few specific details beyond the nature of the attack were ever revealed.

But that’s been changing as cyber-attacks have become more brazen and threaten the public at large. For example, on February 5, 2021, we learned about the remote access intrusion into the control system at a water treatment facility in Oldsmar, Fla., about 13 miles from Raymond James Stadium in Tampa where the Super Bowl was held just two days later.

As an industry observer, one of the more shocking aspects of the Oldsmar hack is that the only thing that stopped it was an observant operator who noticed some unusual changes being made to the facility’s control system. Though remote access to this system was allowed, apparently no user authentication or high-level security methods were employed to restrict access by unauthorized users. And because the operator who noticed the changes received no alerts about them—he just happened to notice that the changes being made were unusual—it’s not unreasonable to assume the facility had no effective anomaly detection or intrusion technologies in place either.


   

Listen to this "Automation World Gets Your Questions Answered" podcast on control system cybersecurity in light of the increasing use of remote connections.


Earlier this week, cyber-crime gang DarkSide claimed responsibility for compromising the Colonial Pipeline Company—one of the largest fuel pipelines in the United States. As a result, fuel outages are being experience across states in the eastern U.S. supplied by the Colonial Pipeline. While we don’t yet know all the details of how DarkSide compromised Colonial Pipeline’s network, we do know that it is a ransomware attack involving the theft of nearly 100 gigabytes of data from the company’s IT network. Information issued by Colonial indicates that it’s OT (operations technology) network was not affected.

Advice for industry

Considering the ongoing rise in cyber-attacks on industry, Ron Brash, director of cybersecurity insights at Verve Industrial, a supplier of industrial control system security, highlighted five key areas of focus to help industrial companies mitigate the threat of a cyber breach affecting their operations. “The financial impact of a shutdown can be significant,” he said. “Cyber now needs to be a primary component of all disaster recovery planning and must become a larger area of management focus, even for organizations that don’t see themselves as a natural target.”

Ron Brash, director of cybersecurity insights at Verve Industrial.Ron Brash, director of cybersecurity insights at Verve Industrial.His recommendations for effectively focusing industrial cybersecurity are:

Realize that industrial cybersecurity is not IT vs. OT, as operations can be affected by attacks on both sides of the system. “Organizations need to work on bringing these two organizations together to protect the entire system. Billing and pricing systems and the data needed to operate them are critical processes, just as critical as the SCADA network operating the pumps and valves. Visibility and protection across the IT-OT landscape is key to protecting operations,” he said.
 
The largest security gaps in industrial companies tend to be in the management and maintenance of security. “Firewalls may exist, but personnel have adjusted rule settings to allow remote access and created servers that route around critical protection layers; patching policies may exist, but the manual tasks that are often standard do not get completed given the urgencies of operations; and standard secure configurations may exist, but exceptions are made, users adjust them, new software is allowed, and ports are opened, leaving gaps in that secure structure,” said Brash.  “[But often] there is no central visibility of these gaps.”

He also noted that availability of robust and timely backups can significantly reduce downtime in case of a ransomware attack. “But are these backups up to date? Do they restore quickly? Without management, the backups you thought you had may not be ready in case of emergency,” he said.
 
Rapid response and recovery are critical. The real advantage a company can have is the immediate ability to take actions across endpoints—IT or OT—to stop the spread of malware, Brash said. “This integration of detection and response actions allows industrial organizations to significantly reduce the spread—and cost—of ransomware attacks.”
 
Have a plan for a conscious shutdown. Brash explained that having a plan for “conscious shutdown” to avoid an OT incident while balancing loss is an acceptable alternative to a major incident. “Incidents like the Colonial crisis have become the new norm within the critical infrastructure cybersecurity community,” he said. “As such, organizations should be adequately trained and prepared to handle incidents like this via a well-defined procedure.”

Brash noted that the ability to consolidate the security status across all systems into a common database to track and ensure protections are maintained is critical to strong cybersecurity protections. “Owners must patch, segment, harden configurations, ensure appropriate backups, and limit access to least privilege,” he said. “These core, fundamental elements of security can be the difference between being a victim or not.”


   Learn why the age of industrial IoT demands a new security paradigm.


IoT device security

Chris Hickman, chief security officer, at Keyfactor.Chris Hickman, chief security officer, at Keyfactor.The general rise in cyberattacks threatening industry underscores the fact that “security cannot be an afterthought but rather needs to be designed and planned for at every step,” said Chris Hickman, chief security officer, at Keyfactor, a supplier of cryptography technology used to prevent network outages and secure machine identities in multi-cloud enterprises and IoT supply chains. “Good security is rarely retrofittable, especially when it comes to IoT devices. It needs to be built in as a core fundamental and planned for to exceed the anticipated lifetime of the product it is securing.”

Mark Thompson, VP of product management at Keyfactor highlighted three common mistakes Keyfactor sees being made in industry as they relate to IoT device security, and how to avoid them:

  • Hardcoding credentials onto the device: Some IoT devices are limited due to hardcoded credentials, Thompson said. “This is a common outcome when manufacturers embed passwords or shared keys into firmware to help simplify development or deployment at scale. If [these keys are] accidentally leaked, threat actors or individuals without proper authority can access an entire fleet of devices.” To avoid this problem, Thompson recommends using strong mutual authentication between any connected devices or applications within the overall deployment.Mark Thompson, VP of product management at Keyfactor.Mark Thompson, VP of product management at Keyfactor.
  • Unsigned firmware: Many IoT devices go to market with unsigned firmware, according to Thompson. This problem only grows as more devices connect and need firmware signing. “It’s strongly recommended that device makers sign firmware with a tightly controlled code signing certificate that only permits access to authorized individuals,” he said. “Another critical step is to keep an internal audit trail of all code signing activities. Using a trusted public-private key pair is the most effective means to secure device firmware and have the ability to check and verify the device’s signature before booting the device or installing firmware updates.”
  • Weak authentication and encryption: “Implementing strong cryptographic keys and algorithms that match the device’s use case applications are critical to hardening its long-term security,” Thompson said. “Equally important is ensuring sufficient entropy to produce an encryption key; randomness in key generation is a priority through this process.”


       Read more about the trends around embedded cybersecurity in automation technologies.


Companies in this article