For Ken Harper, a good change-management system is like a comfortable chair. “You don’t think about the benefits. You simply enjoy them,” says the senior engineer at Trace Automation LLC (www.traceautomation.com), a Pittsburgh-based systems integrator specializing in data collection and inspection.
One thing for sure about automation software is that it will evolve over time—whether through periodic upgrades and patches or accommodations for changing conditions. For this reason, more automation professionals are coming to the same conclusion as Harper and his colleagues: Using the latest change-management tools is the best way to avoid the pain that can result when people mistakenly install the wrong version of the code.
To illustrate why this is true, Harper paints a scenario involving a series of modifications to a hypothetical process, some made in September and a few more in December. Without effective version control, it would be easy for a new programmer in December to pull the original, pre-September version of the code and work on it, instead of the December version.
“When you discover your mistake, you have to redo your work with the corrected code, essentially putting in the same amount of hours all over again on the correct version of the software,” says Harper. “For an integrator like us, failure to manage software changes costs us engineering time.”
Of course, if no one ever noticed the mistake and the new code actually found its way onto the equipment, it could very easily throw the process out of spec and even halt production. “In a worst case scenario, the user could lose control of the equipment and cause damage,” notes Harper.
Knowing just how crucial and costly maintaining careful control over changes is, Harper and his colleagues looked for a way to tighten their version controls without making them difficult to use. Among their goals was to automate such manual tasks as entering notes on modifications in a log. The question before them was whether they should buy a commercially available code-management system or create their own. “We discovered that neither approach met the needs of industrial SCADA [supervisory control and data acquisition] programming,” says Harper.
The reason is that most commercial code-management software tends to be third-party packages originally designed for the office environment. In Harper’s estimation, most of this software lacks the rigor necessary for ensuring safety on industrial equipment, as well as the necessary integration into the programming suite for making it easy to use.
>> Click here to read one approach to code-change tracking.
Another limitation on some of this software is that it tracks changes on the execution or runtime operations of deployed applications, notes Marcos Taccolini, CEO of Tatsoft LLC (www.tatsoft.com), a provider of software and cloud services for industrial automation. Such packages tend not to track modifications made at the engineering configuration level.
Because of these deficiencies, Harper and his team concluded that code-management tools should really be embedded into the programming suite. This conclusion narrowed their search to only a few vendors and eventually led the team to select FactoryStudio from Tatsoft. Because the developer built change control into its software from the beginning, the software tracks any modifications to the code from the time that a programmer logs into the engineering environment.
Enforcing Order and Control
Besides embedding change control, FactoryStudio leverages the .NET Framework, object-oriented inheritance, and open technologies such as extensible markup language (XML) files and structured query language (SQL) databases. “These technologies allow application files to be centralized,” says Taccolini. “For instance, a SQL database can hold an entire project configuration in one file.”
Not only does this strategy help in maintaining the integrity of the developed code, but it also makes it much easier to incorporate strategies for change tracking and backward compatibility.
“In previous generations of automation software tools, the configurations were composed of hundreds of files with closed custom formats,” notes Taccolini. “Some vendors even try to litigate against users who try to open the data structures.” As a result, it was difficult to add management tools or switch to another programming solution.
>> How to Assess Change Management Software: Click here
In addition to the growing use of embedded tools to manage software change, a number of other trends are also having a significant impact. Three of these trends are the proliferation of networks in industrial automation, the growing use of PC-based controls, and greater functionality and sophistication in automation programs, according to Bob Trask, PE, senior engineer at Beckhoff Automation LLC (www.beckhoff.com/usa).
“Perhaps the single most important factor is that automation systems are becoming more networked than ever before,” he observes. “Networked PC-based control systems typically run more than one major kind of software.”
And with automation becoming increasingly more complex over the years, collaboration has become standard practice, especially among large manufacturers and their vendors. Consequently, not only do more people have the ability to access the code, but they also have the need to do so. The resulting environment has only compounded the need to track changes.
For this reason, end users have turned to a number of software tools to manage revisions and log changes to code on their machines. “Popular packages include Visual SourceSafe and Team Foundation Server [both from Microsoft], and Subversion from Apache,” reports Trask.
Code integrity
Beyond revision management tools, auditing tools used to discover any changes are also used to a large degree. The two primary audit methods involve controller-based auditing and server-driven approaches. Examples of these include the ability of Rockwell Automation’s Logix controllers to track as many as 36 functions on a machine rollup without a server-based audit engine. On the server-based side, there is a server-driven approach such as Rockwell Automation’s AssetCentre product, which is used to manage, rollup and automate audit functions.
These tools have value beyond audits because they can also help users and OEMs keep their code libraries up to date and simplify field maintenance by identifying areas of change quickly. Dave Rapini, product manager for RSLogix 5000 controllers at Rockwell Automation (www.rockwellautomation.com), points to one end user in particular who upgrades his machines regularly. This user, according to Rapini, has developed a library of machine code with unique signatures on each critical segment.
“He has built modular code to use the signature value for add-on instructions as a form of code integrity check,” explains Rapini. “The code polices itself. If someone uses the wrong version of a specific code segment, the machine shuts down.”
>> Virtual Technology Equals Real Code Control at a Gas Plant. Click here for more information.
As an alternative approach, some machine builders have been dividing their code into stand-alone modules to replicate the success they have had in applying modularity to mechanical and electrical components.
Because these self-contained building blocks have been validated and certified independently of one another, any revalidation process triggered by subsequent changes are limited to the modified module. Another benefit to modular design is that it permits teams to work on their portions of the project in parallel, which compresses time to market.
>> Sales vs. Engineering: Read-only CAD drawings can help Sales communicate changes to Engineering without incident. Read more at http://bit.ly/awslant68
Machine builders and automation vendors, however, have not been quite as successful at applying the principle to automation software as they have to physical components, according to Robert Muehlfellner, automation director at B&R Industrial Automation Inc. (www.br-automation.com) “Although you can develop the code in parallel and then bring the modules together through your export-import mechanisms, they have to come together into a single access point—the controller,” he explains. “This becomes a bottleneck during the testing phase of your code because typically only one person can do that at any time.”
To permit the kind of parallel testing necessary to alleviate this bottleneck, B&R Automation has introduced a tool called Application Modules. “It modularizes the run time on our controllers, as well as provides the necessary modularization in the engineering tool,” says Muehlfellner. “Everything can be done in parallel.” This includes doing iterative design on individual modules, debugging the code, and verifying the results before the modules ever reach a piece of equipment.
Accommodating upgrades is easier, too. Consider a packaging machine that fills and seals containers. “If I want to optimize my sealer somehow, I can now just change that application module,” says Muehlfellner. “As long as the interfaces between the modules remain stable, I don’t need to touch the code in the other modules."
