Does the potential for failure somewhere in your automated process worry you? Will the unexpected downtime and spoilage cost you in money, time and aggravation?
If so, perhaps you ought to consult with experts like those at the Center for Intelligent Maintenance Systems (IMS Center, www.imscenter.net) at the University of Cincinnati. Professor Jay Lee is concentrating research there on what he calls “the worry-free factory.” Patrick Brown, the IMS Center’s program manager, says, “We want you to have all the information available for making the right decisions.”
This focus is indicative of an important trend among providers of diagnostic and predictive maintenance technology. Researchers at academic institutions and commercial vendors alike have been busy marshalling advances in sensing, networking and algorithms in holistic ways to streamline and automate the necessary flow of information. They have been aiming both their individual and collaborative efforts toward the common goal of improving predictive maintenance.
An example is the work at the IMS Center for installing a prototype of a wireless vibration sensor in the machine tools producing motorcycle engines at the Harley Davidson plant. Because spindle vibration greatly influences the quality of the engines, the manufacturing engineers at Harley Davidson wanted to monitor the health of the spindle motors so they could take corrective action early. The cutting fluids, chips and motion of the machine make installing wired sensors extremely difficult, so the engineers decided to explore wireless technology.
They turned to the IMS Center to solve the important problem of powering the sensor. Not having any wires to connect it to an external power source, the sensor must store its own power. Consequently, energy conservation was a central concern for the project. “The sensor has to be turned on at specific times, give out a few lines of data, and go back to an idle state to conserve energy,” explains Edzel Lapira, PhD and research associate at the IMS Center.
Besides looking for ways to add sensors for gathering prognostic and diagnostic data, Edzel and his colleagues are also working on nonintrusive methods for monitoring the health of automated equipment. This line of research is investigating how to extract more information from the signals and data that already exist in the machine and its controller. Not only does the approach save the cost of adding sensors and other hardware, but it also permits offloading the computations to another computer on the network or even to the cloud.
Industrial robots have been a good application for this approach. Because the multi-axis motion and close quarters makes attaching sensors to these machines quite difficult, researchers at the IMS Center often analyze the torque signals coming from each axis.
As components start to go bad, the torque signals change in predictable ways. “There is a lot of information in the patterns in these signals and in the relationships among them,” says David Siegel, senior PhD graduate researcher.
To interpret what these changes mean for the health of the robot, the IMS Center has been continuing to develop statistical-analysis and pattern-recognition tools for its Watchdog Agent software. The tools can process various kinds of signals to detect faults, diagnose problems and even predict performance. The latest of these tools uses a new technique called trajectory-based prediction, which matches a degradation pattern to previous patterns to determine how far along the degradation trajectory a developing problem is. With this intelligence, the software can predict how much useful life is left in the mechanism, so the user can schedule the necessary repairs.
Other tools help the researchers to develop good predictive models when too much data exists. “It could be that the duration of the data set is too long, or that there are too many variables, such as in a heavily instrumented system,” notes Wenyu Zhao, senior PhD graduate researcher. “The challenge in these cases is to figure out which variables are the most relevant to the failure mode or to the overall health of the equipment.”
GlobalFoundries Inc., a semiconductor manufacturer based in Milpitas, Calif., ran into this problem when it decided to develop a predictive model for an electrostatic chuck on etching equipment. Because manufacturing was monitoring about 100 signals and calculating more than 1,000 statistics throughout its processes for control and fault detection, the GlobalFoundries Inc. engineering staff was overwhelmed. Consequently, they turned to the IMS Center for help in sifting through all that information in a systematic way, and developing the prediction model.
The IMS researchers used the data-analysis and parameter-selection tools in its software to reduce the signals to a manageable set of about five. By picking the key signals and variables, they were able to generate the prediction model.
In the continuous process industries, vendors are developing tools help users to track the health of their field devices from the control room and diagnose most problems before sending technicians into the field. The AMS Device Manager and DeltaV historian from Emerson Process Management (www.emersonprocess.com), for example, are monitoring and troubleshooting the field devices in the steam-and-solvent process that Calgary-based Laricina Energy Ltd. is using to extract oil from bituminous-sand deposits in northeastern Alberta, Canada.
Because the field devices are smart transmitters equipped with self-diagnostics, not only do they regularly report their status to the device manager software, but they also send alerts when they are having problems. Technicians, therefore, can look at the data from the control room before going into the field. “And they can take along what they need,” says Russ Ritchie, the now-retired Laricina Energy Ltd. automation project manager responsible for the AMS Suite and DeltaV startup.
Ritchie reports that his counterparts in the mechanical group will eventually make use of the predictive maintenance capabilities of the asset-management software. For example, they will be able to be proactive by performing such tasks as vibration analyses on pumps to look for wear, cavitations and misalignments. “We on the automation side have put all the pieces in for them” says Ritchie. “The software and end devices run on the wireless [network], and the plants are already wireless-capable.”
Vendors specializing in connecting field devices to distributed control systems (DCSs) like DeltaV are also offering tools for monitoring the physical layer of networks and diagnosing developing problems on them. Among the first of these vendors was Pepperl+Fuchs (www.pepperl-fuchs.com) of Twinsburg, Ohio, which released its Advanced Diagnostics module about five years ago. Its module ties into an asset-management system and sounds an alarm when it detects deviations from a known baseline. This way, the maintenance department can correct developing problems before they take the network down.
The release of such products has bridged a gap that had existed in monitoring programs for a while. “End users have embraced field bus technology in large part because of the added diagnostic information that they could get from their instrumentation,” explains Bernd Schuessler, business development manager for Pepperl+Fuchs’ field bus remote I/O and wireless Hart products. “This information, however, can get back to a DCS or asset management system only if the physical layer remains intact.”
Before the release of tools for monitoring this layer, most users could only react to problems in it when they lost communications. They would have to send maintenance into the field armed with handheld tools for troubleshooting the problem. “By then, it’s already too late,” notes Scheussler; the process is down.
Tools like P+F’s Advanced Diagnostics Module, on the other hand, alert maintenance whenever a performance parameter exceeds a limit—well before a failure has a chance to occur. Using a built-in oscilloscope, engineers and technicians can look at segments of the network and take screen shots of noise, jitter and framing errors on the line.
“You can see exactly what the signal looks like at all times before you send somebody into the field,” says Aaron Severa, a product engineer at P+F. Besides preventing unplanned downtime, “this capability saves unnecessary trips,” he says.
Diagnostics modules can pay dividends in other ways, according to Chris Williams, an electrical engineer in the Controls Technology Dept. at the DAK Americas plant in Gaston, S.C. He has installed several of the modules in a process producing purified terephthalate acid (PTA) for making polyethylene terephthalate (PET), the plastic used for beverage bottles. The installation was part of an expansion in which the company installed advanced instrumentation that used Foundation Fieldbus.
Because the process involves various acids that eventually corrode instruments and cables over time, Williams wanted the ability to monitor their health from the control room and streamline their replacement in the field with a quick-change method used by a sister facility. To avoid the risk of igniting volatile organic vapors, yet permit having up to 16 instruments on each line running the long distance to the field, he specified a combination of high-power trunk lines and the Fieldbus intrinsically safe concept from Pepperl+Fuchs. With this design, the lines operate at full power until they reach a field barrier near the instruments, but outside the hazardous area. From there, they branch out to the field instruments with too little energy to ignite the vapors.
To mitigate the risk of a bad line taking down all of the instruments on it and shutting down the process, Williams installed a P+F diagnostics module on every four lines to monitor their health. “We installed the diagnostics on our Foundation Fieldbus with the primary intent to catch failures in the cabling systems before they could affect our live instrumentation and process control,” he reports. So far, the investment has not had a chance to pay for itself in this way because the cabling has yet to fail, he says.
However, the system has more than paid for itself in other ways: It has been able to prove to instrument vendors and contractors that the DCS and Fieldbus lines are working well and that the problem lies with a field device. “The first thing that vendors say is that you have a network problem,” notes Williams. “I simply show them screen shots of the oscilloscope traces and diagnostic fault logs built into the advanced diagnostics module to prove that the field instrument is indeed the problem, not the network.”
At startup, the module caught some termination errors and crossed wires missed during the physical checkout of the system, says Willams. Later, after the process had been online for a few months, it also found corroding wires inside a device not sealed properly—long before the DCS was aware that there was a problem.
While a growing body of users is installing diagnostics technology for their own operations and maintenance staffs, others are installing the technologies for use by remote service providers. An example is a recycling plant making packaging material in South Carolina. That site used a beta version of a new ServicePort portal from ABB Inc. (www.abb.com) to gain access to the automation vendor’s experts, configuration tools, diagnostic applications and other services from afar.
The mill had been a user of ABB’s DCS for a while and had just installed some of the vendor’s advanced quality control systems (QCSs). As part of its continuous-improvement program, engineers at the site consulted with their counterparts at ABB after installing the QCSs for help with cutting service costs. The portal seemed to be a way of making those cuts without sacrificing their ability to tap the vendor’s expertise in optimizing process performance.
Through the portal, ABB specialists were able to monitor, troubleshoot and optimize the process remotely. Not only did providing service online generate considerable savings in time and money, but it also has helped the site to increase product quality by reducing variability (the average paper basis weight 2-Sigma is below 0.5 and the average paper moisture level 2-Sigma is below 2).
The portal is a concept that grew over 15 years, evolving from the various service tools that ABB’s engineers had developed and refined for their colleagues who traveled from site to site troubleshooting problems on automated equipment. The tools automated the collection of data from process control systems, as well as its subsequent analysis. Although the software tools and associated hardware were originally designed to go to the job, their developers realized that service technicians could access and analyze the data remotely just as easily if the hardware were left at the customer site.
“Rather than building unique hardware, we developed a common infrastructure that interfaces nicely with anyone’s DCS,” says Kevin Starr, senior optimization engineer for the Process Automation Service at ABB Inc. in Wickliffe, Ohio. “This has allowed us to leverage our service delivery talent, grow our knowledge base and resolve problems in days that could have taken weeks, if not months, in the past.” Remote access also cuts travel costs, making it much more cost effective to offer these services more often on a regular basis.
“In short, we put the human back into service,” notes Starr, and there are advantages of having an expert in the loop. He believes that ABB’s decision to buck the trend of completely automating preventive maintenance will reduce the risk of false positives—a risk that has led to the death of many service products. What often kills them are the nuisance alerts triggered by false positives. To avoid wasting time on these alerts, users set alarm thresholds higher and “then the inevitable happens: A failure does occur, and no one gets notified,” he says.
Starr says that automating the human being out of the loop through continuous monitoring is sometimes the right answer, and points to his company’s 800xA Asset Monitors. “However, there are other instances where degradation over time needs to be identified by human experts,” he explains. “Process degradation is, or can be, more expensive than hardware failures.”
ServicePort can automate this kind of monitoring by dividing raw data into asset classes and aligning them with key performance indicators connected to alarms that experts adjust based on historical performance. An example is loop optimization, a task that Starr notes was nearly impossible to do on large-scale automation. “Now, we can provide ROI-based solutions for thousands of control loops per day,” he says.
Starr points to a mill making fine writing paper in the Netherlands. The engineering staff there wanted ABB’s help in identifying loop inefficiencies on two paper machines. They gave ABB a list of control loops that they wanted analyzed on each machine, which uses ABB’s QCS, DCS and drives and motors.
A 10-person ABB team worked together remotely from locations in the United States, Canada and India to format, compile and scrutinize the data that they collected through ServicePort. The performance problems found by the service team ranged from oscillations to interactions. The team members also learned which key performance indicators were stable and which were not. Hence, they were able to provide the mill’s engineering staff with action items for improving the process.
