Inversion: Designing Industrial Automation By Studying How It Fails

"The most reliable automation projects I have seen did not begin with a feature list. They began with a failure list."

Key Highlights

  • Start with how it breaks
  • Pair inversion with a margin of safety 
  • Design for the person who recovers it

 

The most reliable automation projects I have seen did not begin with a feature list. They began with a failure list.

Before anyone drew a sequence or named a PLC tag, the team sat with operators, maintenance and a few engineers and asked one awkward question: If this line must fail, how will it fail?

Not in theory. On this floor, with this product, on a Friday night when the night shift is short-handed and the spare-parts locker is half empty.

That question is inversion. Carl Jacobi told his students, "Invert, always invert." Charlie Munger later made the idea familiar outside mathematics: Instead of asking only how to succeed, ask what would make you fail, then refuse those things. In a plant, that habit beats another slide of happy-path logic.

Start with how it breaks

A failure list is not a risk register that dies in a shared folder. It is a short, ugly set of scenes the team can picture. A sensor stuck high after a wash-down. A conveyor that keeps running after a jam because the jam detector was "temporarily" bypassed. A recipe download that partially succeeds and leaves the batch half old, half new. A valve that opens while a broken feedback wire makes the HMI show closed.

Write those scenes in plain language. Rank them by how often they happen and how much damage they do. Design against the top ones first. Features that do not protect a named failure can wait.

Put the failure list on the wall in the design review. The conversation stops being a tour of screens and becomes a walk through the ways the line can hurt people, scrap product or stop for a shift.

Pair inversion with a margin of safety

Inversion tells you what to avoid. A margin of safety tells you how much room to leave when you are wrong.

Benjamin Graham used the phrase for investing: Leave a gap below what you think something is worth, so a mistake does not wipe you out. Engineers know this already. A bridge rated for ten tonnes is not built to fail at ten and a half.

On an automated line, that margin can look like waste when everything is calm: an interlock that rejects a cycle before the part is damaged; a short buffer so a brief upstream stop does not cascade; a second confirmation of a critical state because sensors get dirty; a safe stop operators trust, so they use it instead of jumping a wire.

Ask both questions in the same review. What would make this fail for certain? And where is the gap that lets us be wrong without scrap or injury? If the answer to the second is "We hope it does not happen," you do not yet have a design. You have a wish.

Design for the person who recovers it

Hardware and software fail. People recover them, or they do not.

The operators who recover fastest are not the ones who memorized every alarm code. They understand why the system exists: what the interlock protects, which state is safe to leave the line in, and which "quick fix" creates a worse failure tomorrow. When operators help write the failure list, they already know the scenes. Under pressure they recognize the jam-bypass story.

Train recovery on the failure list, not only on the happy path. Walk the bypasses that must never become permanent. Show the reason for a stop in words the night shift uses. Leave a written path back to a known-good state.

I once sat with a night crew after a long stop caused by a "temporary" override that had lived for weeks. It had solved a noisy sensor on a busy day, and the failure list never named that path. After that, every review asked: Which temporary changes are still alive, and which failure do they hide?

The next override had an expiry date on the whiteboard.

A short practice exercise

On one cell or one sequence:

1. Write ten ways this automation fails on a bad shift. Use plant language, not vendor language.

2. Mark the three that hurt most. Design or revise against those first.

3. Name one margin for each: extra confirmation, buffer, interlock, or a clear safe state.

4. Ask one operator to walk the recovery for each of the three. Change the design until that walk is boring.

Inversion does not make automation pessimistic. It makes it honest. Failure modes on a real floor are easier to see than a perfect success path drawn in advance. Study them first, leave room for being wrong, and give the people who run the line a way back.

Industrial automation earns trust not by promising it never fails, but by showing it was designed with failure in mind.

Dr. Krisztián Galánthai Fekete is the author of "Find Your Meaning in 60 Days, In the Age of AI and Robotics"

Sign up for our eNewsletters
Get the latest news and updates