Moving Beyond Checklists to Evidence-Based Commissioning of Manufacturing Startups

Factory-acceptance testing verifies equipment, but capability-based startup readiness proves integrated automation systems can deliver sustainable production performance.

Key Highlights

  • Traditional readiness checklists focus on hardware and software completion, but do not guarantee production capability.
  • Defining specific production capabilities and mapping dependency chains helps verify operational readiness across the entire system.
  • Scenario-based testing exposes how the integrated system responds to real-world operational disturbances and constraints.
  • Evidence states with explicit definitions improve clarity and prevent false assumptions about system readiness.
  • A comprehensive readiness package supports traceability, accountability and informed decision-making during plant startup.

An automated line can complete factory-acceptance testing, arrive at the plant, pass individual equipment checks and still be unready for production. The gap is not always unfinished hardware or incomplete code. More often, it is unproven behavior across the installed system.

Factory-acceptance testing answers an essential but bounded question: Does the supplied automation perform the specified functions under the conditions available at the supplier’s facility? Startup readiness asks a broader operating question: Can the plant execute complete production capabilities when controllers, robots, safety systems, utilities, manufacturing software, material flow and people must all work together?

The distinction is reflected in the International Society of Automation’s ISA-105 framework, which treats factory-acceptance testing, site-acceptance testing, site-integration testing and commissioning as related but distinct activities. It is also consistent with ISA-95, which addresses information exchange between manufacturing-control and enterprise functions. Together, these frameworks point to a practical conclusion: verification of a machine or software package is not the same as proof that the installed production system is ready to operate.

For automation teams, the answer is not another longer checklist. It is a readiness method built around production capabilities, dependencies and demonstrated evidence.

Define readiness in terms of production capabilities

Traditional readiness lists are usually organized around project deliverables: equipment installed, power available, network connected, code loaded, training scheduled and punch-list items closed. These are useful indicators of progress, but they do not prove that production outcomes can be achieved.

A stronger approach begins by defining the capabilities the plant must perform. Examples might include releasing and executing a production order, starting the line from an idle state, maintaining required output, completing a controlled stop, continuing in an approved reduced-capacity mode and restoring operation after a system interruption.

Each capability should have a readiness record containing:

  • The operating boundary and expected result
  • The systems and conditions required for execution
  • The test environment and production scenario
  • The evidence collected during demonstration
  • Any remaining restriction and its operational consequence
  • The individual authorized to accept the result

This structure changes the discussion from, “Is the controls work complete?” to, “Has the plant proved that it can execute this production capability under installed conditions?”

Map the dependency chain from command to confirmed outcome

Production capabilities depend on chains of decisions and confirmations. A request may begin in an enterprise or manufacturing-operations platform, move through line supervisory control, pass into several PLCs and motion systems, and depend on valid recipes, material availability, quality status, safety conditions and downstream capacity.

The readiness team should map the runtime chain for each critical capability. The map should identify the initiating command, prerequisites, decision points, acknowledgements, completion signals and the system that owns the final state.

Consider the release of a production order. The order may be valid in the manufacturing execution system, yet execution can still fail because a controller has an old product code, an inspection system has not loaded the correct parameters, a packaging cell is unavailable or the warehouse system cannot accept finished inventory.

None of those conditions may appear during isolated equipment testing.

This is why site integration must test the complete command path. The objective is not simply to show that data can be transmitted. It is to prove that every receiving system interprets the transaction correctly, acts on the same production intent and reports a consistent result.

Prove state and data reconciliation across systems

Many difficult startups are not caused by a complete loss of communication. They are caused by disagreement after communication is restored.

A message may be delivered twice. A controller may restart while a supervisory application retains the prior state. A product may physically cross a transfer point while its digital record remains upstream. A higher-level system may issue a new command before the equipment has reconciled the previous transaction.

Readiness testing should therefore examine state ownership and reconciliation. For every important cross-system transaction, the team should determine:

  • Which system is the authoritative source for the current state?
  • How are duplicate or delayed commands recognized?
  • What happens to a transaction that is interrupted after physical movement begins?
  • How are product identity, counts and quality status reconciled?
  • What must be true before a new command is accepted?
  • Is manual correction controlled, recorded and visible to the other systems?

NIST research on interaction-driven manufacturing systems has long emphasized that reliability depends on the behavior created when software components are combined, not only on the conformance of each component. In today’s automation architectures, that principle applies directly to PLC-to-PLC transactions, MES-to-line exchanges, warehouse interfaces and the digital records associated with physical production.

Use scenario-based site integration testing

Component tests are built around devices and functions. Startup-readiness tests should be built around operating scenarios.

A scenario defines an initial plant state, a production objective, a deliberately introduced condition and an expected system-wide response. It should also define what evidence will demonstrate successful completion.

A useful scenario set can include:

  • Startup from a fully stopped condition with all required services restored
  • Production with one noncritical subsystem unavailable
  • A downstream constraint that forces controlled reduction of upstream output
  • Loss and restoration of communication between control and manufacturing software
  • Interruption during an active product or material transaction
  • Restart after a coordinated plant stop
  • Restoration of the last approved configuration after an unsuccessful change

The goal is not to create an unlimited catalogue of failures. It is to select scenarios that cross organizational or technical boundaries and expose whether the complete automation architecture responds coherently.

For each scenario, record the initial conditions, event sequence, expected decisions, observed response, data-integrity result, recovery time and any restriction placed on startup. This produces reusable evidence and prevents the same issue from being rediscovered during production ramp-up.

Prove sustainable capacity as a system property

Peak speed is not the same as sustainable production. One machine may achieve its nominal cycle while the overall system loses output through small timing mismatches, repeated waiting states, data delays or insufficient recovery after disturbances.

A startup-readiness test should establish a performance envelope for the integrated system. The test should use a representative production mix and run long enough to reveal recurring constraints rather than only short-duration capability.

Useful measures include:

  • Time spent waiting for upstream or downstream conditions
  • Accumulation growth and depletion by production zone
  • Frequency and duration of short interruptions
  • Time required to return to stable output after a disturbance
  • Transaction latency between automation and manufacturing software
  • Reject and rework synchronization across systems
  • Output stability rather than a single maximum-rate value

The data must be time-aligned. PLC events, robot states, drive information, vision results and supervisory records should be correlated on a common timeline. Without this, a system-level constraint can be misdiagnosed as a local equipment issue.

The acceptance question is not “Did the line touch the target rate?” It is “Can the installed system sustain the required operating envelope and recover without accumulating instability?”

Replace red-yellow-green reporting with evidence states

Red-yellow-green dashboards are convenient, but their meaning often varies by workstream. One team may mark an item green when software is loaded. Another may require a witnessed test. A third may use green even though an operational restriction remains.

A readiness gate should use evidence states with explicit definitions:

Evidence state

Required meaning

Unverified

The capability has not been demonstrated under installed conditions.

Demonstrated with constraints

The capability works, but a documented restriction limits its use, performance or supportability.

Ready for controlled production

The capability has been proven for a defined startup window with accepted monitoring and response controls.

Ready for sustained production

The capability has met integrated, performance and ownership criteria without unresolved critical restrictions.

Status should be assigned to the production capability, not to the engineering discipline. A function cannot be declared ready merely because each contributing team has completed its own task. If one required dependency is unverified, the capability remains unverified.

This approach also avoids false precision. A line described as “92 percent ready” may still be unable to execute its most important production function. Capability-based evidence makes the remaining risk visible in operational terms.

Make the startup decision traceable

The final authorization to begin production should be supported by a concise readiness package, not scattered meeting notes and individual status reports.

For each critical capability, the package should show the accepted test result, applicable restrictions, monitoring requirements, responsible owner and criteria for stopping or rolling back production. It should identify which software and configuration baseline was demonstrated, so that later changes do not invalidate the evidence without review.

The package should also define ownership during the controlled startup period. When an integrated issue occurs, the plant needs a known path for triage across operations, controls, information technology, original equipment suppliers and system integrators. Without that structure, every problem becomes a debate over scope while production waits.

Operational ownership should increase deliberately as performance stabilizes. Project support can then reduce according to demonstrated capability and issue trends rather than an arbitrary calendar date.

Beyond FAT and checklists

FAT remains one of the best opportunities to remove automation risk before equipment reaches the site. Checklists remain valuable for organizing thousands of required activities. Neither is a complete measure of startup readiness.

Readiness is established when the installed production system can execute critical capabilities across technical boundaries; maintain coherent physical and digital state; respond to realistic operating scenarios; sustain its required performance envelope; and transfer into defined plant ownership.

For an automation-centric organization, that is the standard that matters. The question is not how many tasks are marked complete. The question is whether the evidence shows that the whole system is ready to produce.

About the Author

Muhammad Rafay Ikram

Muhammad Rafay Ikram is a Senior Engineering Project Manager whose career spans more than 14 years across industrial automation, controls, commissioning, robotics-enabled material handling, packaging systems and major capital-project delivery. His independent research examines predictive startup readiness, dynamic dependency modeling and constraint-aware industrial automation.

Sign up for our eNewsletters
Get the latest news and updates