Demo-Proof: Five Gates That Separate Working Systems From Stage Theater

Every engineering lab has lived some version of the same story. A prototype performs flawlessly in the conference room. The executive demo lands. Six months later, the same system is either quietly abandoned or — worse — running in production as a fragile liability nobody wants to touch.

The gap between those outcomes is rarely raw technical quality. It is evaluation discipline. Most labs assess prototypes against the question "does it work?" when the operative question is "does it survive contact with production?" The five gates below are designed to answer the second question, and each can be adopted by a lab lead this week.

Gate One: The Reproducibility Audit

The bar: a competent engineer outside the original team can reproduce the system's core result from the repository alone.

Run it by selecting a contributor who did not build the prototype, giving them the repo and nothing else, and timing how long a clean reproduction takes. Under four hours with no verbal handoff is strong. Anything beyond two days signals that the system's real specification lives in the team's heads.

Score it: 3 — reproduced with no assistance; 2 — reproduced with documentation fixes logged along the way; 1 — reproduced only after live help from the original team; 0 — not reproduced.

Gate Two: The Failure-Mode Review

Demos show what a system does under ideal conditions. A failure-mode review forces the team to enumerate what it does under hostile ones: corrupted inputs, network partitions, dependency outages, adversarial users, and the perennial favorite — the happy-path retry loop that becomes a self-inflicted denial of service.

The deliverable is a written catalogue of failure modes, each paired with an observed or tested behavior. Untested guesses do not count. A system whose failure modes are mostly "unknown" is not production-ready, whatever its demo looked like.

Score it: 3 — all major failure modes tested with documented behavior; 2 — majority tested; 1 — catalogue exists but is largely speculative; 0 — no catalogue.

Gate Three: The Dependency Stress Test

Prototypes are built in a generous world: current versions, stable networks, cooperative services. Production is not. This gate subjects the system to deliberately degraded conditions — a dependency pinned two major versions back, a dependency registry outage, a 10x traffic spike, a cold start from an empty cache.

The question is not whether the system degrades — everything degrades — but whether it degrades gracefully and whether anyone knows where the cliffs are. Systems that hard-fail when a single upstream dependency hiccups have a blast radius their owners have not yet reckoned with.

Score it: 3 — survives all degradation scenarios with documented behavior; 2 — survives with known, documented limits; 1 — fails scenarios but failure is contained; 0 — fails uncontained.

Gate Four: The Operator Handoff Trial

The people who will run a system in production are rarely the people who built it. This gate makes that mismatch explicit before it becomes an outage.

Hand the system to the actual on-call or operations team with a runbook and nothing else. No builder in the room. Give them three scripted incidents and one unscripted one, then measure time-to-diagnosis and, more importantly, what they had to invent from scratch to cope. Every improvised workaround discovered in the trial is a gap in the runbook — and a future 2 a.m. page.

Score it: 3 — operators resolve all incidents using documentation alone; 2 — minor gaps filled in real time; 1 — operators require builder support; 0 — no handoff attempted.

Gate Five: The Exit-Cost Assessment

The most neglected gate. Before anything ships, someone writes down — concretely — what it would cost to unwind it. Which systems depend on it? What data would need migration? What is the contractual and organizational surface area? How many quarters to safely retire?

Exit-cost thinking inverts the usual enthusiasm of a successful prototype. A system that is cheap to enter and ruinous to leave is not an innovation; it is a hostage situation with a launch party. The assessment also disciplines architecture: teams that know exit costs are measured design for reversibility from day one.

Score it: 3 — exit plan documented, tested at component level, costed; 2 — plan documented but untested; 1 — informal understanding only; 0 — nobody has discussed it.

Reading the Scorecard

Total the five gates out of fifteen. Twelve or above: production-ready, with residual risks documented. Eight to eleven: promising, but fix the weak gates before rollout — and know which ones they are. Below eight: this is a demo, and it should be treated as one, whatever the roadmap says.

None of these gates is exotic. That is the point. The distance between a compelling prototype and a production system is not a mystery; it is a checklist that most organizations never write down. Labs that adopt this discipline stop shipping theater — and start shipping systems that are still standing a year later.