Innovation without theatre: decision gates for product experiments

The central idea: Innovation becomes operational when experiments have a…

Innovation becomes operational when experiments have a decision, a bounded cost, a named owner, and a clear condition for stopping.

Experimentation programmes are usually judged on volume: tests run per quarter, features validated, learnings captured. That framing is comfortable because every test produces something, and it disguises the question that matters, which is whether any test changed what the organisation would otherwise have done. A programme can run forty experiments in a year and shift no roadmap decision, and its dashboard will look healthy throughout.

What changed, and why it matters now: Track learning velocity, cost per resolved assumption,…

The signature of the problem is the learning that gets filed. When a test result is written up as an insight rather than as a decision, it is because no decision was attached to the outcome in advance. Read any experiment repository and count how many entries end with a choice — we stopped, we shipped, we changed direction — versus how many end with an observation. The ratio is the honest measure of the programme, and it is usually uncomfortable.

Build the operating model: Take the three largest active experiments and write the…

Use gates for problem evidence, solution evidence, delivery feasibility, and scalable economics. Each gate should require a small set of artifacts rather than a presentation performance.

Pre-register the decision, not only the hypothesis. Before the test runs, write what result leads to shipping, what result leads to stopping, and what result leads to a further test — and specify the third narrowly, because it is the escape hatch that swallows programmes. If every ambiguous outcome licenses another round, the gate is decorative and the programme will run indefinitely on inconclusive results.

Measure what the decision produced: The purpose of an experiment is not to learn. It is to…

Track learning velocity, cost per resolved assumption, experiments stopped early, and the time from validated evidence to a funded product decision.

Measure the proportion of experiments that produced a stated decision within two weeks of concluding, the number that ended in a stop, and the cost per decision rather than per test. The stop rate is the diagnostic: a programme that almost never stops anything is not testing, it is gathering supporting evidence for choices already made, which is a legitimate activity that should not be called experimentation.

Where execution breaks: Innovation becomes operational when experiments have a…

Teams can celebrate activity while uncertainty remains unchanged. Prototypes multiply, ownership blurs, and no one closes the loop.

The dominant methodological failure is underpowered tests read as though they were conclusive. A result that could easily have arisen by chance is treated as a signal, and the roadmap moves on it. The organisational failure is worse and more common: the test that contradicts a senior preference is reclassified as directional, and a further test is commissioned, which continues until a result agrees.

What this looks like in practice: An experiment that cannot fail is a demonstration. The…

In practice the change is procedural rather than statistical. A one-page pre-registration before every test, naming the decision rule and the person who will apply it. A standing rule that results are read by someone who was not advocating for the feature. And a repository where the decision, not the insight, is the field that cannot be left empty — which alone removes most of the programmes that quietly stopped deciding.

The strongest argument against this: Innovation becomes operational when experiments have a…

The reasonable objection is that early product work is genuinely exploratory, and forcing a decision rule onto a test designed to build understanding produces false rigour. Discovery work often cannot state in advance what result would matter, because the point is to find out what the question is.

That is true and it argues for labelling rather than for abandoning gates. Exploratory work should be named as such, budgeted separately, and exempt from the decision requirement — and it should be a minority of the programme. The failure to distinguish is what allows a programme to describe all of its work as discovery while presenting its conclusions as evidence, which is the arrangement that produces confident roadmaps built on nothing testable.

What belongs in a one-page pre-registration

Five things and no more: the decision the test informs, the metric and the direction that would count as support, the threshold at which the team acts, the sample or duration required, and the person who applies the rule. Anything longer stops being written before tests that are running next week.

The threshold is the field people resist, because naming it in advance removes the room to interpret afterwards. That removal is the entire purpose. A threshold argued about before the data exists is a design conversation; the same threshold argued about after is a negotiation, and the party with more seniority tends to win it.

Naming the person matters more than it appears. Decision rules with no owner are applied by committee, and committees under ambiguity default to running another test. One named person, applying a rule they helped write, resolves in an afternoon what a group will defer for a month.

Stopping rules, and why they are the hardest part

Every experimentation framework includes stopping in principle and most organisations have never stopped anything. The reason is not stubbornness; it is that stopping is the only outcome with a visible cost and no visible benefit. Shipping produces a feature, continuing produces activity, stopping produces a gap in a roadmap and an awkward conversation.

The counterweight has to be structural. Report stops as an output alongside ships, in the same summary, with the resource they released named explicitly. A quarter in which a team stopped three things and redeployed the capacity is a successful quarter, and it will not read as one unless the reporting says so.

It also helps to separate stopping the experiment from stopping the idea. Many things are worth stopping now and revisiting when a precondition changes; recording the precondition converts an uncomfortable kill into a scheduled review, which is both more accurate and easier to agree to.

Reading a result you did not want

The most valuable capability in an experimentation programme is the ability to accept an unwelcome result quickly, and it is almost entirely a matter of who reads the data. A result interpreted by the person who proposed the feature will be interpreted charitably, not through dishonesty but through ordinary investment in an idea.

The cheap structural fix is to have results read first by someone with no stake, who states the outcome against the pre-registered rule before any discussion of what it means. Once the rule has been applied aloud, reinterpretation becomes visible as reinterpretation.

Teams that adopt this generally report the same thing: the number of tests falls and the number of decisions rises. That is the correct direction, and it is frequently mistaken for a productivity problem in the first quarter, which is worth warning leadership about before it happens.

A 30-day implementation sequence: Innovation becomes operational when experiments have a…

Take the three largest active experiments and write the single decision each must enable. Pause any experiment without one.

Week one, audit the last twenty experiments and classify each as ended in a decision or ended in an insight. Week two, write the one-page pre-registration template and use it for everything starting from now. Week three, appoint the neutral reader and have them apply the rule to the first results aloud. Week four, publish the quarter's stops alongside its ships, with the released capacity named.

Report stops beside ships

Maintain one register with every experiment, its pre-registered decision rule, the outcome, and the decision taken. The register makes two things visible that nothing else does: how often the rule was overridden, and by whom. Neither is intended as an accusation, and both change behaviour simply by being recorded, because overriding a rule you wrote is easy in a meeting and uncomfortable in a log.

Review the register quarterly against decision rate and stop rate, and review the thresholds annually. If almost nothing is stopping, the thresholds are set where they cannot fail. If almost everything is stopping, they may be set for a level of certainty the business does not need. Change one threshold at a time and keep the previous value alongside it.

Editorial conclusion: Innovation becomes operational when experiments have a…

The purpose of an experiment is not to learn. It is to decide, cheaply, before the expensive commitment. Programmes that measure learning accumulate documents; programmes that measure decisions accumulate freed capacity and a roadmap that reflects evidence. The difference between them is one page written before the test rather than a report written after it.