← Glossary
// OUTPUTS

Verdict

The outcome of an experiment after the end date or a threshold is hit. Three options: ship, kill, or review.

// what it is

A verdict is not an opinion. It is what the contract says the result is. Ship if the success threshold was met. Kill if the kill threshold was hit, or if the end date arrived without clearing the success threshold. Review if the data is ambiguous and the contract did not anticipate the case.

The verdict is what makes the experiment compound. Each verdict (ship or kill) becomes a row in the archive that the next experiment can build on.

// when this matters

When to use it

Forced by the contract at the earlier of "kill threshold hit" or "end date reached." Never delayed because the team is uncomfortable with the outcome.

// deeper

What this looks like in practice

The verdict has three valid values and only three: ship, kill, review. Ship means the success threshold was met and the variant becomes the new default. Kill means the kill threshold was hit, or the end date arrived without clearing the success threshold; the variant is discarded. Review means the data is ambiguous and the contract did not anticipate it; the team learns and writes a sharper contract for the next attempt.

A verdict of "kill" is not a failure of the experiment. The experiment did its job: it gave you the answer. The failure mode is running an experiment, learning the answer is "no," and then shipping anyway because someone is attached to the work. The contract exists to prevent exactly that.

Verdicts are also the unit the archive stores. Each row is a contract plus a verdict plus the evidence behind it. Six months later, a new team member can read the archive and understand the team is operating principles without asking — the verdicts speak for themselves.

// pitfalls

Common mistakes

  • Inventing a fourth verdict."Partial ship" or "soft kill" are usually rationalizations. If the contract did not anticipate the case, the verdict is review and the team writes a sharper contract.
  • Not recording the verdict.A verdict that lives in Slack is not in the archive. If the team cannot find it in three months, it might as well not exist.
  • Treating "ship" as the only good verdict.A clean kill, recorded with the contract behind it, is more valuable than a hesitant ship. Both teach. Only one wastes downstream effort.
// related

Related terms

Experiment contractThe agreement, written before an experiment runs, that says what is being tested, how it will be measured, what kills it, and when it ends.Kill thresholdThe metric value that ends an experiment early. You set it up front so you do not negotiate with yourself when the numbers come in.Success thresholdThe metric value above which the experiment is shipped. Set up front, alongside the kill threshold, so the verdict is decided by the rules and not by the room.Experiment archiveThe record of every experiment a team has run, with the contract, the metric movement, and the verdict. The compounding asset of an experiment-driven team.

Pick a hypothesis. Vocabulary done.

The fastest way to learn this vocabulary is to commit one experiment. The contract takes about five minutes to write.

Run your first experimentBrowse the field guides