Marketing experiment

A marketing experiment is the unit of marketing work that has a contract attached. Instead of "let us try this and see," you write down what you are testing, what you are measuring, what would kill it, and when you will decide.

It is broader than an A/B test. An A/B test is a marketing experiment whose surface is a single page. A pricing change, an outbound rewrite, a paid-channel shift, and an onboarding step are all marketing experiments. The shape stays the same: hypothesis, metric, thresholds, dates, verdict.

When to use it

Any time a marketing decision is non-obvious, reversible, and big enough that being wrong would cost more than the test itself. Pricing changes, channel shifts, copy rewrites, onboarding sequences — all candidates.

What this looks like in practice

The defining feature of a marketing experiment is the contract that runs alongside it. Without the contract — written before any data arrives — what looks like an experiment is usually a launch with extra steps. The contract is what forces a verdict at the end instead of a recap.

Marketing experiments are the unit a team can stack. A single experiment is a question. A portfolio of experiments is a learning rate. A team that runs four marketing experiments a quarter and writes each verdict down compounds knowledge faster than a team that runs forty unmeasured campaigns and remembers none of them.

Marketing experiments do not require statistical significance to be useful. Most marketing decisions run on N too small for p-values. The discipline that matters is committing the kill threshold and end date in advance — that is what stops drift, regardless of whether the math is rigorous.

A worked example

Common mistakes

• Calling every change a "test."A change is only an experiment if there is a written contract attached. Otherwise it is a launch. Both are valid; only one compounds.
• Picking a metric that cannot move in the timeframe.If your end date is two weeks and your metric is "annual revenue," the experiment will land in the review bucket and teach you nothing.
• Negotiating with the kill threshold mid-flight.The threshold was set when the data was abstract. The instinct to "give it another week" when the metric tanks is the bias the contract exists to prevent.

Related terms