If version B changes the headline, photograph, discount, and landing page, its result can help you choose between two complete ads. It cannot tell you which ingredient to repeat. Decide which of those questions you need to answer before making the variants.
Connect the test to a concrete decision
Name the choice: “Which headline should we use for the next booking-app campaign?” Then choose the primary measure. If the campaign needs completed inquiries, clicks alone cannot settle that choice. Track the cost per inquiry, and keep inquiry quality as a guardrail if low-quality submissions would make the apparent winner unusable.
Write a hypothesis that identifies the change: “Leading with fewer missed appointments will produce more completed inquiries than leading with easier scheduling.” This gives the designer two messages to express and the buyer a comparison to set up. It makes no promise that the challenger will win.
Change one planned variable at a time
For that headline comparison, keep the image, offer, call to action, destination, audience definition, and placement the same. Put the exact control and challenger copy in the variation matrix, alongside the files that implement them. A designer should not have to guess whether changing the image is part of the brief.
Use the buying platform's experiment controls when they provide the split you need, and read how that experiment assigns delivery. Merely running two ads together does not establish comparable exposure. If one ad ran last week and the other during a promotion, use the report as an observation with those differences attached; it cannot isolate the headline's effect.
Set the review and stop conditions
Before launch, set a review date and delivery or outcome requirements appropriate to the experiment. Use its documented stopping and uncertainty rules rather than inventing a universal conversion count. Also name the smallest difference that would change your buying or production decision; a tiny apparent gain may not justify replacing a proven asset.
Define early stops for a broken link, incorrect offer, tracking failure, or unacceptable spend. Those are operational stops, not evidence that the other creative won. Record edits during the run: replacing a headline halfway through changes what the test compares.
Read the result in context
Read the primary measure and guardrail together. In a constructed example, B brings cheaper inquiries but a much smaller share qualifies. If qualified inquiries are the campaign's job, lower acquisition cost alone does not earn B a rollout.
Check the experiment's uncertainty and delivery report before naming a winner. Distinguish three decisions: adopt a version, keep the current version because no useful difference was established, or rerun because the comparison broke. An inconclusive test does not establish that the ads perform equally.
Keep the conclusion within the tested audience and placement. An unexpected segment pattern can suggest the next test; it should not silently replace the question you planned to answer.
Write the plan in five lines
Copy this record into the campaign brief and replace the examples:
- Decision: Choose the headline for the next prospecting feed campaign.
- Hypothesis: Fewer missed appointments versus easier scheduling; exact approved copy and variant IDs attached.
- Comparison: Same audience definition, placement, offer, image, and destination; experiment setup linked.
- Measure: Cost per completed inquiry, with inquiry quality as the guardrail.
- Review: Named owner, review date, evidence requirements, and operational stop conditions.
After the review, append the result and the action it supports. For a decline in an existing campaign rather than a planned comparison, start with creative fatigue signals.
