How does AI ad testing work? The variant funnel, explained
AI ad testing works by generating a real batch of creative variants off one brief, running all of them against live audience signal, and putting media budget entirely behind whichever cut the data proves — rather than betting the whole budget on a single pre-approved concept.
A traditional agency pitch is a single bet dressed up as a sure thing: one concept, one board, one director's note explaining why this is the idea. It's a good process for winning a pitch. It's a bad process for winning attention, because attention is decided by an audience that was never in the room.
Our process starts from the opposite assumption — that nobody, including us, can reliably predict which cut wins before real people see it. So instead of committing to one concept, we generate a batch: the same brief expressed through different hooks, different pacing, different formats.
Data doesn't have taste. That's exactly why we trust it.
Stage one: generate wide
A dozen cuts isn't a lack of conviction — it's where conviction is supposed to come from. Each variant in the batch shares the brief's non-negotiables (the claim, the tone, the product truth) but is free to differ everywhere the brief left open: the hook in the first two seconds, the pacing, whether it's shot as a TVC, a film, or UGC. That range is deliberate. A batch where every variant looks the same isn't really testing anything — it's the same bet with extra steps.
This is the stage where AI production earns its structural advantage over traditional shoots: generating twelve genuinely different cuts costs a fraction of what one traditional shoot day costs, which means the batch can be wide enough to actually surface a surprise instead of just confirming what the team already assumed would win.
Stage two: test hard
Every cut in the batch runs against real audience data before a dollar of media spend is committed. Most of them die there, and that's the point — a variant that doesn't survive contact with a real feed was never going to survive contact with a real campaign. Signal, not opinion, decides what happens next.
What's left after this stage isn't the cut the team liked best in the room. It's the one the data already proved — which means the argument for scaling it has already been made before anyone has to make it in a meeting.
Stage three: scale narrow
Budget moves last, and it moves in one direction: entirely behind the surviving cut, not split evenly across a portfolio of guesses to hedge against being wrong. Concentrating spend behind a data-confirmed winner produces a materially different return than spreading the same budget thin across several unconfirmed concepts.
That's the whole method in three stages: generate wide, test hard, scale narrow. It's less exciting to describe than 'a big creative idea,' but it consistently outperforms a single bet — because it never actually asks anyone to bet.
How is testing AI-generated ad variants different from A/B testing a single ad?+
Standard A/B testing usually compares small variations of one existing creative — a different headline or thumbnail. Variant-funnel testing generates a genuinely different batch of full creative concepts off the same brief before any of them run, then lets real audience data decide which concept — not just which detail — gets the budget.
How many variants should a first batch include?+
Most first batches we scope land between eight and twenty-four variants, depending on how many formats and hooks the brief can reasonably support. Fewer than that and the batch risks converging on the same idea from slightly different angles instead of genuinely testing different bets.
What happens to the variants that don't win?+
They're not wasted — a variant that under-performs still tells you something concrete about what didn't land, which sharpens the brief for the next batch. But no further budget goes into producing or promoting them once the data has made the call.
Ready to find what wins?
Tell us what you're shipping — we'll scope the first batch.
