Why I built a multi-model AI panel for marketing decisions

Ask one AI model for a marketing strategy and you get something dangerous: a confident, fluent, well-structured answer with no way to tell which parts are insight and which parts are the model's habits. For low-stakes questions, fine. For decisions with money attached, not fine.

So I built a panel.

The design

Three models from three vendors — in the run described below: GPT-5.5, Claude and Grok, same prompt, same day — answer the same question independently. Then the adversarial machinery kicks in:

1. Cross-critique. Each model critiques all three answers — anonymised and shuffled, so no model knows which is its own.
2. Revision. Each model rewrites its answer under the critiques it received.
3. Blind judging. Each model scores the anonymised finals on accuracy, completeness and clarity (each out of ten, unweighted), and picks a winner.
4. Consensus synthesis. The winning model writes a verdict that explicitly separates what all three agreed on from where they diverged and how the divergence was resolved.

To be precise about what's what in that pipeline: the scoring aggregation is deterministic code — three judges × three criteria, summed out of 90, majority pick breaking ties. The critiques, revisions and the synthesis verdict are model-written, and the judges are scoring the quality of the argument, not verifying its facts — fact-checking against real data stays a human step, as the example below shows. The anonymisation matters more than it looks: LLM judges measurably favour their own writing when they can recognise it — blind judging is the counter.

Why the full structure, rather than just pasting one model's answer into another and asking "critique this"? Because the informal version has no defence against self-preference, no revision under pressure, and no record of where the models split — and the split map is the product.

What the disagreement teaches you

The unanimous points are a strong prior — not proof: the three vendors train on heavily overlapping data, so shared blind spots survive consensus (which is why external data can still overrule a unanimous panel). The divergences are the real product: a map of exactly where human judgement is needed, which is the opposite of what a single confident answer gives you.

When I ran this site's relaunch strategy through the panel (July 2026 — the verdict is recorded, and excerpted in the case study), all three models independently converged on the same core positioning. The blind judging scored the winning answer 81/90, picked by two of the three judges — but the consensus verdict, not the winning answer, became the working strategy, because the divergences were where the real decisions lived.

One concrete example. The winning answer named this site's content service page "content and conversion consultant". When I checked the framing against live Google Ads data (DataForSEO, July 2026), that phrasing had no measurable search demand at all — while "content marketing consultant" showed 390 searches a month. The final call went against the winning model, with the data. That's the point of the exercise: the panel doesn't make your decisions, it shows you precisely which ones are yours to make — and external evidence still outranks any model, winner or not.

What it doesn't do

Honest limits, because they shape when it's worth running. Consensus isn't truth — the models share training data and can fail together. The judges score writing, not facts: factual claims still get checked against non-AI sources (keyword data, analytics, the client's own numbers) before anything is implemented. A run isn't free — a few pounds of API calls and a quarter of an hour, against seconds for a single prompt — so I use it for decisions with money attached, not for subject lines. And anything confidential goes to three vendors instead of one, which is a real consideration with client strategy; sensitive inputs get stripped first.

The general principle

This is decision hygiene that predates AI — independent estimates before discussion, blind review, structured disagreement have been standard in forecasting practice since the Delphi studies of the 1960s. What AI changes is the cost of doing it properly: a disciplined multi-model panel costs less than one hour of one consultant.

You don't need my exact tool. But if you're using AI for anything strategic, never accept a single model's answer on a question that matters — get a second vendor's model to attack it, minimum. Full disclosure: I sell the panel treatment as a service, so read this as a builder's argument for the method, and judge the method by the evidence above.

Evidence for this article: the Super AI Panel case study (system design and interface); the recorded relaunch-strategy verdict of 21 July 2026 (scoreline and divergences quoted above); Google Ads UK keyword volumes via DataForSEO, 21 July 2026; self-preference bias: Panickssery, Bowman & Feng, "LLM Evaluators Recognize and Favor Their Own Generations" (2024).