AI Systems · Strategy · 2026 · in use
Super AI Panel: multi-model adversarial review for big decisions
Three frontier AI models answer independently, blind-judge each other's anonymised answers, and a winning model synthesises the consensus. Built because one model's opinion isn't a strategy.
The problem
Ask one AI model for strategy and you get one confident answer — with no way to know which parts are insight and which are that model's habits. For decisions with real money attached, that's not good enough.
The system
I built a zero-dependency tool that runs a panel:
- Independent answers — the same question goes to ChatGPT, Claude and Grok in parallel. None sees the others' work.
- Cross-critique and revision (Strategy mode) — each model blind-critiques all three anonymised answers, then revises its own under the critiques it received.
- Blind judging — each model scores all three anonymised, shuffled answers on accuracy, completeness and clarity, and picks a winner. No model knows which answer is its own.
- Consensus verdict — the winning model writes a synthesis that explicitly separates what the panel unanimously agreed from where it diverged and why the final call went the way it did.
Aggregation is deterministic code, not AI: scores are summed, tie-breaks follow explicit rules, and the whole run streams live so you can watch the disagreement happen.
Real use
This site's relaunch strategy was produced by the panel in Strategy mode: three independent strategies, cross-critiqued, revised, judged, and synthesised into the sitemap, positioning and content plan you're looking at. The panel's divergences — where to put the AI landing page, how many articles to launch with — were exactly the decisions worth human attention.
The recorded verdict — quoted from the run record so it's checkable, not just claimed:
Panel winner: ChatGPT (gpt-5.5), 81/90, picked by 2/3 judges. Recorded 21 Jul 2026.
Consensus rulings (override any single answer): AI landing page at root,
/ai-marketing-consultant, not under /services/ · no indexable blog category/tag pages at launch · 10 launch articles exactly, controlled cadence after · thin AI content → 410 unless a genuine replacement exists.
Where the panel was later overruled, the record shows that too: the winning answer's "content and conversion" page naming was replaced by /services/content-marketing-consultant after live keyword data (390 monthly searches vs none for the panel's framing) outranked the model's judgement — the process working as designed.
That's the pattern I bring to client work as AI marketing consultancy: AI opinions are cheap; structured disagreement between independent models is where the signal is. More on the thinking in why I built a multi-model AI panel.