OpenRouter just published a benchmark that puts numbers on something agent builders have been doing by hand for a year. Run several models on the same task, let a judge model reconcile their answers, and the combined result beats the best single model. They call it Fusion. The headline: a two-model panel scored 69.0 percent on a deep research benchmark, above every individual model they tested, including the strongest frontier model at 65.3 percent.
The result is not the interesting part. The interesting part, if you run an AI platform and pay the inference bill, is the cost curve underneath it.
What they measured
The benchmark is DRACO, 100 deep research tasks from Perplexity AI across 10 domains: academic research, finance, law, medicine, technology, UX, general knowledge, retrieval, personalized assistance, and product comparison. Each task is graded against roughly 39 weighted criteria covering factual accuracy, breadth and depth, presentation, and citation quality. Some criteria carry negative weights, so a confidently wrong, verbose answer loses points instead of gaining them. Each response is judged three times. The reported number is the mean.
The rows that matter:
- Fable 5 + GPT-5.5, synthesized by Opus 4.8: 69.0 percent
- Opus 4.8 + GPT-5.5 + Gemini 3.1 Pro: 68.3 percent
- Fable 5 alone: 65.3 percent
- Budget panel, Gemini 3 Flash + Kimi K2.6 + DeepSeek V4 Pro: 64.7 percent
- GPT-5.5 alone: 60.0 percent
- Opus 4.8 alone: 58.8 percent
Two findings carry the whole post. Every panel beat its own strongest member. And a panel of three budget models beat both GPT-5.5 and Opus 4.8 running solo, landed within about one point of the top frontier model, and did it at half that frontier model’s cost.
How fusion works
The mechanics are simple. You send one prompt. It fans out to a panel of models in parallel, each with web search and fetch. A judge model reads every response and writes a structured analysis: consensus points, contradictions, partial coverage, unique insights, blind spots. A final model writes the answer grounded in that analysis. One API call, all server-side.
Teams building agent systems already do a version of this by hand. Tools like Hermes Agent and OpenClaw let a human run several named LLM agents against a shared kanban board. Cards move across lanes, from triage to ready to in progress to done. Agents claim a task, hand work back and forth, and review each other’s output before it ships. People who run these setups learned early that bouncing a problem between several agents beats handing it to one. They call it a model panel, an ensemble, or agent ping-pong.
Fusion automates the same idea inside one server-side call. A panel of models and a judge replace the board and the human. The pattern is the same. The difference is that someone finally put numbers on it.
The synthesis step is doing real work
The most useful data point is the quiet one. They ran Opus 4.8 paired with itself, the same model on both sides of the panel, synthesized by Opus 4.8. It scored 65.5 percent, a 6.7-point jump over solo Opus 4.8 at 58.8 percent. Same model, run twice, then reconciled.
That tells you where the lift comes from. Part of it is model diversity. A large part is the synthesis step itself. Run the same prompt twice and you get different reasoning paths, different tool calls, different source selections, and a reconciliation pass on top of all of it. The orchestration layer is not plumbing around the model. It is part of the capability.
The cost case is the platform decision
Read the budget-panel row again. Three cheap models, fused, beat two frontier models running on their own, came within a point of the best frontier model, and cost half as much. For a class of work, routing everything to the single most expensive model is now the expensive way to get a result that is no better.
This is the same lesson as matching model spend to the job in distillation. The arrangement changes, the discipline does not.
- For commodity work that is high-volume and checkable, a fused budget panel can clear the bar for less than your current frontier bill.
- For the work your product competes on, frontier still earns its premium, and a frontier panel pushes past any single frontier model.
The architecture question is no longer only which model. It is how many, in what arrangement, judged by what. That arrangement has a measurable price and a measurable quality, and most teams have never benchmarked their own against a single-model baseline.
The caveats an engineer should not skip
The OpenRouter post is honest about its limits, and the limits matter before you copy the pattern into production.
- Latency and token cost multiply. You now pay for several panel models plus a judge plus a synthesizer on every call. Fusion wins on quality per task, not on tokens. On interactive, latency-bound, or high-throughput paths, the math can flip against you. The launch FAQ also notes DRACO has no long-horizon tasks, which is where a single strong model still shines.
- Judge choice swings the absolute score. The authors report 10 to 25 point shifts between judges. Relative rankings held, but if you benchmark your own panels, lock the judge and treat the absolute numbers with suspicion.
- Eval hygiene is a real risk. Once the panel had web search, models started finding the benchmark’s grading rubric online. The team had to exclude the hosting domains from search and fetch. If you run your own evals with web tools, your scores are contaminated until you do the same.
- Read the footnotes. One model’s content filters blocked 7 of the 100 tasks, so its score reflects 93 tasks, not 100. Cross-model comparisons are rarely apples to apples.
What this means for your AI platform
The takeaway is not to switch everything to Fusion. It is that the way you arrange models is now a measurable engineering decision, with real cost and quality stakes, and the default of one model per task leaves both on the table.
Two questions are worth answering before your next AI infrastructure commit:
- Which of your workloads are quality-bound enough to justify a panel, and which are cost-bound enough that a budget panel would beat your current frontier bill?
- Have you ever measured your own orchestration against a single-model baseline, on your own tasks, with a locked judge?
Most teams have not. That gap is where an independent benchmark and review pays for itself, before the inference bill compounds.
If your team is deciding how to route work across models, or watching an inference bill climb, an outside read before you commit is usually cheaper than the bill. Book a 30-minute scoping call, or take a focused hour on the decision with an Expert Call.
0 Comments