Take a real deliberation. The prompt: "Review this system design. Identify failure modes, scaling bottlenecks, and missing operational guardrails." The user didn't attach a design. They left the placeholder [describe] in the input box. The Council ran anyway. The result is a mess, but it's instructive. Watching the engine process an underspecified prompt reveals the pipeline more clearly than a well-formed question ever could.
Round 1. Opening Statements. Four models receive the same prompt, independently. Each gets a cognitive persona assigned at engine start:
GPT-5.5 Pro → The Synthesizer: Find patterns, use analogical reasoning
DeepSeek R1 → The Epistemologist: Distinguish known from assumed
Claude Sonnet → The Contrarian: Challenge consensus, find counterexamples
Qwen 3.7 Max → The Systems Thinker: Second-order effects, feedback loops
These are not decorative labels. They are system prompts. The engine injects them directly into the model's instruction context. Each model sees the question, sees its role, and generates independently. No model knows what any other model wrote.
The code that runs this has a single yield point:
// src/lib/debate-engine.ts. Round 1: all models in parallel, no cross-talk
const round1Results = await Promise.allSettled(
modelsWithPersonas.map(async (model) => {
const systemPrompt = buildRound1Prompt(model, effectivePrompt);
const result = await sendModelTokens(model.openRouterId,
[{ role: 'user', content: systemPrompt }], timeoutMs, onEvent, ...);
// emit + persist response, accumulate citations
return response;
})
);
Promise.allSettled runs all four models independently. If one fails, the other three continue. No shared context. Source: MDN: Promise.allSettled. The engine fires them all at once and waits. If every model fails, the deliberation halts immediately. No point running cross-examination with nothing to cross-examine.
What the models actually produced on the [describe] prompt: GPT-5.5 Pro politely asked for the design description. R1 hallucinated a NiraNexus tech stack and ran analysis against it. Sonnet correctly observed that analyzing a missing artifact guarantees fabrication, then contradicted itself by importing the NiraNexus execution context. Qwen identified real failure modes (write amplification, quota race conditions) grounded in the actual platform.
Four opening statements. Three useful. One empty. The engine does not grade them. It stores them and moves to the next round.
Round 2. Cross-Examination. Models are paired. Each critiques a specific peer's Round 1 output. The pairings shuffle every debate:
const pairings = shufflePairings(modelsWithPersonas);
// GPT-5.5 critiques R1, R1 critiques Sonnet,
// Sonnet critiques Qwen, Qwen critiques GPT-5.5
Each model receives its target's full Round 1 response and a critique prompt. The instruction: find factual errors, logical gaps, and missing perspectives. Not opinion. Specific, targeted criticism.
This is where the pipeline produces its most valuable output. Qwen accused Sonnet of the exact error Sonnet warned against: "hallucinating specifics onto a blank canvas." R1 pointed out that Qwen's apocalyptic connection exhaustion analysis ignored Supabase's PgBouncer pooling layer. Sonnet caught R1 inventing failure modes from nowhere while simultaneously disclaiming fabrication.
The cross-examination loop takes what would be four siloed analyses and forces them into direct confrontation. Positions that survive Round 2 carry more weight because they were tested. Positions that fold, like R1's self-contradiction about hallucinating while hallucinating, are flagged for the verdict.
Round 3. Rebuttal (conditional). After Round 2 completes, the orchestrator runs a convergence check. Opus 4.8 reads every response and decides: continue or synthesise.
If the orchestrator returns CONTINUE, all four models receive the full debate transcript and write rebuttals. They address challenges to their original positions, concede where appropriate, sharpen where evidence supports them. If it returns SYNTHESIZE, the engine skips directly to verdict generation. Roughly half of standard debates proceed to Round 3. Document-grounded or MCP-augmented debates that run in compact mode (2 rounds) never reach it.
Why three rounds, not two or four?
Two rounds produces challenged but unrefined positions. A claim gets attacked in cross-examination, the original model never responds. The reader doesn't know whether the challenge was valid or the defence would have held. The pipeline produces noise, not signal.
Three rounds forces each attack to face a rebuttal. Weak claims are abandoned. Strong claims survive with evidence. The final synthesis works with tested, not raw, material.
Four rounds runs into diminishing returns. Models recycle arguments. New content falls below 10% of total output. The cost scales linearly while insight plateaus. Three rounds is the minimum for this dynamic to complete, confirmed across double-digit deliberation batches. This is not an arbitrary choice.
How does the orchestrator decide when to stop?
The orchestrator is a separate model. Not a timer. Not a quorum count. A model reading the complete debate transcript and answering a structured prompt:
After 2 rounds of debate, evaluate whether to continue.
Round 1 responses covered these perspectives. Round 2 critiques revealed:
[Summaries of cross-examination exchanges]
Respond with EXACTLY one word:
- "CONTINUE" if productive tension remains
- "CONVERGED" if substantial agreement reached
- "DEADLOCKED" if positions fixed with no new arguments
This is the actual prompt from src/lib/prompts.ts. The orchestrator can end a debate after Round 2 if positions are stable. It can force Round 3 if material disagreements persist. The code that evaluates its response:
const convergenceVerdict = await sendModelTokens(ORCHESTRATOR_MODEL,
[{ role: 'user', content: buildConvergenceCheckPrompt(r1Responses, r2Responses) }],
timeoutMs, onEvent, ...);
// Productive tension check: Only CONTINUE forces a 3rd round.
// Everything else prioritizes synthesis speed: CONVERGED, DEADLOCKED, or
// any response that isn't exactly CONTINUE skips straight to verdict generation.
if (convergenceVerdict.content.trim().toUpperCase() === 'CONTINUE' && roundNum < maxRounds) {
roundNum = 3;
// Run Rebuttal round...
} else {
// Skip to synthesis — no further rounds
}
The engine treats CONVERGED, DEADLOCKED, or any response that isn't exactly CONTINUE as a signal to synthesise. No ambiguity. No scoring. A single string comparison gates a round of deliberation that costs four model calls.
The orchestrator is not a judge. It does not evaluate correctness. It evaluates whether the deliberation has produced diminishing returns. Two rounds of genuine debate with exhausted positions beat three rounds of recycled arguments. The prompt reflects this. The decision gate is about productivity, not quality.
Convene a Council deliberation to see the three-round pipeline in operation. Every deliberation produces a public evidence record.
What survives and what gets discarded?
The synthesis step produces claims. Not all of them survive. The claim lifecycle in a 3-round deliberation follows a predictable pattern:
A proposed claim that no model challenges enters the verdict directly. A claim with one dissenter carries a DISPUTED label but is preserved. The dissent is not buried. A claim that the proposer retracts under cross-examination is discarded entirely. The verdict includes both surviving claims and surviving dissent, assigned to specific source models.
Disputed status is the operational output of the dissent preservation protocol. A claim faced resistance but remained standing. Dissent is the panel position itself. A claim can be DISPUTED without dissent being unanimous, and dissent can exist on claims that ultimately appear VERIFIED in the verdict.
This is the pipeline's core differentiation from averaging or voting architectures. Disagreement is structured evidence, not noise to be filtered out. The evidence basis labels (VERIFIED, DISPUTED, UNVERIFIED) come from this lifecycle. This is the architectural distinction between adversarial deliberation and model voting — cross-examination produces falsifiable claims, not correlated averages.
Where does this architecture fail?
Honest limitations. Three are documented and mechanically mitigated but still real.
Correlated training data. All four models are frontier LLMs trained on overlapping internet-scale corpora. When all models agree on a factual claim, that agreement may reflect shared training data, not independent verification. The Consilium Protocol independently confirmed this. RLHF alignment creates measurable, domain-specific epistemic blind spots. The MCP research pipeline partially addresses this. Web search introduces external evidence. But for non-MCP debates, model agreement is convergent training, not verified truth.
Anchoring effects. A confident wrong model in Round 1 can influence all subsequent rounds. Other models spend their critique tokens addressing the wrong claim rather than searching for the right one. The random pairing in Round 2 helps: no model can systematically anchor every peer. But a single persuasive error in Round 1 carries weight through the full pipeline.
Synthesis collapse. The orchestrator reduces 12+ model responses (4 per round × 3 rounds) into a structured verdict. Nuance is compressed. Minority positions with strong evidence but weak rhetoric can be overshadowed by majority positions with confident delivery. The dissent section preserves these, but only if the synthesis model correctly identifies them as dissent rather than noise. The engine tracks dissent counts. If a model dissents on more than 40% of claims, the verdict carries a warning flag. This has triggered in production.
The three-mode comparison in Log #6 confirms the pattern. Claude Sonnet 4.6 held an identical unrebutted position across all three execution architectures: Standard, Extended, Graph. Same model. Same concern. Nobody engaged it.
Log #7: The Roster Problem explains how that same model earned its seat. Six selection axes. Zero benchmarks. The roster is a maintenance contract, not a one-time choice.
Log #8: The Green Checkmark Lie covers the failure this pipeline exists to prevent: a single model grading its own output.
Provenance
