There is no single decision. The roster is the residue of six independent constraints, each enforced at a different layer of the system.
The naive version is obvious. Pick the four best models on the leaderboard. Wire them in. Done.
That version fails within a week. Because a model that tops a reasoning benchmark can still hang on streaming output. A model that scores brilliantly can share training data and RLHF methodology with the model sitting next to it, so their errors correlate and the whole adversarial premise collapses. A model that works today can be renamed, deprecated, or pulled tomorrow, and the debate engine will not notice until a user does.
The roster problem is not "which models are best." It is "which models can be trusted to argue with each other, reliably, every day, without a human watching." That is a different question, and it has six answers.
The persona comes before the model
Each council seat is a job description, not a model. The system defines four cognitive personas, and a model is selected to fill each one.
The Synthesizer finds patterns across domains and uses analogical reasoning. The Empirical Adjudicator takes a falsifiable position on every claim and backs it with citable evidence. The Contrarian challenges consensus and hunts counterexamples. The Systems Thinker traces second-order effects and feedback loops.
These are not decorative labels. They are system prompts injected directly into each model's instruction context. The persona constrains what the model is allowed to do.
Look at the Empirical Adjudicator persona. It is a paragraph, and it is specific: take a clear position on every claim, support it with at least one citable piece of evidence, state explicitly when you lack evidence, never hedge with "it depends" without naming the dependency, concede specifically when another model's evidence is stronger.
That persona was written in response to a real failure. A model on the old roster was treating the deliberation as a vibes exercise. Restating the question as a position, hedging behind "it depends," never committing. The persona is the correction. You do not pick a model for a seat. You define the seat, then find a model that can hold it.
The council spans four providers: OpenAI, Qwen, Anthropic, and Google. This is not a stylistic preference. It is a correlation hedge.
The entire adversarial premise depends on models making independent errors. If two models are trained on overlapping corpora and aligned with similar RLHF methodology, their errors correlate. When one gets something wrong, the other gets the same thing wrong. Cross-examination produces agreement, not correction. The council degrades into a single model with extra steps.
Provider diversity attacks the correlation directly. Different providers use different training data, different architectures, different alignment procedures. Their blind spots do not overlap perfectly. According to MindStudio's multi-model council guide, models from different providers with distinct training approaches add the most diversity to a deliberative panel. Running two models from the same provider at different capability tiers adds less, since they share training data and RLHF methodology.
Source: Liminal: Multi-Model vs Single-Provider AI Platforms makes the same case from the infrastructure side. Provider diversity is resilience. If one provider experiences downtime, the platform routes workloads to alternative models automatically. This redundancy does not exist in single-provider solutions.
There is a second-order benefit that is rarely discussed. If a single provider goes down, the council does not die. It degrades. Three of four seats keep arguing while the fallback chain routes around the outage. A four-model council from a single provider is a single point of failure wearing four costumes. Why independent verification matters in the first place is covered in Log #8: The Green Checkmark Lie.
Streaming verification is the gate that benchmarks miss
Benchmarks measure reasoning. Production measures whether the model can stream tokens to a browser before the user loses patience.
DeepSeek V4 Pro was a strong candidate on paper. It scored well on reasoning tasks. It produced coherent analysis. It worked perfectly in non-streaming mode.
It hung on streaming output.
The Model Council is a real-time system. Models stream their responses token by token through server-sent events so the user can watch the deliberation unfold. A model that buffers its entire response and only releases it at the end breaks that contract. The debate appears frozen. The user assumes the system is dead.
So the roster has a hard gate: every model is tested with a direct streaming call before it is added. Works non-streaming but hangs on streaming? Rejected. No exceptions. DeepSeek V4 Pro never made the council because it could not clear this gate, not because it was not smart enough.
Source: O'Reilly: LLM System Design and Model Selection confirms the pattern. Choosing an LLM is no longer about chasing the highest benchmark score. It is about finding the balance point where capability, latency, and reliability align with your use case. Streaming failure at the application layer does not appear on any leaderboard.
Every seat has a fallback chain
No model is trusted unconditionally. Every council seat has a two-tier fallback chain.
Tier one is same-provider. If GPT-5.6 Terra fails, the engine tries GPT-4o and o3-mini before looking elsewhere. This is fast and keeps the persona's reasoning style roughly consistent.
Tier two is cross-provider. If the same-family alternatives also fail, the engine routes to a different provider entirely. The cross-provider chain is ordered by preference and deduplicated against the live roster, so the engine never asks a model to critique itself.
The fallback chain is not a nice-to-have. It is a selection criterion. A model earns its seat partly on the quality of its alternatives. A seat with no viable fallback is a single point of failure, no matter how good the primary model is.
A cron job checks the roster every day
Model IDs change. Providers rename models, deprecate old ones, pull endpoints. A model that worked last month can be a 404 this month, and the debate engine will not know until a user triggers a debate that fails mid-flight.
The Model Council does not wait for that to happen. A cron job runs every day. It reads every model ID from the roster file, hits the live OpenRouter endpoint list, and diffs the two sets.
Dead models get flagged and reported. The audit does not fix anything. It surfaces the drift before a user pays for a failed debate.
This is the difference between a demo and production. A demo uses models that worked during development. Production verifies, every day, that they still work. The roster is not a configuration file. It is a maintenance contract.
The circuit breaker in Log #4 enforces the same philosophy at the debate level. Tripwires prevent broken processes from burning capacity. The cron audit prevents broken rosters from burning user trust.
Consumption weight is a real axis
Models do not cost the same to run. That is not the interesting part. The interesting part is that consumption weight shapes the roster.
When the council swapped DeepSeek R1 for Llama 4 Maverick, per-debate consumption dropped by roughly a third. The persona slot stayed the same. The reasoning quality held. But the same capacity tier now stretched further.
Consumption weight does not decide which models make the council. It decides the trade-offs. A more expensive model has to justify its seat with measurably better adversarial value. A cheaper model that holds its persona gets the seat. The roster is a portfolio, and consumption weight is one of the constraints the portfolio balances.
I am not going to print raw per-token prices here. That is internal accounting. What matters is the shape: consumption weight is a selection axis, and it is weighed against adversarial value on every seat.
Three swaps, three reasons
The roster is not static. It has changed three times since launch, each swap driven by a different failure mode.
DeepSeek R1 to Llama 4 Maverick. The swap was about consumption and persona fit. R1 held the seat competently but the replacement held it for less, so it moved.
Qwen 3.8 Max to Gemini 3.6 Flash. The swap was about provider diversity and a fresh independent perspective. The old seat and the new seat came from different training ecosystems.
Claude Opus 4.8 to Opus 5. The orchestrator seat. The upgrade was about synthesis depth. The newer model produces measurably richer verdicts, which changed the synthesis token budget upward to match.
Three swaps, three different reasons. Consumption. Diversity. Depth. None of them was "the benchmark score went up." The roster does not change because a model got better on a leaderboard. It changes because a production constraint shifted.
What survives the swaps
The roster is a moving target, but the selection axes do not change. Persona coverage. Provider diversity. Streaming reliability. Fallback depth. Daily verification. Consumption weight.
Those six axes survived three model swaps. They will survive the next three. The models are interchangeable parts. The axes are the architecture.
That is the point. A council that cannot explain why a model is on the roster cannot be trusted to deliberate. The Model Council can, because the answer is not "it scored well." It is six constraints, each enforced mechanically, each verified daily.
Provenance
See also: Log #12: The Code Is the Policy: the governance manifesto. The model audit cron is one of four enforcement layers.
