Multi-Model Approach as Epistemic Insurance: When It Works and When It Doesn't

You diversify your portfolio so you don't share a single risk. You don't buy stock in one company — you buy an index. Multi-model AI approach works on the same principle. But only if the models actually carry different risks, not the same ones differently named.

The argument for multi-model approaches is straightforward: one model can err, but it's unlikely that three independent models make the same error. If GPT-4, Claude, and Gemini agree, we have higher confidence. If they disagree, we have a signal to verify.

Claims Framework

• What this article claims: Multi-model approach reduces unsystematic errors (random hallucinations) but does not protect against systematic gaps in training data. Model consensus is a strong signal only for verifiable factual questions with independent evidence sources. Disagreement between models is often more valuable than agreement.
• What it is based on: Condorcet's jury theorem (1785), portfolio theory (Markowitz 1952), systematic vs. unsystematic risk analogy, Farquhar et al. (2024) on semantic entropy.
• Where it simplifies: The article assumes models from different vendors have sufficiently independent training data; actual degree of independence is not empirically measured. The financial portfolio analogy is illustrative, not formal.

This argument has solid theoretical foundation. Condorcet's jury theorem. Wisdom of crowds. Portfolio theory. But also specific validity conditions. Diversification protects against unsystematic risk, not systematic risk. If all models share the same blind spot — training on similar data, similar biases — their agreement isn't proof. It's an echo.

Condorcet's Theorem and Its Assumptions

Condorcet's jury theorem dates to 1785. It states: if each jury member has greater than 50% chance of deciding correctly and decisions are independent, then as the number of members increases, the probability of correct majority decision approaches 100%.

Application to AI looks appealing. If GPT-4, Claude, and Gemini each have more than 50% chance of correct answer and their errors are independent, their consensus dramatically increases reliability. Three models are better than one. Five better than three.

The problem is the independence assumption. Models are trained on overlapping data. Common Crawl, Wikipedia, arXiv, books. Their errors are correlated. If information is missing from all three models' training data, all three err. Not independently — systematically.

Condorcet's theorem works for complementary expertise. A cardiologist, neurologist, and oncologist have different specializations and different blind spots. Their agreement on a diagnosis is a strong signal. But three copies of the same encyclopedia aren't three independent sources. They're one source three times.

The question isn't "do models agree?" but "are their reasons for agreeing independent?"

Systematic vs. Unsystematic Risk

Portfolio theory distinguishes between risk that can be diversified away and risk that diversification won't eliminate. Unsystematic risk is specific to one stock — a scandal at a particular company. Systematic risk affects the entire market — recession, interest rates.

This distinction applies to AI models too.

Unsystematic AI risk includes random errors of individual models. Hallucinations specific to a particular version. Sampling noise. Architectural idiosyncrasies. This risk multi-model approach actually reduces. If GPT-4 occasionally hallucinates a specific detail that Claude and Gemini don't have, their disagreement reveals the problem.

Systematic AI risk includes errors shared by all models. Gaps in training data — nobody had the current information. Shared biases — RLHF at all vendors rewards similar-sounding responses. Fundamental architecture limits — transformers can't reliably count.

Example: a question about 2025 events asked to models with 2024 knowledge cutoff. All three models will guess or hallucinate. Their agreement means nothing because they share the same information deficit. Diversification doesn't protect against systemic risk.

Diagnostic: if models agree on something verifiable that they could have known independently, agreement is a signal. If they agree on something all could have derived from the same (potentially wrong) sources, agreement is an echo.

When Consensus Protects and When It Deceives

Consensus value depends on question type and whether models have independent access to evidence.

Consensus as signal — works:

Factual questions with verifiable answers in training data. "What is Australia's capital?" If all say Canberra, it's a strong signal. This information is widely represented in all models' training data and is unambiguous.

Logical and mathematical tasks. "Is 17 a prime number?" Models derive independently based on definition. Agreement validates the derivation, not just repetition of training data.

Procedural knowledge. "How do you set up a Git remote?" If all give the same procedure, it's probably correct. The procedure is well-documented in many independent sources.

Consensus as illusion — fails:

Recent events after knowledge cutoff. Models share the same ignorance. Their agreement on what happened in 2025 is joint guessing, not independent verification.

Obscure facts poorly represented in training data. If information comes from a single source (e.g., a rarely cited paper), all models may have interpolated from the same (potentially wrong) reference.

Value-laden questions. Agreement on whether something is "good" or "bad" may reflect all vendors' RLHF alignment, not objective truth. Models are trained to answer in ways humans rate as "helpful" — which can converge.

Complex causal conclusions. "What caused the 2008 financial crisis?" All models may share the dominant narrative from training data. Agreement reflects the consensus of historians and economists that models read — not independent analysis.

Disagreement as Information

Disagreement between models is often more valuable than agreement. It signals that the question has multiple legitimate perspectives, or that models have different access to evidence.

Disagreement on facts — signal to verify. If GPT-4 says 2019 and Claude says 2021, at least one is wrong. Don't resolve by voting. A third model may share one's error. Verify with primary source.

Disagreement on interpretation — legitimate diversity. If models disagree on "Is agile development effective?", don't try to pick one answer. The topic is controversial. Different models activated different parts of training data with different perspectives. Synthesize, don't choose.

Disagreement as stress-test. If model A in optimist role finds strong arguments and model B in skeptic role can't find strong counterarguments, asymmetry is information. The topic probably doesn't have strong opposition.

Multi-model approach has greatest value not when it produces consensus, but when it reveals where consensus doesn't exist. And why.

How to Weight Consensus

Consensus isn't binary. Degree of agreement and question type determine what weight to assign it.

Rule 1: High agreement on factual question with verifiable answer. Strong signal. 3/3 models say Canberra — high confidence. But note: if the question requires information after knowledge cutoff, agreement means nothing. Verify whether models could even have known the answer.

Rule 2: High agreement on interpretive question. Weaker signal. May reflect shared bias or dominant narrative in training data. Ask: why would all models have the same opinion? Is it logical, or is it an echo? Consider whether legitimate alternative perspectives exist that models didn't capture.

Rule 3: Low agreement — escalate. If models disagree, don't try to "vote." Identify the area of disagreement. Either verify with primary source, or accept that the question has no unambiguous answer. Disagreement is information, not a problem to solve.

Tools like CrossChat quantify degree of agreement as consensus score. 90% agreement on a factual question is a strong signal. 60% agreement on a strategic question is a signal to explore divergence, not to take a majority vote.

Conclusion

Multi-model approach is epistemic insurance, not truth guarantee. It protects against unsystematic errors of individual models — random hallucinations, sampling noise, architectural idiosyncrasies. But it doesn't protect against systematic gaps in training data or shared biases.

Effective diversification requires three things. Source independence — different vendors, different training mixes, different alignment procedures. Calibrated interpretation — distinguishing factual questions from interpretive, verifiable from speculative. And willingness to accept disagreement as information — not as a problem to solve.

The portfolio analogy holds, but with an important caveat: diversification protects against risk that isn't shared. If all models share the same blind spot, even their unanimous agreement won't protect you. Three copies of the same encyclopedia are still one encyclopedia.

Sources

• Condorcet, M. de (1785). Essay on the Application of Analysis to the Probability of Majority Decisions.
• Surowiecki, J. (2004). The Wisdom of Crowds. Doubleday. ISBN: 978-0385503860.
• Markowitz, H. (1952). Portfolio Selection. The Journal of Finance. DOI: 10.2307/2975974.
• Farquhar, S. et al. (2024). Detecting hallucinations in large language models using semantic entropy. Nature. DOI: 10.1038/s41586-024-07421-0.

Editorial History

Concept: Claude Code + Anthropic Sonnet 4.6 Version 1: Claude Code + Anthropic Sonnet 4.6 Version 2: Codex + GPT-5.2 Quality audit (2026-03-23, Claude Code + Claude Opus 4.6): added Claims Framework, verified sources, language polish.