Your Agents Voted 3-0 — And They Were All Wrong

Day 13: Consensus Is a Trap — Your Multi-Agent System Needs a Tiebreaker, Not a Vote

Everyone reaches for 'majority vote' the first time hallucinations bite them in production. Run the same prompt through three agent instances, take whichever answer two of three agree on, ship it. It feels like triple-checking. It's actually one opinion asked to repeat itself — and it costs three times as much to find that out the hard way.

The false confidence of 3/3

Say you're extracting a shipping deadline from a contract. Three agent instances, same base model, same system prompt, sampling temperature doing the only differentiating work. Majority vote, because more eyes catch more mistakes — that's the theory. The clause reads 'delivery within 30 days of execution,' ambiguous whether anyone meant it that way or not, and all three instances anchor on the signature date instead of the effective date. Same misread, three times over. 3/3 agreement. Vote passes. The wrong date ships straight into the customer-facing summary, and nothing in the pipeline so much as blinked — unanimous agreement was the only signal you were checking for.

That's the trap. 3/3 feels like triple-checked. It's one opinion, sampled three times.

Why voting doesn't do what people assume it does

Majority vote as a reliability mechanism rests on exactly one assumption: errors are independent. Three reviewers, each with a 10% independent chance of missing a bug — the odds all three miss it fall to 0.1%. That's the entire argument for 'more voters, more reliable.' Solid math. It just doesn't apply here.

Agents built on the same base model don't fail independently. They share training data, architecture, the same blind spots on ambiguous phrasing, the same habit of anchoring on the first plausible date in a clause, the same failure to flag ambiguity as ambiguity in the first place. Sampling temperature changes the words, not the judgment underneath them. Voting cancels out noise — random slips. It does nothing against bias — a consistent misread baked into the model. A contract-clause misread is bias, not noise.

Where 'more nodes' came from, and why it doesn't transfer

Classical consensus algorithms earn their safety guarantees by assuming a specific failure model. Paxos and Raft assume nodes fail independently — a crashed disk on node A tells you nothing about node B. Byzantine Fault Tolerant protocols go further and assume nodes can fail adversarially, lying about their state on purpose. Different failure models, but both guarantees rest on the same load-bearing assumption: failures are uncorrelated. LLM agents sharing a base model break that assumption on day one. They're correlated by construction, not independent by design.

The cost nobody puts on the slide

Even if voting worked, look at what it costs. Running N agents to vote means:

• ▹N× the token spend — three full calls where one used to do the job, on every request, forever
• ▹N× the latency if run serially, or worse: your p50 becomes your slowest voter's p50, and the tail compounds across all three
• ▹Zero improvement on the failure mode that actually hits you in production, because that failure mode is correlated, not random
• ▹A false sense of coverage that makes the real bug harder to find later — nobody goes back and re-checks a 3/3 unanimous answer

You're paying 3x for a check with structurally zero chance of catching your dominant failure mode. That's not a reliability investment. It's a latency-and-spend tax wearing a placebo.

What actually catches correlated errors

Catching a systematic bias takes something that doesn't share the bias. Two patterns actually work, and both work through asymmetry, not repetition:

• ▹A cheap deterministic check — schema validation, running the actual test suite, a regex or business rule ('delivery date must be after effective date, not signature date'). No model bias to share, because it isn't a model.
• ▹A designated arbiter with a genuinely different setup — different model family, different context window, or a prompt built specifically to challenge the primary output ('here is an extracted date; find the two candidate anchor dates in the source and confirm which one this refers to'). Its job isn't to re-derive the answer — it's to attack the specific failure mode the primary agent is prone to.

Neither of those is 'ask the same question three times.' Both put a genuinely different vantage point in the loop, on purpose.

# Vote (looks safe, isn't)
answer_1 = agent(same_model, prompt).run(contract)
answer_2 = agent(same_model, prompt).run(contract)
answer_3 = agent(same_model, prompt).run(contract)
result = majority(answer_1, answer_2, answer_3)
# cost: 3x tokens, 3x latency
# catches: random sampling noise
# misses: systematic misreads shared by all three

# Tiebreaker (asymmetric, cheap)
draft = agent(model_A, extraction_prompt).run(contract)
ok = deterministic_check(draft, rule="delivery_date > effective_date")
if not ok:
    draft = arbiter(model_B, challenge_prompt).run(contract, draft)
# cost: 1x + occasional 2nd pass, only on flagged cases
# catches: the specific bias the primary agent is prone to

The design rule

Before you bolt a vote onto a pipeline, ask one question: would these agents fail the same way? Same base model, same prompt family, same blind spot on ambiguous input — if yes, a vote buys you nothing but latency and spend. Replace the quorum with a tiebreaker: a deterministic check where one exists, or an arbiter built deliberately to disagree with the primary path. If the answer is genuinely no — truly different models, different context, different failure surface — then a vote might catch something real. That's rare. Check first.

Extend your knowledge

• ▹Take any place in your own pipeline where you run N parallel agent calls and vote. Check whether they share a base model and prompt family — if they do, swap one voter for a deterministic rule check and compare cost against catch-rate.
• ▹Read up on the failure-independence assumptions in classical consensus — Paxos/Raft's independent-failure model versus BFT's adversarial one — to see exactly which assumption LLM agents break.
• ▹Look at how well-designed LLM-as-judge setups use a different model family for the judge, never the same one grading itself — that's the arbiter pattern, already in production.
• ▹Next in this series: designing the arbiter's prompt so it's actually adversarial to the primary agent, instead of just re-asking the same question in different words.