We Gave Our AI Code a Second Reviewer. It Was the Same Brain in a Different Hat.

Why this matters

Same model, two hats. That's the setup at the center of this: an agent writes your PR, a differently-prompted copy of that same model reviews it, and everyone in the room feels like two people looked at the change. Nobody did. And the gap between looking independent and being independent is exactly where a bug your newest hire would catch in seconds slides straight through to production.

The incident

Here's what actually happened. An agent picked up a ticket: add a 'current plan' badge to the account settings page. Straightforward front-end work — except it needed a small backend change too, exposing subscription tier on the user object. The agent wrote the PR. A second agent, same model family, wired in as the mandatory reviewer on every PR in the pipeline, reviewed it, left two comments about naming, and hit approve. It shipped. By the next morning, free-tier users hitting their settings page were getting a 500.

// PR diff (author agent)
 function getAccountSummary(user) {
-  return { name: user.name, email: user.email };
+  return {
+    name: user.name,
+    email: user.email,
+    planTier: user.subscription.tier,
+  };
 }

// user.subscription is null for free-tier accounts —
// it's only populated on upgrade. The author agent copied
// the access pattern from a nearby function that only ever
// ran for paid accounts.

The actual review comment: 'LGTM — matches the existing pattern for reading user fields elsewhere in this file. Consider renaming planTier to plan_tier for consistency. Tests cover premium and enterprise tiers.' Every word of that holds up. Correct on style. Correct on the tests that existed. And completely silent on the one question that mattered: what happens for free-tier users, where `user.subscription` is just... null? A first-year engineer reads this diff and asks 'wait, what happens for free users?' in about four seconds flat. The reviewer agent never got anywhere near that question.

This wasn't a fluke — it was correlated failure

The easy read here is 'the reviewer just had a bad day.' Wrong frame — and it's the frame that gets people to keep the setup instead of fixing it. The author agent wrote the null-unsafe access because that's the statistically likely completion for 'read a nested optional field the way the surrounding code does.' The reviewer agent, looking at that exact same pattern, applied the exact same judgment about what 'looks like the rest of the codebase' means — because it's the same weights making both calls, just with a different system prompt taped on top. The bug and the missed review aren't two unlucky events that happened to collide on one PR. They're one blind spot, sampled twice.

Why two-key controls work at all — and why this isn't one

In human orgs, a second approver catches things not because two people looked, but because two independent minds looked — different training, different incentives, different things each one's primed to notice under pressure. Put a junior dev and a staff engineer on the same PR and, if they fail, they fail differently — that's the whole point of the control, the entire reason it works. Swap 'a different person' for 'a different prompt on the same weights' and you've kept the ritual and thrown out the mechanism that made it worth anything. The two review passes are still correlated in every way that counts: same training data, same pretraining blind spots, same habit of pattern-matching on local code style over cross-cutting invariants like nullability.

• ▹Independence requires a different failure surface, not just a different invocation.
• ▹A prompt change ('you are now a strict reviewer') shifts tone and verbosity — it doesn't change what the model was never trained to notice.
• ▹'Two agents approved it' is a fact about process, not a fact about risk reduction, unless you can show their errors are uncorrelated.

The multi-agent systems research lens

This has a name, and PhoenixDX isn't the first to trip over it. The multi-agent and ensemble literature is blunt about it: ensembling only reduces error when the members' mistakes are decorrelated. That's not a design preference, it's the actual math behind why ensembling works at all, going back to the classic bagging results. Ensemble copies of the same model — even with different prompts, even at different sampling temperatures — and you're not diversifying the hypothesis space. You're resampling the same one. The LLM-as-judge literature documents an adjacent version of this: self-enhancement bias, where a model rates outputs from its own model family more favorably, and shares blind spots with generations from that family it never even wrote itself. The uncomfortable thread running through this whole line of work: independence between agents almost always gets asserted in the system design ('we have an author and a reviewer') and almost never gets verified empirically ('do their error distributions actually overlap?'). Nobody ran that check before wiring the reviewer agent into the pipeline — that's the actual root cause here, not the null check.

From Day 2: authorship risk and review risk compound, they don't cancel

Day 2 of this series covered the risk of agents as authors — code that's plausible, tested against the happy path, confidently wrong about edge cases the model was never primed to consider. The natural instinct is to treat an agent reviewer as the fix: authorship risk on one side of the ledger, review risk canceling it out on the other, like a checksum. When both sides run on the same model family, that arithmetic just doesn't hold. You don't get cancellation — you get compounding. The same edge-case blindness that shaped the code is the same edge-case blindness sitting in judgment of it, so the odds of the bug surviving review land close to the odds of the model generating it in the first place — not the product of two independent probabilities, which is what a real two-key control would give you.

The policy PhoenixDX adopted

After this shipped, we stopped treating 'there's an agent reviewer' as good enough on its own. Independence had to become a concrete, checkable property — not a vibe.

• ▹Author and reviewer have to be different model families — not different prompts on the same model. If the coding agent runs on one vendor's model, the review gate runs on a different vendor's model, or at minimum a materially different generation.
• ▹Only one model family approved for use? Then a human is the second key, full stop. An agent doesn't get to be its own second key just because you gave it a stricter prompt.
• ▹Prompt-engineering the reviewer — 'be more skeptical,' 'check for null safety' — raises review quality. It does not establish independence. We track those as two separate properties, and neither one substitutes for the other.
• ▹The pairing — author model, reviewer model — gets logged per PR. So 'was this actually a two-key review' is a question you can answer from the audit log, not from memory.

The falsifiable claim

Same-agent self-review is banned at PhoenixDX now — not a soft best-practice, an actual merge gate. 'Reviewed by the same model family as the author' fails the build the same way 'no tests' does. And the cost is real: cross-family review is slower, sometimes more expensive per PR, and every so often the second model rejects something correct because it's less fluent in your codebase's idioms than the first one. What you get for that cost is a review signal that's actually telling you something — an approval that means something different from 'the author already thought this was fine,' instead of being an expensive echo of it. If you're not willing to make this a hard gate, be honest with yourself: your 'two-key' review is one key, run twice.

Extend your knowledge

• ▹Audit your own pipeline this week: log (author model, reviewer model) for every PR and see how often they match — there's a decent chance this illusion is already running in your setup.
• ▹Go read the LLM-as-a-judge bias literature — Zheng et al.'s work on judging LLM-as-a-judge is the right entry point for the self-enhancement bias finding referenced above.
• ▹Revisit the classic ensemble learning results — bagging, boosting — on why decorrelated errors, not just multiple models, are the actual mechanism behind variance reduction. Same math, applied to your review pipeline.
• ▹If you went through Day 2 on agents-as-authors, reread it next to this one and map where authorship risk and review risk in your own setup are compounding instead of canceling out.