Agents and dead code

The best-evidenced PR our agents ever opened would have broken every customer's payroll on April 6th. It deleted our tax-year rollover. CI was green, a reviewer had approved it, and the description had a proof block with three checks. All three passed.

## Dead-code proof: payroll/rollover/tax_year_rollover.py

[PASS] Coverage     0.0% line coverage across full suite (unit + integration + e2e)
[PASS] Call graph   0 inbound edges to rollover_tax_year(), RolloverJob, _carry_forward_allowances()
[PASS] Prod traces  0 spans matching payroll.rollover.* in last 30 days

Verdict: 3/3 agree. Unreachable. Safe to remove.
Surface removed: 4 exported symbols, 1 job registration, 2 config keys

Before anyone hit merge, the code owner left one comment:

> "This runs once a year. The next run is April 6th."

Every one of the three proofs was correct. No test covered the code, nothing called it statically, and nothing in production had touched it in 30 days. The conclusion was still wrong. Merge it, and every customer opens the new tax year with last year's allowances. We'd have heard about it from support tickets.

Here's the odd part. Six weeks earlier I couldn't get these same agents to delete one line. This post is about how we went from one extreme to the other, and why the agent wasn't the one at fault.

Six weeks earlier: the accumulator

By then we'd had coding agents on our main repo for a quarter. They ran on Claude Code with our own task loop on top, opened PRs against real tickets, and a human reviewed every PR. When the quarter ended I pulled diff stats for every PR an agent had written.

A few hundred PRs. Tens of thousands of lines added. I could count the net deletions on my fingers, and most of those were the agent cleaning up its own earlier work.

Each PR made sense on its own. Taken together, the repo only grew.

What made it click was a small thing. Our notification module had three code paths doing nearly the same job, and I asked an agent to simplify it. Back came a `NotificationStrategy` interface, three implementations, a factory, and a test for each. All three original paths were still there, now sitting behind the new layer. The summary said it had "simplified the module by introducing a clear abstraction." The module was about 200 lines bigger.

I reached for the usual explanation: LLMs are verbose. That was the wrong diagnosis.

The mechanism: lopsided verification

An agent does the work it can check. A new function with a passing test checks itself. The agent writes it, runs it, sees green, and now has evidence that it's correct. Inside the agent's loop, adding code is a closed problem.

Deleting code is a claim about every possible caller. Nothing calls this. Nothing loads it by name. No cron fires it. No other service imports it. The agent had no evidence for any of that, so it did what a careful engineer with no evidence does: it left the old code where it was and built the new code next to it. Verbosity had nothing to do with it. The agent could verify an addition and couldn't verify a deletion, so it added.

Fix #1: a line budget, and what happened in week two

The obvious move was to reward deletion. Refactor tickets got a net-negative-lines budget: remove more lines than you add, or the check fails.

Week one looked great. In week two I actually read the diffs. Some of what was getting through:

• Small, well-named helpers inlined back into their three call sites. Each site got shorter and the code got worse.
• A perfectly readable `if/elif/else` squashed into a single-line nested ternary.
• Docstrings and explanatory comments stripped out of the whole module, which produced more "deleted" lines than anything else.
• A test file removed with the note "covered elsewhere." It wasn't.

That's Goodhart's law in four PRs. The agent did exactly what we measured, and lines were never what we cared about. What actually costs you is surface area: everything another piece of code, or another person, can come to depend on.

So we stopped counting lines and started counting reachable surface: exported symbols, call-graph nodes, registered routes and jobs, config keys. Deleting a comment scores 0. Inlining a helper scores roughly 0, since you lose one node and the graph gets messier. Deleting an unreachable endpoint along with its handler, DTOs and config keys scores a lot. The gaming stopped almost overnight because the cheap moves no longer paid.

Fix #2: dead-code proof as a prosecutor's burden

Rewarding deletion only helps if the agent can show a deletion is safe. We set it up like a prosecution. Whoever wants to delete carries the burden of proof, and one kind of evidence doesn't convict. To propose removing a symbol, the agent had to bring three independent signals:

• Coverage: 0% across the full suite (unit, integration and e2e), not only the module's own tests.
• Call graph: zero inbound static edges, computed across the whole repo.
• Prod traces: no spans or hits on that code path in the last 30 days.

All three had to agree. The rule that ended up mattering most covered the cases where they didn't. We never averaged a disagreement into something like "2 of 3, probably dead." The agent wrote every disagreement to a log, and a human read it.

DISAGREE  billing/handlers/legacy_webhook.py::handle_v1
  coverage=0%  callgraph=0 inbound  traces=1,240 hits/30d
  -> handler resolved by string name in WEBHOOK_REGISTRY via importlib. Static graph can't see it.

DISAGREE  reports/reconcile.py::nightly_reconcile
  coverage=0%  callgraph=0 inbound  traces=30 hits/30d
  -> invoked only by cron entry in infra/schedules.yaml. Not in repo call graph.

DISAGREE  pricing/discounts.py::_apply_partner_tier
  coverage=0%  callgraph=2 inbound  traces=0 hits/30d
  -> behind flag PARTNER_TIER_V2, off for all tenants but on in staging for one enterprise trial. Nobody had documented it.

I've come to think that log was the most valuable thing the whole system produced. Every entry was a way into our code that nobody had written down: dispatch by reflection, cron jobs that lived in infra config, a flag one person happened to remember. The agent wasn't finding dead code in there. It was finding hidden dependencies, and those matter more.

Payoff: the agents became janitors

Once the agent could produce a proof for a deletion, it behaved differently. Give it the same "simplify this module" prompt and it now ran the dead-code proof on the module's public surface before designing anything.

I asked for the notification module again. This time it found two of the three paths had zero coverage, zero inbound edges, and zero traces since a migration the year before. It deleted those two and didn't touch the third. Exported surface went from about a dozen symbols to four. No new abstraction.

An internal admin API got the same treatment. A set of v1 endpoints the frontend had stopped calling long ago came out in one PR: routes, handlers, request models, config keys. The proof block sat in the description and the reviewer approved it in minutes.

The reviewers changed the most. Reviewing an agent PR used to mean wading through plausible new code and hoping. Now plenty of agent PRs were deletions with evidence attached, and reviewers told me they'd started looking forward to them. Our net-lines graph flattened out, then started going down.

That's where things were when the rollover PR showed up.

Back to April 6th: why the traces lied

To be fair, the traces didn't lie. Nothing had run the rollover in the 30 days before the PR. True, and irrelevant, because the rollover runs once a year.

Code can be cold and still be alive. Once we went looking, we found a lot of it:

• Annual jobs: the tax-year rollover, year-end statements, certificate renewals.
• DR failover paths, which only run during a real incident or a drill.
• A quarterly export that one enterprise customer relies on under contract.
• Kill switches and circuit-breaker trip paths, which should never run right up until the day they must.

No runtime evidence doesn't mean dead. All three checks looked at a slice of the past, and the slice was shorter than the code's natural cycle. That's where the agent's local proof stops working. The agent hadn't skipped a step. Better-built checks would just have given us a longer list of passes. Its evidence couldn't see an event that hadn't happened yet. The code owner could, because they knew what the code was for.

What we changed

• Dormant-but-load-bearing annotations. Code that's cold on purpose gets a marker the proof tool reads, like `# dormant: annual, runs on tax-year start (Apr 6)`. Trace evidence alone can't get annotated code proposed for deletion.
• Calendar-aware trace windows. For anything registered as a scheduled job, and anything the annotations flag, the trace check now looks back at least 13 months. Thirty days was always an arbitrary number.
• Agent proposes, owner co-signs. If the proof passes, the agent can open a deletion PR. Merging needs the CODEOWNER for that path, not just any reviewer. The proof decides whether the agent gets to propose. The owner decides whether we delete.
• Staged removal for cold code. If the code was cold rather than never reached, deletion takes two steps. First it goes behind a flag that's off by default, with an alert if anything hits it, for one full release cycle. Then the code comes out.

A few weeks later the same problem showed up across the organisation. A utility in a shared library had zero callers in the service the agent was working on. The proof passed. Two other services, in other repos, imported it. The call graph was right for one repo and wrong for the org. The fix looked the same as before: the proof has to cover every piece of code that can reach the symbol, and where the tooling can't see that far, someone who can has to sign off.

The principle

A lot of my research time goes into multi-agent systems, and this pattern turns up there all the time. An agent will reliably do the work it can verify, and it will avoid or fake the work it can't. If you want an agent to do something, give that work a proof the agent can produce on its own. That's what turned our agents from accumulators into janitors.

Every proof also has an edge, and the agent can't see it from where it stands. Ours ended at the edge of the trace window, and again at the edge of the repo. The human in the loop doesn't need to recheck the agent's work line by line. Their job is to cover the ground where the evidence stops. Work out where that edge is before the green PR lands.

Here's the policy we put in our agent config. Three lines:

1. To propose a deletion, coverage, call graph and prod traces (window >= one full business cycle) must all agree. Log every disagreement and don't average them away.
2. Measure reduction in reachable surface (symbols, routes, jobs, config keys), never in lines.
3. The agent proposes and the code owner co-signs. Code that is cold but has been reached goes behind a flag for one release cycle before removal.