Comprehension debt

Day 22: Comprehension Debt, the Debt Agents Make Cheap to Create

Your team will ship more code this year than last year. The number of people who understand that code hasn't changed. The debt playbook you inherited tracks code quality and has no column for how much of the code your people can actually explain. With agents writing a growing share of what merges, that missing column is where the cost builds up.

The 2am incident: where the three hours went

You've probably seen this one. At 2am an alert fires in a payment reconciliation service. The on-call engineer opens the module and finds clean, typed, well-tested code with sensible names. An agent wrote it about six weeks ago. Someone reviewed it, CI passed and it merged. Nobody has touched it since.

The fix is one line: a retry window that's too short now that an upstream API has slowed down. Writing it takes 15 minutes. Getting confident it's the right line takes three hours. That time doesn't go on debugging in the usual sense. It goes on rebuilding a theory of the code that nobody on the team ever had:

• ▹Why is the retry logic a custom backoff when everyone else uses the shared retry library?
• ▹Does the idempotency check run before or after the ledger write, and is that order deliberate?
• ▹What else reads the table this module writes to? Changing the timing could break a consumer nobody remembers.
• ▹Then the original PR thread. The review comments are about naming and one missing test case. Nobody asked why it was built this way.

The code wasn't bad. It reached production before any person had built a mental model of it, and the engineer on call at 2am paid for that, with interest.

Name it: comprehension debt

Comprehension debt is the gap between the code you've shipped and the code someone on your team can explain: what it does, why it has this shape, and what breaks if you change it. It grows whenever code merges faster than people can form a model of it. The interest comes due when someone has to change, debug or remove that code and first has to reconstruct the reasoning behind it. Peter Naur made the case in 1985, in 'Programming as Theory Building': a program's real value lives in the theory its programmers hold, not in the text. Agents produce the text. They don't produce the theory.

| Dimension           | Classic code debt                                   | Comprehension debt                                         |
|---------------------|-----------------------------------------------------|------------------------------------------------------------|
| Who creates it      | Humans cutting corners under deadline, usually knowingly | Anyone merging code they didn't build a model of; agents make it near-free |
| What it looks like  | Messy code, duplication, TODOs, slow builds, flaky tests | Clean-looking code nobody can explain; long silences in incident channels |
| Visible in the code?| Usually yes: linters, smells, complexity metrics    | No. The code can be perfect. The gap is in people's heads    |
| When it hurts       | Every time you work near it, gradually              | Suddenly: incidents, urgent changes, when the one reviewer leaves |
| How you pay it down | Refactoring                                         | Building understanding: explanation, walkthroughs, ownership |

In the code review lesson we said review is how a team shares knowledge, not only a correctness gate. When review stops sharing knowledge, comprehension debt is what piles up.

Why the old playbook breaks

The '20% of each sprint for debt' rule depends on something nobody says out loud: somebody noticed the debt when it was created. A person who cut a corner knew they'd cut it. They left a TODO, mentioned it at standup, or the team felt the pain and raised a ticket. Your debt backlog was a list of things people already knew about.

Comprehension debt never gets written down like that. Agents produce dozens of PRs a week, and each one passes CI and looks reasonable in the diff. Nobody raises a ticket saying 'I approved this without understanding it'. The debt grows quietly until an incident or an urgent change exposes it. At that point there's no list to work through, just an unknown number of modules nobody understands.

The bottleneck has moved. It used to be finding time to pay debt down. Now it's noticing that the debt exists at all. Your job as an EM moves from scheduling paydown sprints to spotting the gap early and keeping it small.

A flashback from my CTO years: bus factor 1 vs bus factor 0

When I was a founder and CTO, my comprehension risk was a single person. One engineer had built the core sync engine. It worked well, and they were the only one who really understood it. Every time they went on leave I was a little nervous. That's a bus factor of 1. It's a familiar problem and the fixes are familiar too: pairing, documentation, rotating ownership.

But someone did hold the theory. You could take them for coffee and get it out of them. With agents, a module can have a bus factor of 0 from the day it merges. The agent doesn't remember the session that wrote it, the reviewer skimmed it, and the engineer who prompted it has moved on to the next ticket. Nobody can lose the knowledge because nobody ever had it. That's new for engineering leaders. Our instincts don't cover it, because every one of them assumes there's a person to transfer the knowledge from.

Three signals you already have comprehension debt

You don't need new tooling for this. The signals already show up in things you have, so pick one and check it this week.

• ▹Signal 1: in incidents, time-to-understand dwarfs time-to-fix. Open your last three to five incident timelines and mark when someone said 'found it' or 'I see what's happening', then when the fix shipped. If most of the time sits before 'found it' and the fix was small, the bottleneck was comprehension. Add 'time to understand' as its own field in your postmortem template.
• ▹Signal 2: review comments cover style and tests and never the design. Pull ten recently merged agent-written PRs and read the comments. Count the ones asking 'why this approach?', 'why not use X?' or 'what else depends on this?'. If you get close to zero, your reviews check correctness without building any understanding.
• ▹Signal 3: nobody can answer 'what breaks if I delete this?' Choose three modules that changed a lot recently. In standup or a 1:1, ask the people closest to them exactly that. Hesitation, 'I'd have to look' and 'probably nothing?' all mean the module is red. The question works because answering it needs a model of the dependencies, and reading the code alone won't give you one.

The practice, step 1: change what review approves

Merge time is the cheapest place to stop comprehension debt. Before agent-written code merges, the reviewer writes a three-sentence explain-back in the PR covering what it does, why this approach, and what it touches. If they can't write it, the PR waits. That isn't a punishment. The next step is to ask questions, pair with the author, or get the agent to explain its reasoning until the reviewer can write those three sentences.

This changes what 'approve' means. It used to mean 'I see nothing wrong'. Now it means 'I could debug this at 2am'.

## Reviewer explain-back (required for agent-written changes)

**What it does** (one sentence, in your own words, no copy-paste from the PR description):
Adds exponential backoff to the reconciliation fetcher so upstream timeouts retry up to 5 times before marking the batch failed.

**Why this approach** (including what was considered and rejected):
Uses a local backoff instead of our shared retry lib because the lib retries on all 5xx, and the upstream returns 503 for permanent account-closed errors that must NOT retry.

**What it touches** (callers, data, side effects, what breaks if removed):
Writes to `recon_batches.status`; the nightly ledger job reads that column. Removing this reverts to single-attempt fetches and the ledger job will flag false mismatches.

- [ ] I could explain this to on-call without opening the PR thread

Some practical notes:

• ▹The reviewer writes the explain-back, not the author. An author can paste agent output, and the whole point is that a second human holds the model.
• ▹'Why this approach' matters most, and it's the sentence people skip. Agents often pick a reasonable-looking shape for reasons nobody wrote down. If the reviewer can't say why, they should ask the agent and then check the answer.
• ▹Match it to risk. Copy changes and dependency bumps can skip it. Anything touching money, auth, data writes or cross-service contracts always gets one.
• ▹Watch for the agent writing the explain-back on the reviewer's behalf. You end up with the form filled in and nobody understanding the code. Reviewers can use the agent to ask questions, but they write the three sentences themselves.

The practice, step 2: put a number on the gap

If you don't measure it, it won't get managed, and you can't see comprehension debt in the code. So build a lightweight ownership map in your retro. Keep it light. A heavy process will be abandoned within a month.

• ▹Pull the top 20 files (or modules) by change rate over the last 30 to 60 days. High churn shows you where the team actually works and where agents generate the most code.
• ▹Go through them one by one and ask the room: 'Can someone here explain this: what it does, why it's shaped this way, what breaks if it's deleted?' A named person saying yes makes it green. Silence or 'sort of' makes it red.
• ▹Write down the red count. Over time, that's your comprehension-debt metric.
• ▹Track it every sprint. One snapshot tells you little. A red count that climbs sprint after sprint means code is merging faster than people understand it.

# Top 20 most-changed files in the last 60 days
git log --since="60 days ago" --name-only --pretty=format: \
  | grep -v '^$' \
  | sort | uniq -c | sort -rn | head -20

| File / module                  | Changes (60d) | Can explain? | Who      | Notes                         |
|--------------------------------|---------------|--------------|----------|-------------------------------|
| recon/fetcher.ts               | 31            | RED          | -        | agent-written, reviewer left  |
| billing/invoice_builder.py     | 24            | GREEN        | Linh     |                               |
| auth/session_refresh.go        | 19            | RED          | -        | 'probably safe to delete?'    |
| ...                            |               |              |          |                               |

Red count this sprint: 7 / 20   (last sprint: 5)

Three pitfalls. Don't turn the red count into a target people get punished for, or modules will start going green when they shouldn't. Don't let the most senior person in the room claim every file: someone who 'owns' eight modules is a bus factor of 1 again. And be honest about 'sort of'. At 2am, partial understanding counts as red.

For something more rigorous later, Adam Tornhill's hotspot analysis (from 'Your Code as a Crime Scene' and the CodeScene tooling) combines change frequency with complexity. Start with the 20-file conversation, though. What makes the exercise work is people asking each other the question out loud.

The practice, step 3: pay it down by building understanding, not by refactoring

Most teams react to a red module by refactoring it, and that's usually the wrong move. If an agent rewrites code nobody understands, you get different code that nobody understands. The debt lives in people's heads, so that's where you pay it down.

• ▹Walkthroughs (30 minutes, one red module). One person studies the module beforehand, maybe questioning the agent along the way, then explains it to a teammate who asks 'why?' at every step. The explainer learns the most and the listener checks the explanation. At the end, two people understand the module. One or two a week is plenty. This doesn't need to become a programme.
• ▹On-call shadowing on red modules. When an alert touches a red module, a second engineer joins as a shadow, even if the fix is trivial. People learn fastest during incidents, so use that deliberately.
• ▹Intent notes, drafted by the agent and signed off by a human. Ask the agent for a short note on the module covering its purpose, design decisions and rejected alternatives, invariants, and what depends on it. A human then edits it, corrects it and signs it. The editing is the valuable part. The draft will be confidently wrong in places, and finding those places is how the human builds the model. An agent note nobody has signed is documentation debt, not understanding.

# INTENT: recon/fetcher.ts
_Drafted by agent 2026-10-02. Edited & verified by: Minh (2026-10-06)_

**Purpose:** Fetch upstream settlement records and stage them for nightly ledger reconciliation.

**Key decisions:**
- Local backoff, not shared retry lib: upstream uses 503 for permanent errors. (Agent draft said 'for performance' — WRONG, corrected.)
- Idempotency check runs BEFORE ledger write; reversing this caused duplicate entries in staging.

**Invariants:** Never mark a batch `failed` before 5 attempts; never retry on `ACCOUNT_CLOSED`.

**Dependents:** nightly ledger job reads `recon_batches.status`.

**What breaks if deleted:** ledger job flags false mismatches; finance gets paged.

When a module has a named person who can explain it and a signed intent note, mark it green at the next retro. That's comprehension debt being paid down.

The trade-off, honestly

This costs something. Explain-backs slow merges down. A PR that would have merged in ten minutes might now wait a day while the reviewer actually works through it. Engineers will push back, and the strongest ones often push hardest because they're used to trusting their skim. Expect to hear 'the tests pass, so why does it matter whether I can explain it?'

When I introduced this, I didn't sell it as quality or discipline. That sounds like a manager adding process. I sold it as a trade: 'We're buying back incident hours.' Each explain-back is a small payment now so nobody pays three hours at 2am later. Then I showed the team our own incident timelines and the gap between time-to-understand and time-to-fix. That data did more to convince people than anything I said.

Did we lose velocity? Yes. Merge throughput dropped for the first few weeks, and I didn't measure it carefully enough to give you a clean number. Explain-backs got quicker over time because reviewers started asking 'why this shape?' earlier, often while the agent was still working. Authors also began prompting agents for simpler, more conventional designs, because they knew a human would have to explain the result. I didn't see that second effect coming, and it may be the biggest win. Be wary of anyone who quotes you a precise ROI for this. What you can honestly track is your own red count and your own incident timelines.

Some code isn't worth it: prototypes and spikes you really will throw away, internal scripts that never go near production, and generated boilerplate whose shape the framework dictates. Spend your explain-back budget where 2am incidents actually happen.

Where this fits in the 30 days

Today's lesson in one line: on an agentic team, managing tech debt means managing what your team understands. Code quality still matters, and agents keep getting better at producing clean code. Your team's understanding of that code doesn't improve on its own. Improving it is the leader's job, and tomorrow's lesson builds on it.

Take one question into your next 1:1. Ask it plainly and listen to the answer: 'Which part of our system would you be scared to debug alone?' Whatever they name is probably your first red module.

Extend your knowledge

• ▹Read Peter Naur's 1985 essay 'Programming as Theory Building'. It's short and makes the clearest case that a program's real asset is the theory in its programmers' heads, which is exactly what agents don't produce.
• ▹For hotspot analysis (change frequency combined with complexity), read Adam Tornhill's 'Your Code as a Crime Scene'. Think of it as the rigorous version of the top-20 ownership map.
• ▹This week, add a 'time to understand' field to your postmortem template and run the git-log top-20 command on your main repo before your next retro.
• ▹Before rolling the explain-back template out widely, trial it for two sprints on one high-risk area (payments, auth or data writes) and compare the red count before and after.