Failure-first specs

Day 29: Write What Breaks Before What It Does

Your coding agent has already decided what your users see when the payment provider goes down. You didn't ask it to, and you probably didn't review that decision, but it's in the code. Users judge a product mostly by what happens when something fails. Write only the happy path and the agent fills in every failure you skipped, and none of those choices gets reviewed as a product decision.

The $0.00 PR

I've seen this PR more than once. CI is green and the reviewer approves it in four minutes. Two days later a support ticket lands: "Checkout says my order total is $0.00. Is this free?" It wasn't free. The payment provider's quote endpoint had been timing out for about twenty minutes, and the agent's change had turned the outage into a zero.

The ticket said: "show the order total from the payment provider on the review page." The happy path worked fine. Here's the bit of the diff that mattered:

   async function getOrderSummary(cartId: string) {
-    const quote = await paymentClient.getQuote(cartId);
+    let quote;
+    try { quote = await paymentClient.getQuote(cartId); } catch (e) { logger.warn(e); }
+    return { items: await cart.items(cartId), total: quote?.total ?? 0 };
   }

A try/catch, a warn-level log, a `?? 0`. It looks defensive and tidy, and it would get through most code reviews. So who decided that "payment provider down" should mean "show the customer zero"? No one. The agent made the call because nobody else had.

The reframe: the agent didn't miss the edge case. It filled it in.

The usual line is "the agent missed an edge case." That gets it backwards. The agent ran straight into the edge case, either because the test mock threw or because the type checker complained about a possibly undefined value. It dealt with it, and it dealt with it in whatever way made the error disappear.

If you remember one thing from today, make it this: every failure your spec leaves out still gets handled. When a dependency fails, the code has to do something. It crashes, retries, shows a default, swallows the error, or blocks the user. If you don't pick one, the model picks whatever looks reasonable and gets the task to done. In practice that means it compiles, the tests pass, and the logs show no red.

• An unwritten failure case is a product decision you handed off without realising it.
• The agent isn't being careless. You asked for a working feature, and to an agent "working" usually means "no exceptions."
• The output looks fine. A `?? 0` reads like hygiene, not like a decision, and that's exactly why it gets through review.

Why humans got away with it: tacit knowledge

Specs were just as happy-path before agents. What changed is who reads them. On one team I worked with, a senior dev added a "We couldn't load your price, retrying..." banner every time checkout called an external service. No ticket ever asked for it. In 2023 a silent pricing fallback had given the team a very bad week, and this dev had sat through the postmortem. The rule was never written down. They just knew it.

That's tacit knowledge: team memory, old incidents, the gut reaction that money plus a fallback means trouble. It filled the holes in every spec for free, which is why nobody noticed the holes.

Your agent wasn't in that postmortem. Multi-agent systems research describes two ways agents come to know norms. Either someone specifies the norm explicitly, or the agents learn it through repeated interaction with the group. A coding agent dropped into your repo for one ticket has no such history. If the rule isn't in the prompt, the context, or the repo, the agent doesn't know it.

Tie back to the course

Earlier in the series we looked at the user's job-to-be-done and how to choose success metrics that reflect it. Today is the uncomfortable other half. Users don't average out their experience of your product. They remember the moments it let them down: the wrong total, the lost draft, the spinner that never stopped. A checkout that works 99 times and shows $0.00 once doesn't get remembered as 99% good. People remember it as the checkout that showed $0.00. Product trust gets won or lost on the unhappy path, so design it as carefully as the main feature.

The change we made: "what breaks" goes first

At PhoenixDX we flipped the order of our agent-bound specs, so the "what breaks" section now comes before "what it does." The order matters. Put it last and people skip it, or they write "handle errors gracefully," which tells the agent nothing at all. Put it first and you're forced to think about failure before you fall in love with the feature.

For each dependency or input, the template asks four things:

• How it fails: timeout, 5xx, bad data, empty result, partial result, slow response.
• What the user should see: the exact experience, not just "an error."
• What must never happen silently: the outcomes that are worse than a visible failure.
• Who gets alerted: a person, channel, or metric, and at what threshold.

Here's the template filled in for the payment example:

## What breaks (write this first)

### Dependency: Payment provider quote API
1. How it fails
   - Timeout (> 3s), 5xx, or 429 rate limit
   - Returns a quote with missing or negative total
   - Returns a quote in the wrong currency
2. What the user should see
   - Review page shows items, but the total area says
     "We couldn't load your price. Retrying..." with a spinner
   - After 2 failed retries: "Pricing is temporarily unavailable.
     Your cart is saved." Place Order button is disabled
3. Must NEVER happen silently
   - Showing $0.00 or any default/stale total
   - Allowing Place Order without a confirmed quote
   - Swallowing the error with only a warn-level log
4. Who gets alerted
   - Error-level log with cartId and provider status code
   - Page on-call if quote failure rate > threshold for 5 min

### Input: cartId
1. How it fails: cart expired or belongs to another session
2. User sees: "Your cart expired" + link back to cart
3. Never silently: render someone else's cart, or an empty total
4. Alert: none (expected), but count it as a metric

## What it does
Show the order total from the payment provider on the review page.

Notice where the work went. "What it does" is one line, and the agent can handle that by itself. "What breaks" is where the product thinking is, and only you can write it. Once you think like a product owner, more of your job is writing down what the user should experience when things break and less of it is writing code.

What happened next

Rework on agent PRs dropped noticeably. I don't have a clean before-and-after number, but the review comments changed. We got far fewer "why does this default to X?" threads, and far fewer PRs went back to the agent two or three times. If the spec says "never show $0.00," the agent doesn't come up with one.

We also got something we hadn't expected. Writing out failure cases turned up product questions nobody had ever answered. "If pricing is down, do we block checkout, or let it through and charge later?" Engineering can't answer that. It's a trade-off between revenue and trust, and product owns it. Our PMs had never been asked, because the senior dev's banner had always answered it quietly for them.

So the spec stopped being a ticket you toss to the agent. It turned into a conversation with product first. That's the real step up. An engineer who writes "what breaks" well is doing product work, not just prompt prep.

Reviewer's companion habit: read the unhappy path first

The spec is one half of the system and review is the other. Before you read the feature logic in an agent PR, scan the diff for the places where failure decisions tend to hide:

• New catch blocks: what happens inside? Is the error re-thrown, shown to the user, or swallowed?
• Default values: `?? 0`, `|| []`, `?? ''`, `.get(key, None)`, fallback configs. Each one decides what the user sees when data is missing.
• Skipped or loosened tests: `.skip`, a deleted assertion, `toBeDefined()` where there used to be an exact value, a wider tolerance.
• Mocks: new or changed mocks that always succeed, meaning the failure path never actually ran.
• Log levels: an error downgraded to `warn` or `debug` is a decision about who finds out.

Check each hit against the spec's "what breaks" list. A catch block or default that the list doesn't cover means one of two things: the spec is missing a case, or the agent invented a policy. Either deserves a comment. On the $0.00 PR this scan takes about ten seconds and catches the bug before it ships.

# Quick pre-review scan of an agent PR's added lines
git diff main...HEAD -U0 | grep -E '^\+' | grep -nE 'catch|\?\? |\|\| |\.skip|xit\(|mock|warn\('

Monday exercise

• Pick one ticket you're about to hand a coding agent this week.
• Write only the "what breaks" section: five lines max, no implementation detail. Describe what the user experiences, not how to code it.
• Give the agent the ticket with your section at the top.
• When the PR comes back, do the reviewer scan first and list every catch, default, mock, and loosened test.
• Label each one: matches my spec, contradicts my spec, or wasn't in my spec. Every "wasn't in my spec" is a product decision the agent made for you. Decide whether you agree with it.

Leading into Day 30

A product mindset means you decide what the user experiences, bad moments included, instead of leaving it to whoever or whatever writes the code last. Tomorrow we pull the 30 days together.

Extend your knowledge

• Read Michael Nygard's 'Release It!', especially the stability patterns: timeouts, circuit breakers, fail fast. It gives you the standard vocabulary for deciding how a dependency should fail.
• Read the Google SRE Book chapter on postmortem culture. Then ask which lessons from your own team's postmortems exist only in people's heads and belong in your agent context files (CLAUDE.md, AGENTS.md, or similar).
• Add a 'What breaks' heading to your team's ticket or PR template, and add this rule to your agent instructions: 'Never introduce a default value or swallowed error for a dependency failure unless the spec names it.'
• Look up the Peak-End Rule from Daniel Kahneman's work on remembered experience. It explains why people judge an experience by its most intense moment and its ending rather than its average. When a flow breaks, the worst moment is often the peak.