Your Postmortem Might Be Training Your Copilot to Fail the Same Way Twice

Day 15 — One Incident, Two Postmortems

Most teams write exactly one postmortem per incident, then send that same document to customers, to leadership, and to the engineer who ran the command that took prod down. One doc, three audiences, three different jobs: protect trust, cover legal, teach the fix. Ask one sentence to do all three and it collapses to the safest common denominator — vague. Split it into two documents with two owners, and both jobs actually get done.

The moment this breaks

A postmortem goes out to customers: "A configuration issue was identified and has been resolved." Fine — that's what customer-facing prose is for. Then someone forwards the exact same doc internally as the incident writeup. The engineer who ran the command that caused the outage reads that sentence about their own mistake and learns nothing from it — not because anyone's protecting them from blame, but because the sentence was built to not spook a customer, not to name what actually happened. Three months later, a different engineer hits the same fork in the same decision tree. Nobody warned them, because the internal doc never described the fork in the first place.

The mechanism: blameless got confused with vague

"Blameless postmortems" entered engineering culture to stop teams from punishing people for honest mistakes — punish the mistake and people stop reporting incidents, and now you're flying blind. Somewhere along the way, that got overcorrected. Avoiding blame turned into avoiding specifics, period. If naming what someone did felt anywhere near blaming them, the safe move was to describe the outcome instead of the decision. That habit then fused with the legal instinct to keep customer language noncommittal, and one style of prose — passive, safe, outcome-only — started doing double duty for two audiences that need opposite things.

Same incident, sentence by sentence

Take a shape of incident that's showing up more now that infra changes get authored by copilots, not just humans: an on-call engineer approves a Terraform plan an internal LLM infra-agent generated, without diffing it, because the PR is tagged "hotfix" — a label that skips the mandatory human-diff-review gate normally required for agent-authored changes. The plan deletes a security group. Prod goes dark for 40 minutes.

CUSTOMER VERSION:
"A configuration issue was identified during a routine infrastructure
update and was resolved within 40 minutes. We have added additional
safeguards to prevent recurrence."

INTERNAL VERSION (what should be written instead):
"On-call approved a Terraform plan generated by the infra-copilot
agent without reviewing the diff. This was possible because PRs
tagged 'hotfix' bypass the mandatory human-diff-review gate that
otherwise applies to all agent-authored infra changes. The on-call
assumption was: 'hotfix-tagged PRs from the copilot have already
been validated by the agent's own plan-check step' — that check
only validates syntax, not intent, and nothing in the tooling
surfaces that distinction at approval time."

The customer sentence isn't false. It's also useless to every engineer who will ever stand at that same fork — approve a hotfix-tagged agent PR under time pressure, or stop and diff it first. The internal sentence names the exact gate, the exact label, and the exact false assumption someone made. That's the only version a future engineer can recognize before they walk straight into it.

Why one document can't do both jobs

• ▹The customer doc is built for trust and liability: say as little as possible, sound confident, commit to nothing that could be used against you.
• ▹The internal doc is built for pattern recognition: name the exact decision point, so the next engineer's brain lights up at the same fork before they take the wrong branch.
• ▹Those two goals pull the same sentence in opposite directions. Try to satisfy both at once and you get a document that satisfies neither — legal still flags anything that reads like an admission, and engineers still can't pull a decision tree out of it.
• ▹This matters even more on an AI-first team. Internal postmortems increasingly become the raw material fed into on-call copilots and incident-response RAG indexes. A vague postmortem doesn't just fail to teach a human — it quietly poisons the corpus an agent will later query when it's trying to figure out whether this has happened before.

The split, in practice

• ▹Two templates. A customer template — impact, timeline, resolution, high-level remediation. An internal template — the exact commands run, the exact alerts ignored or missing, the exact assumption that turned out wrong, and the decision tree with the fork marked.
• ▹Two owners. The customer doc belongs to whoever owns customer trust — support lead, PM, founder. The internal doc belongs to engineering, full stop.
• ▹Two review gates — and the internal one is the interesting call. Have it reviewed by the engineer who nearly made the same mistake last quarter, not by a manager. That person knows exactly which sentence is still too vague to be useful, because they stood one branch away from this exact incident.
• ▹Write the internal doc first, straight from the raw timeline and logs, holding nothing back. Derive the customer doc by stripping detail from that — not the other way around. Redacting from something specific is easy. Adding specificity to something vague after the fact almost never happens.

The trap: don't let the internal doc become a blame document

Specificity isn't blame. "On-call approved an unreviewed diff because the hotfix label bypasses the review gate" names a decision and a gap in the system. "Jane should have known better" names a person and teaches nobody anything repeatable. Here's the test: swap the engineer's name for anyone else on the team — does the sentence still hold up, still teach the same thing? If yes, you named the decision, not the person. That's what blameless was supposed to mean all along. It never meant "don't say what happened."

The test for whether it worked

Hand your internal postmortem to a new hire. If they can tell you exactly what they'd do differently next Tuesday — which gate they'd check, which assumption they'd verify, which alert they'd stop ignoring — it worked. If all they can do is summarize what happened, you've written a customer doc twice, and someone on your team is going to live through this incident's twin.

A postmortem is where a team writes down what it learned. Day 16 is about what happens when nobody actually opens the file again.

Extend your knowledge

• ▹Go read the source: John Allspaw's Code as Craft posts and Etsy's Debriefing Facilitation Guide, the origin of 'blameless postmortem' — worth a reread now that the term's gotten diluted.
• ▹Pull your last three postmortems and run the new-hire test on each: could someone act differently next Tuesday from what's written, or does it only describe the outcome?
• ▹If your team already runs an on-call copilot or incident search tool, check what it actually returns when queried about a past incident. Fast way to find out if your internal docs are specific enough to be useful as retrieval data.
• ▹Draft both templates — customer and internal — before your next incident, not during one. Decide the two owners and two review gates now, while there's no pressure to rush a single doc out the door.