Three Names, 78% of Your Sev1s: The On-Call Audit Most EMs Never Run

Day 17: Your On-Call Rotation Is an Org Chart in Disguise

Every EM eventually checks whether the on-call rotation is fair. Almost nobody checks who actually fixes the outage once the page goes off — and that's the question that decides whether your next resignation letter is a shrug or a five-alarm fire.

Run this audit before you read the rest of this lesson

Go open your incident tool right now — PagerDuty, Opsgenie, incident.io, whatever your team pays for. Skip the schedule tab. Pull the resolver field instead: for every sev1 and sev2 in the last 90 days, who actually closed it, not who happened to be on shift when it fired. Five minutes, one query. Do it before you finish this lesson, because the story below only lands once you already suspect what your export is about to show you.

• ▹Filter to the last 90 days, sev1 and sev2 only
• ▹Export who resolved each incident, not who was paged
• ▹Count closes per person, divide by rotation size
• ▹If three people account for more than 60% of closes on an 8-12 person rotation, keep reading — I had the exact same number

The week I paged around my own rotation

I was on-call. A sev1 fired at 2am — a queue backup in a service I hadn't touched in months. I looked at the alert, then paged a teammate who wasn't scheduled that week, because I knew he'd fix it in ten minutes and I'd need forty. In the moment that felt like good judgment. It kept feeling like good judgment for months. It only stopped feeling normal when I noticed how often it was happening — not just from me, from the whole rotation, everyone quietly routing around the schedule to the same two or three names.

The schedule said the load was spread evenly. The behavior said the opposite: the team had already voted, page by page, on who the real responders were. The rotation was theater. The escalation shortcut was the actual system running underneath it.

What the resolver data showed

Once I was CTO I finally ran the numbers properly — not a hunch, an actual export. Ten people on the rotation. Three names closed 78% of every sev1. Not one of those three was labeled 'senior' anywhere — not in the HR system, not in the leveling doc, not in the on-call runbook. On paper they were mid-level engineers taking shifts like everybody else. In practice they were the only three people who could read a stack trace from our legacy billing service and know what to do about it in under fifteen minutes.

That gap — between what the org chart says and what the incident data says — is the whole lesson. It stays invisible until you go looking for it, and it never fixes itself on its own.

Why this stays invisible by design

PagerDuty, Opsgenie, and every AI incident copilot bolted on top of them — Rootly, incident.io's AI triage, whatever LLM runbook assistant your team built — are all optimized to answer one question: whose turn is it. They're genuinely good at that. None of them default to answering the question that actually matters: whose turn does it become forty minutes in, once the scheduled person is stuck. Fairness and bus factor are two different metrics that happen to live on the same schedule screen, and if that screen is the only thing you look at, fairness is the only thing you'll ever measure.

• ▹Fairness metric: pages per person, shift count, off-hours load — this is what your rotation tool shows you by default
• ▹Bus-factor metric: sev1/sev2 resolutions per person, weighted by how often they were the escalation target instead of the scheduled owner — this is a query nobody builds for you

The AI layer makes this harder to see, not easier. A triage bot that auto-summarizes a sev1 and drafts a first response looks like progress — and it is — but it also raises the bar for what counts as 'stuck enough to escalate.' The scheduled engineer sits with a convincing AI-generated summary a little longer before admitting they don't actually know the fix, and the page still lands on the same three people in the end. The bot doesn't shrink your bus-factor problem. It just delays the moment you find out you have one. Route a multi-agent incident pipeline the same way — dispatcher hands off to the 'next available' agent, but every low-confidence handoff quietly funnels to the same specialist agent or the same human-in-the-loop — and you've built the identical pattern on different hardware. Measure who dispatch picked and the load looks balanced. Measure who actually closed the ticket and you'll find your bottleneck standing right there.

The reframe: rotation as org chart, not staffing tool

Stop treating the on-call schedule as a fairness problem to solve. Treat the resolver data as the org chart your HR system doesn't have — the real map of who understands which system deeply enough to be trusted with it at 2am. That map exists whether you go find it or not. The only choice you actually get is when: on a calm Tuesday, during an audit you ran yourself, or the week one of those three names hands in their notice and walks out the door with a decade of tribal knowledge about the billing service.

What I did once I saw the number

I broke the shortcut on purpose. Told the team: no more off-rotation pages to Priya or Sam without a documented reason, and every incident they touched had to end with a runbook entry, not just a closed ticket. I accepted worse MTTR for about two months — pages sat longer, some escalated a step further than they used to, a couple of sev1s ran longer than they should have. That was the price of finding out, on my own schedule, whether the rest of the rotation could operate without three specific people. It turned out it could — once the knowledge actually got written down. It couldn't before.

• ▹Turned every incident Priya or Sam resolved into a runbook entry within 48 hours, no exceptions
• ▹Paired the next scheduled engineer with them live on the next similar page, instead of letting it get reassigned away
• ▹Tracked resolver concentration monthly instead of shift fairness — watched that 78% on three names shrink as the runbooks filled in
• ▹Told the team up front why MTTR would look worse for a while, so nobody mistook it for regression

Your move for Day 17

This week, run the audit from the top of this lesson for real: resolver data, not the schedule, last 90 days, sev1 and sev2 only. Then say your three names out loud to yourself. Not in a doc yet, not in a Slack post — just say them. That's the actual bus-factor conversation, and you need to have it with yourself before you have it with them. Tomorrow picks up exactly here: what to do with those three names once you've got them, and how to spread concentrated knowledge around without burning out the people who happen to be good in a crisis.

Extend your knowledge

• ▹Pull your incident tool's resolver export this week and calculate the concentration ratio — top 3 resolvers over total sev1+sev2 closes — before your next rotation planning cycle
• ▹If you run AI-assisted triage (Rootly, incident.io AI, or an in-house LLM runbook bot), check whether its escalation logs record which human it ultimately handed off to — that log is your resolver data for the AI-assisted era
• ▹Read the 'On-Call' chapter in Google's SRE Workbook for the classic framing of escalation paths, then hold it up against your own resolver export — the gap is usually where your bus factor is hiding
• ▹Preview Day 18: once you've named your three people, the next move is turning what they know into something the other seven can actually act on