I Promoted My Best Engineer. Velocity Dropped. The Scorecard Never Saw Why

Why this matters

If your review process only scores what got shipped, you'll keep promoting people — and rating agents — for the wrong reason. In a human+agent team it's worse: coordination work is already harder to see than it was for a lone IC, and now half your "team" isn't even human.

The promotion that looked obvious on paper

Two years before PhoenixDX, I signed off on a promotion that every number on the page supported. Six months later the team was slower than before I promoted our best engineer, and I couldn't explain why using the same scorecard that had justified the promotion in the first place.

The scorecard that said yes

The engineer — call him Minh — had the highest PR count on the team, the lowest bug-reopen rate, and had shipped two of the quarter's headline features solo. The new role was tech lead: unblock four other engineers, review their designs before code got written, catch integration problems early. Saying no would have meant telling my best performer that his best quarter wasn't good enough. On paper, no was indefensible. I said yes.

The first crack: velocity, not laziness

Within two months, team throughput dipped. Minh wasn't slacking — if anything he was in more meetings, more Slack threads, more design docs than anyone. This wasn't a motivation problem. It was that the thing he now spent most of his time on had no name on our scorecard, so from the outside it looked like nothing was happening.

The diagnosis: a scorecard with no column for multiplying others

When I pulled the review template apart, the gap was obvious once I went looking for it:

• ▹Measured: PRs merged, lines of code, features shipped, bugs closed by the person
• ▹Measured: individual code quality, test coverage on their own diffs
• ▹Not measured: how many other people's PRs got unblocked because of a review or a design call he made
• ▹Not measured: how much faster the team moved because he caught an integration problem before four people built on top of it
• ▹Not measured: time spent in a design doc that never became his own PR

The review had a column for output and zero for leverage. Minh's new job was almost entirely leverage. We were grading him against a job he no longer had.

The mirror: this is exactly how teams misgrade a multi-agent system

Once I named it, I recognized the pattern somewhere I hadn't expected: our own agent pipelines at PhoenixDX. When an orchestrator agent breaks a task into subtasks and dispatches them to worker agents, every dashboard we had — tokens spent, PRs opened, tests passing, diffs merged — credited the worker agent that produced the artifact. The orchestrator's actual work — deciding a task was scoped wrong, retrying a worker that stalled, catching that two workers were about to collide on the same file, killing a bad plan before it burned budget — left no artifact. It's coordination, not output, and coordination doesn't show up on a scoreboard built to count things. The failure mode is identical: any system that only instruments the unit of output will starve the unit of coordination — whether that coordinator is a promoted engineer or an orchestrator agent — because the starving is invisible until throughput drops and nobody can say why.

What changed in the review doc afterward

I added one question to every review, human or agent-team retro alike: "Name three pieces of work that shipped faster, or shipped at all, because of this person's/agent's involvement — and what would have gone differently without them?" It's deliberately falsifiable. "Great mentor, very collaborative" is a vibe — nobody can be wrong about it. "This PR would have shipped two weeks later without the design review that caught the schema conflict" is a claim with a counterfactual attached, and the people named can confirm or deny it on the spot. If nobody can name three concrete instances, the multiplier claim wasn't real — it was a story we told about a busy calendar.

Day 6 preview

If your review process can't score coordination, it can't score the actual job most senior engineers do today — and it definitely can't score what an orchestrator agent contributes versus a worker agent. Tomorrow: how to build a review or eval that scores coordination without turning it into another vibe metric.

Extend your knowledge

• ▹Read Camille Fournier's "The Manager's Path," chapter "Being a Tech Lead" — it's the pre-agent version of the same scorecard problem this lesson describes.
• ▹Look at how the Claude Agent SDK's subagent/orchestration docs describe orchestrator vs subagent responsibilities, and check whether your own tracing setup (tokens, cost, latency) attributes coordination decisions anywhere or only attributes worker output.
• ▹In your next 1:1 or agent-team retro, try asking the falsifiable question from this lesson — 'name three things that shipped faster because of X' — and see how many people can actually answer it with specifics.
• ▹Audit one existing performance review template at your org: count how many line items measure output versus how many measure unblocking, reviewing, or coordination.