Stop Asking If a Task Is Agent-Ready. Ask If You Can Grade It.

Three days. That's what it cost us to merge a PR an agent had knocked out in about twenty minutes, on the API client refactor at PhoenixDX. The diff wasn't the issue — four hundred lines, clean, tests green. The issue was that nobody, including the engineer whose name was on it, could say whether it was actually correct.

The refactor collapsed three near-identical retry wrappers into one client. One of our senior backend engineers took the review and kept running into the same wall: the agent had made a string of judgment calls — which errors deserved a retry, whether idempotency keys still carried through, what should happen to the one caller that depended on the old client quietly swallowing a specific timeout — and not one of those calls existed anywhere except as something you could infer from reading the diff. Every comment turned into a Slack thread. "Did you mean for this to retry on 409s now?" "Let me check what it used to do." "Why did the timeout caller's behavior change?" "I'd have to ask the agent." We weren't reviewing code at that point. We were reverse-engineering a spec nobody had ever written, one comment at a time, from the one person even less equipped to answer than the reviewer.

That's the failure I actually want to talk about, because the agent wasn't wrong, exactly. The refactor mostly held up. The failure happened earlier — we handed off a task before anyone, human or model, had settled what "correct" meant. So here's the check I run now before I write a single prompt: could I write the acceptance criteria right now, before any code exists? Not a fuzzy description — an actual rubric, a test list, a set of before/after invariants I could hand a stranger and let them verify the work without ever talking to me. If the answer's no, I stop. The task isn't agent-ready. It isn't even ready for a person. It's spec-not-ready, and no prompt fixes that.

It's not size, and it's not risk

The first pushback I get from other EMs is usually a proxy variable wearing a disguise. "Sure, but that was a big cross-cutting refactor — of course it's risky." It isn't about size. I've watched 800-line migrations sail through because the schema diff and the rollback condition were nailed down before anyone touched a keyboard, and I've watched three-line config changes turn into the exact same multi-day slog because nobody could actually say what the flag was supposed to gate. It's not "high-risk code" either — some of the scariest-looking infra changes have a crisp definition of done ("p99 latency unchanged, zero dropped connections during cutover"), while plenty of harmless UI tweaks don't ("make the settings page feel cleaner"). And it's not a junior-vs-senior split — hand a senior engineer an unspecified task and you get the same ambiguous artifact you'd get from an agent, just slower and delivered with more confidence. The variable is verifiability. Full stop. Difficulty is a distraction.

Three pairs, same task category held fixed each time, so the one thing that actually moves is easy to see:

• ▹Migrations — Agent-ready: "Backfill `orders.currency` to 'USD' for all rows before 2024-01-01, verify count matches the audit log, rollback if row count mismatches by more than 0." Not ready: "Clean up the currency data, some of it looks off." Same operation, but the second one has no stopping condition — the agent (or a human) has to invent one, and whatever it invents is now unreviewable without a debate.
• ▹Bug fixes — Agent-ready: "Repro: POST with an empty `items` array returns 500 instead of 400. Fix so it returns 400 with the existing validation error shape, add a test for the empty-array case." Not ready: "Users are saying checkout feels flaky sometimes, can you look into it." The first hands over a falsifiable claim. The second hands over a feeling, and the agent will confidently produce a fix for whatever it decides 'flaky' means.
• ▹Feature asks — Agent-ready: "Add a `?sort=recent|popular` query param to `/search`; default `recent`; `popular` orders by the existing `popularity_score` column; add cases to the existing search test file for both values plus the invalid-value fallback." Not ready: "Let users sort search results better." 'Better' is doing all the load-bearing work here, silently, and it's exactly the kind of gap an agent fills in without telling you it filled it in.

"I'll just review the diff carefully" doesn't scale

The obvious pushback: isn't careful review the safety net? You don't need the spec up front if you're disciplined about scrutinizing what comes out the other end. I used to buy this. It's exactly what our senior engineer tried on the API client refactor, and it's exactly why the review ate three days instead of thirty minutes. Reviewing a diff against a spec that only exists in someone's head means the reviewer has to reconstruct that spec first — from the code, from memory, from asking the author questions the author often can't answer either — and only then check the diff against their own reconstruction. That's one thinking task turned into two people doing it in sequence, with a lossy handoff sitting in the middle. It might work once, if the reviewer happens to know the domain cold and guesses right. It can't be a policy, because a policy has to survive the reviewer without that context, the review that lands Friday at 5pm, the PR from someone two time zones over who's asleep by the time the question gets asked. "Review carefully" isn't a substitute for a spec. It's the same failure, just wearing a review hat.

This lines up with something I keep bumping into in my multi-agent systems research at RMIT: agents don't resolve ambiguity in what you hand them, they amplify it. Underspecify a task for a single model and you get one plausible guess. Run the same underspecified task through a pipeline — planner, coder, reviewer agent — and each stage resolves the ambiguity a little differently, so the guesses compound instead of cancelling out. That's not a model-capability gap the next release quietly fixes. It's structural, baked into delegation itself, and it doesn't care whether what's downstream of you is silicon or a direct report. Any EM who's watched a vague Jira ticket come back three different ways from three different engineers has already seen this movie. The agent version just runs faster and cheaper, which is exactly why it looks like a new problem when it isn't.

So the one line I'd actually put in a team norm or a PR template this week isn't "is this a good agent task." It's "write the acceptance check before you assign it — to an agent or to a person." Concretely: nobody types a prompt until there's a rubric, a test list, or a before/after invariant sitting next to it, and PR descriptions link to that spec instead of narrating the diff after the fact. What actually shifts once that's the default: review time stops tracking diff size and starts tracking spec clarity, disagreements happen before the work starts instead of after it's already sitting in a PR, and "the agent got it wrong" mostly stops coming up — because most of what looked like the agent getting it wrong was really just nobody having decided what right looked like yet.