Acceptance criteria for coding agents

Day 25: Acceptance Criteria for a Reader Who Can't Ask Questions

Every acceptance criterion you've ever written came with a safety net: if it was unclear, the reader could ask you. A coding agent usually doesn't ask. It fills each gap with a confident guess and keeps going. Anything you leave out still gets decided, just not by you, and you usually find out after it's shipped.

The hook: a one-line CSV export ticket

I've seen some version of this ticket on every team I've run. It looks done, and that's exactly the problem.

Story: As an account owner, I want to export my invoices to CSV.

AC:
  Given I am on the Invoices page
  When I click "Export CSV"
  Then I download a CSV file of my invoices

Give this to a mid-level engineer and you'll have three questions in Slack before lunch:

• ▹"All invoices, or only the ones matching the current filter and date range?"
• ▹"Should voided and draft invoices be in there?"
• ▹"Amounts as raw numbers, or formatted like the UI ($1,234.50)? Which date format?"

Give it to a coding agent and nobody asks anything. The agent makes three guesses without telling you, and each one sounds reasonable:

• ▹Guess 1: export every invoice on the account and ignore the active filter. "My invoices" means all of them.
• ▹Guess 2: include voided invoices. They're invoices, so they go in the file.
• ▹Guess 3: reuse the UI formatter, so amounts come out as locale-formatted strings like "$1,234.50".

The code was clean and the tests passed. Then a finance customer's accountant pulled the file into their reconciliation spreadsheet, and the totals didn't match the ledger because voided invoices had been counted as revenue. Nothing in the agent was broken. It answered a question nobody had asked out loud.

The reframe: ACs were never complete

Acceptance criteria were never a full spec. Conversation filled in the rest: the Slack thread, the aside after standup, the "quick question" at someone's desk. Agents took the conversation away and left the ambiguity behind. It's the same idea we've used since the start of this course: write for the user's outcome, not the feature. "Export a CSV" is a feature. "The accountant can reconcile this month's revenue without editing the file" is an outcome. You can't resolve the ambiguity until you know the outcome.

The 'no questions allowed' test

Before a ticket goes to an agent, a new hire, or a contractor in another timezone, run one test on each AC:

"If the reader couldn't message me, what would they have to guess?"

• ▹Write every guess on the ticket as a list. If it only exists in your head, it doesn't count.
• ▹Turn each guess into either a new AC (you've decided) or an explicit "your call" note (you've decided it doesn't matter, and you say so).
• ▹Leaving a guess off the list doesn't give the reader a free choice. It means you skipped a decision.

Don't skip the "your call" note. Telling an agent "column order is your call" shows it where it's allowed to choose. Without that note, a gap you left on purpose looks exactly like a gap you forgot.

Rewrite pass 1: the positive path

Start with what has to happen. Here's the bar: you should be able to write each criterion as a failing test before any code exists. If you can't picture the assertion, the criterion is still too vague.

BEFORE                                  | AFTER
----------------------------------------|----------------------------------------------------
Then I download a CSV of my invoices    | Export contains exactly the invoices matching the
                                        |   current filter and date range on the page
                                        | Voided and draft invoices are excluded
                                        | Columns: invoice_number, issue_date, customer_name,
                                        |   status, currency, subtotal, tax, total
                                        | Amounts are plain decimals with 2 places, no
                                        |   currency symbol or thousands separator (1234.50)
                                        | Dates are ISO 8601 (2026-10-13)
                                        | File is UTF-8 with header row; values containing
                                        |   commas or quotes are RFC 4180 quoted
                                        | Zero matching invoices -> file with header row only

Every line on the right becomes one assertion. They fall into three groups. Scope says which rows. Format says how the values look. Edge cases cover empty results, commas in customer names, and non-ASCII characters. In my experience, those three groups catch most of what agents get wrong on data features.

Rewrite pass 2: 'must NOT' criteria, the line most teams skip

In my experience, agents rarely miss the goal. The problem is how far they reach to hit it. To get the CSV amounts right, an agent will happily edit the shared money formatter, and the PDF export that uses the same formatter breaks. It'll add a CSV library because that's the clean option. It'll add a required parameter to an endpoint a mobile app depends on. Every one of these changes makes sense inside the task and causes damage outside it.

People leave these lines out because teammates already know the unwritten rules. An agent only knows what's in the ticket, the repo, and whatever context file you've given it, like CLAUDE.md or AGENTS.md. So write the boundaries down:

• ▹MUST NOT change the existing PDF export: same output, and its existing tests pass unmodified.
• ▹MUST NOT add a new dependency. Use the standard library's CSV writer.
• ▹MUST NOT change the public API signature of GET /invoices. Any new parameter is optional with the current default behaviour.
• ▹MUST NOT modify shared formatting utilities. If CSV needs different formatting, add a CSV-specific function.

A good 'must NOT' is as checkable as a positive AC. You can point to a test that stays the same, a lockfile with no diff, or an API schema check. "Don't break anything" doesn't qualify, because the agent has no way to check it.

Rewrite pass 3: machine-checkable vs judgment calls

Now sort what's left. Some criteria the agent can check on its own in a loop: run the test, read the result, fix, run it again. Others need a human to judge, and no test the agent writes will settle them honestly.

• ▹Machine-checkable: row scope, excluded statuses, column list, number and date format, quoting, empty file, PDF tests unchanged, no lockfile diff. The agent can run these without supervision.
• ▹Judgment: "export feels fast enough for a 10k-row account", "button placement is obvious next to the existing PDF button", "filename makes sense to an accountant". A human reviewer owns these.

Put the split on the ticket. Labelling the judgment ACs does two things: the agent won't fake them with a meaningless test, and the reviewer knows exactly what they're approving. Sometimes you can turn a judgment call into a check, like "10k rows export in under 3 seconds on staging". Only do that if you actually know the threshold. A number you made up just to get a test is still a guess you didn't list. It just has a decimal point.

The finished ticket: a template you can copy

Outcome: An account owner can hand this month's invoices to their accountant,
who reconciles them against the ledger without editing the file.

MUST (machine-checkable)
- Rows = invoices matching the page's current filter + date range
- Excludes status in (voided, draft)
- Columns: invoice_number, issue_date, customer_name, status, currency, subtotal, tax, total
- Amounts: plain decimal, 2dp, no symbol/separators. Dates: ISO 8601
- UTF-8, header row, RFC 4180 quoting. Zero rows -> header only

MUST NOT
- Change PDF export or its tests; add dependencies; modify shared formatters
- Change GET /invoices signature (new params optional, default = current behaviour)

JUDGMENT (human reviewer signs off)
- Feels fast enough for a 10k-row account; button placement next to PDF is obvious

YOUR CALL: column order after invoice_number; filename format

That's three blocks, with an outcome line on top and a 'your call' line at the bottom, and it fits on one screen. It works for people too. A new joiner gets as much out of it as the agent does.

The product-mindset point

Writing ACs like this makes you settle intent before anyone writes code. Should voided invoices count? Is the export for the account owner or for their accountant? Engineers used to put that PM conversation off until a Slack thread in the middle of the sprint. The agent didn't add this work. The work was always there, and the agent made it visible. It also raised the price of skipping it: before, a missed decision cost a two-minute question; now it costs a finance customer a broken reconciliation. If you can't answer one of the guesses yourself, you have a product question, not a ticket-writing problem. Take it to your PM before the ticket goes to anyone.

Today's exercise

• ▹Pick a ticket from your current sprint, ideally one you'd consider giving to an agent.
• ▹Run the no-questions test and write every guess on the ticket.
• ▹Turn each guess into an AC or an explicit 'your call' note.
• ▹Add at least two 'must NOT' lines you can check: a test that must not change, a file that must not move, or a signature that must not break.
• ▹Label each AC as machine-checkable or judgment.

Tomorrow, on Day 26, we move to the other end of the loop: reviewing what comes back. A diff can pass every AC and still be the wrong product, so we'll look at how to review for intent, not just correctness.

Extend your knowledge

• ▹Gojko Adzic, 'Specification by Example': the classic on turning examples into precise, testable criteria. Read it with 'the reader can't ask' in mind.
• ▹Dan North, 'Introducing BDD': the article that introduced Given/When/Then, developed with Chris Matts. It explains why the format was meant to drive a conversation, not replace one.
• ▹Anthropic's Claude Code best-practices guidance on CLAUDE.md files: put team-wide 'must NOT' rules (no new deps, don't touch shared utils) in the repo context so you don't have to repeat them on every ticket.
• ▹Try it yourself: on a scratch branch, give a coding agent the original one-line CSV AC, then your rewritten version, and compare the two diffs. The gap between them is what the rewrite got you.