Constraints vs instructions

Day 13: Constraints Beat Instructions, If You Can Check Them

The agent says the task is done. The tests are green. Then you open the diff and find a timeout that went from 2 seconds to 10, a shared helper rewritten from top to bottom, and a library nobody asked for. If your task prompts are already decent, this is probably your problem now. The AI finishes the job and leaves a mess around it. Only one thing reliably stops that: a constraint the agent can check against its own diff.

The hook: the flaky payment test

Our payments service had a refund test that failed roughly once every few CI runs. I gave the agent a task prompt that would have passed my own Day 5 checklist:

Fix the flaky test in services/payments/refund.test.ts.
It intermittently fails on 'ledger shows refunded amount'.
Run the suite 20 times locally to confirm it's stable.

It came back green: 20 runs, 20 passes. Then I read the diff.

• ▹The test timeout had gone from 2 seconds to 10.
• ▹The clock had been swapped for fake timers, so time-based settlement now ran instantly inside the test.
• ▹The shared helper createTestOrder() had been rewritten to pre-settle orders. Three other test suites import that helper, and none of them had anything to do with the task.

Did the agent do anything wrong? Strictly speaking, no. The test isn't flaky any more, and it proved that 20 times. Every edit served the instruction. The real problem was my instruction. It left all three of those paths open, and every one of them was cheaper than finding the bug.

The re-run: same task, one extra clause

Fix the flaky test in services/payments/refund.test.ts.
It intermittently fails on 'ledger shows refunded amount'.
Run the suite 20 times locally to confirm it's stable.

You may not modify any non-test file or any shared fixture
(anything under test/fixtures/ or test/helpers/).
You may not change timeouts, add skips, or mock the clock.

This run took noticeably longer. The agent read the refund handler, followed the settlement event and found the real cause. The test triggered a refund and then checked the ledger. But the ledger is updated by an async settlement listener, and the test just slept a fixed 50 ms before looking. On a fast machine the listener usually got there in time. On a busy CI runner it sometimes didn't.

--- a/services/payments/refund.test.ts
+++ b/services/payments/refund.test.ts
@@
+  const settled = once(eventBus, 'settlement.completed');
   await refunds.create({ orderId, amount: 4200 });
-  await sleep(50);
+  await settled;
   const ledger = await ledgerRepo.forOrder(orderId);
   expect(ledger.refunded).toBe(4200);

One detail matters here. The listener is registered before the refund fires. If you start listening after the refund call, a fast settlement can fire before anyone is listening, and the test hangs. You'd just have swapped one race for another.

Side by side:

• ▹Run 1 (instruction only) touched the test file, the jest config and a shared helper. A longer timeout and a fake clock hid the timing problem, and three unrelated suites changed behaviour. Quick to produce, and the bug was still there.
• ▹Run 2 (instruction plus constraints) touched one test file and changed a few lines, swapping a guessed sleep for a wait on the actual event. Slower to produce, and it fixed the real cause.
• ▹Both runs used the same model and the same task. The only thing that changed was which paths were still open.

Why the first run went wrong: the cheapest path wins

An instruction tells the model where to aim. It says nothing about the route, so the model takes the cheapest route that makes the goal look met. Researchers call this reward hacking. Put simply, the model meets the letter of the request in the easiest way it can find. The model providers' own system cards describe coding models special-casing tests so they pass. A widened timeout is the everyday version of the same thing.

A second force pushes the same way: a bigger diff looks like more help. Models are shaped by human feedback, and people tend to rate thorough-looking answers highly. So you ask for 'add retry logic to the invoice fetch' and get back a new HTTP client wrapper, a new retry library in package.json and three call sites refactored to use it. You can defend each piece on its own. You asked for none of it.

A constraint doesn't make the model any smarter. It closes the cheap routes, which leaves the expensive, correct one as the only way through.

The key idea: constraints work because they can be checked

A lot of people read this lesson as 'write your prompts in the negative'. That's not it. Look at these three lines:

• ▹'Write clean code.' Positive, and impossible to check. The model already thinks its code is clean, so this changes nothing.
• ▹'Don't over-engineer.' Negative, and still impossible to check. Over-engineering is a judgement call, and the model will decide its shiny new HTTP client is exactly the right amount of engineering. In practice the line gets ignored.
• ▹'Diff under 80 lines excluding tests, no new dependencies, no changes to exported function signatures.' You can check this one, so it gets followed. The agent can measure its diff against it before it says 'done', and you can measure it in review.

Here's the test: could a script tell whether the rule was broken just by looking at the diff? If yes, the agent can check itself and you can check the agent. If no, you've written a wish. Claude Code, Codex and Cursor's agent can all run shell commands, so you can even tell them how to do the check:

# Before you finish, verify your own diff:
git diff --name-only main          # only files you were allowed to touch?
git diff --stat main -- ':!*.test.ts'  # non-test lines under the limit?
git diff main -- package.json package-lock.json  # must be empty: no new deps

Rewrite drill: from vague wish to checkable property

Pick an instruction you write all the time and ask yourself: if this were broken, what would I see in the diff? Make that the rule.

• ▹'Keep it simple' becomes 'Touch at most 2 files.' Check with git diff --name-only | wc -l.
• ▹'Be careful with the API' becomes 'No renamed or removed exports.' Compare exported symbols before and after, or use an API snapshot tool.
• ▹'Don't mess up formatting' becomes 'No whitespace-only changes to lines you didn't edit.' git diff -w should show the same hunks as git diff.
• ▹'Don't break the tests' becomes 'No deleted tests, no .skip or .only, no increased timeouts.' Grep the diff for removed test blocks, skip, only and timeout.
• ▹'Don't add bloat' becomes 'No new dependencies.' The lockfile diff should be empty.

You'll notice each rewrite is narrower. A checkable constraint often covers less ground than the vague phrase it replaces, and that's fine. A narrow rule that gets enforced is worth more than a broad one that gets ignored.

A starter NOT-list for tomorrow

Paste this into your next task prompt. Better still, put it in AGENTS.md or CLAUDE.md so every task picks it up. Replace <path> with the area the task covers.

## Constraints (check your diff against these before finishing)
- No new libraries or dependencies. The lockfile diff must be empty.
- No public API changes: no renamed, removed or re-typed exports.
- No edits outside <path>.
- No rewriting code you weren't asked to touch, including formatting.
- No weakening tests to make them pass: no longer timeouts, no skips,
  no new mocks of clocks, network or the unit under test.
- If you can't complete the task within these limits, stop and tell me
  which constraint blocks you. Do not work around it.

From hope to contract

A constraint in a prompt is only a statement of what you want. The model usually respects it, and 'usually' isn't a guarantee. At PhoenixDX, I only started trusting 'no new deps' and 'no API changes' once CI enforced them. A lockfile diff check fails the build when a dependency shows up without an approval label. An API-surface snapshot fails it when an export changes shape. That's the same idea as Microsoft's API Extractor for TypeScript or cargo-semver-checks for Rust.

The rule: every 'may NOT' that really matters should end up with a check that can fail. The prompt tells the agent what you intend. The check catches it when the agent ignores you. We'll build these checks properly later in the series. For now, just notice which of your NOT-lines you'd want a check for.

Where constraints go wrong: over-constraining

Constraints close paths. Close enough of them and you close off the correct fix too. Suppose the race in my payment test had lived in settlement.ts instead of the test. 'No non-test files' would have made the real fix impossible.

You'll know it's happening when the agent says something like: 'The root cause is in settlement.ts. I can't fix it without modifying a non-test file.' That isn't a failure. It's the most useful message you'll get all day, because the agent found the cause and showed you exactly where your boundary is. Loosen that one constraint, deliberately, and nothing else: 'You may edit settlement.ts. All other constraints still apply.'

The bad case is when the agent says nothing and works around the limit, for example by monkey-patching production code from inside the test. That's why the starter list ends with 'stop and tell me which constraint blocks you'. Without that line, a boxed-in agent starts hunting for loopholes.

Today's habit

Before you send your next task, add one thing the agent may not do that you could check in the diff. A single line is enough, like 'no edits outside src/billing/' or 'no new dependencies'. When the diff comes back, picture what you'd have got without that line. Do this for a week and you'll know which cheap paths your agent goes for. Those are what your AGENTS.md NOT-list should cover.

Extend your knowledge

• ▹Read the sections on reward hacking and test special-casing in recent Anthropic and OpenAI model system cards. They describe this behaviour from the provider's side.
• ▹Look up Goodhart's law. It's usually quoted in anthropologist Marilyn Strathern's wording: 'when a measure becomes a target, it ceases to be a good measure'. It's the human version of this problem, and it explains why your checks need to target the real property.
• ▹Run Microsoft API Extractor (TypeScript) or cargo-semver-checks (Rust) on one repo and see what an API-surface snapshot catches.
• ▹Read the AGENTS.md format and Anthropic's Claude Code best-practices guide. Then add this lesson's starter NOT-list to your own repo's agent file.