The Prompt Worked Perfectly — Until Someone Else Ran It

Day 2: The Prompt-as-Spec Test

A prompt that only works when you're the one running it isn't a spec. It's a note to yourself that happens to be shaped like instructions. If you can't hand it to someone else and get the same result, you don't have reusable leverage — you have a habit.

The incident

At PhoenixDX, one of our senior engineers had a prompt for adding a new API endpoint. He'd run it a dozen times over a sprint, tweaking a word here and there, and it kept producing clean, on-pattern code. He was proud of it — fairly, it was a good prompt. He dropped it into our shared prompt library so the rest of the team could reuse it. Two days later a teammate ran the exact same text. Same repo, same day, nothing changed in between. The agent produced an endpoint that technically compiled but used the wrong auth middleware, invented its own error-handling pattern instead of the one the codebase actually used, and shipped with no tests. Same input. Worse output.

Diagnosing it live

We put the original prompt and the failed run side by side. The text matched exactly. What didn't match was everything that happened in the original engineer's head before he hit enter. He knew, without ever writing it down, which controller file to model the new endpoint after — he'd been living in that code all week. He knew what "proper error handling" meant, because there was exactly one pattern the team used and he'd long since internalized it. He knew what "done" looked like, because he was mentally checking the diff against a definition he'd never typed anywhere. Every time he ran that prompt, he was quietly patching those three gaps from memory. It never felt incomplete to him — because he was the one completing it, just not on the page.

The actual failure mode: tacit context

This isn't a wording problem. It's tacit context — knowledge the author has that never made it into the artifact, because the artifact was never the only thing carrying the intent; his memory was doing half the work. Day 1 covered why this cuts deeper with AI specifically: a prompt is a spec with no compiler. Ambiguous code fails loud, at build time. An ambiguous prompt fails quiet — the agent doesn't flag the gap, it fills it with a guess, and that guess looks like working software right up until someone checks it against the pattern the author actually had in mind. A human reader sometimes rescues tacit context by asking a question in Slack. An agent doesn't ask. It just picks something plausible and ships it.

The test

A prompt is a spec only if it transfers intent with zero shared context. That's the whole rule, and it's falsifiable: hand it to someone who wasn't in the room — a teammate who's never touched that code, or a brand-new agent session with no prior turns — and watch what breaks. If the output degrades, the prompt was never a spec. It was a trigger that only worked because you were standing next to it.

Before / after: the same prompt

Here's the original, tacit-context version — this is what was actually typed:

Add a new endpoint for cancelling a subscription. Follow our existing patterns 
and make sure error handling is proper. Should be done when it works.

Read that back now that we know where it broke. "Existing patterns" assumes the reader already knows which file is the reference implementation. "Proper error handling" assumes the reader already knows there's exactly one accepted pattern, not a menu of options. "Done when it works" assumes the reader shares the author's private checklist for what "works" even checks. None of that is on the page — it was in his head. Here's the rewrite that names all three out loud:

Add POST /subscriptions/:id/cancel.

Model it directly on billing/controllers/pause_subscription.py — same file 
structure, same service-layer call pattern, same auth middleware import 
(requireOwner, not requireAuth).

Error handling: use the ApiError class from lib/errors.py exclusively. 
Do not write ad-hoc try/except blocks or return raw dict error responses 
-- that was the old pattern before the ApiError migration and it still 
shows up in some files; don't copy it.

Done means: a passing test in tests/billing/test_cancel_subscription.py 
that covers (a) successful cancel, (b) already-cancelled subscription 
returns 409, (c) non-owner request returns 403 via requireOwner.

Nothing in the second version is clever. It just says out loud the three things the original author was silently supplying every single time: the file to copy, the pattern to follow (plus the deprecated pattern to explicitly avoid — the "thing everyone just knows" trap), and the acceptance check. That's the whole delta between a prompt that worked once and a prompt that works for anyone.

Checklist for auditing your prompts

• ▹Does it name a concrete target file or reference implementation, not just "our patterns" or "the usual way"?
• ▹Does it state a definition of done someone else could check without asking you — a test to pass, a specific behavior, not "it works"?
• ▹Does it call out the thing every regular on the team already knows not to do (the deprecated pattern, the file not to touch) — exactly the knowledge a newcomer or fresh agent session won't have?
• ▹Have you actually run it in a context with zero shared history — a teammate who wasn't in the discussion, or a brand-new agent session with no prior turns — before calling it reusable?

The reframe

Stop scoring a prompt by whether it produced good output for you. Score it by whether it produces good output without you in the loop to quietly patch the gaps. That's the real bar for anything you hand to a teammate, drop in a shared library, or expect to still work in next week's agent session. Day 3 pushes this one step further: what happens when the "second reader" isn't a person who might ask a clarifying question, but another agent that will confidently fill the gap wrong and just keep going.

Extend your knowledge

• ▹Pick your best-performing prompt from the last two weeks and hand the exact text to a teammate who hasn't touched that code — don't explain anything verbally first. Log every question they ask back; each one is a tacit-context gap.
• ▹Run that same prompt in a brand-new agent session (no prior chat history) instead of continuing an existing one, and compare the diff against what you got in your original session.
• ▹For any prompt you plan to put in a shared team library, run it through the four-item checklist above before you share it, not after someone reports it broke.
• ▹Revisit Day 1's 'spec with no compiler' framing and think about which of your recent prompts you've been unconsciously 'compiling' in your head every time you run them.