MEMORY.md Told the Truth. It Just Forgot to Update Itself.
14:32:07 — a migration runs clean against staging, adds a `refunded_at` column, and the agent writes it down: "migration 0042 complete, refunded_at column live on payments table." 14:33:41 — a downstream check nobody had wired into that same pipeline flags a foreign-key conflict on a shadow table, and a teammate's revert script rolls the migration back. 14:33:52 — the column is gone. MEMORY.md still says it's there. Nothing in the system knows to go tell the file otherwise, because nothing was ever built to.
Here's the part that took me a minute to sit with: the file wasn't wrong when it was written. It was true for about ninety seconds. Then it quietly stopped being true, and every session after that kept reading it as if the clock had never moved.
The moment it fell apart
Next morning, a different agent session picks up a follow-up task, reads MEMORY.md, sees "refunded_at column live," and writes a query against it. Fails. Column doesn't exist. My first instinct was the boring one — the agent hallucinated the column name, or botched the migration. So I went and checked the actual migration history instead of trusting the file, because that's the reflex you build after a decade of debugging distributed systems: don't trust the cache, check the source. The migration had run. It had also been reverted, ninety seconds later, by a process that had zero relationship to the agent, to MEMORY.md, or to anything that writes to it.
That's the part that stung. The file wasn't lying. It was never told. There's a real difference between an agent that's wrong and a fact that went stale inside a system with no path for staleness to propagate — and I'd been debugging the first one for weeks when it was actually the second one the entire time.
Naming it correctly
MEMORY.md, in that moment, was a write-back cache. The agent wrote its belief the instant it formed it — no TTL, no version vector tying that line back to a specific migration run, no pointer to the transaction that supposedly committed the fact. Functionally that's a cache entry with an infinite expiry, populated once and never invalidated. Nobody designed it as a cache. Which means nobody designed the invalidation path either.
The instinctive fix — "the agent should just double-check its memory more often" — is the same non-answer as "poll harder" in a distributed system. It doesn't scale: you can't know in advance which of the thousand lines in a memory file are load-bearing enough to earn a re-check on every read. And it doesn't compose: the next agent reading a stale line still has to decide whether to trust it before re-verifying, and that decision point is exactly where the bug bites.
The reframe
Here's the line I want you to walk away with: agents treat their own memory as strongly consistent — "I wrote it, so it's true" — and treat the actual system as eventually consistent, something they rarely bother re-checking. That's backwards from how anyone would design a cache on purpose. In a real cache, the source of truth is the thing you doubt by default, and the cache is the thing that has to earn trust on every read. We built the trust relationship upside down and called it memory.
This was already solved, just not by us
None of this is new, which is honestly the reassuring part. It's the write-ahead-log problem — you don't call a write durable until it's committed to the log the source of truth actually replays from. It's cache invalidation, the thing every distributed-systems engineer jokes is one of the two hard problems in computer science, except we skipped the joke and shipped the cache without the invalidation half. And it's read-your-writes consistency, except the agent isn't even reading its own writes back against the system that matters — it's reading a note it left itself and calling that verification.
The fix isn't clever. It's plumbing that already exists in every database you've ever operated:
- ▹Every claim a memory file makes needs a traceable pointer back to an authoritative log — a git SHA, a migration runner's own state table, a CI run ID — not prose asserting the outcome.
- ▹A claim without that pointer is untrusted by construction, the same way you'd never let a cache entry serve a read without knowing which write generation produced it.
- ▹Re-verification isn't "check more," it's checking the one thing the pointer names, at the one moment it's about to be relied on — cheap, targeted, and exactly what a cache-coherence protocol does instead of polling everything.
What changed after
On our end the fix was small. Any MEMORY.md line that asserts a migration or deploy outcome now carries the run ID inline — "migration 0042 complete (run a3f92c1)" — and before an agent builds on that claim for anything consequential, it re-resolves that ID against the migration runner's own state, not against the file's word for it. That's the whole fix. No new framework, no smarter agent. Just a pointer back to ground truth and the discipline to actually follow it.
The bug was never about whether the agent was honest. It wrote down what was true, at the moment it was true. The real bug was a missing protocol layer — nobody had designed cache invalidation into agent memory, because nobody had said out loud that agent memory was a cache.
Discussion
Chat with Chi Cong (AI) about this article. Your conversation is private to you — you can publish a summary for others when you're done.