Reflection steps in agent pipelines

Day 20: Reflection Without New Evidence Only Adds Confidence

Your self-critique step is probably approving bugs. A lot of agent pipelines now have a 'reflect', 'critique' or 'think again' pass, and it costs tokens and latency on every run. If that pass can't see anything the first one couldn't, it won't catch the bugs that matter. It just makes a wrong answer look reviewed.

The hook: 'verified, safe to apply'

We asked our migration agent to enforce uniqueness on the email column of the users table ahead of a login change. It wrote the migration, and then a reflection step reviewed it. That step used the same model and saw the same context. Its prompt was: 'Review your migration carefully for correctness and safety.' Here's what it said, and next to it, what staging said:

--- Reflection step output ---
Reviewed migration 0142_unique_user_email.
- Index is created CONCURRENTLY, so no long table lock.      OK
- Emails are validated as unique at signup.                   OK
- Down-migration drops the index cleanly.                     OK
Verdict: verified, safe to apply.

--- Staging ---
ERROR:  could not create unique index "users_email_lower_key"
DETAIL: Key (lower(email))=(<redacted>) is duplicated.
-- and because it was CONCURRENTLY, Postgres left an INVALID index behind
-- that someone had to find and drop by hand.

Read the critique line by line and every point holds up. Signup validation really does exist. What neither pass knew was this: older accounts were created before that validation existed, and some emails differ only by case. Staging runs on a copy of production data, so it had both. The fact that would have caught the bug lived in the database. It wasn't in the diff and it wasn't in the model. So the reflection step read the same diff, shared the same assumptions, agreed with them, and came out sounding more certain than the first pass did.

The puzzle

Reflexion, Self-Refine and similar papers report real gains from reflection. Plenty of agent frameworks let you add a critique loop in one line. So if self-critique missed a bug this ordinary, what are those papers actually measuring? Before you answer, look at what each result depended on.

What the evidence says

Reflexion (Shinn et al., 2023). On coding tasks the agent writes its own unit tests, actually runs them, and reflects on the results. The execution feedback is the new signal, and that loop is how it gets 91% pass@1 on HumanEval against 80% for the GPT-4 baseline. Reflections also go into episodic memory, so each retry starts with the evidence from the last failure. This isn't one model reading the same answer twice. In fairness, the framework also allows 'internally simulated' feedback, and the abstract doesn't break down gains by feedback source. The authors do report that the choice of feedback signal changes performance.

CRITIC (Gou et al., 2023). Here the model critiques its output with tools: a search engine for facts, a Python interpreter for math, a toxicity API for safety. My reading of their ablations is that the gains mostly vanish once you remove the tool feedback and let the model critique itself alone. Check the paper for the exact figures.

Huang et al., 'Large Language Models Cannot Self-Correct Reasoning Yet' (ICLR 2024). On reasoning benchmarks like GSM8K, CommonSenseQA and HotpotQA, intrinsic self-correction (no oracle labels, no external feedback) left accuracy flat or made it worse. Some earlier positive results had used oracle labels to decide when to stop correcting. That leaks the answer into the loop.

'Decomposing LLM Self-Correction' (arXiv 2601.00828) adds a twist. It's one preprint, one benchmark, three models, so read it as a signal, not settled law. On GSM8K-Complex, stronger models fixed a smaller share of their own errors. DeepSeek, at about 94% accuracy, corrected 16.7% of its errors intrinsically. GPT-3.5, at about 66%, corrected 26.8%. The authors call this the 'Error Depth Hypothesis': stronger models make fewer mistakes, but the ones they still make are deeper and harder to fix alone. Two more findings: detecting an error didn't predict correcting it, and hints about where the error was hurt every model. My interpretation, not theirs: a pointer with no evidence attached pushes the model to change something, and what it changes may be a step that was already right.

Self-Refine (Madaan et al., 2023) and MCQG-SRefine (arXiv 2410.13191) do show gains from self-critique. The biggest ones are on output judged by preference, like dialogue, readability or question quality, graded by humans or an LLM judge. Self-Refine's gains on math reasoning were small. In MCQG-SRefine, expert-written prompts also put domain knowledge into the critique step. That's real value. But it's mostly polish on subjective output, not correctness checked against ground truth.

| Study                        | Reflection signal                         | Result                                         |
|------------------------------|-------------------------------------------|------------------------------------------------|
| WITH EXTERNAL SIGNAL                                                                                                    |
| Reflexion (coding)           | Executed (self-written) unit tests        | 91% vs 80% pass@1 on HumanEval                 |
| CRITIC                       | Search, Python interpreter, toxicity API  | Consistent gains; shrink without tools         |
| MASTER (2512.13930)          | DFT physics simulation results            | Up to 90% fewer simulations vs trial-and-error |
| NO EXTERNAL SIGNAL                                                                                                      |
| Huang et al. (ICLR 2024)     | Model re-reads its own answer             | Flat or worse on reasoning                     |
| 2601.00828                   | Model re-reads its own answer             | Low correction rates; stronger models fix less |
| Self-Refine / MCQG-SRefine   | Self-critique, mostly preference-graded   | Gains mainly in quality/style; small on math   |

So here's the answer to the puzzle. The strong reflection results mostly measure one of two things: a loop fed by an external verifier, or polish on tasks graded by preference. 'Reflect on your answer' with nothing new to look at is the weakest condition in the literature. It's also one of the most common setups in production.

The mechanism: correlated errors

Earlier in the course we saw that ensembles and voting only help when the members make different mistakes. Put five agents on the same model with the same context and they tend to make the same mistake five times, then outvote the truth. A critic running on the same model with the same context is that problem shrunk down to one agent. It's an ensemble of size one with a sterner persona. Both sides are conditioned on the same weights and the same tokens, so its blind spots mostly line up with the generator's.

That's why their agreement tells you so little. In the migration, both passes believed the same thing: 'signup validation means the data is unique'. The critic didn't test that belief. It repeated it and added a checkmark. And if the Error Depth Hypothesis holds, this gets worse over time. As your base model improves, the errors left over are more and more the ones it can't see from its own point of view. Those are exactly the errors a self-critique loop exists to catch.

Reframe: reflection is a multi-agent design problem

Stop thinking of reflection as a prompt trick. Think of it as adding an agent to the system. A critic agent earns its place when both of these are true:

• ▹It has different information. That could be evidence (test output, type errors, logs, query results) or different context (another model, another spec, the production schema, the incident history). The multi-agent RL communication literature makes a related point (see the survey in arXiv 2203.08975): communication helps by widening an agent's view of the environment, which means carrying observations the receiver doesn't already have.
• ▹It can block. If the critic's output lands in a log, or in a context window the generator is free to ignore, it's commentary. Wire it to a required status check and it becomes control.

The same survey proposes nine dimensions for analysing communication design, covering what gets sent, to whom, and under what constraints. In other words, 'add a critic' is several decisions, not one. I learned the human version of this as a founder. A cofounder who knows things you don't and can overrule you is worth far more than an advisor who reads your deck and nods. Most reflection steps in production are the advisor.

Taxonomy: from strongest to weakest reflection signal

• ▹1. Executed tests or the type-checker. Value: highest. For the property under test, the result is ground truth, and the failure output tells the model what broke. Cost: compute and test runtime, often cheaper than another big LLM call. Limit: only as good as your test coverage, and flaky tests become false vetoes.
• ▹2. Static analysis and linters (ruff, mypy, Semgrep, Squawk for Postgres migrations). Value: high for known bug classes. Deterministic and fast. Cost: almost nothing per run, plus setup. Limit: blind to data-dependent and semantic bugs. Squawk would not have caught our duplicate emails.
• ▹3. Retrieved prod or observability data: schema and data snapshots, error logs, traces, a prod-vs-branch diff. Value: high for bugs that depend on the real environment, which is the class that bit us. Cost: plumbing, access control, and masking sensitive data before any agent sees it. Limit: a stale snapshot gives you false confidence.
• ▹4. A critic with a different model or context. Value: moderate. A different model family decorrelates some errors, and different context (a spec, a runbook, past incidents) brings real information. Cost: one more LLM call, plus the effort of building that context. Limit: it's still an opinion. Make it cite evidence, and don't let one model family generate, critique and grade.
• ▹5. Same-model 'think again'. Value: low for correctness. It can help with tone, format and completeness when output is judged by preference. Cost: a full extra LLM call, and its latency, on every run. Limit: correlated errors, plus it makes wrong answers sound verified.

The rule for Monday: no new evidence, no reflection step

Go through every reflect, critique or review step in your pipeline and ask three questions:

• ▹What new information does this step see that the first pass didn't? Name the artifact: a test result, a linter finding, a query result, a different spec. 'Its own previous output' doesn't count.
• ▹Can it block? Is a fail verdict wired to something that actually stops the change, like a required check, a merge gate or a halted plan? Or is it just advice?
• ▹Is it the same model with the same context as the generator?
• ▹If you got 'none', 'no' and 'yes', delete the step. Put those tokens and that latency into a cheap verifier from tiers 1-3.
• ▹Keeping an LLM critic? Then every blocking claim has to cite a specific piece of evidence. Objections without a citation are advisory only.
• ▹Give the generator the raw failure output (stderr, the failing assertion), not a summary like 'there's an issue near line 40'. In 2601.00828, bare location hints hurt every model tested.
• ▹Cap retries. A verifier loop with no limit just burns tokens on a model that can't fix the problem.

So when is a second pass with no new evidence fine? When you want polish, not correctness: tone, formatting, checking that a summary covers every section. Call it polish, and never let it output the word 'verified'.

How we rewired the migration pipeline

We deleted the 'review your migration carefully' pass. In its place we put evidence first, then a critic with different context and the power to veto:

def review_migration(sql: str, snapshot: Snapshot) -> Verdict:
    # Tier 2 + 3: run it, don't just read it
    lint = run_squawk(sql)
    dry = dry_run(sql, db=snapshot.clone())  # disposable clone; CONCURRENTLY can't run in a txn
    if lint.blocking or dry.failed:
        return Verdict(block=True, evidence=[*lint.findings, dry.stderr])

    # Tier 4: critic only sees things the generator never saw
    critique = critic.review(
        migration=sql,
        dry_run_log=dry.stdout,
        table_stats=snapshot.stats(dry.touched_tables),  # row counts, null rates, dup checks
        incidents=incident_log.related(dry.touched_tables),
    )
    blocking = [i for i in critique.issues if i.blocking and i.cites_evidence]
    return Verdict(block=bool(blocking), evidence=[i.citation for i in blocking])

# CI: Verdict.block -> required status check fails -> merge is impossible

Here's what changed, qualitatively:

• ▹The snapshot is masked, but the masking keeps the properties that matter. Emails are lowercased and then hashed, so case-variant duplicates still collide. The agent never sees a real address, and the duplicate-key error still reproduces exactly as it did on staging.
• ▹Failures now surface in CI, before a human opens the PR, instead of on staging. The generator gets the actual Postgres error. On the next attempt it usually adds a dedup data migration before the index, which is what a human would have written.
• ▹Critic output got shorter and more concrete. Blocking claims have to cite the dry-run log or the table stats, so the generic 'consider edge cases' remarks disappeared.
• ▹The cost moved. It used to be an extra LLM call on every migration. Now it's restoring and refreshing a snapshot, which is infrastructure work we can see and reason about.
• ▹We took the word 'verified' out of every LLM-produced message. Only a passing check gets to say that.

Common pitfalls

• ▹A critic that only sees the diff. If the bug lives in data, config or runtime behaviour, a diff-only critic can't see it, by design.
• ▹One model family generating, critiquing and grading (LLM-as-judge). Your eval ends up with the very blind spots it's supposed to measure.
• ▹Advisory critics. If a fail verdict doesn't stop anything, the pipeline will route around it the first time a deadline gets tight.
• ▹Stale or unrepresentative snapshots. Dry-run on an empty schema and you'll catch syntax errors and miss everything that depends on the data.
• ▹Flaky verifiers. A test that fails at random teaches the team to override the gate. After that, the gate has no authority.

Next: Day 21

If critics only help when they have different information, the next question is how that information moves between agents. Who produces the evidence? Where does it live? How does another agent check it's real and not just one more confident claim? Tomorrow covers shared blackboards and verification protocols, the plumbing that lets agents trade evidence instead of opinions.

Extend your knowledge

• ▹Read Huang et al., 'Large Language Models Cannot Self-Correct Reasoning Yet' (ICLR 2024), together with Gou et al., 'CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing'. Side by side, they show the no-signal and with-signal conditions.
• ▹Kamoi et al., 'When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs' (TACL 2024) is a careful taxonomy of which self-correction claims hold up.
• ▹Exercise: list every reflect, critique or review step in your current pipeline and run the three-question audit on each. For every one that comes out 'none / no / yes', name the cheapest tier 1-3 verifier that could replace it.
• ▹If you ship Postgres migrations, put Squawk in CI and add a dry-run against a masked snapshot. Then go back through past incidents and check which ones each would have caught.