Failure handling in agent fleets

Day 30: Failure Handling in Agent Fleets: Containment, Not Recovery

The most expensive failure in your agent fleet won't throw an error. It comes back as a confident, well-formatted answer, and every agent downstream treats it as fact. If your failure handling only catches crashes, it guards against the cheap failures and lets the expensive ones straight through.

The all-green run

Here's a real run from our PhoenixDX fleet. It had the standard shape: a planner, a schema agent, and two coder agents that open PRs. The task was to add tier-based discounts to the order service. Step by step:

• The planner writes a plan that says: 'Use the existing `customer_tier` column on `customers` to select the discount rule.' There is no such column. The planner made it up, most likely because a customers table usually has something like it.
• The schema agent exists to confirm schema assumptions. It reads the shared plan, sees `customer_tier` mentioned, and reports 'confirmed'. It never touched the database. It checked the planner's claim against the planner's own output.
• Coder agent A builds the discount resolver against `customer_tier`. Coder agent B writes the migration-free API change plus tests, with fixtures that include `customer_tier`. The tests pass. Of course they do, because the fixtures and the code agree with each other.
• The orchestrator reports 0 errors and 4 PRs, all green. About 40 minutes after the run started, a human reviewer opens the first PR, checks the real schema and stops everything.

Nothing crashed and nothing timed out. No retries fired, and the circuit breakers never moved. Our failure handling did its job perfectly. It just wasn't looking for this kind of failure.

The wrong mental model: we copied SRE

Most of us built agent reliability from the playbook we already had: crash, restart, retry with backoff, add a timeout, add a circuit breaker. They're good patterns, but every one of them assumes failure is loud. The service throws, the socket hangs, or the status code starts with a 5. Agents don't fail like that.

| Dimension            | How services fail                  | How agents fail                              |
|----------------------|------------------------------------|----------------------------------------------|
| Signal               | Exception, timeout, 5xx            | 200 OK                                       |
| Shape of output      | Missing or malformed               | Plausible, well-formatted, schema-valid      |
| Detectable by        | The caller, automatically          | Only by checking against reality             |
| Retry behavior       | Same request often succeeds        | Same context often fails the same way        |
| Spread               | Bounded by the call graph          | Unbounded: wrong claims enter shared context |
| What 'healthy' means | Up and responding                  | Up, responding, and wrong                    |

Microservice patterns protect availability. The main risk in an agent fleet is correctness. An agent that is available, fast and wrong does more damage than one that is down.

Reframe: failure as contagion, not downtime

On the shared memory day we put agents on a common blackboard so they could build on each other's work. That same blackboard is how a wrong claim spreads. Once the planner's invented column landed there, it was no longer just the planner's mistake. It was something every agent 'knew'. Each consumer stacked its own work on top, which made the error harder to see and more expensive to remove.

So ask a different question. Instead of 'is the agent up?', ask 'how far can a wrong claim travel before something checks it?' The metric for that is blast radius per handoff: how many agents, artifacts and minutes consume an unchecked claim before a verification step catches it. In the all-green run, the blast radius was 3 downstream agents, 4 PRs and about 40 minutes.

Containment means placing checks so this graph stops at the first edge. Each of the three controls below handles part of that.

Control 1: prove-it handoffs

At every boundary between agents, the producer attaches evidence the consumer can check cheaply. A claim without evidence is just an assertion. Good evidence points outside the conversation, at the database, the test runner or the file system. Another agent's text doesn't count. That's exactly where our schema agent went wrong: its 'evidence' was the planner's plan, which is circular.

• Claim 'the column exists' → evidence: the information_schema query and the rows it returned.
• Claim 'the tests pass' → evidence: the command, the exit code, and a hash of the test output that the consumer can re-run or look up.
• Claim 'I read the config' → evidence: file path, line range and the content hash at read time.
• Claim 'the API returns X' → evidence: the recorded request/response pair itself, not a summary of it.

from dataclasses import dataclass
from typing import Literal, Optional

@dataclass
class Evidence:
    kind: Literal['query_result', 'test_run', 'file_read', 'http_exchange']
    source: str          # e.g. the SQL, the test command, path:line
    payload_hash: str    # lets the consumer re-check cheaply

@dataclass
class Claim:
    statement: str
    evidence: Optional[Evidence]
    status: Literal['verified', 'unverified', 'refuted'] = 'unverified'

def receive(claim: Claim, verifier) -> Claim:
    if claim.evidence is None:
        return claim  # stays 'unverified'
    if claim.evidence.source.startswith('agent:'):
        return claim  # another agent's text is not evidence
    claim.status = 'verified' if verifier.check(claim.evidence) else 'refuted'
    return claim

def can_start(task) -> bool:
    # Unverified claims may inform work. They may not be its only basis.
    return all(c.status == 'verified' for c in task.load_bearing_claims())

The rule is simple. A claim without evidence is labelled 'unverified', and an unverified claim can never be the only basis for downstream work. A refuted claim is the loud failure you were missing. It fires at the first boundary, where it costs the least.

Pitfalls: don't demand evidence for everything. Only load-bearing claims need it, meaning the ones a later agent will build on. Keep the checks cheap: a query, a hash, a file read. If verifying costs as much as redoing the work, you haven't built a contract. You've built a second fleet.

Control 2: retries that change something

Running the same agent again on the same context with the same prompt isn't a retry. It's another vote. For a transient infra error, that's fine. For a reasoning error, you're sampling the same distribution again, and it usually lands in the same place. When it lands somewhere else, you have a coin flip you can't explain. A real retry changes the conditions.

• Retry: only for transient or infra failures, such as rate limits, timeouts or a dropped tool connection. Same context, with backoff. This is the one place the SRE playbook carries over directly.
• Reframe: change what the agent sees. Inject the ground truth it was missing (the actual schema, the failing test output), or shrink the task down to the part that failed.
• Reroute: change who does the work. That could be a different model, a specialist agent, or a deterministic tool in place of an LLM.
• Escalate: stop and hand a human a failure packet (more on that below).

def next_step(failure, attempts):
    if failure.kind in TRANSIENT:
        return Retry(context=attempts[-1].context, backoff=True)

    # Signal rule: agreement between different attempts is information.
    if len(attempts) >= 2 and attempts[-1].prompt != attempts[-2].prompt \
       and failure.signature == attempts[-2].failure.signature:
        return Escalate(build_failure_packet(attempts))

    if not any(a.reframed for a in attempts):
        return Reframe(add_ground_truth=True, shrink_scope=True)
    if not any(a.rerouted for a in attempts):
        return Reroute(to=alternate_agent(failure))
    return Escalate(build_failure_packet(attempts))

The signal rule matters most. When two attempts with different prompts or different context fail the same way, that agreement is telling you something. The problem is probably in the task, the spec or the environment, not in the sampling. More attempts only burn tokens and delay the moment a human looks. Stop and escalate.

Control 3: make 'I'm stuck' cheap

This one is a confession from the EM side. Years before agents, I watched engineers hide blockers because status meetings punished red. People reported green right up to the day it turned into a fire. Then I found our orchestrator doing the same thing to agents. Our eval scored 'escalated' as a failed task, and our prompts said things like 'complete the task; do not give up.' We had built a status meeting that punished red, and then we wondered why the agents kept reporting green.

We changed two things. Abstention became a routed outcome with its own queue instead of a failure state. And the prompts now say plainly that stopping with a good failure packet counts as a correct completion. I don't have a clean before/after dataset I'd put my name on, so I'm not going to invent percentages. The direction was clear, though. Early escalations went up, often within the first couple of steps. Late disasters, where a wrong output reaches a PR or a reviewer, went down. The escalation queue got busier and the reviews got calmer. I'll take that trade every time.

Pitfall: cheap doesn't mean free. If escalating costs nothing, a lazy agent will escalate everything. The price of escalating is the failure packet. An agent only gets to stop by handing over something useful.

The failure packet: what an agent hands a human

Our escalations used to be 3,000-line transcript dumps: every tool call, every retry, every self-correction, and no conclusion. The human had to redo the agent's reasoning just to find the one fact that mattered. In every fleet, the human reviewer is the real final failure handler, and usually the least-designed part of the system. Design their input.

failure_packet:
  task: Add tier-based discount to order service
  tried:
    - Resolve discount rule via customers.customer_tier (from plan)
    - Reframed with live schema from information_schema
  ruled_out:
    - hypothesis: column exists under another name
      how: searched customers, accounts, loyalty_* for tier-like columns; none found
    - hypothesis: tier is computed elsewhere
      how: grep for 'tier' in services/ and db/migrations; no producer found
  evidence:
    - query: SELECT column_name FROM information_schema.columns WHERE table_name = 'customers'
      result_hash: 9f3a...e1
    - plan_claim: 'customers.customer_tier exists' (status: refuted)
  smallest_unblocking_question: >
    Where does a customer's tier come from today: a new column we should add,
    or the loyalty_points threshold in pricing-config.yaml?

The packet has four fields: what I tried, what I ruled out and how, what evidence I have, and the smallest question that unblocks me. The last field does most of the work. A human can answer a one-line question in 30 seconds. Nobody can answer a transcript.

Replay: the same run with containment on

Same planner, same plan, same invented `customer_tier` column. This time the claim is load-bearing and has no evidence, so it reaches the schema agent marked 'unverified'. The schema agent isn't allowed to cite the plan as evidence, so it runs the information_schema query, gets zero matching rows and marks the claim 'refuted'. That's a loud failure at the first boundary. The orchestrator doesn't re-run the planner on the same context. It reframes instead: the planner gets the actual `customers` schema and re-plans, and the new plan comes back with a claim that cites the query. No coder agent ever starts on the bad plan. Blast radius: 1 agent, about 2 minutes. (Suppose the reframed plan had also needed a tier source that doesn't exist. The second, differently prompted failure would have triggered the packet above, and a human would have answered one question instead of reviewing 4 PRs.)

Monday checklist

• Map every handoff boundary in your fleet and mark which ones carry checkable evidence for load-bearing claims.
• Audit your retry logic for same-context re-runs on non-transient failures.
• Find every place where 'escalated' counts as a failure, whether in metrics, evals or prompts.
• Ship a four-field failure-packet template and make it the only way an agent can stop.

Day 30 close

For 29 days we worked on making agents capable: planning, tool use, orchestration, shared memory, evaluation. The last skill is making them honest about when they aren't capable. A fleet you can trust still fails. What earns the trust is that its failures are loud, stay local and leave something useful behind.

Extend your knowledge

• Read 'Why Do Multi-Agent LLM Systems Fail?' (Cemri et al., 2025) and its MAST failure taxonomy. Map its inter-agent misalignment and task verification failures onto your own handoff boundaries.
• Re-read the 'Addressing Cascading Failures' chapter of Google's SRE book, then sort its techniques into the ones that carry over to agents (load shedding, retry budgets) and the ones that don't (assuming the same request succeeds).
• Read Richard Cook's short paper 'How Complex Systems Fail' with your fleet in mind. Pay close attention to the points on latent failures and hindsight bias in post-incident review.
• Exercise: take one real escalation transcript from your fleet and rewrite it by hand as a four-field failure packet. Then time how long a teammate needs to act on each version.