Sagas for LLM agents

One refund request. By the end of the run there were two charges on the card, a ticket nobody closed, and an apology for a charge the customer never saw. It started when the support agent asked Jira to create a refund ticket for order 1182. Jira returned a 504. Behind the gateway, the ticket got created anyway.

The model sees an error and does what it was trained to do: try again. This time the summary says 'Order #1182 refund' instead of 'Refund for order 1182'. Our harness deduplicated retries by hashing the arguments. Different words give a different hash, so the call goes straight through. Now there are two tickets in the queue for one refund.

Next the agent charges a re-shipping fee through Stripe. Stripe succeeds. Then the worker pod gets evicted before the tool result is written back. On resume, the harness feeds the transcript back to the model, and the last thing that transcript knows is that the agent decided to charge the card. So it charges the card again. The ticket call gets replayed too. This time the argument-hash cache does catch it and hands back the result from the second ticket. The transcript now shows three successful create_ticket calls.

Two steps later the warehouse API rejects the reship, and the harness rolls back by walking the transcript. It closes the second ticket. Then it 'closes' the third, which never existed. It was only a cache echo of the second. The first ticket, the one behind the 504, stays open. The unwind refunds one charge, since the transcript only knows about one, and sends the customer a pre-written apology: 'We charged you a re-shipping fee by mistake, and it has been refunded.' Stripe receipts were turned off. The customer never got a receipt for either charge and hadn't noticed them. What they have now is an apology for something they didn't notice, and a second charge still sitting on their statement.

The model didn't hallucinate anything. Every step looked reasonable from inside the transcript.

Here's what I'll argue for the rest of this post. In a multi-step agent run, the step that errors isn't the hard failure. The hard one is the step that timed out and might have happened. You need three things in place before you write any clever undo logic. Write an intent to a durable log before every side-effecting call. Derive the idempotency key from the run ID and step index, never from the model's arguments. And check the real systems before you compensate anything. The LLM transcript is not your log, and replaying it is how side effects get doubled.

The three outcomes nobody draws

Every tool-calling tutorial draws two arrows out of a call: success and error. Production has a third one: unknown. A 504 means the gateway stopped waiting. It tells you nothing about whether the service behind it finished. A connection reset after the request body went out tells you the same nothing. So does a process that dies between getting a response and writing it down. In all of these the side effect may or may not exist, and nothing you have locally can tell you which.

Ordinary services deal with this too, but their retry logic is boring, and boring helps. A payment client retries the same request, with the same body, a fixed number of times. LLM agents make the unknown case worse in three ways:

• ▹They retry eagerly. To a model, an error string in the context is an invitation to be helpful. So it tries again, often in the same turn and sometimes more than once.
• ▹They rephrase between retries. The arguments get regenerated every time, so 'Refund for order 1182' turns into 'Order #1182 refund', and any dedup built on hashing the arguments sees two different requests.
• ▹They read silence as success. Give a model a 204, an empty body or a proxy's blank page, and it'll usually write 'Ticket created!' and carry on. The unknown is now recorded as confirmed, in the one place you were planning to trust.

You could live with any one of these. Put them together and every timeout becomes a coin flip, and the agent then builds the rest of the run on whichever way it landed.

Step 1: sort your tools before you build anything

Before you design logs or keys, put every tool into one of four classes. The class decides how much ceremony a call needs. These four classes are the only taxonomy I'll use for the rest of the post.

• ▹Pure: search_orders, get_customer, lookup_policy. No side effects, so retry as often as you like. There's nothing to undo, so they skip the saga log. If you want an audit trail, trace them somewhere else.
• ▹Reversible: create_email_draft, create_ticket_draft, add_internal_note on a draft. A delete undoes it completely, and nobody outside the run has seen it.
• ▹Compensatable: create_ticket (the support team sees it in their queue), charge_card (it lands on the customer's statement). You can't undo these. What you can do is issue a counter-action, like closing the ticket with a reason or refunding the charge, and the original stays in the record.
• ▹Irreversible: send_email, send_sms, a post to a customer-facing Slack channel. Once someone has read it, all you can do is send another message.

The one teams get wrong most often is charge_card. A refund is not an undo. The customer may see both the charge and the refund on their statement. Processors often keep the original processing fee when you refund (Stripe does). And your finance team has to reconcile both lines. Register it as compensatable, not reversible, and the rest of the design follows from there.

Step 2: log the intent, not just the call

Databases solved this a long time ago with the write-ahead log: write down what you're about to do somewhere durable, then do it. The transactional outbox pattern brings a similar idea to services. Your harness needs the same thing. This is the smallest table I'd ship:

CREATE TABLE saga_log (
  run_id           TEXT        NOT NULL,
  step_idx         INT         NOT NULL,
  tool             TEXT        NOT NULL,
  tool_class       TEXT        NOT NULL,  -- pure | reversible | compensatable | irreversible
  idempotency_key  TEXT        NOT NULL UNIQUE,
  args             JSONB       NOT NULL,
  status           TEXT        NOT NULL,  -- INTENDED -> CONFIRMED | FAILED | UNKNOWN
                                          -- later: NOT_EXECUTED | COMPENSATED
  external_ref     TEXT,                  -- ticket id, payment intent id, message id
  compensation     JSONB,                 -- {"tool": "refund", "args": {...}}, written before the call
  audience         TEXT[],                -- who could have seen this: {agent}, {support_team}, {customer}
  created_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
  updated_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (run_id, step_idx)
);

There's one rule: no INTENDED row, no tool call. Commit the row, then make the call, then move the row to CONFIRMED, FAILED or UNKNOWN. If you crash between any two of those, you're left with a row that tells you exactly what you don't know. That's the information the transcript lost in the incident above.

def dispatch(run, step_idx, call):
    spec = REGISTRY[call.tool]
    if spec.cls == 'pure':
        return spec.fn(**call.args)

    row = log.get(run.id, step_idx)
    if row and row.status == 'CONFIRMED':
        return row.result                      # already done, hand back the real result
    if row and row.status in ('INTENDED', 'UNKNOWN'):
        row = reconcile(row)                   # Step 6: ask the real system first
        if row.status == 'CONFIRMED':
            return row.result

    key = step_key(run.id, step_idx, call.tool)
    log.insert(run.id, step_idx, call.tool, spec.cls, key, call.args,
               status='INTENDED',
               compensation=require_compensation(spec, call, key),  # Step 4
               audience=spec.audience(call.args))
    try:
        result = spec.fn(**call.args, idempotency_key=key)
    except (Timeout, ConnectionReset, GatewayError):
        log.update(run.id, step_idx, status='UNKNOWN')
        raise ToolOutcomeUnknown(call.tool)    # the model sees 'unknown', not 'error'
    except ClientError as e:                   # 4xx: the server said no, nothing happened
        log.update(run.id, step_idx, status='FAILED', error=str(e))
        raise
    log.update(run.id, step_idx, status='CONFIRMED',
               external_ref=result.id, result=result)
    return result

Notice what the model gets on a timeout: ToolOutcomeUnknown, not a generic error. In the system prompt I tell the model that an unknown outcome means it must not retry with new arguments, because the harness will resolve it. The prompt isn't what makes this safe. The harness enforces the rule either way. What the prompt buys you is less of the model arguing with itself in the transcript.

Step 3: build keys from the run, not the model

Fixing the reworded retry takes one line:

def step_key(run_id: str, step_idx: int, tool: str) -> str:
    return hashlib.sha256(f'{run_id}:{step_idx}:{tool}'.encode()).hexdigest()[:32]

Hash the arguments, and 'Refund for order 1182' and 'Order #1182 refund' get two keys, so you get two tickets. Hash the run and the step instead, and both phrasings land on the same key, because they're the same intent at the same point in the plan. The model owns the wording. The harness owns identity.

There's a catch. This only works if the harness assigns step_idx, not the model, and if step_idx doesn't advance while the current row is UNKNOWN. A retry of an unresolved step is still the same step. If the model really wants to do something different, it has to wait until the harness has worked out what already happened.

Then the key has to actually reach the other side:

• ▹Stripe supports keys natively. Send it as the Idempotency-Key header and a repeat with the same key gets the original result back instead of a new charge. If the parameters differ under the same key, Stripe returns an error rather than quietly creating a second charge, so a reworded retry fails loudly. Stripe can prune keys once they're at least 24 hours old, so write the key into the PaymentIntent's metadata as well. Then a resume the next day can still find the charge through the Search API. Stripe's search isn't real-time either (new objects can take up to about a minute to show up), so the re-check advice below applies here too.
• ▹Most ticketing systems have no idempotency header at all. With Jira or Linear, put the key in a label (saga-3f9a...) or a custom field, and search for it before you create. Zendesk tickets have an external_id field that does the job. Search indexes can lag a little behind writes, so don't fully trust a lookup that comes back empty straight after a timeout. I re-check once after a short pause before I decide a ticket doesn't exist.
• ▹Email and messaging APIs differ a lot here. Some accept an idempotency key on send. Plenty don't. If yours doesn't, put the key in a custom header or your own message metadata so the reconciler can find it in your send logs.

Step 4: write the compensation before the call

Compensation logic written in the middle of an incident is how you end up closing ghost tickets. Write it before the side effect, while you still know exactly what you're about to do, and store it in the row next to the intent.

The compensation points at the idempotency key, not the external ID, because the ID doesn't exist yet. That's why it says refund(charge_key) and not refund(pi_...). The reconciler turns the key into a real ID later.

@tool(cls='compensatable', audience=lambda a: ['support_team'],
      compensate=lambda args, key: {'tool': 'close_ticket',
                                    'args': {'by_key': key, 'reason': 'rolled back by run'}})
def create_ticket(summary, order_id, idempotency_key): ...

@tool(cls='compensatable', audience=lambda a: ['customer'] if a.get('receipt') else ['finance'],
      compensate=lambda args, key: {'tool': 'refund',
                                    'args': {'charge_key': key, 'amount': args['amount']}})
def charge_card(customer_id, amount, reason, idempotency_key, receipt=False): ...

@tool(cls='irreversible', audience=lambda a: ['customer'],
      compensate=None)   # no automatic undo exists
def send_email(to, template, vars, idempotency_key): ...

def require_compensation(spec, call, key):
    if spec.cls in ('pure', 'reversible'):
        return None
    comp = spec.compensate(call.args, key) if spec.compensate else None
    if comp is None or spec.cls == 'irreversible':
        # no pre-written undo: a human approves before the call, not after
        approval.require(call, reason='no compensation available')
    return comp

Check twice. At registration time, refuse to register a compensatable tool that has no compensate function, so it gets caught in code review. At dispatch time, refuse the call if the compensation can't be built for these particular arguments, say because the refund amount can't be computed. For irreversible tools the only compensation is a correction message: a template plus a recipient list, written now. If the agent can't fill that in, the step goes to a human.

Step 5: order the plan around a pivot

In saga terms, the pivot is the step after which you stop rolling back and only roll forward. For an agent, design it so everything before the pivot is pure or reversible, and everything that touches the outside world happens in one late commit.

Here's the refund-and-reship run, reordered:

• ▹Search the order and the policy (pure).
• ▹Draft the customer email (reversible: delete the draft).
• ▹Create the ticket in a draft or staging state that isn't in the team queue yet (reversible).
• ▹Create the Stripe PaymentIntent with capture_method set to manual, so you authorise but don't capture. Cancelling an authorisation releases a pending hold, so the statement never shows a charge-and-refund pair. Authorisations do expire, typically after about seven days for online card payments, so the pivot can't sit around for long.
• ▹Pivot: one human approval, showing everything staged above.
• ▹Commit: capture the payment, move the ticket to open, send the email. Each of these still goes through the intent log with its own key.

Putting approval at the pivot instead of on every step is what makes human-in-the-loop bearable. The reviewer sees one complete proposal, not six dialogs they'll soon be clicking through without reading. Track one number per run: irreversible steps executed before the pivot. It should be zero. When it isn't, the usual cause is that the model reordered the plan, and you want your evals to catch that before a customer does.

Step 6: reconcile, then compensate

When a run fails or resumes, walk the log backwards. For any row that isn't settled, ask the real system first and decide after. Compensations get their own idempotency keys, because an unwind can crash halfway just like the run did.

def reconcile(row):
    found = REGISTRY[row.tool].lookup(row.idempotency_key)  # Stripe search, JQL label, external_id
    if found is None:
        found = retry_after(2.0, REGISTRY[row.tool].lookup, row.idempotency_key)
    if found:
        return log.update(row.run_id, row.step_idx, status='CONFIRMED',
                          external_ref=found.id, result=found)
    return log.update(row.run_id, row.step_idx, status='NOT_EXECUTED')

def unwind(run_id):
    for row in log.rows(run_id, order='step_idx DESC'):
        if row.status in ('INTENDED', 'UNKNOWN'):
            row = reconcile(row)
        if row.status != 'CONFIRMED' or row.tool_class == 'pure':
            continue
        if row.tool_class == 'irreversible':
            corrections.queue(row)       # see next section, never automatic
            continue
        comp = row.compensation
        comp_key = step_key(run_id, row.step_idx, 'compensate:' + row.tool)
        REGISTRY[comp['tool']].fn(**resolve_refs(comp['args']), idempotency_key=comp_key)
        log.update(run_id, row.step_idx, status='COMPENSATED')

Now run the opening incident again with this loop in place.

• ▹Step 2, create_ticket: the INTENDED row goes in with key k2 and the label saga-k2. Jira returns a 504, and the row moves to UNKNOWN.
• ▹The model retries with 'Order #1182 refund'. Same step_idx and same tool, so it gets the same key k2. The harness sees UNKNOWN and reconciles before calling anything. JQL finds the label saga-k2 on SUP-4470, the row becomes CONFIRMED, and the model gets SUP-4470 back as its result. One ticket.
• ▹Step 3, charge_card: the INTENDED row goes in with key k3, and Stripe succeeds. The pod is evicted before the row reaches CONFIRMED.
• ▹On resume, the harness reads the log, not the transcript. Row 3 is INTENDED, so it reconciles. A Stripe search on metadata saga_key k3 finds the PaymentIntent, and the row becomes CONFIRMED. Nothing gets charged again. Even if something did resend the call within 24 hours, Stripe would return the original result for k3.
• ▹The warehouse rejects the reship, so the run unwinds. Row 3 is refunded once, under its own compensation key. Row 2 closes SUP-4470, the only ticket that exists. Step 1 was pure and never got a log row, so there's nothing to undo there. No ticket gets closed twice and none is left open.
• ▹Receipts were off, so the charge row's audience was ['finance']. The customer gets no apology for a charge they never saw.

Nothing in this loop ever reads the transcript. The transcript is still useful for debugging why the model did something. It's no use for deciding what actually happened.

The part you can't undo

Emails, SMS and chat messages change what people know, and nothing takes that back. 'Compensating' one means sending another message. That message is a new side effect with its own audience, and it can make things worse.

This is what the audience field is for. When an unwind has several things to correct, rank them by blast radius. Internal-only effects get compensated first, silently: tickets only the support team saw, notes on the CRM record. Customer-facing corrections go last. Each one gets a human look, and only happens if the customer actually saw the original. A refund with receipts turned off needs no letter. A refund where a receipt went out might need a one-line note. A wrong email to the customer needs a person to decide what to say.

Some steps should be gated rather than compensated, because the apology costs more than the failure. Bulk sends are the obvious case. You don't want 'please ignore our previous email' going to four thousand people because the send to three of them failed. The same goes for messages to anyone with legal or contractual weight, and for anything posted in a public channel. For these, approval at the pivot is your compensation strategy. Get it right before it goes out, because you can't recall it afterwards.

Checklist for your harness README

• ▹Every tool is registered with a class: pure, reversible, compensatable or irreversible. Registration fails without one.
• ▹An INTENDED row is committed to the saga log before every non-pure tool call. No row, no call.
• ▹Idempotency keys come from hash(run_id, step_idx, tool), never from the model's arguments, and step_idx doesn't advance while a row is UNKNOWN.
• ▹The key reaches the external system as a native header where that's supported (Stripe), and also as a label, custom field, external_id or metadata so you can find it later.
• ▹Every compensatable step has a pre-written compensation stored in its row. Irreversible steps, and steps with no compensation, go to human approval before the call.
• ▹Irreversible steps sit after a single pivot, and you track per run how many irreversible steps executed before the pivot.
• ▹Before any compensation runs, the reconciler checks every INTENDED or UNKNOWN row against the real system by key.
• ▹Compensations carry their own idempotency keys.
• ▹Customer-facing corrections are ranked by audience and gated on a human.
• ▹The transcript is never the source of truth for what happened. Resume reads the log, not the chat.