Guardrails on agent handoffs

Day 21: In a Multi-Agent Pipeline, Safety Belongs on the Handoffs

You can have every agent in a pipeline pass every guardrail and still ship the exact change those guardrails were written to stop. If your planner → coder → reviewer → deployer chain is only guarded inside each box, you've tested the parts and assumed the whole is safe. Today is about the gap between the boxes.

Hook: the incident log

This is a simplified version of a pipeline we ran, with the names changed. There were three agents. Each had the same hard rule in its system prompt, backed by a tool allowlist or a pre-action check: never touch prod config.

• ▹Template writer: edits config templates in templates/. It has no write access to config/prod/.
• ▹Renderer: fills in template variables and writes the rendered files into a build artifact. It only writes to build/.
• ▹Deployer: ships artifacts. Its guardrail blocks the deploy if the change set touches config/prod/.

09:12  template-writer  task: 'make DB pool size configurable'
09:12  template-writer  edit templates/app.yaml.tmpl: pool_max: 40 -> pool_max: {{ DB_POOL_MAX }}
09:12  template-writer  guard[no-prod-config]: touched config/prod/? NO  -> PASS
09:14  renderer         render templates/*.tmpl with env of runner 'ci-build-03'
09:14  renderer         DB_POOL_MAX resolved from env = 200
09:14  renderer         guard[no-prod-config]: wrote to config/prod/? NO (wrote build/) -> PASS
09:15  deployer         change set: templates/app.yaml.tmpl, build/artifact.tar
09:15  deployer         guard[no-prod-config]: change set touches config/prod/? NO -> PASS
09:15  deployer         status: prod config UNCHANGED. deploying.
09:16  prod             app.yaml live: pool_max 40 -> 200

Every check came back green. Prod config changed anyway. Before you scroll on, pick one: which agent broke the rule?

The answer is none of them, and that is the lesson

The template writer edited a template. Allowed. The renderer rendered into a build directory. Also allowed. The deployer checked its own definition of 'touches prod config', and the honest answer was no. Each agent did the right thing by its own rule, and the rule still got broken.

The failure mode fits in one line: a forbidden action got split into steps that were each allowed. Nobody owned the whole action, so nobody checked it.

Back in the orchestration days of this series, we gave each agent a role, a toolset and a clear output. We drew the arrows between them as plain plumbing. Those arrows had no job, and the bug lived there.

Why per-agent guardrails feel safe

Once you've shipped a single agent, per-agent guardrails are the obvious next move. They do have real advantages:

• ▹They're easy to test. Feed the agent a bad prompt, confirm it refuses, and you have a clean unit test.
• ▹They demo well. 'Watch, the deployer won't touch prod config.' And on stage, it won't.
• ▹Audit is simple: one rule per agent, one log line per check, one owner per agent.

PIPELINE GUARDRAILS                     status
--------------------------------------------------
template-writer  no-prod-config          PASS
template-writer  tool-allowlist          PASS
renderer         no-prod-config          PASS
renderer         no-network-egress       PASS
deployer         no-prod-config          PASS
deployer         approval-gate(prod)     PASS (auto: no config change)
--------------------------------------------------
6/6 green

What this dashboard really shows is six answers to six local questions. Not one row asks whether prod config changed. Look at the approval gate. It auto-passed because the deployer reported no config change. A human gate that depends on an upstream claim can't be more trustworthy than that claim. All that green just makes you feel safe.

The trust chain problem

Each agent takes whatever the upstream agent produced as fact. That isn't a weakness in the model. The pipeline is built to work that way: the deployer gets a change set and a status, and it reasons inside that frame. LLM agents are very good at reasoning consistently from the context they're given, and nothing gives them a reason to question it.

Put what the deployer saw next to what was actually true:

WHAT THE DEPLOYER SAW (its context)        WHAT WAS TRUE (artifact vs live prod)
-----------------------------------        -------------------------------------
change_set:                                 build/app.yaml (rendered)  vs  prod app.yaml
  - templates/app.yaml.tmpl                   db:
  - build/artifact.tar                          host: db.prod.internal     (same)
config/prod/* in change_set: false            -   pool_max: 40
renderer.status: 'ok'                         +   pool_max: 200
upstream summary:                               timeout_ms: 3000           (same)
  'refactor: make pool size configurable,
   no behavior change'                       config delta: 1 key, prod-impacting

The deployer's view was internally consistent, and it was wrong. 'No behavior change' came from the template writer's commit message, which was written before any value existed. That phrase travelled two hops untouched and ended up as the deployer's belief. In an LLM pipeline, trust rides on natural-language summaries from agent to agent, and nobody re-checks them on the way.

The research lens: safety is emergent

In a multi-agent system, safety is emergent. It comes from how the agents interact, and you won't find it inside any single one. Distributed systems engineers learned this long ago: correct components don't add up to a correct composition. Take a microservice saga. The order service validates its input. So do the payment service and the inventory service. Nobody validates the saga as a whole, so you can still charge a card for an item that was never reserved. Every service is correct, and the business invariant is still broken. An agent pipeline is a saga whose steps are non-deterministic and explain themselves in prose. The fix has the same shape too: put invariants and checks on the composition as well as on the components.

Whiteboard model: nodes and edges

Draw your pipeline as a graph, with agents as nodes and handoffs as edges. A node guardrail checks behaviour, meaning what this agent is allowed to do with its tools. An edge guardrail checks the payload: what's crossing this boundary, and whether its claims hold up. Here's the pipeline with the locks on the boxes:

And here it is with the locks on the arrows:

• ▹Node guards ask: 'Did this agent do something it isn't allowed to do?'
• ▹Edge guards ask a different question: 'Is this handover what it claims to be, and what will it actually do to the world?'
• ▹You need both. Node guards cap the damage any single agent can do, and edge guards catch the problems that only appear once the steps combine.

What a handoff contract actually looks like

A handoff contract is a typed, versioned agreement for one edge. It has four parts:

• ▹Typed output schema: the next agent consumes a structured payload, not free text. Prose summaries can ride along, but no decision should depend on them.
• ▹Declared side effects: what this output will change if it ships, chosen from a closed set like none | config | data | infra, with the specific keys where they matter.
• ▹Provenance: the upstream inputs it depends on, such as the template commit, where the variable values came from, and the runner environment. This is how you spot that a value came from a CI box and not from prod's secrets store.
• ▹Independent edge check: code at the boundary that recomputes the critical claim from ground truth rather than trusting the producer's word. Make it deterministic where you can, and never let the producing agent grade its own work.

Here's the contract for the renderer → deployer edge:

edge: renderer -> deployer
contract: RenderedArtifact
version: 3

schema:
  artifact_uri: string
  artifact_digest: sha256
  target_env: enum[staging, prod]

declared_side_effects:          # producer must fill this in, never defaults
  class: enum[none, config, data, infra]
  config_keys: list[string]     # required if class == config

provenance:
  template_commit: git_sha
  values_source: enum[prod_secrets, staging_secrets, runner_env]
  runner: string

edge_check:                     # runs outside both agents
  recompute: diff(render(artifact).config, live(target_env).config)
  pass_if: actual_keys is subset of declared_side_effects.config_keys
  block_if:
    - actual_keys not subset of declared
    - target_env == prod and values_source == runner_env
  on_block: route_to_human(with: diff, provenance)

def check_renderer_to_deployer(handoff: RenderedArtifact) -> EdgeVerdict:
    # Recompute the claim from ground truth. Do not read handoff.status or summaries.
    rendered = load_config(handoff.artifact_uri, digest=handoff.artifact_digest)
    live = fetch_live_config(handoff.target_env)
    actual = changed_keys(rendered, live)

    declared = set(handoff.declared_side_effects.config_keys or [])
    undeclared = actual - declared

    if handoff.target_env == 'prod' and handoff.provenance.values_source == 'runner_env':
        return EdgeVerdict.block('prod values resolved from runner env', actual, handoff.provenance)
    if undeclared:
        return EdgeVerdict.block('undeclared config delta', undeclared, handoff.provenance)
    return EdgeVerdict.allow(actual)

The line that matters is fetch_live_config. The check compares against what's running in prod right now. It ignores the last commit, the previous artifact and the producer's report. 'Unchanged' gets computed at the edge and is never taken on faith from upstream. Point this check at our incident and it sees pool_max: 40 → 200, finds no declared config key for it, and blocks the deploy.

The fix we shipped and what changed

The deployer no longer accepts 'unchanged' as an input. We took that word out of its contract. On the renderer → deployer edge, we diff the rendered artifact against live prod and block any config delta the contract didn't declare. When the template writer really does mean to change prod config, it declares that upstream, and the declaration flows through the contracts all the way to the approval gate. The human then approves a real diff, not someone's summary of one.

It wasn't free. Here's what it cost us:

• ▹Latency. Every deploy now fetches live prod config and diffs it. In a pipeline where most of the wall-clock time already goes to LLM calls that's small, but it's still a real round trip to prod on every run. It also needs read credentials we'd never given that stage before.
• ▹A week of false positives. The first version blocked on noise: key reordering, defaults the app fills in at runtime, timestamp fields. We had to normalize both sides before diffing. People were annoyed that week, and there was real pressure to add an 'override' flag. We kept the block and fixed the normalizer.
• ▹Contract upkeep. Every new artifact type needs a schema and a side-effect class. That work never stops, and it's exactly the job the edges were missing.

What we got back: the approval gate meant something again, because it now shows a recomputed diff instead of an upstream summary. And when we later added a fourth agent, the edges spelled out exactly what it was allowed to hand over.

Common pitfalls

• ▹Verifying the contract with an LLM that has the same context as the producer. If it reads the same summaries, it'll come to the same belief. Recompute from ground truth wherever you can.
• ▹Diffing against the last artifact instead of live state. Drift, manual hotfixes and env-sourced values only show up when you compare against what's actually running.
• ▹Letting 'declared_side_effects' default to none. With that default, every undeclared side effect passes as a declared non-effect. Make the field required.
• ▹Weighting every edge equally. Start with the edges right before irreversible actions (deploy, migrate, send, pay) and work backwards from there.
• ▹Over-building where it doesn't pay. In a read-only pipeline, say a research agent feeding a summarizer, a schema check is usually enough. Full recompute-at-the-edge earns its cost where a handoff leads to a side effect in the world.

Three edge checks to add to your pipeline this week

• ▹1. Declared vs. actual side effects at every handoff. Have each producer declare what its output touches (none / config / data / infra). At the edge, compute what it really touches and block on any mismatch.
• ▹2. Re-derive at least one critical claim at the edge rather than passing it through. Pick the claim your riskiest downstream decision leans on, like 'unchanged', 'tests passed' or 'no PII', and recompute it from source data at the boundary.
• ▹3. One whole-pipeline invariant, tested end to end. Write a single sentence that has to hold for the whole composition, for example 'prod config only changes through a declared, approved contract'. Test it by running the full pipeline against staging, including adversarial tasks that try to split a forbidden action into allowed steps. Testing agent by agent won't catch this.

Bridge to Day 22: who checks the checker?

Once your edges carry contracts, the next question is who verifies them. Some claims can be recomputed deterministically, like a config diff. Others can't: 'this refactor preserves behavior', or 'this plan matches the ticket'. For those, teams bring in verifier and critic agents, and those agents fail in their own ways. They share blind spots with the producer, they defer to confident upstream prose, and they get gamed once the producer learns what they check for. Day 22 covers verifier and critic agents and how to keep them independent.

Extend your knowledge

• ▹Read up on the Saga pattern (Chris Richardson's writing on microservices.io and his book 'Microservices Patterns') with your agent pipeline in mind. Map compensating actions and saga-level invariants onto your handoffs.
• ▹Reread Anthropic's 'Building effective agents' post and go through each workflow pattern (prompt chaining, orchestrator-workers, evaluator-optimizer). For every arrow, ask what crosses it and who checks it.
• ▹Try Open Policy Agent (OPA) / Rego for writing edge checks as policy code that runs outside your agents, so your verifier isn't an LLM sharing the producer's context.
• ▹Exercise: take your riskiest handoff and write its contract in the YAML shape above. Then try to push a forbidden outcome through it in three allowed steps. If you manage it, your edge check needs work.