Pipeline vs swarm

Day 24: Pipeline or swarm? Build a pipeline and contain one swarm

When a pipeline fails at 2am, you get a stack trace and a stage name. When a swarm fails at 2am, you get a transcript of agents politely agreeing with each other, and no name on the decision. That's what's really at stake in the 'pipeline or swarm' choice. It matters less for reasoning quality than most demos suggest. Most teams pick one shape for the whole system. In production you usually want both, with a hard wall between them.

Hook: the swarm that quarantined a real bug

I've seen this setup play out. A CI triage system runs three agents to decide whether a failing test is flaky: a Log Reader, a History Analyst and a Code Reviewer. They talk freely, then vote. If they agree it's flaky, the test is quarantined automatically. One night they agreed on a test that was actually catching a real regression in a refund retry path. The test went quiet and the bug shipped. This is what on-call found in the logs:

[run 7f3a] triage.flaky_test  test_refund_retry_idempotent
LogReader:    Failure is a timeout in the HTTP client. Pattern matches known infra noise.
HistoryAgent: Agree. This test has failed before and passed on retry.
CodeReviewer: Recent diff touches retry logic, but the change looks safe. Agree: flaky.
LogReader:    Consensus reached: FLAKY. Recommend quarantine.
-> action: quarantine(test_refund_retry_idempotent)   owner: <none>

There's no stack trace and no failing assertion. There's just a conversation, and every line in it sounds reasonable. The History Analyst's 'passed on retry' was about an older commit. The Code Reviewer saw the risky diff and still went along with the two agents before it. Re-run the triage and you get a different conversation, so you can't replay the decision. The owner field is empty because a consensus doesn't have an owner. That's what today's lesson is about.

The question this lesson answers

Earlier in the series you saw how agents talk to each other: messages, handoffs, shared state. Yesterday you saw how they fail. Today's question is about architecture: where in your system can agents talk freely, and where can't they? My answer is a default plus one exception. The default is a deterministic pipeline. The exception is a short-lived, bounded swarm at the one stage where the solution space really branches, and the only way out of it is through a contract.

Two failure shapes, side by side

| Aspect                 | Pipeline                                 | Swarm                                          |
|------------------------|------------------------------------------|------------------------------------------------|
| Where it fails         | At a named stage boundary                | Inside the consensus, between messages         |
| How it fails           | Loudly: schema error, exception, timeout | Quietly: a confident, agreed, wrong answer     |
| Error propagation      | Stops at the next validator              | Amplified: agents build on each other's claims |
| Responsibility         | Stage N owns it                          | Diffused: 'the agents decided'                 |
| Replay                 | Same inputs -> same path                 | Same inputs -> different conversation          |
| 2am artifact           | Stack trace + stage name                 | Transcript                                     |

This table doesn't compare capability. Swarms can find answers that pipelines miss. The point is that the two shapes break in different ways, and you'll be the one operating whichever shape you ship.

A 15-minute test: has your swarm already turned into a DAG?

Plenty of production 'swarms' stopped being swarms weeks ago. The agents are allowed to message anyone, but in practice they take the same few paths on every run. If that describes yours, you're paying for coordination you don't use: extra tokens, extra latency and harder replay, all for a fixed graph you could just write down. To check:

• ▹Log every agent-to-agent message for a week as JSONL with run_id, ts, from and to. Most frameworks can emit this from a callback or trace exporter.
• ▹Turn each run into a path signature, meaning the ordered list of from->to edges.
• ▹Count the distinct signatures. Then check how many runs the top few cover.
• ▹If a handful of paths cover nearly every run and the order stays stable, your swarm is a DAG. Write it down as a pipeline, and keep free-form talk only where the paths actually vary.

import json
from collections import Counter

runs = {}
with open('agent_messages.jsonl') as f:
    for line in f:
        m = json.loads(line)
        runs.setdefault(m['run_id'], []).append((m['ts'], m['from'], m['to']))

paths = Counter()
for msgs in runs.values():
    msgs.sort()
    paths[' > '.join(f'{a}->{b}' for _, a, b in msgs)] += 1

total, cum = sum(paths.values()), 0
for i, (sig, n) in enumerate(paths.most_common(10), 1):
    cum += n
    print(f'{i:>2}. {n:>5} runs  ({cum/total:.0%} cumulative)  {sig[:120]}')
print(f'distinct paths: {len(paths)} across {total} runs')

While you're looking at the output, check where the variation clusters. Usually the paths are identical up to one stage, spread out there, and come back together afterwards. That stage is your candidate. It's the only place where a swarm might be worth what it costs.

The pattern: a pipeline spine with one bounded swarm

• ▹Spine: deterministic stages with typed inputs and outputs. Every failure here has a stage name.
• ▹Branch point: the one place where the solution space actually fans out.
• ▹Swarm pod: agents talk freely, but within a budget and a round limit. The pod has a named owner.
• ▹Checkpoint: deterministic code, not an LLM judge. It accepts or rejects whatever the swarm produced.
• ▹Reject path: falls back to a safe default that doesn't depend on the swarm. The swarm can propose an action, but it never takes one on its own.

How to find the one stage that earns a swarm

Some tasks really do branch: root-causing, generating hypotheses, comparing design alternatives. Several plausible answers compete, and letting different perspectives argue can improve the result. How much it helps depends on the task, so measure it on yours. Other tasks only feel hard because they're long, the input is messy, or they need a lot of tool calls. Those need a better single agent or a better pipeline stage. Ask three questions of each stage:

• ▹1. Are there several plausible answers that need to compete? If one careful agent with the right context would reach the same answer every time, it isn't a branch point.
• ▹2. Does disagreement carry information? If agents arguing would surface evidence that a single pass misses, a swarm can help. If they'd just paraphrase each other, it won't.
• ▹3. Can a deterministic check judge the output? If you can't write code that rejects a bad answer at this stage, you're not ready for a swarm there, because nothing would contain it.

A stage needs a yes to all three. In most systems I've reviewed, one stage qualifies at most. If you count three, look again. You're probably counting stages that are just long.

The exit contract: a copyable checklist

The exit contract keeps the swarm's emergent behaviour inside the pod. Nothing leaves the pod unless it meets the contract.

• ▹Token and wall-clock budget: hard limits enforced by the orchestrator. Asking nicely in the prompt doesn't count.
• ▹Max rounds: a fixed cap on exchanges. If the cap is hit, the output is 'unknown', not a forced consensus.
• ▹Structured output schema: a typed verdict, not prose.
• ▹Evidence for each claim: every claim cites at least one artifact (log line, run ID, diff hunk, repro output).
• ▹Required dissent field: the strongest counter-argument. It fails if it's empty or says 'none'.
• ▹Named owner: a human or an accountable agent/team that gets paged when the checkpoint rejects.
• ▹Deterministic validator: code that can reject the result and send it to a safe default.

# Example contract (values are illustrative, tune for your system)
swarm: flaky_triage
owner: ci-reliability-oncall
budget:
  max_tokens: 60000
  max_wall_clock_s: 300
  max_rounds: 3
output_schema: TriageVerdict@v2
evidence: every claim cites >= 1 artifact id
dissent: required, non-empty, not 'none'
validator: checkpoints.triage_v2   # pure code, no LLM
on_reject: do_not_quarantine; page owner

from typing import Literal
from pydantic import BaseModel, Field

class Claim(BaseModel):
    statement: str
    evidence: list[str] = Field(min_length=1)   # artifact ids

class Repro(BaseModel):
    commit: str
    runs: int
    failures: int

class TriageVerdict(BaseModel):
    verdict: Literal['flaky', 'real_bug', 'unknown']
    claims: list[Claim] = Field(min_length=1)
    dissent: str = Field(min_length=1)
    repro: Repro | None = None
    rounds_used: int
    tokens_used: int

def checkpoint(v: TriageVerdict, head_commit: str, budget) -> tuple[bool, str]:
    if v.rounds_used > budget.max_rounds or v.tokens_used > budget.max_tokens:
        return False, 'budget_exceeded'
    if v.dissent.strip().lower() in {'none', 'n/a', 'no dissent'}:
        return False, 'empty_dissent'
    if v.verdict == 'flaky':
        if v.repro is None:
            return False, 'consensus_without_evidence'
        if v.repro.commit != head_commit:
            return False, 'repro_on_stale_commit'
        # flaky means it both passes AND fails on the same commit
        if not (0 < v.repro.failures < v.repro.runs):
            return False, 'not_nondeterministic'
    return True, 'ok'

The flaky-test system after the redesign

The agents and prompts didn't change. The structure around them did. Log normalization, history lookup and diff fetching became deterministic stages. The three agents still debate, but only inside the pod and only for up to three rounds. A 'flaky' verdict now has to include a reproduction attempt on the current commit that shows both passes and failures. The checkpoint rejects 'consensus without evidence'. When it rejects, the test isn't quarantined. It stays red, and the CI-reliability on-call gets paged.

• ▹The refund-retry case now fails at the checkpoint. The repro fails every time on HEAD, so 'not_nondeterministic' blocks the quarantine.
• ▹On-call gets a named seam: checkpoint triage_v2 rejected with reason X, owner Y. The transcript is still there, but now it's supporting evidence rather than the main artifact.
• ▹Replay gets simpler. You don't need to replay the conversation to understand the decision. You replay the checkpoint against the stored verdict, and that's deterministic.
• ▹The blast radius shrinks. At worst, the swarm uses up its budget and returns 'unknown'.

Where this pattern breaks

• ▹Open-ended research tasks. When the whole task is exploration, as with a deep-research agent, there's no single branch point to contain the swarm in. Anthropic's write-up of their multi-agent research system describes an orchestrator-worker shape with parallel subagents for this reason. You can still budget it, but the pipeline spine gets thin.
• ▹The branch point moves. If the hard stage changes from request to request (sometimes the plan, sometimes the code, sometimes the test), one pod in a fixed place won't fit. You'll be tempted to add pods everywhere, and then you've rebuilt the swarm you started with.
• ▹You can't write a deterministic validator. If every useful check on the output needs judgment, your checkpoint turns into an LLM judge and containment gets weaker. Be upfront about that, and keep a human in the loop.
• ▹You've outgrown it. You'll know because your checkpoint rejects a large share of pod outputs for reasons the swarm couldn't have avoided, or the path-count test shows variation across many stages. At that point you need a real dynamic orchestrator with its own observability. That's a different design. More swarm won't fix it.

Bridge to Day 25

Containing the swarm leaves you with a new problem: the pod is still a black box. The checkpoint tells you that the swarm failed, but not why. Tomorrow we look inside the pod: tracing each message as a span, linking claims to evidence, and replaying a conversation well enough to debug it.

Exercise

• ▹Draw your current agent graph, or run the path-count script and let real traffic draw it for you.
• ▹Mark the one stage that passes all three branch-point questions. If none does, that's a valid result, and you should ship a pipeline.
• ▹Write that stage's exit contract in five lines: budget and rounds, output schema, evidence and dissent rule, owner, and validator plus reject path.
• ▹Post it to your team or the series thread, and include one case your validator would reject.

Extend your knowledge

• ▹Anthropic, 'Building effective agents' (Dec 2024): explains the difference between workflows and agents, and why you should start with the simplest pattern that works.
• ▹Cognition (Walden Yan), 'Don't Build Multi-Agents': makes the case for context continuity over free-form multi-agent chatter.
• ▹Cemri et al., 'Why Do Multi-Agent LLM Systems Fail?' (2025): introduces the MAST failure taxonomy, which helps when you're deciding what your checkpoint should reject.
• ▹Anthropic, 'How we built our multi-agent research system': a production example of open-ended research that does justify parallel agents, including what it costs.