Trade-off analysis in ADRs

Day 30: Trade-off analysis is the part you can't hand to the agent

Your agent can write a flawless ADR in four minutes. You can approve it in two. And it's quite possible neither of you made a decision. Every artifact in this series can now be drafted on demand, so the document isn't the hard part anymore. The hard part is whether it contains a call that someone on your team will put their name to.

The hook: a flawless ADR with no decision in it

At PhoenixDX we needed an event pipeline. Product events had to land somewhere durable, and a couple of downstream jobs had to react to them. We had two realistic options: put a message queue in the middle, or write events straight to Postgres and have consumers poll a table. One of our engineers asked an agent to draft the ADR, and it came back clean. Context, options, a pros and cons table for each, a consequences section, and a recommendation for direct DB writes. The formatting was perfect. Nothing in it was wrong.

In review I asked one question: "If we'd picked the queue instead, which sentence in this document would become false?" Not one. The pros and cons were symmetric, so you could flip the recommendation without touching anything else in the doc. We had a nicely formatted summary of the option space, and no decision anywhere in it.

Name the failure: balanced-list syndrome

Balanced-list syndrome is a trade-off section where every pro and con would hold for any company, any team, in any year. "Kafka offers high throughput and replayability but adds operational complexity." True. It's also true at Shopify, at a three-person startup, and in 2016. A statement that's true everywhere tells you nothing about your situation.

Here's the test. Swap your company name for a competitor's. If the section still reads as true, you haven't analysed a trade-off. You've reproduced the vendor docs.

• Symptom 1: every option gets roughly the same number of bullets. Once your context is applied, real trade-offs are rarely that symmetric.
• Symptom 2: the recommendation lives in its own paragraph and doesn't follow from any line in the table.
• Symptom 3: no line mentions a person, a date, an incident, a budget, or a number specific to your system.
• Symptom 4: reviewers approve it without asking a single question. A real decision usually draws at least one objection.

Why agents produce this by default

An agent writing an ADR is optimising to be complete and defensible. Listing every consideration evenly is the safest thing it can do. No reviewer can say it missed something, and none of its claims can be shown wrong. Committing is the risky move, and committing well takes information the agent doesn't have.

The agent knows the general trade-offs very well. It doesn't know your constraints: who on the team has actually run the technology in production, what the hard deadline is, which incident from last quarter still makes everyone nervous, what's cheap for you to undo and what isn't. These are the quality attributes and constraints from the early days of the series. The model supplies the generic attributes. You supply their weights and the constraints. Leave those out of the prompt and the agent fills the gap with balance.

The rule: three lines an agent can't fill in on its own

After that review I added one rule to our ADR template. Every ADR must contain these three lines, written by a human who owns the decision:

• Rejected option: what we said no to, by name. "Alternatives were considered" doesn't count. "We are not using Kafka" does.
• Why it lost here: the specific local fact that decided it. It has to be about your team, system, deadline, or history, not a general property of the technology.
• Regret condition: an observable signal, with a threshold, that would tell us we chose wrong. A dashboard, a log query, or a calendar check should be able to detect it.

Each line forces something the balanced list gets to skip. The first forces a commitment, and the second forces context. The third makes the decision falsifiable. You write down in advance what would prove you wrong, so later you can check the decision against reality instead of arguing about it again.

Before and after: the event-pipeline ADR

Here's the agent's original trade-off section, condensed:

## Trade-offs

| Option               | Pros                                              | Cons                                              |
|----------------------|---------------------------------------------------|---------------------------------------------------|
| Kafka                | High throughput; replay; decoupled consumers;     | Operational complexity; learning curve;           |
|                      | mature ecosystem                                  | infra cost                                        |
| Managed queue (SQS)  | Low ops; scales automatically; simple API         | Limited replay; vendor lock-in; ordering caveats  |
| Direct DB writes     | Simple; transactional with app data; no new infra | Polling overhead; harder to scale; tight coupling |

**Recommendation:** Direct DB writes offer the best balance of simplicity
and capability for current needs, while leaving room to evolve.

Every cell is accurate. You could also paste this table into any company's wiki. "Best balance for current needs" never says which needs. Now the same ADR with the three lines added:

## Decision
Events are written to an `events` table in the existing Postgres cluster,
in the same transaction as the business write. Consumers poll by cursor.

## Trade-off (owner: Chi Cong, reviewed 2026-10)
- **Rejected:** Kafka.
- **Lost because:** two engineers will own this pipeline and neither has run
  Kafka in production. We already operate Postgres with on-call runbooks.
  The transactional write also removes the dual-write bug class we hit last
  quarter.
- **Regret if:** sustained > 5k events/sec for 15 min on the events table,
  OR a second consumer team asks for independent replay of the stream.

## Also rejected
- Managed queue: lost on the same dual-write problem; no team-specific
  advantage over Postgres at our current volume.

Now the reviewer has something to work with. They can challenge the local fact ("Actually, Minh ran Kafka at his last job"). They can challenge the threshold ("Our batch import already spikes past 5k"). They can check the regret condition against a dashboard today. The original gave them nothing to push on. This version makes three specific claims, and each one can be checked. That's what makes it a decision. It being longer has very little to do with it.

Note: the 5k events/sec figure is our own threshold. It's roughly where our current Postgres setup would start to strain and where polling lag would start hurting consumers. Yours will be different, and it should be. If you could have copied the number from a blog post, it's the wrong number.

How to split the work with the agent

You don't need to stop using the agent for ADRs. Let it do what it's good at, and keep the part only you can do.

• Agent: generate the option space. It often surfaces something you missed. Here, a transactional outbox with a relay was worth naming.
• Agent: draft the generic pros and cons for each option. It's fast, broad, and mostly right. Use it as a checklist. It isn't the analysis.
• Human: write the three lines (rejected option, local reason, regret condition). This is where you commit.
• Agent: stress-test those three lines. Give it your context and ask it to attack them. Don't ask it for praise.

These stress-test prompts work well for us:

Here is our ADR and our team context (attached).

1. Argue that the regret condition is ALREADY true today, using
   anything in the context or metrics I've provided.
2. Argue the strongest case for the rejected option, assuming our
   'lost because' fact changes in 6 months. What would change it?
3. Is the 'lost because' line a local fact, or a generic property of
   the technology dressed up as one? Quote the line and classify it.
4. Name one observable signal we should monitor that our regret
   condition does not cover.

Agents are good in the adversarial role because they have no stake in the decision. Point that same bias toward balance at your commitment, ask it to find holes, and the bias starts working for you.

Regret conditions become monitors

Most ADRs get reread during an incident, when someone asks why the thing was built this way. By then changing course is expensive. A regret condition with a threshold can become a monitor, so the ADR resurfaces when the condition first shows up, before anything breaks. This is the fitness function idea from earlier in the series, applied to a decision instead of a quality attribute.

Regret conditions based on metrics become alerts. Here's an illustrative Prometheus rule. Your metric names will differ:

groups:
  - name: adr-regret-conditions
    rules:
      - alert: ADR014_EventPipeline_RegretCondition
        expr: sum(rate(events_inserted_total[5m])) > 5000
        for: 15m
        labels:
          severity: decision-review   # routes to the ADR owner, not on-call
        annotations:
          summary: "ADR-014 regret condition met: sustained >5k events/sec"
          adr: "docs/adr/0014-event-pipeline.md"
          action: "Revisit Kafka rejection. Owner: Chi Cong"

Organisational regret conditions, like "a second consumer team appears", don't show up in any metric. Make those a calendar check with a named owner: a 10-minute quarterly review where the owner reads each open regret condition and answers yes or no. On an agent-heavy team you can automate the first pass. A scheduled agent reads the ADR folder, pulls the relevant metrics and the list of services consuming the events table, and opens a ticket when a condition looks close. A human still makes the call.

Send these to a decision-review channel, not the pager. When a regret condition fires, the system isn't down. The assumptions behind the decision have changed.

Common ways it goes wrong, and the one-line fix

• Vague regret condition ("if scale grows significantly"). Fix: give it a metric, a threshold, and a duration, or make it a yes/no question someone answers on a set date.
• Straw-man rejected option ("We rejected building our own message broker"). Fix: name the option a smart dissenter on your team would actually argue for.
• Generic 'lost because' ("Kafka is operationally complex"). Fix: rewrite it until it would be false at a competitor, e.g. "nobody here has run Kafka in production".
• Nobody owns the regret conditions. Fix: put a person's name next to each one, not a team's, and reassign it when that person leaves.
• The regret condition is already true. Fix: run the 'argue it's already true' prompt before you merge the ADR.

Course close: what's left when everything can be drafted

Think back over the 30 days: context diagrams, C4 views, quality attribute scenarios, NFRs, risk registers, ADRs, architecture reviews, fitness functions. A good agent can draft a reasonable first version of every one of them today. That's genuinely useful. It frees up hours a week, and the drafts are often more thorough than what a tired human writes at 6pm.

What it can't draft is a commitment grounded in your context: "We're saying no to this, because of this fact about us, and we'll know we were wrong when we see this." All the other architecture work leads up to that sentence, and trade-off analysis is where it gets written down. That's why this lesson comes last. Everything before it feeds into this one.

In an ASE team, this is also how you tell whether a solution architect is adding value. The value isn't in producing documents. It's in being willing to sign their name to a call that can be proven wrong.

Monday action

• Copy the block below into your ADR template, directly under Decision.
• Open your most recent agent-drafted ADR and try to fill it in. Leave everything else alone.
• Can't name the local fact or the threshold? Then the decision isn't finished. Book 30 minutes with the people who have that context.
• Before Friday, turn one regret condition into an alert or a calendar invite.

## Trade-off (owner: <person>, date: <yyyy-mm>)
- **Rejected:** <named option a smart dissenter would pick>
- **Lost because:** <fact about OUR team/system/deadline/history>
- **Regret if:** <metric + threshold + duration, or dated yes/no question>
- **Monitored by:** <alert name or calendar check + owner>

Extend your knowledge

• Michael Nygard, 'Documenting Architecture Decisions' (2011 blog post). The original lightweight ADR format. Look at how much weight he gives Context and Consequences, because that's where your local facts go.
• MADR (Markdown Any Decision Records). An open ADR template with explicit 'Considered Options' and 'Decision Outcome' sections. Add the three-line block and compare the result with what your agent produces.
• Neal Ford, Rebecca Parsons and Patrick Kua, 'Building Evolutionary Architectures' (O'Reilly). Covers fitness functions in depth. Use it to turn regret conditions into automated checks.
• Gary Klein's pre-mortem technique (Harvard Business Review, 'Performing a Project Premortem'). A structured way to imagine the decision has already failed, and a good place to find regret conditions.