Per-agent context tax

Day 29: The Per-Agent Context Tax, or Pricing Every Read Your Agents Make

Your warehouse bill has been flat for months. Your LLM bill goes up every month. That's one problem showing up on two invoices, and if you budget infra and tokens as separate lines you won't see it. When several agents share the same tables, the biggest cost in your data plane is no longer the warehouse. It's tokens. You can't fix a cost you haven't priced.

This lesson covers one idea. In an agent system, the real data cost is a context tax: every agent pays again, in tokens, to read, check and summarise the same shared data. We'll name it, model it and measure it. Then we'll remove most of it with an architecture change.

1. The hook: a flat bill next to a climbing one

I keep running into the same pattern, at PhoenixDX and in the teams I advise. The warehouse bill stays flat for months while the LLM bill climbs. Different people own the two bills, so nobody connects them. Then someone opens the trace for one scheduled run and finds 12 agents reading the same 5 tables. Each agent pulls rows, pastes them into its prompt, checks whether the numbers look right and writes its own summary. The result is 12 summaries of the same data, worded 12 ways. Sometimes they give 12 different answers to 'how much is overdue?'.

The warehouse did its job, and it did it cheaply. The money went on what happened after the query, where every agent turned raw data into context on its own.

2. Name the tax: four parts of one agent's data cost

Take one agent reading one data surface (a table, view, index or API you treat as a single source). That read breaks into four steps:

• ▹Fetch: the query or retrieval call. You pay in infra: warehouse compute, vector DB reads, egress. It's usually cheap, and it grows with data volume.
• ▹Serialise: rows or chunks become prompt text. You pay in input tokens. The cost grows with how much of the surface the agent pulls, and every agent pays it again on every run.
• ▹Validate: the agent checks the data. 'Are these totals plausible?' 'Is this column null?' 'Does this match last week?' You pay input and output tokens, plus retries when a check fails or the model second-guesses itself. Grows with agents × runs.
• ▹Summarise: the agent compresses the surface into something it can reason with. You pay in output tokens, which usually cost several times more than input. This grows with agents × runs too, and each agent comes away with a slightly different interpretation.

Look at how each step scales. Only fetch scales mainly with data. Serialise, validate and summarise scale with agents × runs × surfaces touched. Your storage line tracks how much data you hold. Your context tax tracks how many agents look at that data and how often. In agent-heavy teams, the second number is the one that keeps growing.

3. The cost model you can paste into a spreadsheet

monthly_data_plane_cost ≈ storage + compute + egress
                        + Σ_agents ( runs_per_month
                                     × surfaces_read_per_run
                                     × tokens_per_surface_read
                                     × blended_price_per_token
                                     × retry_factor )

tokens_per_surface_read = serialise_in + validate_in + validate_out + summarise_out
(price input and output tokens separately; output is usually several times dearer)

A worked example. The prices are illustrative assumptions: $3 per million input tokens and $15 per million output tokens. Use your provider's current rates instead.

• ▹Warehouse (storage + compute + egress): about $1,800/month, roughly flat.
• ▹Agents: 12, each running 30 times a day, which is 900 runs per agent per month.
• ▹Each run reads 3 of the 5 shared surfaces.
• ▹Per surface read: 6,000 tokens serialised in, 2,000 tokens of validation input, 400 validation output, 800 summary output. That's 8,000 in (= $0.024) and 1,200 out (= $0.018), so about $0.042 per surface read.
• ▹Retry factor: 1.15, covering failed checks, re-reads and self-corrections.
• ▹Per agent per month: 900 × 3 × $0.042 × 1.15 ≈ $130.
• ▹Across 12 agents: about $1,565/month, already close to the warehouse bill.
• ▹Crossover: $1,800 / $130 ≈ 13.8. At 14 agents, the context tax is bigger than the whole warehouse bill. The two agents already on next quarter's roadmap take you past that point.

Now see what moves the curve. Doubling the data pushes the warehouse line up a bit with more storage and bigger scans. It raises tokens per read only until agents hit their context limit or start truncating. Doubling the agents doubles the tax. Adding one surface to every agent's toolset multiplies it again. That's why nobody notices. The cost grows with the number of agents, not with the amount of data.

4. How to measure it this week

You don't need a new platform. Add two tags to every tool call that touches data, surface_id and agent_id. Then attribute model tokens back to those surfaces. You only have to wrap your tools once:

import functools, json, time
from contextvars import ContextVar

current_agent = ContextVar("agent_id")
current_run = ContextVar("run_id")
surfaces_this_turn = ContextVar("surfaces_this_turn", default=None)

def surface_tool(surface_id: str):
    """Tag a tool as reading one data surface; log what it injects into context."""
    def wrap(fn):
        @functools.wraps(fn)
        def inner(*args, **kwargs):
            result = fn(*args, **kwargs)
            payload = result if isinstance(result, str) else json.dumps(result, default=str)
            log_event({
                "ts": time.time(),
                "kind": "tool_read",
                "agent_id": current_agent.get(),
                "run_id": current_run.get(),
                "surface_id": surface_id,
                "tool": fn.__name__,
                "tokens_in": count_tokens(payload),  # provider token counter or tokenizer estimate
            })
            surfaces = surfaces_this_turn.get()
            if surfaces is None:
                surfaces = []
                surfaces_this_turn.set(surfaces)
            surfaces.append(surface_id)
            return result
        return inner
    return wrap

def on_model_response(usage):
    """Attribute the model's reasoning/summary tokens to the surfaces it just read."""
    surfaces = surfaces_this_turn.get() or []
    share = 1 / max(len(surfaces), 1)
    for s in surfaces:
        log_event({
            "ts": time.time(),
            "kind": "model_turn",
            "agent_id": current_agent.get(),
            "run_id": current_run.get(),
            "surface_id": s,
            "tokens_in": usage.input_tokens * share,
            "tokens_out": usage.output_tokens * share,
        })
    surfaces_this_turn.set([])

@surface_tool("finance.invoices_v")
def get_invoices(customer_id: str):
    ...

Splitting tokens evenly across surfaces is crude, but it's enough for a first pass. Watch out for double counting. A model turn's input tokens already include the tool payloads it just read, so don't add both. Count input tokens from tool reads and output tokens from model turns. Then build the 'same surface, many agents' heatmap, with surfaces on one axis, agents on the other and tokens in each cell.

-- Heatmap cells: who pays how much to read what
-- tokens_in comes from tool reads only; model_turn input already contains those payloads
SELECT surface_id,
       agent_id,
       COUNT(DISTINCT run_id)                                        AS runs,
       SUM(CASE WHEN kind = 'tool_read' THEN tokens_in ELSE 0 END)   AS tokens_in,
       SUM(COALESCE(tokens_out, 0))                                  AS tokens_out
FROM agent_surface_events
WHERE ts >= now() - interval '7 days'
GROUP BY 1, 2;

-- Ranking: the surfaces where the tax concentrates
SELECT surface_id,
       COUNT(DISTINCT agent_id)                                      AS agents_touching,
       SUM(CASE WHEN kind = 'tool_read' THEN tokens_in ELSE 0 END)
         + SUM(COALESCE(tokens_out, 0))                              AS total_tokens
FROM agent_surface_events
WHERE ts >= now() - interval '7 days'
GROUP BY 1
ORDER BY total_tokens DESC;

Look for rows that are hot across most of the columns. Say 8 or more agents read a surface and each spends thousands of output tokens on it. You're paying for the same summary 8 or more times. Start with those rows.

5. The fix, part 1: read once, digest once

Take validation and summarisation out of the agents and put them in the data plane. What comes out is a context artifact: a materialised, versioned, pre-digested view of one or more surfaces, built for an LLM to read. It works like a materialised view, except the consumer is a model rather than a dashboard.

{
  "artifact_id": "ctx.finance.receivables.digest",
  "version": "2026-10-13T06:00Z#41",
  "source_surfaces": ["finance.invoices_v", "finance.credit_notes_v"],
  "schema_hash": "sha256:9c1e…",
  "freshness": { "as_of": "2026-10-13T06:00Z", "max_age_minutes": 60 },
  "validation": {
    "row_count": 18422,
    "checks_passed": ["no_null_amounts", "totals_reconcile_to_ledger"],
    "checks_failed": []
  },
  "facts": { "overdue_total_aud": 412300, "overdue_customers": 37 },
  "summary": "Receivables are stable week on week; overdue concentrated in 5 accounts…",
  "caveats": ["Credit notes after 05:00 not included"],
  "token_count": 1140
}

• ▹Schema hash: when the source schema changes, the artifact stays invalid until it's rebuilt. It catches the silent column rename that used to give you 12 different wrong answers.
• ▹Freshness stamp: agents and tools can reject a stale artifact instead of guessing. It's the freshness contract from the earlier freshness day, applied to digests rather than chunks.
• ▹Validation results: checks run once, deterministically where you can (SQL assertions, dbt-style tests). An LLM only handles the judgement calls. Agents inherit the result and don't repeat the work.
• ▹Facts plus summary: hard numbers go in structured fields and the narrative goes in text. Agents quote the facts instead of recomputing them.
• ▹Token count: you know what each artifact costs to read, so you can price it.

Artifacts aren't free to build. In the example, 5 artifacts rebuilt hourly, at roughly 20k tokens in and 1.5k out per build, cost about $300 a month. That cost scales with surfaces × refresh rate. It doesn't scale with the number of agents, which is the whole point.

6. The fix, part 2: agents consume artifacts, not tables

An artifact only helps if agents read it. Leave a raw SQL tool lying around and the model will often reach for it. So change the tool contract. Return the digest by default, and make raw rows something an agent has to ask for:

from typing import Literal

@surface_tool("ctx.finance.receivables.digest")
def read_receivables(scope: str,
                     detail: Literal["digest", "rows"] = "digest",
                     reason: str | None = None):
    art = artifacts.latest("ctx.finance.receivables.digest", scope)

    if art.schema_hash != registry.schema_hash("finance.invoices_v") or art.is_stale():
        return {"status": "stale", "header": art.header(),
                "hint": "Report staleness, or escalate with detail='rows' and a reason."}

    if detail == "digest":
        return art.body()  # ~1.1k tokens, validated, versioned

    if not reason:
        return {"status": "rejected", "hint": "Raw rows need a reason."}
    log_escalation(current_agent.get(), "finance.invoices_v", reason)
    return warehouse.query_rows("finance.invoices_v", scope, limit=200)

Next, turn on prompt caching for the stable prefix. Order the prompt as tools, system instructions, artifact bodies, then the task. Caching works on prefixes, so you save on repeated runs of the same agent against the same artifact version. There are two catches. Every artifact version bump invalidates the cache, so refresh cadence and cache hit rate pull against each other. And agents with different system prompts won't share a cache entry. Check your provider's current cache pricing and minimum cacheable length before you count on the savings.

Now rerun the example. Building the artifacts costs about $300. Agents reading ~1.2k-token digests cost about $120, less with cache hits. A 5% escalation rate to raw rows adds about $70. That's roughly $500 against $1,565, and the 13th and 14th agents now cost a fraction of what they would have.

Task success improves too. Agents that share a digest give the same answer, so they agree with each other more often. Smaller, cleaner context usually helps reasoning as well, which is the context-rot point from the chunking days.

The trade-off has two parts. The first is staleness. An hourly digest can be wrong for up to an hour after something important changes. The second is lost nuance. A summary keeps what its designer thought mattered. An agent that needed the odd edge case in row 14,203 won't see it unless it escalates. Escalation is your safety valve, so make it cheap to use and easy to see in the logs.

7. Before and after at PhoenixDX, in rough proportions

These are directional observations from our own rollout, not benchmarks. Your proportions will differ.

• ▹Token spend per run: for agents working mostly on shared operational tables, it fell to roughly a third to a half of what it was. Most of the saving was output tokens, because agents stopped writing their own summaries.
• ▹Distinct interpretations of the same table: we went from one per agent to one canonical digest, plus whatever escalating agents produce. Disagreements became visible instead of buried.
• ▹Incidents: 'two agents gave the customer different numbers' mostly disappeared.
• ▹What got worse: stale digests became a new incident class. The worst case was an artifact that kept passing its own checks while the upstream load had quietly stalled. Freshness now reads the source's load timestamp, not the build time.
• ▹Also worse: the artifact pipeline became one more thing to own, monitor and get paged for. The context tax moved into data engineering's on-call.
• ▹And one agent started escalating to raw rows on nearly every run, which wiped out its savings. The escalation log caught it. We fixed it with a better digest, not a stricter tool.

8. When not to do this

• ▹You have one or two agents: the tax is small, and the pipeline costs more to run than it saves.
• ▹The data is highly volatile: if a surface changes meaningfully faster than any sensible refresh cadence (live order books, minute-level telemetry), every artifact is stale by design. Give agents narrow, fresh queries instead.
• ▹The agents are exploratory or analyst agents: their job is finding what nobody has summarised yet. Give them raw access with budgets, not digests.
• ▹Runs are one-off or infrequent: a monthly report agent won't pay back a pipeline.

Here's a simple decision rule. For each surface, compute the overlap ratio R = (number of distinct agents reading it) × (runs per refresh window). R is how many times the digest gets read before it's rebuilt. If R ≤ 2, leave the surface raw. If R is 3–5, digest it when the data is stable and the summaries are expensive. If R > 5 and the data holds steady within the refresh window, digest it now. Apply the rule to the top 10 rows of your heatmap and the result is your roadmap.

9. Monday checklist

• ▹Wrap every data-touching tool with surface_id and agent_id tags, and attribute model tokens back to surfaces.
• ▹After a week of logs, build the 'same surface, many agents' heatmap and rank surfaces by total tokens.
• ▹Put the context tax and the warehouse bill on one budget line, and work out your crossover point.
• ▹For the top 1–2 surfaces with R > 5, ship a versioned context artifact with a schema hash, freshness stamp, validation results and token count.
• ▹Switch those tools to digest-by-default, require a reason for raw rows, log every escalation, and order prompts for prefix caching.

Tomorrow is Day 30. Once you can price every read, you can stop optimising storage, retrieval and LLM spend separately. Instead, you design the whole data and retrieval stack around one number, cost per correct answer. That's the capstone.

Extend your knowledge

• ▹Read your LLM provider's prompt caching docs (Anthropic's prompt caching guide is one example), then model cache hit rate against your artifact refresh cadence.
• ▹Use dbt tests or Great Expectations to move the deterministic half of agent validation into data-plane checks, so the LLM only handles the judgement calls.
• ▹Instrument agent traces with OpenTelemetry's GenAI semantic conventions so surface_id and agent_id travel with token usage into the observability stack you already run.
• ▹Go back over the earlier days on caching and freshness, and map each concept to a field on the context artifact before Day 30.