Context engineering

Day 25: Context Engineering Is Mostly Subtraction

Your agent just did something dumb. You probably want to add a rule, paste in another doc or give it one more example. Most of the time the fix goes the other way: take things out. Knowing what to take out is a skill you can learn, and this lesson teaches it.

The diff that was mostly red

One afternoon at PhoenixDX I opened a pull request against the setup of one of our internal coding agents, and nearly every line of the diff was red. We'd deleted most of its instruction file. We'd trimmed each tool output it received down to a few lines. And we'd removed about half of the documents our retrieval step was attaching to every request.

Someone on the team asked, only half joking, if we'd just broken it. We ran the same task set we'd been using to check the agent. It finished more tasks and took fewer wrong turns on the way. The model was the same and there was no clever new prompt. It just had less to read. I won't quote a percentage because our task set was small, but the improvement was clear and it held up. The rest of this lesson explains why.

Prompt vs context: one-sentence definitions

• ▹A prompt is what you ask the model to do.
• ▹Context is everything the model can see when it decides what to do next: system instructions, files like CLAUDE.md, retrieved documents, tool results, conversation history and examples.
• ▹Context engineering means curating that whole picture so the right things are in it and the wrong things aren't.

Earlier in the course you learned that a context window is a fixed budget of tokens. This lesson is about how you spend that budget. Having room for something doesn't mean it belongs there.

Why more can be worse

A model can't skip irrelevant text the way your eyes slide past an email signature. Any token in the context can shift what it produces next, and irrelevant text still shifts it, often in the wrong direction. A useful way to picture it: every piece of context is a vote for some action.

• ▹A stale doc. An old README says the service uses REST, but you moved to gRPC months ago. The agent writes a REST client. Nothing malfunctioned. It did what the README told it.
• ▹A 400-line log. The one error that matters is on line 312, and everything else is warnings, retries and timestamps. The agent fixes a deprecation warning on line 40 because it was loud and came first.
• ▹A 'helpful' example from another task. You included a sample migration that adds a column with a default value. Now the agent puts defaults on columns that shouldn't have them, because the example looked like the house pattern.

There's also a broader effect, usually called context rot: models get worse at using what's in the input as it grows longer, even when the right answer is in there somewhere. Chroma's 'Context Rot' report measured this degradation across many models as input length went up. The earlier 'Lost in the Middle' paper found something related: models use information at the start or end of a long input far better than information buried in the middle. In practice this means every token has a cost, including the ones that fit comfortably in the window.

What we actually deleted: three cuts

Cut 1: a CLAUDE.md that described an architecture we'd already left behind.

• ▹Before: the file had a long section on our old monolith, covering its folder layout and how services shared one database. We'd split things up since, and nobody had updated that section.
• ▹Symptom: the agent kept putting new code in paths that only existed in the old layout, and kept suggesting cross-service database queries we no longer allowed.
• ▹After: we deleted the history section and kept a short description of the current layout plus the three rules that still apply. The wrong placements stopped.

Cut 2: verbose tool outputs, truncated to the lines that matter.

• ▹Before: test runs and build commands sent their entire output to the agent, including passing tests, progress bars and dependency noise.
• ▹Symptom: the agent would wade through all of it and then chase the wrong problem. Sometimes it summarised the run as passing when one important test had failed.
• ▹After: we wrapped the tools so the agent sees failures, their stack traces and a one-line summary. If it needs the full log it can still ask for it.

def summarise_test_output(raw: str, max_lines: int = 40) -> str:
    """Return what the agent needs to decide its next action, not the whole log."""
    lines = raw.splitlines()
    failures = [l for l in lines if l.startswith(("FAILED", "ERROR", "E  "))]
    summary = next((l for l in reversed(lines) if "passed" in l or "failed" in l), "")
    kept = failures[:max_lines]
    note = f"\n[{len(failures) - len(kept)} more failure lines omitted; call get_full_log() if needed]" if len(failures) > len(kept) else ""
    return "\n".join([summary, *kept]) + note

Cut 3: few-shot examples the agent copied too literally.

• ▹Before: we included two complete example pull requests to show the agent 'how we write code here'.
• ▹Symptom: it copied surface details from them, like variable names, comment style and even a helper function from an unrelated module. It was using the examples as templates when we meant them as illustrations of a style.
• ▹After: we removed both examples and wrote three plain sentences describing our conventions. The output started to look more like our code, because the agent was no longer imitating two specific pieces of it.

The manager analogy: context is the agent's inbox

Good engineering managers don't forward every Slack thread to their team. A lot of the job is deciding what to hold back: the reorg rumour, the half-finished argument in another channel, the customer complaint that's already sorted. Everything you forward takes up some of the team's attention and nudges their priorities, whether you meant it to or not.

An agent's context is its inbox, and you're the manager deciding what lands there. So when an agent misbehaves, look at what you sent it before you blame its judgement. In my experience a lot of 'the model is dumb' moments turn out to be noisy inboxes.

A simple test: the 3-question checklist

For every piece of context, ask one question: would this change the agent's next action on this specific task? If it wouldn't, cut it. In practice that one question splits into three.

• ▹Relevance: does it affect what the agent should do on this task, not on tasks in general? A deploy guide is just noise when the agent is fixing a CSS bug.
• ▹Freshness: is it still true today? A stale fact does more damage than a missing one, because the agent will act on it with full confidence.
• ▹Detail: does the decision need this much detail? A one-line summary, or a pointer like 'see docs/auth.md if you touch login', often works better than pasting the whole file.

Try this today

• ▹Choose one agent setup or long prompt you use a lot: a CLAUDE.md, a system prompt, a custom GPT's instructions.
• ▹Choose a real task you know well and run it once with the current setup. Save the result and write down any wrong turns.
• ▹Go through the context line by line using the three questions and cut it roughly in half. Be ruthless with history, background and examples.
• ▹Run the same task again with the shorter version.
• ▹Compare the two runs. Were there fewer wrong turns? Did it stop copying patterns from other tasks, or mentioning things that no longer exist? And did it lose anything it actually needed?

One comparison proves nothing statistically. It does give you a feel for the effect, and that feel is the skill you're building here.

Honest limits: subtraction is not starvation

We went too far once. Our instruction file had a line saying we avoid lazy-loaded database relations in the reporting service, because they'd caused slow N+1 queries on a dashboard. During the cleanup someone shortened it to 'avoid lazy loading', and later the line was deleted as 'obvious'. A few days after that, the agent reintroduced exactly that pattern in a new report. It had no way of knowing we'd already made that call, or why.

You can tell the two failures apart by how the agent behaves:

• ▹Too much context (noise): the agent wanders, fixes the wrong thing, copies patterns that don't apply, or acts on outdated facts.
• ▹Too little context (starvation): the agent undoes decisions you've already made, guesses at constraints, or asks questions the context should have answered.
• ▹Rule of thumb: cut descriptions, history and raw dumps, and keep decisions along with the reasons behind them. A rule without its reason only covers the cases it names. With the reason attached, the agent can handle cases nobody wrote down.

Where this goes next

Today you curated what one agent sees. It gets harder when agents work together and you have to decide what one agent passes to the next. If it hands over everything, the next agent drowns in noise. If it hands over too little, the next agent remakes decisions that were already settled. The upcoming lessons on multi-agent handoffs use the same three questions.

Extend your knowledge

• ▹Read Anthropic's engineering post 'Effective context engineering for AI agents' for a practitioner's take on curating context inside agent loops.
• ▹Skim Chroma's 'Context Rot' technical report and the paper 'Lost in the Middle: How Language Models Use Long Contexts' (Liu et al.) to see the long-context effect actually measured.
• ▹Audit your own CLAUDE.md or AGENTS.md. Label each section as decision, description or history, then delete most of the history.
• ▹Wrap one noisy tool (test runner, linter or build) so it returns only failures and a summary, then compare how the agent behaves before and after.