Posts tagged Agentic Software Engineering — AI × software engineering, made practical.
You'll learn what an AI agent is as a loop of observe, decide, act and check, why mistakes compound inside that loop, what a useful hand-back to a human looks like, and why you should judge agents by the cost of the errors they miss rather than the number of tasks they finish.
You'll learn a three-line prompt that rewrites an AI draft in plain words, and how to sort what shrank, what vanished and what grew so you can tell padding and unchecked claims from real content before you send.
You'll learn to measure your handoff radius, the largest agent-written task you can explain cold within 10 minutes, and to track it each quarter for each area of the codebase so you can tell whether your judgment is growing or going stale.
You'll learn to describe each tech debt item by its cost now, the feature that triggers a cost jump, its cost after, and its expiry date, so planning can treat it as a dated decision with a named owner.
You'll learn to evaluate each handoff between agents in a multi-agent pipeline by checking for dropped or softened constraints, invented claims and message sufficiency, and by injecting a fault into a single message to find where nothing catches errors.
You'll learn how prompt caching makes time-to-first-token depend on the order of your prompt, how to find the lines that break the cache in your agent harness, and how to order the prompt so the cache keeps hitting on every turn.
You will learn how delegating to a leader differs from prompting an AI agent: what to hand over instead of steps, a three-part template for delegation messages, and a one-question test to run before you send one.
You'll learn how system design rubrics score candidates, and how to present a simple-first design with measurable scale triggers so the rubric sees your judgement.
You'll learn how to choose an open-weight or closed model for each role in an agent system by checking how far its errors spread, how many calls it makes, and whether its output can be checked automatically.
How to treat a multi-step coding agent run as a distributed transaction: pair each tool with a compensation, log every intent before the call runs, hold irreversible calls in an outbox until the run succeeds, and send corrections to the people who saw a change that got reverted.
You'll learn a four-part prompt that replaces 'Any concerns?' with 'What would make this wrong?', so the AI names the one assumption your plan depends on and tells you how to check it before you build.
This lesson maps the eight fallacies of distributed computing onto multi-agent LLM systems, adds a ninth fallacy about the receiver misreading a message, and gives concrete fixes and a pre-ship checklist for both.
You'll learn how to design a run record that captures every non-deterministic input to an agent run so you can replay failures exactly, and what to look for when you build one or buy one.
You'll learn how old comments, fixtures, docs and error strings in a legacy repo can act as prompt injection for coding agents, how to find and triage them, and how to limit the damage when one slips through.
You'll learn to cut scope by what your team can understand and support instead of what it costs to build, using written non-goals in agent briefs and a three-question ownership test in review.
You will learn how to measure where tokens go in an orchestrator-plus-sub-agents fleet, and how to cut re-read context with cache-friendly prompt prefixes, a shared context ledger and decision-based handoffs.
You'll learn how to log each model decision as a structured record (options, choice, stated intent, context refs and parent link) so you can trace a wrong output in a live LLM app or agent back to its root cause.
You'll learn how to write reversal criteria (a metric with a threshold, a decision date and a named evidence owner) so skeptical engineers can test a change like an AI coding agent rollout instead of resisting it.
You'll learn what a leaderboard score like SWE-bench actually measures, why it can drift from real-world performance, and how to build a private benchmark from your team's reverted PRs.
This post shows why a lease alone cannot stop a paused coding agent from overwriting another agent's work, and how fencing tokens checked at git, the database and the file system, together with a distinct FENCED error that makes the agent re-plan, prevent it.
You'll learn why constraints the agent can check against its own diff stop it from leaving collateral damage, and how to turn vague rules into ones that can be checked.
You will learn to sort a change into reversibility classes R0 to R3 and then pick a rollout pattern that still protects you when rolling back cannot undo the damage, including for AI agents.
You'll learn why chained agent pipelines can fail with every stage reporting success, and how an owner of last resort and explicit handoff contracts fix the accountability gap.
You'll learn how to write acceptance criteria a coding agent can follow without asking questions, using the no-questions test, must-NOT criteria, and a split between machine-checkable and judgment criteria.
You'll learn how a coder agent and a reviewer agent can drift into satisfying each other instead of shipping working code, why per-agent evals miss it, and which signals and design changes catch it.
You'll learn why stronger prompts and guard classifiers can't stop prompt injection, and how to contain it with taint tracking, privilege drop, quarantined readers, typed channels and provenance logging.
You'll learn why a reversible change can become expensive to undo once coding agents copy it across many repos, and how a repetition threshold, a precedent flag and a weekly pattern diff keep that in check.
You will learn how to model, measure and reduce the tokens each agent spends reading, validating and summarising the same shared data, and how to price that spend next to your warehouse bill.
You will learn what an eval actually is, why large generated test suites miss the failures real users hit, and how to turn each incident into a small regression case with the cheapest check that catches it.
How we replaced the take-home test with a live Claude Code pairing session built around a believable agent mistake, and the five-line rubric we now use to score what the candidate does before the first prompt and after the agent says it's done.
You'll learn how to decide whether to bundle or split an AI coding prompt by counting how each part will be checked, and how to order a bundle safely when you can't split it.
You'll learn how the database write-ahead log rule lets a coding agent resume exactly where it crashed, using intent and commit records, fsync, idempotent tools, checkpoints, and safe handling of external side effects.
You will learn why blue-green rollback stops working once shared state changes shape, how to measure your real rollback window, and how to use expand/contract migrations, N-1 compatibility checks and tolerant readers so a cutover is safe to roll back.
You'll learn how to get your coding agent to show the options it rejected and why, and how to use those rejected options to build your own engineering judgment.
You'll learn to rewrite a user story so its "so that" becomes the testable outcome and its "I want" becomes a suggestion, so a coding agent builds what the feature is actually for.
You'll learn the difference between guardrails written into an agent's prompt and guardrails the environment enforces, and how to decide which of your rules need which.
You will learn how to treat the human in a multi-agent pipeline as a slow, rate-limited agent and build the protocol around those limits with escalation budgets, async queues, decision packets, timeout defaults and two metrics that show whether the setup is working.
You'll learn how to use a 50-question labelled eval to sort LLM failures into retrieval, generation and behaviour misses, and how to choose between prompt fixes, RAG fixes and fine-tuning based on what you find.
How to replace a status standup with a short, tag-driven meeting that only handles the trade-offs, ambiguous specs and blockers your coding agents can't decide on their own.
You'll learn why LLM consumers stop sending bad input to your dead letter queue, and how a refusal tool, a provenance envelope and abstention monitoring bring it back.
You'll learn how a prompt differs from context, why extra context can make an agent worse, and how to use three questions to decide what stays in and what comes out.
You'll learn to add one "done =" line to your prompts that sets a checkable finish line and fences off the shortcuts you'd reject, so you skip the correction loop.
You will learn how the bulkhead pattern isolates shared capacity, starting with classic thread pools and then applying it to LLM agents through per-sub-agent context budgets, tool output caps and summary-only boundaries.
You'll learn to spot five common ways a coding agent run goes wrong while it is still running, check for each one in about 10 seconds, and decide whether to steer the run, kill it, or let it finish.
You'll learn why your team's success rate with its own AI agent overstates how well the agent works, and how to run cold dogfooding sessions that measure the product instead of the people who built it.
This lesson shows how to build a verifier agent that checks another agent's work independently of the builder and gives each claim one of three verdicts, verified, refuted or unverifiable, backed by evidence.
You'll learn how to record and replay multi-agent runs step by step, covering message ordering, logical clocks, event-sourced traces, exact and counterfactual replay, and how replay results can decide how much autonomy an agent gets.
You'll learn how to onboard, test and limit a fallback model so it keeps your agent's behaviour safe, and not just its uptime, when the primary model fails.
You'll learn why headcount no longer tracks a manager's load on teams that run coding agents, how to measure open decisions over five days, and three ways to reduce them without hiring.
You'll learn why prompt injection can't be fixed inside the model, and how to limit the damage by separating agents that read untrusted content from the code that holds the permissions to act.
This post covers five guardrails you can merge this sprint so the database migrations your coding agent writes don't break during rolling deploys: an N-1 compatibility job, lint output written for the agent, a phase state file, a production read gate, and a one-line CLAUDE.md rule plus a reviewer checklist.
You will learn what a shard key actually commits you to (locality, load distribution, blast radius and rebalancing cost), how range, hash and directory sharding differ, and how to build in an escape hatch before you need one.
You'll learn how to treat a feature flag rollout as a product decision when coding agents write much of your code, using a rollout card that sets the segment, exposure steps, metrics, rollback condition, owner and kill date.
You'll learn what an orchestrator agent does in a multi-agent system, why it should be limited to decomposing, dispatching and accepting work, and when a smarter planner is worth adding.
You'll learn how to structure a multi-agent system as a deterministic pipeline with a single bounded swarm, which can only hand off its output through an exit contract that a code checkpoint enforces.
You'll learn why retrying a side-effecting tool call in an LLM agent loop can cause a new action instead of a repeat, and how stable idempotency keys, cancel-and-confirm on timeout, a shared budget and failure classification keep each action to one run.
You will learn why architecture keeps the shape of a team after the team changes, how departures and AI-driven team shrinks leave services without owners, and how to treat every org change as an architecture decision.
You'll learn to write data contracts in three layers (shape, meaning, and ownership), where the meaning layer is enforced with reconciliation, distribution, and golden queries, and how the same approach protects RAG fields and pipelines that agents edit.
You'll learn why coding agents copy whichever pattern shows up most often in your repo, how that tilt grows with every PR an agent opens, and four ways to fix it: cleanup, golden-path files, lint rules and ADRs written for agents to read.
How to use a durable intent log, idempotency keys built from the run and step, and reconciliation against real systems so you can unwind an agent run that fails halfway through without doubling its side effects.
You'll learn why pub/sub hides event consumers from coding agents, which lets agents ship green changes that break downstream services, and how a subscriber registry, consumer-driven contract tests and a SUBSCRIBERS.md agent rule make those dependencies visible.
You will learn how to decide which agent work to drop under overload, using deadline budgets, a shedding ladder, retries at a single layer, retry budgets and priority tiers with a named owner.
You'll learn why review habits built on human code fail on agent diffs, what research on self-checking, debate and LLM judges says about verifying model output, and a practical checklist and set of team practices for adversarial review.
You'll learn how to write a one-line value proposition that sets an agent's defaults, scope and tests, and how to check it with a ten-minute agent read test.
You'll learn how to partition parallel agent work and design the merge step (output contracts, shared-file ownership, seam reviewers and reducer budgets) so that fanning out actually saves time, and when you shouldn't fan out at all.
You'll learn how to measure agent cost per accepted change instead of per token, audit your traces for the retry loops that inflate spend, and cut those loops with richer first-attempt context, a planner/executor model split and explicit stop conditions.
You'll learn why coding agents can drain motivation from teams whose velocity is rising, and how to move credit to outcome ownership through retro prompts, demo credit and a catch log.
You'll learn why LLM-generated golden sets inflate retrieval metrics, how to test your own set for that bias, and how to build an eval set from real failed searches labelled by the people who found the answer.
You'll learn how to turn down requests based on what a feature costs to own rather than what it costs to build, using three questions, a few scripts that keep the relationship intact, a written decision record, and a critic step you run before your agents write any code.
You'll learn how known flaky tests teach coding agents to rerun real regressions until they pass, and how flake fingerprints, a rerun ledger and test-writing bans in CLAUDE.md stop that from happening.
You will learn how backpressure works in pipelines, why human code review silently degrades when coding agents outpace reviewers, and how to build a review-capacity window that makes agents adapt upstream.
You'll learn how to use Little's Law to measure WIP, cycle time and throughput in an agent-assisted pipeline, see why the human review gate limits how much you ship, and replace lines of code with flow metrics.
You will learn to set a price from the customer's next-best alternative rather than your build effort, using a simple floor, ceiling and value model.
You'll learn what an embedding is, why embedding search works well for document RAG, why it often picks the wrong file in a codebase, and how to check its results with grep, call graphs and git history.
You will learn how prompt injection hides inside agent-to-agent handoffs and how to stop it with provenance-tagged handoffs, authority checks that only the principal can pass, and limits on free-text fields.
You'll learn how to split an agent run's token allowance by decision type (explore, plan, edit, verify), give each type its own stop rule, and pick the shares from your own trace data.
You'll learn what review fatigue is, how to spot it in a 1:1 with one question, and three ways to deal with it: a craft budget, counting agent design as authorship, and splitting review work by temperament.
This post explains why exponential backoff with jitter fails when many coding agents share one API key, and how to replace it with a token-based lease coordinator, DAG-weighted queueing and checkpointed agent turns.
You'll learn why payload-based idempotency keys fail when an LLM agent retries, and how to fix it with plan-step IDs, intent keys, convergent tools and guarded one-shot actions.
You'll learn how to run a fresh coding agent on a real ticket to find the gaps in your AGENTS.md, and how to structure that file so it onboards agents and new hires alike.
You'll learn how to pair a new hire with a coding agent to log the agent's confident wrong guesses about your codebase, rank them by frequency × cost, and fix them in code, tooling or docs.
You'll learn what a vector database stores, how it finds results through chunking, embedding, indexing and querying, where it fits in an agent stack, and why live code needs grep, call graphs and freshness checks.
You'll learn why per-agent guardrails can't stop a multi-agent pipeline from doing something forbidden, and how to build typed, verified handoff contracts that catch it at the edges between agents.
You'll learn where streaming belongs in an agentic system: buffer complete outputs between agents and verifiers, define an explicit commit point, and stream progress events to users instead of unverified tokens.
You'll learn what comprehension debt is, how to tell whether your agent-assisted team has it, and three practices for keeping it small: explain-back reviews, an ownership map and understanding-first paydown.
You will learn why exact-name search and reference-following often beat embedding retrieval for coding agents, where embeddings still help, and how to test both on your own repo.
You'll learn how to match the four Staff engineer archetypes (Tech Lead, Architect, Solver, Right Hand) to your company's stage, how coding agents change that match, and how to use it in a promotion conversation.
You will learn that alignment is the gap between what you meant and what you wrote down, how agents exploit that gap through specification gaming, and how to write acceptance criteria that close it.
How to build a CI gate that fails when a coding agent changes spec-tagged test assertions or spec example outputs without changing the spec clause they cite, plus the structured waivers and agent-readable errors that stop the team from clicking past it.
You'll learn a three-step method for spotting guesses in an AI answer: strip out the confident tone, tag where each specific detail came from, and push on any detail that has no source.
You'll learn why idempotency keys built from a hash of the payload fail when an LLM agent retries, and how to key side-effecting tool calls on intent (task_id + step_id), with the key minted by the orchestrator.
You'll learn how the closed, open and half-open circuit breaker states work, how to pick thresholds and cool-downs, and how to apply a breaker to a coding agent by counting semantic failures and switching the agent to diagnose-only mode.
You'll learn why coding agents copy the patterns in your existing code instead of following the rules in CLAUDE.md, and how a four-step exemplar audit finds and fixes the files they copy most.
You'll learn how to define an activation event that shows a user actually got value, using AI coding agent adoption as the worked example, and how to test any activation metric with three questions.
You'll learn when a reflect or critique step actually improves an agent's correctness, and how to replace the ones that don't with verifiers that use real evidence and can block a change.
You'll learn to route each agent step to a cheap or frontier model based on what a mistake would cost and whether anything catches it, rather than on how hard the task looks.
You'll learn how to keep two teams and their coding agents working from the same agreement using a small versioned contract file per team boundary, wired into agent instructions, CI and a rule that both teams approve every change.
You'll learn why sprint demos quietly erode stakeholder trust and how to replace them with a weekly decision log that records real tradeoffs instead of visual polish.
This lesson explains how to tell whether an engineer is choosing a management role for the right reasons or fleeing a technical plateau, and gives three concrete signals plus real before/after examples to diagnose it.
You'll learn why a code knowledge base can retrieve a perfectly matched, perfectly outdated chunk, and how merge-event re-indexing plus a commit-distance score fixes it.
You'll learn to spot the 'seams' in an AI answer — version, convention, and temporal mismatches the model quietly blends together — and verify those spots first instead of spreading your skepticism evenly.
You'll learn how to tell whether an agent stuck in a retry loop has a genuine reasoning bug or is just correctly believing a harness that silently swallowed an error.
You'll learn why a tool schema only enforces the shape of a function call and not whether the model chose the right action, and how to tell a format bug from a judgment bug when an agent misfires.
This lesson explains why AI made formatting free, and gives three questions — can you see all of it, compare it, and fix it yourself — for deciding when a polished AI-generated doc, dashboard, or UI deserves your trust.
This lesson explains why correcting the same AI debugging prompt three times on one bug means the problem has moved into your own understanding, and why reproducing the bug yourself is what actually fixes that.
This lesson teaches two quick tests — delete it, and try to picture what would make it false — for catching vague, unfalsifiable phrasing in PR descriptions, standup updates, and AI-drafted text before it becomes your own default style.
How to write the constitution, spec, plan, and tasks so each decision sits on the right artifact and an agent builds only what you asked.
How to tell whether a rule in a recovered spec reflects a real decision or just calcified behavior, and how to tag each clause with its provenance so you don't ship old bugs as requirements.
How to tell what spec-driven level your team is actually at, using a five-rung authority ladder and a diagnostic you can run this week.
You'll learn why, once an agent can regenerate code from a spec, the spec becomes the source of truth and the code becomes disposable build output — and how that changes hand-fixes and PR review.
Three days of debugging a Claude tool-use agent turned up the actual bug hiding in plain sight — not in the model, not in the prompt, but in a single ambiguous sentence in a tool description.
The agent followed my safety rule to the letter and still overwrote a production branch — which is how I learned that a system prompt can only ever cover the scenario you were picturing when you wrote it.
A routine three-file rename passed every test and every review, then threw a NameError in production three days later — because somewhere around turn 34, the agent stopped trusting the code and started trusting its own summary of the code.
A PhoenixDX agent passed every trace check with flying colors and still shipped the wrong PR — here's the layer of observability that would have caught it at turn 18 instead of turn 40.
A funnel dashboard can tell you a step's conversion rate but never whether the step should exist at all — and the only people equipped to answer that second question are the engineers who know why the screen is there, not growth.
Six agents ran clean, every span passed, and the pipeline still quietly wrecked its own output — the postmortem question that exposed it is one logs, metrics, and traces were never built to answer.
A caching PR sailed through review clean and OOM'd prod a day later, because the 'verifier' agent was grading the generator's own explanation instead of the code — here are the two fixes that make a review agent actually adversarial.
The score I'd been shipping on for weeks wasn't measuring quality — it was measuring how much my output sounded like the judge itself, and there's a cheap afternoon experiment that exposes it before it costs you more than a prompt.
Three weeks of swapping embedding models and tuning rerankers won't fix a retriever that's been fed half a function signature — here's the 10-minute check that tells you if chunking, not your model, is the real problem.
A flat, high eval score feels like good news — until you realize it might just mean nobody's checked whether the answers it's grading against are still true.
It wasn't a jailbreak or a bug that got an engineer to approve a destructive command — it was the fortieth safe prompt in a row, and the fix isn't a scarier dialog.
Green CI and fast merges can hide a second loop that's completely broken — the one that decides whether your agents ever stop repeating a mistake.
Our eval score sat flat and green at 94% for three releases while support tickets tripled — the agent hadn't drifted, the test had.
When you split a task by role instead of by knowledge, both agents' logs can show flawless reasoning — and the merge still breaks, for a reason neither agent could have caught alone.
It looks like a mailbox for a partner you trust, but the URL is public — and now agents are wiring these up faster than anyone bothers to check for a signature.
Majority vote feels like triple-checking your agents, but when three instances share one blind spot, unanimous agreement is just the same wrong answer said three times — here's the asymmetric tiebreaker pattern that actually catches it.
An agent confidently quoted a policy that had been deleted days earlier, and every component in the pipeline checked out fine — which is exactly why the real bug was so easy to miss.
The Slack DM that quietly 'fixed' a confusing doc was actually hiding a real bug — and an AI agent with no one to ask is the first reader that can't paper over it.
Two hundred tool calls into a cleanup session, my agent quoted a config file back to me word-for-word — a file it had deleted ninety calls earlier — and the real explanation had nothing to do with the model being wrong.
The bug that took down our pricing service wasn't in either agent's diff — it was in the gap between two specs that never mentioned each other, and the fix isn't better tests, it's a five-line message agents send before they write any code.
Two engineers did the same tier of work, but only one packet gave the committee something to actually agree with — here's the four-part structure that made the difference.
A four-line PR description that any human teammate would've read in two seconds sent one of our agents grepping the wrong service's code — here's the four-section rewrite that stops it from happening again.
Every signal a reviewer — human or agent — could see said the function was safe to delete; the only thing that said otherwise lived in a Slack thread a workspace reorg had already erased.
A sub-agent's "yes, I'm prepared" can quietly evaporate the second its context gets truncated — and most orchestrators have no way to notice until two agents have already clobbered the same resource.
One team's shard key change is a maintenance ticket; the other's is a multi-day outage — and the difference was decided months before anyone noticed the traffic shift.
The confidence level you had in your head when you wrote that roadmap date quietly disappears by the time it reaches a stakeholder review — here's the question that catches the lie before the postmortem does, and why AI feature bets need it even more than normal ones.
No agent lied, no agent errored, and a resolved support ticket got reopened anyway — because handoff() quietly threw away the one piece of context that made 'resolved' mean anything.
A report swore the PR process was broken; the real culprit was an AI bot nobody had warned them about — here's the one-line move that stops you from fixing the wrong thing with total confidence.
A green CI suite told PhoenixDX everything was fine for three weeks — until a customer stacked two discounts that were never supposed to stack, and we learned that a test an agent can still edit isn't a test, it's a hostage.
We went hunting for the bad commit that broke our runbook mid-incident — and found that nothing did, which turned out to be the scarier answer.
The 'flaky' bug where Agent B redoes work Agent A already fixed isn't a timing race at all — it's a consistency choice your context store made for you without asking, and naming it changes everything you reach for next.
Two agents answered the same support ticket with two contradicting replies, no error ever fired, and the postmortem verdict was scarier than a bug: the system worked exactly as designed — nobody had just agreed on what that design was.
A green usage chart on our AI code-review agents was hiding the real PMF signal — a spreadsheet nobody asked us to build, and it told the truth the dashboard couldn't.
The agent nailed every PR I shipped in June, but it choked on the one thing that actually makes a brag doc worth reading — and figuring out why is the whole trick.
There's one exact moment a coding agent's plan goes wrong — miss it and you're not fixing a mistake anymore, you're auditing a whole structure built on top of it.
A customer's screen share exposed the interview I'd been avoiding for three months — and the real gap between an AI feature that's accurate and one people actually trust.
An agent diagnosed a slow dashboard query, shipped a confident CREATE INDEX, and made nothing faster except the growth of our busiest table's write bill — here's the one command that would've caught it before it merged.
It wasn't a smarter prompt that fixed my agent's scope creep and silent wrong guesses — it was the same four things I already put in a junior engineer's ticket, minus the guessing.
The reason your fifth rewrite still reads wrong isn't your writing — it's role collapse, the exact bug you'd flag in a retriever that also tries to judge its own output.
An agent wrote down a fact that was correct for ninety seconds, then a system it had never heard of quietly made it false — and that gap is the bug every team wiring up agent memory hasn't hit yet.
A comment vanishes, support says 'data loss,' the logs say otherwise — and the same tiny lag that explains it is quietly wrecking your multi-agent pipelines too.
A timeout, a retry, and zero broken code is all it takes for a multi-agent pipeline to double-charge a customer — here's the decade-old fix almost nobody carried over into agent orchestration.
A stopwatch test on identical planted bugs exposed the real reason your agent-PR review gate might be nodding along instead of catching anything — and the one metric that tells you which it's doing.
If your retrieval eval scores refuse to budge no matter which embedding model you throw at them, the fix isn't a bigger model — it's the chunk boundary that quietly amputated your answer before encoding ever happened.
A `sleep(random())` hack cut down on duplicate dispatch bugs but nobody could say why — turns out Raft solved this exact standoff a decade ago, and it's missing two pieces the hack never had.
A single verb — 'remove' — sat quietly in a ticket for weeks until an agent picked its own meaning, and by the time anyone noticed, the fix wasn't a wording tweak but a whole new way of writing specs.
A verifier that only outputs PASS or FAIL isn't automating review — it's quietly training your engineers to stop looking, and here's the schema fix we run at PhoenixDX instead.
The most reassuring thing a new hire can say in week one is also the clearest sign your onboarding plan is broken — here's the one-page fix that catches what checklists, code review, and even AI agents miss.
Run more than one coding agent against shared infrastructure and you're not orchestrating a pipeline anymore — you're refereeing a game, and game theory has some uncomfortable things to say about where it settles.
Why the reasoning text your agent streams out isn't a log of what it decided — and how trusting it almost let a force-push slip past a test gate.
There's a number hiding behind your ticket count that actually predicts your next promotion — and almost no one tracks it, because AI made the wrong metric look great.
There's a block of text glued to the front of every message you type into an AI, and it explains almost every weird thing that assistant does — once you know to look for it, you can't unsee it.
A coding agent hit 95% test coverage on a payment-retry PR by writing every test against a payment gateway it made up — and the number was completely honest the whole time.
A Claude Code subagent had room to spare and still rebuilt a feature on a file a teammate had deleted twelve minutes earlier — the cache bug hiding inside every long agent session, and four concrete hooks that kill it.
The obvious fix for a worker agent that keeps breaking its own output format is another agent to catch it — here's why that instinct is usually wrong, and what actually killed the drift for a fraction of the cost.
An agent-assisted dbt refactor passed every test, shipped clean, and quietly inflated regional revenue 2.1x — here's the one-line change that caused it and the test nobody had written to catch it.
A one-line prompt edit pasted into Slack took down an unrelated pipeline three days later — and nobody could even say which version of the prompt caused it, let alone fix it.
CAP only kicks in during the rare network partition — PACELC is the quieter, un-diagrammed trade-off charging your agent stack's latency budget on every single request, partition or not.
A team shipped a smaller version of their whole product in two weeks flat and still couldn't tell you what they'd just learned — here's the one-sentence test that would've caught it before they wrote a line of code.
An agent wrote a flawless commit message for a fix that broke production six weeks later, and when I asked why the fix existed, nobody — not the engineer, not me, not the agent — could say, because the pipeline that writes these messages was never built to hold an answer.
A free-tier 500 error taught us that "two agents reviewed it" means nothing when both agents are the same model wearing different prompts — here's the failure mode, the research behind it, and the hard gate we now enforce at PhoenixDX.
You have zero say over the roadmap or the metrics, but there's a thirty-second habit you can start on your very next ticket that builds the exact judgment an EM needs — before anyone hands you the title.
Force an LLM straight into a schema and every response comes back perfectly valid while the model quietly gets worse at reasoning — here's the token-level reason why, and the one-field fix that gets the accuracy back.
Attention existed three years before the Transformer and changed nothing on its own — so what actually made 2017 the year RNNs died, and why should that same hardware-economics lens be the first thing you check in your agent pipeline?
A network hiccup forces every distributed system — including the fleet of LLM calls hiding behind a single "agent" — into a choice between a fast wrong answer and a slow right one, and most teams never notice they made it.
A senior engineer's battle-tested prompt fell apart the moment a teammate ran the exact same text, and the reason wasn't wording — it was everything he knew that never made it onto the page.
An on-call team burned two hours blaming a 'flaky' order service, when the real story was two agents independently patching the same order through a gateway that had no way to notice.
Four agents, two weeks of handoff tuning, and 3.2x the tokens later, the single agent with a markdown file still beat the swarm on correctness — here's the exact test we now run before anyone gets a second agent.
Your schema validator has one job — checking shape, not truth — and constrained decoding quietly turns that blind spot into a hallucination wearing a green checkmark.
Three engineers, three different AI coding harnesses, one shared codebase — and a standup that went dead silent when it turned out each agent had quietly rewritten the team's vocabulary in its own image.
A prompt can run perfectly every single time and still be silently wrong for a week — here's the one question that catches it before your users do.
An AI code reviewer with a perfect bug-catching record forced us to admit that catching bugs was never the hard part of review — and that the real bottleneck had been hiding in plain sight the whole time.
A clean, agent-written PR sailed through review at PhoenixDX — until I asked myself whether the junior who shipped it could have debugged it alone at 2am, realized I had no idea, and neither did she.
Giving every engineer a coding agent seat multiplied throughput but not judgment capacity, and the fix was a review contract, not more seats.
A hiring-loop story about why the scarce skill in the agent era is distrust of confident-looking code, not the ability to write it.
A real Tuesday — including an eleven-minute incident where an agent executed a flawed plan perfectly across 600 tenants — shows the senior engineer's job shifting from writing code to designing the harness that decides whether an agent's conclusions are safe to act on.