Your 'cheap' AI agent just cost you $42 — and the invoice never showed it

Why this matters

Here's the number nobody puts in the deck: the agent's task cost $2. Fixing what it got wrong cost your senior engineer twenty minutes. If your 'cost per task' metric only counts the API bill, it'll tell your FinOps partner the cheap agent is winning — right up until someone asks why your best people are spending their afternoons rewriting agent output instead of shipping their own work.

The $2 task everyone celebrated

Picture it: budget review, someone throws up a dashboard, average cost per completed task just dropped from $8 to $2 after a model swap. The room nods. Somebody claps. What's missing from that slide is the twenty minutes a senior engineer spent re-reading the diff, chasing an edge case the agent missed, and rewriting two functions before it was safe to merge. That time happened. It just didn't happen anywhere a spreadsheet was watching.

Name the trap

'Cost per task' quietly turned into 'API cost per task' — and nobody voted on that. It's not a harmless shortcut, it's a metric that only sees one side of the ledger. Token usage is metered, itemized, sitting on a vendor invoice with your name on it. Review time is scattered across Slack threads, PR comment chains, and the part of a code review nobody timestamps. Easy-to-measure beat true-to-measure, and now the number in your deck is optimizing for the wrong thing.

Why this is the natural failure mode right now

This isn't a one-off measurement bug — it's the direction the whole field is heading. As agents get faster and cheaper at generating code, the bottleneck stops being 'can we produce this diff' and becomes 'can we trust this diff enough to merge it.' Generation is commodity-priced now. Trust isn't. Trust gets built by a human reading the code, running it mentally, checking it against context the agent never had — and that's human time, not tokens. Any cost metric that skips the trust-building step is still measuring last decade's bottleneck.

The math, made concrete

Take the same ticket, run through two different agent passes:

• ▹Task A: $2 in API cost. The diff looks plausible — no tests, a one-line rationale. Your senior engineer needs twenty minutes to verify it: reading unfamiliar logic, checking edge cases, maybe running it locally.
• ▹Task B: $8 in API cost. It ships with tests that actually exercise the change, a short rationale for the approach, and a diff small enough to genuinely review. Your senior engineer needs three minutes to approve it.

Put a loaded senior engineer at roughly $120/hour — a reasonable blended figure for salary, benefits, and overhead, adjust it to your market. Task A: $2 + (20/60 × $120) = $2 + $40 = $42 in true cost. Task B: $8 + (3/60 × $120) = $8 + $6 = $14. The '$2 task' everyone clapped for actually cost three times more than the '$8 task,' once you count the side of the ledger that was doing the real damage. The cheap one lost, badly, and the invoice never said a word.

How review cost hides

Nobody logs '20 minutes reviewing agent-generated PR #4821' as a line item. That time gets smeared across a dozen invisible corners: a Slack thread where a teammate asks 'does this handle the null case?', a PR thread that drags three rounds, the fifteen minutes someone spends locally reproducing a bug the agent's 'fix' didn't actually fix, the silent 'let me just double-check this' before anyone hits approve. None of it reaches a spreadsheet. All of it eats the sprint.

The fix: what actually belongs in the denominator

You don't need a dashboard to start. You need a formula and a habit.

true_cost_per_task = api_cost + (reviewer_minutes / 60 * loaded_hourly_rate)

cost_per_task_ratio = true_cost_per_task / api_cost_only
# if this ratio is consistently high, your 'cheap' agent is expensive

Here's the version you can start this sprint with zero tooling: add one required field to your PR template — 'review time (minutes, rough guess is fine)' — for anything tagged agent-generated. That's the whole lift. Now you have a denominator. Pull it weekly, multiply by your team's loaded rate, and sit it right next to the API bill on the same chart. The gap between those two bars is the real story.

The second-order effect

Once reviewer-hours sit in the denominator, what you're optimizing for changes. You stop asking agents — and the tools and prompts driving them — to be cheap, and start asking them to be reviewable: smaller diffs, tests that demonstrate correctness instead of just asserting it, a short rationale so the reviewer isn't reverse-engineering intent. That's a real shift in what 'good agent behavior' means on a team — from 'produced working code' to 'produced code a human can trust fast.' It's a better target. It's also the one your reviewers have been asking for the whole time.

Close

Cost per task was never really a pricing question — it's a trust question wearing a pricing question's clothes. Until reviewer-hours make it into the denominator, the number in your deck rewards agents for looking cheap, not for being cheap. Day 18 picks up right here: once you can actually price trust, how do you build it into the agent's workflow instead of bolting it on after the fact.

Extend your knowledge

• ▹Add a 'review time' field to your PR template this sprint and pull the first week's numbers before your next budget review — don't wait for a dashboard to tell you what you can measure by hand today.
• ▹Line up your true-cost-per-task trend against change failure rate and lead time for changes — two of the four DORA metrics. A cheap-looking agent that spikes change failure rate is failing the same trust test from a different angle.
• ▹Go back to your Day 16 notes on agent output quality — reviewability (diff size, test coverage, rationale) is the lever that moves both sides of this equation at once.
• ▹Next time you evaluate a coding agent or model, ask the vendor for reviewer-time data, not just token pricing. If they can't answer that, that's the answer.