The RICE Score Lied — Because Claude Wrote the Code in an Hour

Why This Matters

You can score a RICE backlog half-asleep by now — Reach, Impact, Confidence, Effort, done, next ticket. Here's what nobody flagged: the moment your team started shipping with Claude or Copilot, one of those four numbers quietly stopped measuring what you think it measures. It's also the one you trust the most.

The One-Hour Feature

A few months back, one item hit the backlog: AI-drafted replies for churn-risk customer emails. Reach: every at-risk account, so high. Impact: tied straight to retention, so high. Confidence: we'd seen the pattern work elsewhere, so high. Effort: basically nothing — one of our engineers had Claude spin up a working version in about an hour. RICE score off the chart. We shipped it first.

Then we spent three weeks arguing about whether it actually worked. Not whether it ran — it ran fine from day one. Whether the drafts were safe to send. One email implied a refund policy we don't have. Another read as dismissive on an account that was one bad interaction from walking. Support, legal, and the engineer who built it went back and forth for weeks over edge cases nobody had scoped — because the RICE score said ship it, and scoping felt like the friction we were allowed to skip.

Name the Mechanism

RICE = (Reach × Impact × Confidence) / Effort. Effort was never really measuring "how long this takes to build." It was standing in for something the formula never says out loud: how much risk and review this item is going to cost you. That substitution worked for years because, for a human typing code by hand, effort and risk rode the same curve — the harder something was to build, the more surface area there was to get wrong, and the more time got spent catching it along the way. Building and verifying used to be the same act.

• ▹RICE's hidden assumption: effort ≈ risk ≈ review burden.
• ▹That held as long as a human's typing speed was the bottleneck on both writing the code and thinking it through.
• ▹It breaks the second an agent writes the diff — typing time collapses to near-zero, but the risk and the review burden don't move an inch.
• ▹Low Effort now just means "cheap to produce." It says nothing about whether you can trust the output.

The Inversion — Two Real Backlog Items

Set the churn-risk email feature next to a second item that shipped around the same time: rewriting the billing reconciliation job to handle multi-currency rounding correctly. Estimated at three days — slow, careful work through currency edge cases. RICE scored it low: three days of Effort dragged the denominator up and sank it below the email feature in the queue.

• ▹Billing reconciliation (RICE said "high effort"): three days to build, but the correctness bar was fully mechanical — a golden set of currency test cases already existed, a domain expert could verify it was right in about an hour by running them, and it got reviewed line by line as it was written.
• ▹AI-drafted churn emails (RICE said "low effort"): one hour to build, but correctness meant judging tone, implied promises, and edge cases across an open-ended space of customer situations. There was no test suite for "does this email sound right." Trust got built the hard way — case by case, in production.

We shipped in RICE order: email feature first, billing job later. That's exactly backwards from what actually cost the team time. The billing job, once it landed, got trusted almost immediately. The email feature, built in an hour, cost three weeks of review — because nobody had budgeted for the fact that "cheap to produce" and "cheap to trust" had quietly split into two different numbers.

The Fix — Swap Effort for Verification Cost

We now score backlog items as (Reach × Impact × Confidence) / Verification Cost. Verification Cost isn't dev-hours. It's an estimate of what it takes a skeptical reviewer — someone who assumes the agent might be quietly wrong — to actually trust the output before it ships. Three things drive it: how many test cases or golden examples it takes to cover the correctness space, how much domain risk sits behind getting it wrong (money, safety, legal, customer trust), and how big the blast radius is if a wrong answer slips through unnoticed.

The 60-Second Checklist

Before you trust an Effort or Verification Cost number on your next roadmap review, run the item through three questions:

• ▹Does a golden set already exist? If a skeptical reviewer has to invent test cases from scratch to know it's right, Verification Cost is high — no matter how fast the code got written.
• ▹What's the blast radius if it's subtly wrong? Silent data corruption, money movement, or a customer-facing message you can't unsend scores high. A glitch someone spots and fixes in five minutes scores low.
• ▹Is correctness checkable by a machine, or does it need human judgment across a long tail of cases? Deterministic logic with clear invariants is cheap to verify even when it's slow to build. Tone, nuance, and open-ended judgment calls are expensive to verify even when they're instant to produce.

If the answers come back "no golden set, high blast radius, judgment call" — treat that RICE score as noise until you've priced in the review. Doesn't matter how good the Confidence number looks.

The Arc

The lesson here isn't "RICE is broken, burn it down." It's that every prioritization framework bakes in an assumption about where the cost in your system actually lives — and RICE was built for an era where that cost lived in typing the code. On an AI-augmented team, the cost moved to verifying the code, and the formula never got the memo. Your job isn't to memorize a better formula. It's to keep noticing when the cost has quietly moved somewhere else — because it will keep moving. Tomorrow: why every other framework you inherited from PM culture is making a similar bet you haven't checked in years.

Extend your knowledge

• ▹Re-score your next five backlog items with Verification Cost instead of dev-hour Effort estimates and see which ones flip position — that flip is the conversation worth having with your team.
• ▹Look at how LLM-ops teams build eval suites and golden datasets to make verification cheap and repeatable — same mental model, just applied to Verification Cost.
• ▹For anything with high blast radius and judgment-based correctness, scope the golden set or review checklist before writing the ticket — not after the agent hands you a diff.
• ▹Go reread the original RICE writeup (Intercom's product blog, credited to PM Sean McBride) and notice how many of its assumptions were written for a world before agents wrote code. That's the habit Day 12 builds on.