My Adoption Dashboard Said 100%. My Engineers Were Still Doing It By Hand.

Day 9: The PMF Signal Hiding in Your Workarounds

Your dashboard is green. Daily actives up, agent runs up, review time per PR trending down. And right now, somewhere on your team, someone is copy-pasting your agent's output into a spreadsheet and rebuilding it by hand. That spreadsheet is the real signal. Your dashboard is measuring exposure, not fit — and exposure doesn't pay the bills.

The dashboard vs. reality gap

We'd built an internal review pipeline — three agents stitched together. One flagged risk in the diff, one caught test coverage gaps, one wrote a plain-English summary for the reviewer. The usage chart was a straight line up and to the right: daily active engineers climbing, agent runs climbing, review time per PR dropping. If I'd put that chart in a board deck, it would have said this works. Then I sat next to an engineer during an actual review. I watched them run the agent, read the output — and then rebuild the whole thing by hand in a shared spreadsheet. Severity column. Owner column. A checklist copied from last sprint's doc. The agent's output was a first draft they didn't trust enough to act on directly. It counted as "used" on my dashboard. It wasn't trusted anywhere that mattered.

The trap: usage measures exposure, not fit

Almost every founding EM who ships an internal, semi-mandated tool falls into this one. Once the tool is the default — or the mandate — "usage" stops measuring preference and starts measuring compliance. People open it because it's in the workflow, not because it's winning them over. AI tooling makes this worse: agent calls get wired into CI, pasted into standups, written into onboarding docs, and the exposure numbers climb on their own without anyone actually choosing anything. An agent that fires automatically on every PR shows 100% adoption even if every output gets thrown away. "Agent ran" and "agent's output shaped a decision" are two completely different events. Most telemetry only ever counts the first one.

The tell: look at what people build around your product

The thing that told me the truth wasn't in any dashboard. It was the spreadsheet. So I went and looked — really looked — at what my team was building around the tool, not just what they clicked inside it. That spreadsheet had columns our agent output didn't: an owner, a priority the team actually used day to day (not the agent's generic severity label), a status for "waiting on design" that our tool had no concept of. That workaround wasn't laziness. It was a precise, unpaid spec. It told me exactly which job our tool failed to do. The Slack threads said the same thing in a different accent — engineers pasting agent summaries into a channel, then rewriting the actionable part themselves, because the raw output wasn't shaped like something you could assign to a person and track to done.

Why interviews wouldn't have caught this

I asked directly, in 1:1s: "how's the review tool working for you?" Almost every answer: "yeah, it's fine, catches the obvious stuff." Nobody was lying. Stated preference and revealed preference just live in different parts of the brain — judging a tool in the abstract and quietly routing around it in practice are two separate tasks. "Do you like this?" gets a polite, generalized answer. "What do you actually do when the job still has to get done and no one's watching?" is a much harder question to fake your way through — and the workaround answers it whether you ask or not. I'd bet the same gap is sitting inside a lot of AI-copilot NPS scores right now: people rate the assistant highly in the survey and quietly don't ship its code unreviewed.

The pivot

We stopped calling the spreadsheet a rogue workaround and started treating it as the real spec. We rebuilt the agent's output around the job the spreadsheet was actually doing: an owner field, status states beyond pass/fail (including "blocked"), priority language lifted from how the team already triaged bugs instead of a generic confidence score. We didn't bolt on features — we swapped the shape of our output for the shape people were already hand-rolling every day. Adoption didn't tick up. It became real. And I knew it because the spreadsheet disappeared on its own. Nobody announced they'd stopped using it. We just noticed, a few weeks later, that it was gone.

The diagnostic: 3 questions to audit your own feature this week

• ▹Where does data leave your product right after someone uses it — into a spreadsheet, a doc, a second chat window with an LLM, a personal note? That export is a job your product didn't finish.
• ▹What manual step survives after you've "automated" something? If someone still hand-verifies, reformats, or retypes the agent's output before acting on it, you haven't built automation — you've built a suggestion generator with extra steps.
• ▹What do your support pings and Slack DMs actually ask for? Not the formal feature requests — the quick "@here, does anyone know how to just get X out of this" messages. Those are workaround requests wearing a disguise.

Tie back to the arc

This is PMF-testing you can run today, with data already sitting in your Slack history and shared drives — no interview script, no NPS survey, no waiting on a research team's calendar. As a founding EM with a captive internal team, I had a version of this most founders never get: I could watch revealed preference directly instead of inferring it from funnel drop-off. Tomorrow: why this kind of fit doesn't hold still — how workflow shifts quietly expire the fit you just found, and how to make sure you're not shipping last lesson's answer to a question that's already changed.

Extend your knowledge

• ▹This week, run the 3-question audit on one feature you own: export destinations, surviving manual steps, and recent support pings. Write down what you find before you form a theory.
• ▹If you run an internal AI or agent tool, add a lightweight "was this output used as-is?" signal — even a thumbs up/down next to the output. It's cheap and it separates exposure from trust in the data itself.
• ▹Look at Teresa Torres's "opportunity solution tree" for a structured way to turn workaround observations into a testable backlog, not just an anecdote.
• ▹Next time you're in a 1:1, skip "is this tool working for you" and ask "walk me through the last time you used it" instead. Behavior recall surfaces the workaround; opinion questions don't.