92% line coverage. Green CI. An agent wrote the PR, the same agent wrote the tests, and nobody blinked. Then we ran mutation testing on it once, mostly out of curiosity, and 41% of mutants were killed. The other 59% were small bugs we planted on purpose, and the suite didn't notice any of them.
Almost every survivor was either a `<` that could become `<=` or a `return err` that the tests executed and never checked. The tests weren't lazy. They were mirrors. One agent wrote the code and the assertions in a single session, so the suite locked in whatever the code did, bugs included. All the coverage number proved was that the lines ran.
Here are three survivors from that run, trimmed down. Every line shown was covered, and every test passed against both the original code and the mutant.
// ── Survivor 1: boundary flip ─────────────────────────────
// settle.go
for attempt := 1; attempt < s.maxAttempts; attempt++ { // mutant: attempt <= s.maxAttempts
// settle_test.go: runs the loop, never reaches the limit
fake := &flakyGateway{failTimes: 2}
svc := NewSettler(fake, WithMaxAttempts(5))
_, err := svc.Settle(ctx, req)
require.NoError(t, err)
require.Equal(t, 3, fake.calls)
// ── Survivor 2: swallowed error ───────────────────────────
// settle.go
if err := s.ledger.Append(ctx, entry); err != nil {
return Receipt{}, fmt.Errorf("append ledger: %w", err) // mutant: if-body removed
}
return Receipt{ID: entry.ID}, nil
// settle_test.go: makes the ledger fail, checks the call, ignores the result
ledger.On("Append", mock.Anything, mock.Anything).Return(errors.New("disk full"))
svc.Settle(ctx, req)
ledger.AssertCalled(t, "Append", mock.Anything, mock.Anything)
// ── Survivor 3: negated condition ─────────────────────────
// settle.go
if inv.Currency != acct.Currency { // mutant: ==
amount = s.fx.Convert(amount, inv.Currency, acct.Currency)
}
// helpers_test.go: shared setup stubs FX as identity "so tests don't need rates"
fx.On("Convert", mock.Anything, mock.Anything, mock.Anything).
Return(func(m Money, _, _ string) Money { return m }).Maybe()Now read them the way you actually review at 6pm. Each one looks fine. The retry test checks the call count, the ledger test checks that the ledger got called, and the FX stub is just a convenience. And not one of them would catch the bug sitting a few lines away.
Here's where we landed. Coverage can't tell you whether agent-written tests are any good. Nightly whole-repo mutation runs are too slow for anyone to act on. What works is a diff-scoped mutation gate that runs on every PR in a few minutes. It mutates only the changed lines and their direct callers, and it fails on any surviving mutant instead of a percentage. The survivor list goes straight back to the agent as its next prompt. The human reviewer's job becomes judging the agent's "equivalent mutant" claims, because those claims are where spec ambiguity shows up.
The rest of this post is the recipe.
Why agent tests are mirrors
When one session writes both the code and the tests, the tests encode what the code does now, not what it was supposed to do. The agent has no independent source of truth. It reads the function it just wrote, works out what it returns, and asserts exactly that. If the function is off by one, the test is off by one in the same direction. Humans who write tests after the code do the same thing. The difference is volume: an agent does it on every PR, every time, and fast enough that nobody reads the output.
Four smells give this away, and you can grep for them tomorrow morning before installing anything:
- **Snapshot-everything.** Whole responses get piped through `toMatchSnapshot()` or a golden file. The snapshot was generated from the code under test, so it's just the code's output with extra steps.
- **Existence-only checks** like `toBeDefined()`, `toBeTruthy()` and `require.NotNil`. As long as something comes back, the test passes.
- **Mocks that repeat the implementation.** These are identity stubs, pass-through fakes, or mocks whose return value uses the same formula as the real code. Survivor 3 is one.
- **Expected values copied from actual output.** Watch for literals with eleven significant digits, specific UUIDs and timestamps. Nobody types `1043.2799999999997` on purpose.
# 1. Snapshot-everything: test files ranked by snapshot assertions
rg -c 'toMatchSnapshot|toMatchInlineSnapshot|cupaloy\.|goldie\.|snaps\.Match' \
-g '!node_modules' | sort -t: -k2 -nr | head -20
# 2. Existence-only checks
rg -n 'toBeDefined\(\)|toBeTruthy\(\)|not\.toBeNull\(\)|(assert|require)\.NotNil\(' -g '*test*'
# 3. Mocks that repeat the implementation: identity and computed returns
rg -nP 'mockImplementation\(\(?(\w+)\)? => \1\)|Return\(func\(' -g '*test*'
# 4. Expected values copied from output: suspiciously precise literals
rg -nP '(toEqual|toBe|Equal\(t,)\s*\(?\s*-?\d+\.\d{6,}' -g '*test*'These are heuristics. They give you a reading list, not a verdict. All four are symptoms of the same problem, and mutation testing catches that problem directly.
Mutation testing in 90 seconds
A mutant is a small bug you add on purpose: flip `<` to `<=` or `!=` to `==`, swap `&&` for `||`, or empty out an `if` body. Then you run the tests. If a test fails, the mutant is **killed**. If everything still passes, it **survived**, meaning the code changed and nothing noticed. Mutation score is the share of mutants killed. A mutant that changes the code without changing the behavior is **equivalent**, and no test can kill it. That's all the theory you need.
Recipe 1: mutate the diff, not the repo
Most teams I've seen try mutation testing once and give up, and it's usually because they pointed it at the whole repo. Ours ran as a nightly job and took about 40 minutes. Nobody reads a nightly report about code an agent merged yesterday and has already moved on from. So run it where someone can still act on the result: on the PR, scoped to what the PR changed.
TypeScript: Stryker
Stryker's `mutate` option accepts line ranges (`src/pricing.ts:40-52`). `--incremental` reuses results for mutants whose code and covering tests haven't changed. With both, the PR run mutates only what the diff touched. The config:
{
"$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
"testRunner": "vitest",
"checkers": ["typescript"],
"tsconfigFile": "tsconfig.json",
"reporters": ["json", "html", "clear-text"],
"incremental": true,
"incrementalFile": "reports/stryker-incremental.json",
"concurrency": 4,
"timeoutMS": 10000,
"timeoutFactor": 2,
"mutator": {
"excludedMutations": ["StringLiteral", "Regex", "ObjectLiteral", "ArrayDeclaration"]
},
"thresholds": { "high": 100, "low": 95, "break": null }
}`break` is `null` on purpose. Stryker's built-in threshold is a percentage, and we don't gate on percentages (Recipe 3 explains why). The TypeScript checker throws out mutants that don't compile before any test runs, which saves a surprising amount of time. On a PR, the mutate list comes from the diff:
npx stryker run --incremental --mutate "$(paste -sd, mutate-set.txt)"
# mutate-set.txt holds lines like: src/pricing/discount.ts:40-52Go: gremlins, plus go-mutesting for error paths
For Go we use gremlins. It has a `--diff` flag that limits mutation to changed code, it writes a JSON report, and it's fast enough for a PR check. If changed lines are all you want, `gremlins unleash --diff origin/main` is the entire recipe. We also mutate callers (Recipe 2), so we run it on each touched package and filter the JSON by line range ourselves.
Know what your mutator set actually covers. gremlins does arithmetic, boundary, negation and increment mutations. It doesn't remove branches, so it will never produce Survivor 2, the swallowed error. go-mutesting's `branch/if` mutator does, and it's the tool we used for that original one-off run. Its output is harder to scope to lines, so we run it nightly in report-only mode on the packages touched that day. Don't skip that job. In our Go services, error paths are where most of the bugs we found were hiding.
The GitHub Actions job
name: mutation-gate
on:
pull_request:
push:
branches: [main]
jobs:
mutation-ts:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 } # need the merge base for the diff
- uses: actions/setup-node@v4
with: { node-version: 22, cache: npm }
- run: npm ci
# Incremental baseline: the branch's own last run first, then main's.
- uses: actions/cache@v4
with:
path: reports/stryker-incremental.json
key: stryker-${{ github.head_ref || github.ref_name }}-${{ github.sha }}
restore-keys: |
stryker-${{ github.head_ref || github.ref_name }}-
stryker-main-
- name: Refresh baseline on main (off the critical path)
if: github.event_name == 'push'
run: npx stryker run --incremental
- name: Build mutate set
if: github.event_name == 'pull_request'
run: BASE=origin/${{ github.base_ref }} ./scripts/mutate-set.sh ts > mutate-set.txt
- name: Mutate the diff
if: github.event_name == 'pull_request'
run: |
[ -s mutate-set.txt ] || { echo "No TS changes to mutate"; exit 0; }
npx stryker run --incremental --mutate "$(paste -sd, mutate-set.txt)"
- name: Gate
if: github.event_name == 'pull_request'
run: node scripts/mutation-gate.mjs stryker mutate-set.txt reports/mutation/mutation.json
- name: Survivors to job summary
if: always() && github.event_name == 'pull_request'
run: cat survivors.md equivalent-claims.md >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || true
- uses: actions/upload-artifact@v4
if: always()
with: { name: stryker-report, path: reports/mutation/ }
mutation-go:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- uses: actions/setup-go@v5
with: { go-version-file: go.mod, cache: true }
- run: go install github.com/go-gremlins/gremlins/cmd/gremlins@latest # pin a version
- name: Build mutate set
run: BASE=origin/${{ github.base_ref }} ./scripts/mutate-set.sh go > mutate-set.txt
- name: Mutate touched packages
run: |
mkdir -p reports/gremlins
cut -d: -f1 mutate-set.txt | xargs -r -n1 dirname | sort -u | while read -r pkg; do
gremlins unleash --workers 4 --timeout-coefficient 3 --invert-logical \
--output "reports/gremlins/$(echo "$pkg" | tr / _).json" "./$pkg"
done
- name: Gate
run: node scripts/mutation-gate.mjs gremlins mutate-set.txt reports/gremlins/*.json
- name: Survivors to job summary
if: always()
run: cat survivors.md equivalent-claims.md >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || trueBefore: one 40-minute whole-repo run every night that nobody read. After: about 3 minutes on a typical PR, and the output is a short list of specific lines. The cache is what holds it near 3 minutes. Without a baseline restored from main, every branch's first run starts cold. Check the flag names against whichever gremlins version you pin, because its CLI has changed between releases.
Recipe 2: widen the scope just enough
A gate that only mutates changed lines has a hole in it, and agents find that hole faster than people do. They aren't being sneaky. They're optimizing for a green check. Tight scoping rewards tiny diffs: change one comparison in a helper, and the behavior change shows up three call sites away, in code the gate never touches. It also rewards pushing logic into places the gate skips, like glue code, config and generated files.
So we also mutate the direct callers of every function whose body changed. We stop there, at one hop. One hop covers the case where a helper's contract changed and the caller's tests should have caught it. Two hops brings back the 40 minutes.
The script reads changed lines from `git diff -U0` and pulls the names of changed functions from the hunk headers. Then it runs `git grep` to find call sites and adds the enclosing top-level block of each one. It's crude on purpose. It's about 40 lines of bash you can review in a PR, which beats a call-graph tool you have to take on faith.
#!/usr/bin/env bash
# scripts/mutate-set.sh <ts|go> -> file:from-to, one per line
# Changed lines in this PR + the enclosing top-level block of each direct caller
# of a function whose body changed. Generated and glue files are NOT excluded.
# Go: add `*.go diff=golang` to .gitattributes so hunk headers name the func.
set -euo pipefail
EXT=$1
BASE=${BASE:-origin/main}
MAX_CALL_SITES=${MAX_CALL_SITES:-15}
NOTEST=(":!*_test.go" ":!*.test.ts" ":!*.spec.ts" ":!*.d.ts")
{
# 1. Changed lines, new side of the diff
git diff -U0 --diff-filter=AMR "$BASE"...HEAD -- "*.$EXT" "${NOTEST[@]}" | awk '
/^\+\+\+ / { file = substr($0, 7); next }
/^@@/ {
split($3, a, ","); start = substr(a[1], 2); n = (a[2] == "" ? 1 : a[2])
if (n > 0) print file ":" start "-" start + n - 1
}'
# 2. Direct callers of functions whose bodies changed
git diff -U0 "$BASE"...HEAD -- "*.$EXT" "${NOTEST[@]}" \
| sed -nE 's/^@@ [^@]* @@ .*\b(func( \([^)]*\))?|function|const|class) +([A-Za-z_][A-Za-z0-9_]*).*/\3/p' \
| sort -u | while read -r sym; do
sites=$(git grep -nE "(^|[^A-Za-z0-9_])${sym}\(" -- "*.$EXT" "${NOTEST[@]}" \
| grep -vE "^[^:]+:[0-9]+:.*(func|function)( \([^)]*\))? +${sym}\(" || true)
count=$(printf '%s\n' "$sites" | grep -c . || true)
if [ "$count" -gt "$MAX_CALL_SITES" ]; then
echo "mutate-set: ${sym} has ${count} call sites, not expanding" >&2
continue
fi
printf '%s\n' "$sites" | grep . | while IFS=: read -r file line _; do
awk -v L="$line" -v F="$file" '
/^(export )?(default )?(async )?(func|function|class|const|let) / { if (NR <= L) s = NR }
/^}/ && NR >= L && s { print F ":" s "-" NR; exit }' "$file"
done
done
} | sort -uTwo things we learned from running this for a while. First, `MAX_CALL_SITES` matters. Without it, a changed `New()` or `toString()` pulls in half the repo, so the script says so on stderr and leaves the decision to the reviewer. Second, the caller blocks are usually unchanged code, so Stryker's incremental mode serves most of their results from cache. Widening the scope costs much less runtime than you'd expect.
Config is the hole this doesn't close, because mutation tools don't mutate YAML. Our rule: if a PR changes behavior through config, it ships with a test that loads the real config file, not a fixture. That keeps the code that reads the config in scope.
Recipe 3: zero survivors, not 80%
Don't gate on "score ≥ 80%". Agents game percentages. They don't mean to, but the cheapest way to move a ratio is to pad the denominator with easy kills: one more validation helper with an obvious test, a getter, a defensive branch anyone could kill. The score goes up while the one boundary that matters still survives.
Our rule is **zero surviving mutants on the scoped lines**. Every survivor gets either a new test that kills it or an equivalent annotation with a one-sentence reason a human can disagree with. `NoCoverage` counts as a survivor, and it's the loudest signal of the lot. One gate script handles both tools:
// scripts/mutation-gate.mjs
// usage: node mutation-gate.mjs <stryker|gremlins> mutate-set.txt report.json...
import { readFileSync, writeFileSync, existsSync } from 'node:fs';
const [tool, setFile, ...reports] = process.argv.slice(2);
const ranges = (existsSync(setFile) ? readFileSync(setFile, 'utf8') : '')
.split('\n').filter(Boolean)
.map((l) => { const [, file, a, b] = l.match(/^(.+):(\d+)-(\d+)$/); return { file, from: +a, to: +b }; });
if (ranges.length === 0) { console.log('Nothing in scope.'); process.exit(0); }
function* mutants() {
for (const path of reports) {
const r = JSON.parse(readFileSync(path, 'utf8'));
if (tool === 'stryker') {
for (const [file, { mutants }] of Object.entries(r.files))
for (const m of mutants)
yield { file, line: m.location.start.line, op: m.mutatorName, to: m.replacement ?? '', status: m.status };
} else {
for (const f of r.files ?? [])
for (const m of f.mutations)
yield { file: f.file_name, line: m.line, op: m.type, to: '', status: m.status };
}
}
}
// Timeouts count as killed. Compile errors / not-viable are ignored.
const SURVIVED = new Set(['Survived', 'NoCoverage', 'LIVED', 'NOT COVERED']);
const sameFile = (a, b) => a === b || a.endsWith('/' + b) || b.endsWith('/' + a);
const claimFor = (file, line) => {
const above = readFileSync(file, 'utf8').split('\n')[line - 2] ?? '';
return above.match(/equivalent-mutant:\s*(.+)$/)?.[1];
};
const survivors = [], claims = [];
for (const m of mutants()) {
if (!SURVIVED.has(m.status)) continue;
const r = ranges.find((r) => sameFile(m.file, r.file) && m.line >= r.from && m.line <= r.to);
if (!r) continue;
const to = m.to ? ` → \`${m.to.replace(/\s+/g, ' ').slice(0, 80)}\`` : '';
const row = `- \`${r.file}:${m.line}\` ${m.op}${to}`;
const reason = claimFor(r.file, m.line);
reason ? claims.push(`${row}: ${reason}`) : survivors.push(row);
}
writeFileSync('survivors.md', `### Surviving mutants (${survivors.length})\n${survivors.join('\n')}\n`);
writeFileSync('equivalent-claims.md', `### Equivalent-mutant claims (${claims.length})\n${claims.join('\n')}\n`);
console.log(`${survivors.length} surviving, ${claims.length} claimed equivalent`);
if (survivors.length) { console.log(survivors.join('\n')); process.exit(1); }We don't use Stryker's `// Stryker disable` comments for this. A lint rule fails the build if one appears. Every exception has to go through a `// equivalent-mutant:` comment on the line above, which means every exception ends up in `equivalent-claims.md` in front of the reviewer. Nothing gets skipped quietly.
**Timeouts.** A mutant that turns `i < n` into an infinite loop will time out. Stryker counts that as detected, and our gate treats gremlins' `TIMED OUT` the same way. That's right. The catch is that on shared CI runners, a slow test can time out by accident and pass for a kill. We set `timeoutFactor: 2` and `timeoutMS: 10000` in Stryker and `--timeout-coefficient 3` in gremlins, and we print the timeout count. If a PR has more than a handful, someone looks.
**Flaky tests.** This one runs against intuition. A flaky test doesn't turn the gate red. It makes the gate lie green, because a random failure "kills" whatever mutant happened to be running at the time. Before you turn the gate on, quarantine known-flaky tests out of the mutation run. A separate vitest project or a Go build tag does the job. Otherwise you're measuring noise and calling it a score.
The loop: survivors are the next prompt
This step is what made the gate useful instead of merely annoying. `survivors.md` is already a precise bug report: file, line, operator, replacement. That's a better prompt than most humans write. So it goes straight back to the agent, usually the same one that opened the PR.
Here's the prompt template. Steal it:
The mutation gate found surviving mutants on lines this PR changed or calls into.
A surviving mutant is a deliberate small bug that no test detected.
For each surviving mutant below, do exactly one of:
1. Add a test that FAILS on the mutant.
- Assert on an observable outcome: return value, returned error, emitted
event, persisted state. Do not assert only that a mock was called.
- Derive the expected value from the requirement or the PR description,
not by running the code and copying its output.
- If that test also fails on the current code, do not bend the test.
Report it as a suspected bug, with the requirement you used.
2. Argue it is equivalent in ONE sentence, with an annotation on the line
above the mutated line:
// equivalent-mutant: <operator> <why no input can tell the difference>
If your sentence depends on a behavior nobody has specified (what happens
exactly at the limit, on empty input, at midnight, in another timezone),
start it with "UNSPECIFIED:" and ask. Do not guess.
Do not edit, weaken or delete existing assertions.
Survivors:
{{survivors.md}}Here's one real round trip on the settlement code from the top of the post, after we turned the gate on. The PR gate caught the first two survivors. The nightly go-mutesting run caught the third, and we added it to the same prompt.
### Surviving mutants (3)
- `internal/settle/settle.go:58` CONDITIONALS_BOUNDARY → `<=`
- `internal/settle/settle.go:71` CONDITIONALS_NEGATION → `==`
- `internal/settle/settle.go:74` branch/if: if-body removed (nightly go-mutesting)// Agent round 1: two new tests, one claim
func TestSettle_GivesUpAfterMaxAttempts(t *testing.T) {
fake := &flakyGateway{failTimes: 100}
svc := NewSettler(fake, WithMaxAttempts(5))
_, err := svc.Settle(ctx, req)
require.ErrorIs(t, err, ErrGatewayUnavailable)
require.Equal(t, 5, fake.calls) // "retry up to 5 attempts", per PR description
}
func TestSettle_ReturnsLedgerError(t *testing.T) {
ledger.On("Append", mock.Anything, mock.Anything).Return(errDiskFull)
receipt, err := svc.Settle(ctx, req)
require.ErrorIs(t, err, errDiskFull)
require.Empty(t, receipt.ID)
}
// settle.go:70, the claim
// equivalent-mutant: CONDITIONALS_NEGATION UNSPECIFIED: the stub makes same-currency
// Convert an identity; is a real FX call for same currency allowed?
if inv.Currency != acct.Currency {The ledger test killed the swallowed-error mutant on the first try. The retry test did something better: it failed against the real code. The loop starts at `attempt := 1` and uses `<`, so it made four attempts, not five. The agent had taken "five" from the PR description, as the prompt told it to, and it flagged the mismatch instead of bending the test. That failure was a real bug. A mirror test would have happily asserted four. We changed the loop to `<=`, reran the gate, and got green: 0 survivors, 1 equivalent claim.
The third survivor was a different kind of finding. The claim wasn't exactly wrong. It was a question. In production our FX client makes a network call even when the currencies match, and nobody had ever decided whether that's allowed. That's the next section.
What changed in review
We stopped reading the agent's test code line by line. The gate already checks the thing we used to squint for, which is whether the tests notice when the code changes. Now we read `equivalent-claims.md`. It's short, every line is a sentence you can argue with, and it's where the interesting problems kept turning up.
Here are two where "equivalent" really meant "nobody decided what happens at the boundary":
- **Overdue invoices.** The mutant: `daysPastDue > 0` → `daysPastDue >= 0`. The agent's claim was "equivalent because daysPastDue is computed from UTC midnight and the overdue job runs at 01:00 UTC, so it is never 0 when the check runs." Each half of that sentence was true. Put together, they hid the real question: is an invoice overdue on its due date, and whose midnight are we talking about? Customers in UTC+7 were watching invoices go overdue at 8am local time on the due date. The fix was mostly a one-line spec decision (the due date is inclusive and evaluated in the account's timezone), plus a test that pins 23:59 and 00:00 local time.
- **Empty batches.** This was `len(lines) > 0` → `len(lines) >= 0`, just before the invoice builder, and the claim was "equivalent because the loop over an empty slice is a no-op." The loop was a no-op. The code after it wasn't. Without the guard, the builder created a zero-amount invoice and the notifier emailed it to the customer. Nobody had written down whether zero-line invoices should exist. Product said no. The spec got a sentence, the code got an explicit early return, and the test now asserts that no invoice is created and no email goes out.
Both times, the code did exactly what the tests said and the tests said exactly what the code did. The gap was in the spec. A one-sentence equivalent claim is the cheapest way I've found to put a spec gap in front of a human. We updated the PR template to match:
## Mutation gate
- [ ] Gate is green (0 surviving mutants in scope)
- [ ] I read every `equivalent-mutant:` claim in this PR
- [ ] Every `UNSPECIFIED:` claim links to a decision (spec, ticket, or comment)Rollout checklist
- **Run it report-only for two weeks.** Put `continue-on-error: true` on the gate step, look at survivors on real PRs, and tune the mutator list before anyone gets blocked.
- **Then gate agent-authored PRs only.** We key on a label the agent sets: `if: contains(github.event.pull_request.labels.*.name, 'agent')`. Humans can opt in. Mirror tests are mostly a problem of agent volume.
- **Keep an allowlist of mutators, not a blocklist.** Start with boundary, negation, logical operator, arithmetic, and block/branch removal. Drop the string literal mutators, which are where the log-message noise comes from, and drop regex and object/array literals too. Stryker's config only takes an exclude list, so write the allowlist down somewhere and exclude everything outside it. Only add a mutator back when a real bug slips through that it would have caught.
- **Track survivors per PR, not a repo-wide score.** The repo-wide number mostly measures legacy code nobody is touching. The per-PR number measures what you're merging today.
- **Watch equivalent claims per PR, and how many get rejected in review.** If the rejection rate climbs, the agent is using claims as an escape hatch. Tighten the prompt.
- **Make equivalent annotations a checkbox in the PR template** so that skipping them is a visible choice.
- **Refresh the incremental baseline on every push to main.** If you don't, the PR job slowly drifts back toward a cold run.
- **Quarantine flaky tests from the mutation run before you gate.** A flaky test reports kills that never happened.
After a quarter
I'll be careful with numbers here. We changed the gate and the review process in the same quarter, and I can't cleanly separate the two effects.
Mutation score on changed code stopped being an interesting number. By construction, the gate makes it 100% minus annotated equivalents on every merged agent PR. What we watch now is equivalent claims per PR and how many get rejected. Review time didn't really shrink. It moved: less of it goes to reading test code and more goes to arguing about boundaries, which is a better use of it. Escaped bugs changed shape too. Off-by-ones and dropped errors mostly vanished from our incident reviews. What still gets through is spec bugs, where the code and the tests agree with each other and are both wrong about what the customer needed.
That's the honest limitation. The gate checks whether the tests notice changes. It doesn't check whether the spec is right. Killing a mutant only pins down current behavior. If that behavior is wrong and the agent takes the expected value from the code, the new test locks the bug in tighter than before. That's why the prompt says to derive expected values from the requirement, and it's why the retry bug only surfaced because the PR description happened to say "five attempts". With no description, there's nothing to check against, and deciding what the code should do is still human work. What the gate does is show you exactly where that work is needed, one equivalent claim at a time.