Slopsquatting defenses

The most convincing paragraph in one of our PRs last month was defending a package that doesn't exist. The agent had added one import to our FastAPI service and, as our AGENTS.md asked, explained why in the PR description. Here it is, word for word:

### New dependency: fastapi-jwt-middleware

Added `fastapi-jwt-middleware` to validate bearer tokens on the /admin routes
instead of hand-rolling the check. The package is under active maintenance and
widely used in production FastAPI services. It keeps auth declarative and
consistent with our existing dependency-injection pattern.

All three sentences sound reasonable. Nobody has ever published anything called `fastapi-jwt-middleware`. The model that made up the name also made up the reason for it, in the same response, from the same weights. It didn't look up "active maintenance" or "widely used in production" anywhere. Justifications usually say things like that, so the model wrote them.

We only caught it because `pip install` failed. That was luck. If someone had registered the name the week before, the install would have worked, the tests would have passed, and the most persuasive thing in the PR would have been the paragraph vouching for a malicious package.

Open your AGENTS.md and find the dependency section. If it says something like "explain any new dependency in the PR description", check whether that's your only control. A lot of teams added that line this year and called the problem handled. It isn't. When you ask a model to justify a hallucinated import, the hallucination shows up a second time, and the second version reads better than the first.

Here's what the rest of this post argues. A prose explanation from an agent is more model output. It becomes a control only when three things are true. It's made of facts a machine can check: registry URL, first-publish date, maintainers, weekly downloads and the exact symbol imported. A tool call fetched those facts. And a separate verifier with no write access checks them again against the live registry in CI. Below is the stack I'd put into a repo this week, with what each layer catches and what gets past it.

Why this works for attackers, in 90 seconds

If hallucinated package names were random noise, nobody would bother squatting them. They aren't random. In "We Have a Package for You!" (USENIX Security 2025), Spracklen et al. took prompts that had produced hallucinated package names and ran each one ten more times. 43% of the hallucinated names came back in all ten runs. So the model produces these names consistently. An attacker can run the prompts developers commonly use, collect the names that keep coming back, register them on PyPI or npm and wait. Seth Larson called this slopsquatting. The name stuck because the attack is cheap and it scales. The attacker doesn't have to fool your engineers, because your agent already picked the name for them.

Step 1: Close the doors. One registry, and the agent installs only through it

Pick one internal registry endpoint. Use whatever you already run: Artifactory, Nexus, a devpi instance, Verdaccio for npm. Every package manager on every machine the agent touches points at that endpoint and nothing else. Never add `extra-index-url`. That flag is how dependency confusion gets in, and agents love copying it out of READMEs.

# /etc/pip.conf (agent sandbox image + CI runners)
[global]
index-url = https://pkgs.internal.example.com/api/pypi/pypi-remote/simple
# no extra-index-url. If you need a private package, publish it to the same proxy.

# pyproject.toml (uv)
[[tool.uv.index]]
name = "internal"
url = "https://pkgs.internal.example.com/api/pypi/pypi-remote/simple"
default = true

# .npmrc (committed at repo root)
registry=https://pkgs.internal.example.com/api/npm/npm-remote/

Then make the registry annoying to route around. Claude Code deny rules work as a speed bump:

{
  "permissions": {
    "deny": [
      "Bash(pip install --index-url:*)",
      "Bash(pip install -i:*)",
      "Bash(pip install --extra-index-url:*)",
      "Bash(uv pip install --index-url:*)",
      "Bash(npm install --registry:*)",
      "Bash(npm config set:*)",
      "Bash(pip install git+:*)",
      "Bash(pip install https:*)"
    ]
  }
}

Prefix rules are easy to get around. Reorder the flags, set an environment variable or call `python -m pip`, and the rule stops matching. Network egress is what actually enforces this. The agent sandbox and the CI runners should be able to reach your internal registry and nothing else: not pypi.org, not files.pythonhosted.org, not registry.npmjs.org. The deny rules give the agent a readable error, and the egress rule is what blocks the traffic.

**What it catches:** installs from rogue indexes, extra-index dependency confusion, and `pip install` from a git URL or tarball the agent found in a blog post. Names that don't exist anywhere fail fast with a clear error. You also get one place that logs every package fetch, which Step 5 depends on.

**What it misses:** any real package on the public registry, including a slopsquatted name someone registered last Tuesday. A pass-through proxy fetches whatever you ask for. Curation features like Artifactory Curation, Sonatype Firewall or a policy layer like Socket narrow that gap, but they cost money and setup time. Step 1 gives you one door you control. It doesn't filter anything.

Step 2: Replace prose with a block a machine can check

Delete "explain any new dependency" from AGENTS.md. Replace it with a structured block whose fields the agent fills by running a tool, not from memory. Give it a script, and tell it to paste the output exactly as printed:

## New dependencies (AGENTS.md)

For every package you add to a manifest or lockfile:
1. Run `scripts/dep-facts <ecosystem> <package>` and paste its output verbatim
   into the PR's `dependency` block. Do not edit the fetched fields.
2. Fill `imported_symbols` with the exact `module:Name` you import.
3. Fill `alternative_considered` with a package already in our lockfile, or `none`.
If `dep-facts` returns NOT_FOUND, remove the dependency. Do not look for a
similarly named package without telling the human in the PR.

<!-- .github/pull_request_template.md -->
### Dependency block (required if lockfile changed)
```dependency
- package:
  ecosystem:            # pypi | npm
  registry_url:
  first_published:      # date of earliest release, from registry
  maintainers: []
  weekly_downloads:
  imported_symbols: []  # module:Name
  alternative_considered:
  evidence: []          # raw URLs dep-facts fetched
```

Below are two filled-in blocks. The values in the first one are illustrative. In practice, whatever `dep-facts` fetched at PR time goes there. Notice how clean the second one looks.

# GOOD: produced by a tool call
- package: pyjwt
  ecosystem: pypi
  registry_url: https://pypi.org/project/PyJWT/
  first_published: 2011-xx-xx        # earliest upload_time in releases
  maintainers: [jpadilla]
  weekly_downloads: <pypistats last_week>
  imported_symbols: ["jwt:decode", "jwt:PyJWKClient"]
  alternative_considered: none (already a transitive dep via msal)
  evidence:
    - https://pypi.org/pypi/pyjwt/json
    - https://pypistats.org/api/packages/pyjwt/recent

# HALLUCINATED: written from the model's memory
- package: fastapi-jwt-middleware
  ecosystem: pypi
  registry_url: https://pypi.org/project/fastapi-jwt-middleware/
  first_published: 2021-03-14
  maintainers: [fastapi-contrib]
  weekly_downloads: 48000
  imported_symbols: ["fastapi_jwt_middleware:JWTBearerMiddleware"]
  alternative_considered: python-jose (less actively maintained)
  evidence: []

A sharp reviewer might spot the empty `evidence` list and the suspiciously round download count. Don't count on that. Tell a model to include evidence and it will write evidence URLs that look right. The block is still a claim. The difference is that its fields can be compared against the registry by a machine. Step 3 is what turns it into a control.

**What it catches:** nothing on its own, because it's only a format. It gives the verifier an exact target, and it makes the agent fetch real data, so honest mistakes show up before the PR opens. **What it misses:** everything, if nobody checks it.

Step 3: The verifier re-fetches every field and fails on any mismatch

The verifier runs in CI with a read-only token. It gets the diff and the dependency block. It never sees the author agent's transcript, plan or reasoning. You might use an LLM here too, say to summarize an odd package's README for the reviewer. If you do, it gets the same inputs, no write tools and no power to approve anything. Deterministic code decides pass or fail. The verifier ignores what the author said and fetches the facts itself.

It does four things. (1) It parses the lockfile diff and fails if any newly added top-level package has no block. Otherwise the agent could pass by leaving the block out. (2) It fetches every field from the registry and compares. (3) It downloads the package as an archive and checks that each `imported_symbols` entry is actually defined there. It never installs the package or runs its code. (4) It prints the facts it fetched, so the human reviewer reads CI's version, not the agent's.

#!/usr/bin/env python3
# scripts/verify_deps.py: runs in CI, read-only token, no package installs.
# Never trusts the PR block: every field is fetched again here.
import ast, io, json, sys, zipfile, urllib.request, urllib.error

PYPI = 'https://pypi.org/pypi/{}/json'
STATS = 'https://pypistats.org/api/packages/{}/recent'

def fetch(url):
    try:
        with urllib.request.urlopen(url, timeout=15) as r:
            return r.read()
    except urllib.error.HTTPError as e:
        if e.code == 404:
            return None
        raise

def first_published(meta):
    uploads = [f['upload_time'] for files in meta['releases'].values() for f in files]
    return min(uploads)[:10] if uploads else None

def defined_names(wheel_bytes, module):
    # Parse source with ast. Never import, never run setup.py.
    names = set()
    path = module.replace('.', '/')
    with zipfile.ZipFile(io.BytesIO(wheel_bytes)) as z:
        for cand in (f'{path}.py', f'{path}/__init__.py'):
            if cand in z.namelist():
                tree = ast.parse(z.read(cand))
                for node in tree.body:
                    if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
                        names.add(node.name)
                    elif isinstance(node, ast.Assign):
                        names |= {t.id for t in node.targets if isinstance(t, ast.Name)}
                    elif isinstance(node, ast.ImportFrom):  # re-exports
                        names |= {a.asname or a.name for a in node.names}
                return names
    return None  # module not in wheel at all

def verify(block):
    errs, name = [], block['package']
    raw = fetch(PYPI.format(name))
    if raw is None:
        return [f'{name}: NOT_FOUND on registry']
    meta = json.loads(raw)

    if first_published(meta) != str(block['first_published']):
        errs.append(f'{name}: first_published claimed {block["first_published"]}, '
                    f'registry says {first_published(meta)}')

    stats = json.loads(fetch(STATS.format(name)) or '{}')
    actual = stats.get('data', {}).get('last_week', 0)
    claimed = int(block['weekly_downloads'])
    if not (0.5 * actual <= claimed <= 2 * actual):  # counts drift; lies don't
        errs.append(f'{name}: weekly_downloads claimed {claimed}, actual {actual}')

    # maintainers: compare against author/maintainer fields + project URLs.
    # PyPI's JSON API does not expose owner roles; flag for human, don't pass.

    wheel = next((u for u in meta['urls'] if u['packagetype'] == 'bdist_wheel'), None)
    if wheel is None:
        errs.append(f'{name}: no wheel, sdist-only: route to human review')
    else:
        wb = fetch(wheel['url'])
        for sym in block['imported_symbols']:
            module, _, attr = sym.partition(':')
            names = defined_names(wb, module)
            if names is None:
                errs.append(f'{name}: module {module} not in published package')
            elif attr not in names:
                errs.append(f'{name}: {sym} not found (dynamic export? needs human)')
    return errs

if __name__ == '__main__':
    blocks = json.load(open(sys.argv[1]))     # extracted from PR body
    new_pkgs = set(json.load(open(sys.argv[2])))  # from lockfile diff
    missing = new_pkgs - {b['package'].lower() for b in blocks}
    errors = [f'{p}: in lockfile diff but no dependency block' for p in missing]
    for b in blocks:
        errors += verify(b)
    print('\n'.join(errors) or 'all dependency claims verified')
    sys.exit(1 if errors else 0)

Run it on the hallucinated block and it stops at the first fetch: `fastapi-jwt-middleware: NOT_FOUND on registry`. Now say an attacker has registered the name. The block's dates and download counts are still the model's inventions, so they won't match the registry. The symbol check fails too, unless the attacker happened to define `JWTBearerMiddleware` in a module at exactly that path. So the attacker has to guess the model's hallucinated API as well as its hallucinated name. Hallucinations repeat, so that's doable, but it's a lot more work.

**What it catches:** packages that don't exist, invented metadata, symbols missing from the published code, and new packages without a block. **What it misses:** an honest, accurate block about a malicious package. If the agent really did fetch the facts, they'll match. The verifier can tell you the claims are true. It can't tell you the package is safe. Dynamically created symbols (module-level `__getattr__`, star re-exports) need a human, and the script sends them to one instead of passing them.

npm works the same way. `https://registry.npmjs.org/<name>` gives you `time.created` and `maintainers`, and `https://api.npmjs.org/downloads/point/last-week/<name>` gives you downloads. To check symbols, read `exports`/`main` and the `.d.ts` files from the tarball `npm pack` downloads. Always pass `--ignore-scripts`.

Step 4: Age and downloads are risk signals, not gates

Once you have verified first-publish dates and download counts, you'll be tempted to write "auto-approve anything older than 30 days with more than 1,000 weekly downloads." Don't.

A patient attacker gets past both thresholds easily. Names cost nothing to register, so they register early and ship harmless releases for a couple of months. A 30-day clock just means setting a calendar reminder. Downloads are even easier to fake. A cron job running `pip download` from a few cloud regions pushes a package past 1k a week, and neither registry can tell a scripted fetch or an uncached CI run from a real user. Hallucinated names that keep recurring also pick up real downloads from other people's agents, so the better the attack works, the more legitimate the package looks. And once a threshold auto-approves, the attacker knows exactly what number to hit.

Use the numbers to decide how closely a human looks, never whether one looks:

# scripts/risk_tiers.yaml: sorts review depth. Nothing here approves anything.
red:      # reviewer reads source of the imported module before approving
  any_of:
    - first_published_days < 90
    - maintainers_count == 1 and first_published_days < 365
    - no_source_repo_link
    - repo_link_does_not_reference_package_name
    - name_edit_distance_to_popular_pkg <= 2
    - has_install_scripts            # npm preinstall/postinstall, sdist-only
yellow:   # reviewer checks repo, release history, maintainer changes
  any_of:
    - weekly_downloads < 10000
    - maintainer_added_in_last_90_days
    - major_version_jump_since_last_seen
green:    # still goes through Step 5 if new to the org
  otherwise: true

**What it catches:** young, single-maintainer and typo-adjacent packages move to the front of the queue. A signal like "maintainer added last month" reaches the person who will actually read it. **What it misses:** anything an attacker has deliberately aged and inflated. That's why this step ranks packages and never approves them.

Step 5: Anything new to the org goes to a named human

The question isn't whether a package is new to this repo. It's whether anyone in the company has ever depended on it. If `httpx` already runs in 40 services, an agent adding it to the 41st is boring. If `fastapi-jwt-middleware` has never shown up anywhere in your org, one specific person should look at it.

Keep a first-seen ledger in a central repo. CI resolves the full lockfile, direct and transitive, against it. If any package isn't in the ledger, the build fails until a ledger entry is merged. The ledger file is owned through CODEOWNERS:

# org/dep-ledger: ledger.yaml
- id: pypi:pyjwt
  first_seen: 2024-05-02
  first_repo: payments-api
  tier: green
  approved_by: '@platform-lead'
  notes: transitive via msal before direct use
- id: npm:zod
  first_seen: 2025-01-17
  first_repo: web-console
  tier: green
  approved_by: '@frontend-lead'

# org/dep-ledger: .github/CODEOWNERS
/ledger.yaml      @acme/dep-stewards   # two named people, not a 40-person team

# every app repo: .github/CODEOWNERS
/uv.lock              @acme/dep-stewards
/poetry.lock          @acme/dep-stewards
/package-lock.json    @acme/dep-stewards
/pnpm-lock.yaml       @acme/dep-stewards

# .github/workflows/deps.yml (excerpt)
- name: New-to-org check
  run: |
    python scripts/lock_diff.py --base origin/main --head HEAD > new_pkgs.json
    curl -sf -H "Authorization: Bearer $LEDGER_RO_TOKEN" \
      https://raw.githubusercontent.com/acme/dep-ledger/main/ledger.yaml > ledger.yaml
    python scripts/check_ledger.py new_pkgs.json ledger.yaml
    # exits 1 with: "pypi:fastapi-jwt-middleware is new to the org.
    #   Open a PR on acme/dep-ledger; a dep-steward must approve."

Turn on "Require review from Code Owners" in branch protection for both repos. CODEOWNERS on the lockfile alone is noisy. Every Dependabot bump pings the stewards, and they learn to approve without reading. The ledger points their attention at the rare case: a package nobody in the company has used before. If the Step 1 registry logs fetches, seed the ledger from that log on day one so you don't start with an empty list.

**What it catches:** the first appearance of any name anywhere in the org, transitive dependencies included. A named person is accountable for it, and you get a dated record you'll want during incident response. **What it misses:** a new, malicious version of a package already in the ledger. The ledger tracks names, not versions.

What this stack still misses

Everything above deals with one problem: an agent pulling in a name that shouldn't be there. Most serious supply-chain incidents look different, so be clear about where this stack stops.

• **A legitimate maintainer gets compromised.** The 2024 xz-utils backdoor came from an attacker who spent roughly two years earning co-maintainer trust. In the September 2025 npm incident, phished maintainer credentials were used to push malicious versions of packages like `chalk` and `debug`. Those packages were old, heavily downloaded and already in every ledger. They'd pass every check in Steps 3 to 5.
• **A transitive dependency gets hijacked.** A malicious patch release three levels down leaves every name in your lockfile alone and changes only versions and hashes. The verifier has nothing to compare, and the ledger already knows the name.
• **Install-time code.** npm lifecycle scripts and Python sdists run code at install time. If the agent's sandbox holds credentials, those scripts can read them.

Each of these needs its own controls, and they're worth having. Install with hashes (`uv lock` and `pip --require-hashes` give you this). Add a cooldown on brand-new versions (uv's `exclude-newer`, pnpm's `minimumReleaseAge`). Default to `--ignore-scripts`. Keep long-lived secrets out of agent sandboxes. Run a behavioral scanner that catches packages that suddenly start making network calls. The stack in this post makes hallucinated imports expensive. It does nothing about trusted code that turns malicious.

The general lesson: explanations count only when built from checkable tool results

Dependencies are just where this is easiest to see. Any time an agent explains itself, the explanation comes from the same process that made the change. Coherent reasoning shows that the model writes well, not that the reasoning is correct. An agent's explanation counts as evidence only when three things hold. It's built from tool-call results. Those results are specific enough for a machine to compare. And something that didn't write them does the comparing.

The pattern transfers almost unchanged. For a database migration, "this is backward compatible" becomes the schema diff plus the queries in the codebase that touch the changed columns, and CI runs it again against a snapshot. For an IAM change, "this only grants read access" becomes the policy simulator's output for a fixed list of actions, and the verifier runs that again. When an agent asks for broader permissions, "I need write access to deploy" becomes the exact API calls it plans to make, checked against what the requested role actually allows. Each time, you replace a paragraph you'd have to trust with fields someone else can re-fetch. In my research on multi-agent systems, I keep finding that this split matters more than how capable either agent is: one agent makes the claims, and a separate agent with no stake in the outcome checks them.

30-minute rollout, ranked by effort vs. coverage

Ship steps 1, 2 and 5 before lunch. Do 3 and 4 this week.

• **Step 1, registry config (5 min, high coverage for its cost).** Commit `.npmrc` and the `[[tool.uv.index]]` block, set `index-url` in the agent sandbox image and add the deny rules. If you can't do the egress rule today, open a ticket for it.
• **Step 2, PR template + AGENTS.md (10 min, zero coverage alone).** Swap "explain new dependencies" for the structured block and the `dep-facts` instruction. Until Step 3 exists it's only a format, but it changes what the agent produces starting today.
• **Step 5, CODEOWNERS + ledger (15 min, high coverage for new names).** Create `dep-ledger`, seed it from your current lockfiles (one script, one commit), add the CODEOWNERS lines and turn on code-owner review. On its own, with nothing else in place, this step would have stopped `fastapi-jwt-middleware`.
• **Step 3, verifier job (half a day, closes the loop).** Start with existence, `first_published` and the symbol check. Add downloads and maintainers once it's stable. Run it as a read-only job with no repo write token.
• **Step 4, risk tiers (an hour, improves human attention).** Post the tier in the PR comment so the steward knows whether this is a two-minute look or a read through the source.
• **Then, separately:** hashes, a version cooldown, `--ignore-scripts` and secrets kept out of the sandbox, for the threats this stack doesn't cover.

When that's done, go back to the line in AGENTS.md. "Justify new dependencies" can stay as a courtesy to reviewers. Just stop counting it as a control. The paragraph defending `fastapi-jwt-middleware` was the best-written thing in that PR, and the package never existed.