Parallel agents and git worktrees

Two of my agents reported green tests that morning. Both were wrong, and git had nothing to do with it. Agent 4 was passing its tests against data that agent 2 had seeded into the shared local Postgres. Earlier, agent 3 hit `EADDRINUSE :3000` and "fixed" it with `kill $(lsof -t -i:3000)`, which took down agent 1's dev server mid-test. Agent 1 retried, its tests flaked, and it filed the failure as "intermittent, unrelated to my change".

Each worktree was isolated at the file level, each branch had a clean diff, and each agent handed me a confident summary. I merged three of them before lunch. Main went red in a way none of the four branches could reproduce.

I lost most of the afternoon before it clicked: the code was fine. The machine wasn't. `git worktree add` gives each agent its own copy of the files. It doesn't give them their own ports, database, Docker network or caches. There's still exactly one of each, and five agents are fighting over them.

**Worktrees isolate files, not runtime.** Everything below follows from that sentence.

What a worktree doesn't own

This is the shared state that has burned me, along with what each one looks like in agent output. Agents almost never say "another process is using my database". They say "flaky test", and you believe them.

• **Ports (3000, 5173, 6006, 8080…).** If you're lucky you get `EADDRINUSE`. If you're not, you get no error at all. Vite with `strictPort: false` (the default) moves to 5174 and logs one line nobody reads, and the agent's e2e run hits *another agent's* server on 5173. You end up with "tests pass locally" on a branch that doesn't work.
• **Local Postgres / MySQL.** You'll see "migration already applied", "relation does not exist", and unique-constraint failures in seed scripts. The nastier version is tests that pass only because another agent's fixtures happened to be sitting there.
• **Local Redis.** Stale cache hits, rate limiters that trip for no reason, and queues where one agent's worker consumes another agent's jobs.
• **Docker Compose project name.** Compose takes the default from the directory name. `api-auth` and `api-billing` get different projects, so far so good. But two worktrees that share a basename in different parents collide, and every one of them still tries to publish `5432:5432`. You'll see "port is already allocated", or `docker compose down` removing someone else's containers.
• **Build caches outside the worktree** (`~/.cache/*`, a turbo `cacheDir` pointed somewhere shared, Playwright browsers in `~/.cache/ms-playwright`). A build "works" off a cache entry another branch produced, or trips over a half-written cache file from two concurrent writes.
• **Global pnpm store and postinstall side effects.** The content-addressed store copes fine with concurrent installs. Postinstall scripts that write outside the worktree (Prisma engines, browser downloads, native rebuilds) don't. The symptom is an install that fails once and passes on retry.
• **The `.env` you copied into every worktree.** It has the same `DATABASE_URL`, `PORT` and `REDIS_URL` in every worktree. This one file causes most of the list above, and it can show up as any symptom on it.

The slot contract: one integer per worktree

The fix is boring, and that's why it works. Each worktree gets one integer `N`, from 1 to 9. Slot 0 is my own checkout. Every piece of shared state is derived from `N`, no exceptions, and the values are written to a file the agent reads. Nobody, human or agent, gets to use a default.

# .env.slot for slot N (generated, gitignored)
WT_SLOT=N
PORT=3000+N*100            # slot 2 -> 3200
API_PORT=3001+N*100        # every extra service gets an offset inside its slot's 100-range
DATABASE_URL=postgres://.../<repo>_wN
REDIS_URL=redis://localhost:6379/N    # Redis ships with 16 logical DBs; 9 slots fit
COMPOSE_PROJECT_NAME=<repo>_wN
TURBO_CACHE_DIR=<main>/.wt/N/turbo

Two details matter. First, `.next/` and `node_modules/.vite` already live inside the worktree, so you get that isolation for free unless you've pointed them somewhere shared. I keep the turbo cache dir in the contract anyway because I want it pinned, not left to whatever this month's turbo decides. Second, if your `docker-compose.yml` publishes ports, parametrise those too (`"${PG_PORT:-5432}:5432"`) and add them to the slot file. A unique project name buys you nothing if every project wants host port 5432.

Why does process env beat your existing `.env`? Because dotenv, Next.js and Prisma all leave a variable alone if it's already set in the process environment. If you source `.env.slot` into the shell first, the slot values win. Your copied `.env` is left holding secrets, and that's the only job it ever should have had.

Copy-paste: `wt new` and `wt rm`

Here's the script I use, trimmed to the parts that matter. Put it on your PATH as `wt`. It assumes three things: pnpm, a local Postgres you can reach with `createdb`/`dropdb` (PG13+ for `--force`), and a `db:migrate` script in package.json. If your stack is different, change those lines.

#!/usr/bin/env bash
# wt — one worktree, one slot.   wt new <task> [--deps] | wt rm <task> | wt ls
set -euo pipefail
ROOT=$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")   # main checkout, even from inside a worktree
REPO=$(basename "$ROOT")
SLOTS="$ROOT/.wt/slots"; mkdir -p "$SLOTS"
export PGHOST=${PGHOST:-localhost} PGUSER=${PGUSER:-postgres} PGPASSWORD=${PGPASSWORD:-postgres}

wt_new() {
  local task=$1 deps=0 n="" dir="$ROOT/../$REPO-$1"
  [ "${2:-}" = "--deps" ] && deps=1
  for i in 1 2 3 4 5 6 7 8 9; do                 # mkdir is atomic: two callers can't claim the same slot
    if mkdir "$SLOTS/$i" 2>/dev/null; then n=$i; break; fi
  done
  [ -n "$n" ] || { echo "wt: no free slots" >&2; exit 1; }
  echo "$task" > "$SLOTS/$n/task"
  git -C "$ROOT" fetch -q origin main
  git -C "$ROOT" worktree add -q -b "agent/$task" "$dir" origin/main
  mkdir -p "$ROOT/.wt/$n/turbo"
  cat > "$dir/.env.slot" <<EOF
WT_SLOT=$n
WT_DEPS_OWNER=$deps
PORT=$((3000 + n*100))
API_PORT=$((3001 + n*100))
DATABASE_URL=postgres://$PGUSER:$PGPASSWORD@$PGHOST:5432/${REPO}_w$n
REDIS_URL=redis://localhost:6379/$n
COMPOSE_PROJECT_NAME=${REPO}_w$n
TURBO_CACHE_DIR=$ROOT/.wt/$n/turbo
EOF
  createdb "${REPO}_w$n"
  cd "$dir" && set -a && . ./.env.slot && set +a
  pnpm install --frozen-lockfile
  pnpm db:migrate
  echo "wt: $task -> slot $n  $dir  :$PORT"
}

wt_rm() {
  local task=$1 dir="$ROOT/../$REPO-$1" f n
  f=$(grep -lx "$task" "$SLOTS"/*/task) || { echo "wt: no slot for $task" >&2; exit 1; }
  n=$(basename "$(dirname "$f")")
  for p in $((3000 + n*100)) $((3001 + n*100)); do  # only listeners on this slot's ports, never anyone else's
    lsof -t -i:"$p" -sTCP:LISTEN | xargs -r kill || true
  done
  docker compose -p "${REPO}_w$n" down -v --remove-orphans 2>/dev/null || true
  dropdb --if-exists --force "${REPO}_w$n"
  redis-cli -n "$n" FLUSHDB >/dev/null 2>&1 || true
  git -C "$ROOT" worktree remove --force "$dir"   # branch stays; you still need to merge it
  rm -rf "$SLOTS/$n" "$ROOT/.wt/$n"
  echo "wt: freed slot $n"
}

case "${1:-}" in
  new) shift; wt_new "$@" ;;
  rm)  shift; wt_rm "$@" ;;
  ls)  grep -H . "$SLOTS"/*/task 2>/dev/null || echo "wt: no slots in use" ;;
  *)   echo "usage: wt new <task> [--deps] | wt rm <task> | wt ls" >&2; exit 1 ;;
esac

Add `.wt/` and `.env.slot` to `.gitignore`, and keep repo directory names lowercase because Compose rejects uppercase project names. Now look at the `lsof` loop in `wt rm`. It's nearly the same command agent 3 ran that morning. The difference is scope. It only touches ports this slot owns, and `-sTCP:LISTEN` limits it to the server actually listening there. Without that flag, `lsof -i:PORT` also returns any process with a client connection to the port. Killing a process is fine. Killing one that belongs to a different slot is the bug.

Make agents obey it

The script sets up the slot, but agents still have to use it. They won't unless the instruction sits somewhere they always read: `CLAUDE.md` for Claude Code, `AGENTS.md` for Codex and most other tools. I keep these lines in both:

## Runtime (multiple agents share this machine)
- Before running anything: `set -a; . ./.env.slot; set +a`. PORT, API_PORT, DATABASE_URL, REDIS_URL and COMPOSE_PROJECT_NAME come from there.
- Never hardcode ports, DB names or URLs, and never fall back to defaults like 3000 or `app_dev`.
- Never kill a process you didn't start. If a port is taken, stop and report it.

Why not just put this in the task prompt? Every task gets a different prompt, and the one time I forget the line is the time an agent reaches for `kill`. The repo file loads every session, for every agent and every tool, and I don't have to remember anything. It also changes what the agent does when it gets stuck. An agent that sees `EADDRINUSE` wants the error gone, and the quickest way to get rid of it is to kill whatever is on the port. A written rule turns "make it pass" into "stop and report". Stopping and reporting is exactly what you want from something that can't see the other four agents.

Lockfiles: the shared state slots can't fix

You can give every agent its own port and its own database. You can't give each one its own `pnpm-lock.yaml` and still merge them. When two branches both add a dependency, you get lockfile conflicts that no agent resolves well and no human wants to touch. Slots don't help here. Ownership does.

**Rule: one dependency-changing branch per batch.** Create that branch with `wt new <task> --deps`. Every other worktree installs with `--frozen-lockfile` (the script already does this) and is blocked from committing changes to manifests or the lockfile:

#!/usr/bin/env bash
# .githooks/pre-commit   (enable once: git config core.hooksPath .githooks)
grep -qx 'WT_DEPS_OWNER=1' .env.slot 2>/dev/null && exit 0
if git diff --cached --name-only | grep -E '(^|/)(package\.json|pnpm-lock\.yaml)$'; then
  echo "pre-commit: this worktree doesn't own dependencies." >&2
  echo "Drop the change and note the package you need in your summary." >&2
  exit 1
fi

`core.hooksPath` lives in the shared git config, so you set it once and every worktree picks it up. The error message is written for the agent. It says what to do next instead of just reporting a failure. Agents can skip hooks with `--no-verify`, so add a fourth line to `CLAUDE.md`/`AGENTS.md`: "Never use `--no-verify`." For a hard guarantee, run the same check in CI against the PR's diff. I also extend the hook to the `migrations/` directory, because a schema change is a lockfile by another name.

Merge order is lock semantics

The second half of the thesis is merge order. With one agent, it doesn't matter. With five, it decides whether main stays green. I treat it like taking locks: branches that change shared foundations merge first, one at a time, and everything else waits and re-checks.

• **1. Lockfile and migration branches first, and alone.** Keep one schema branch in flight at a time, and merge it before any feature branch that might touch the same tables.
• **2. Then leaf features, smallest diff first.** A small diff has the smallest blast radius and leaves the cheapest rebase for whoever's next. `git diff --stat origin/main...agent/<task>` gives you the order.
• **3. After each merge, every remaining branch rebases and re-runs tests.** Every one of them. A green run from before the last merge tells you nothing about main as it is now.
• **4. Two failed rebases means discard and re-prompt from the new main.** Don't hand-fix it. Run `wt rm`, then `wt new` with the same brief plus one line about what changed.

Why rule 4 is cheaper than it feels

Rule 4 is the one people push back on, and I pushed back on it too. Throwing away a branch that's 90% done feels wasteful. That instinct comes from a world where a human spent two days on the 90%. Here an agent spent minutes on it. Running the same brief again against the new main costs about the same few minutes, plus one line of context from me.

Rescuing a branch looks cheap, but it isn't. You read a diff you didn't write and work out why it no longer fits. You fix the conflict, then hit the second conflict the first one was hiding. All of that costs your attention, and when you're the bottleneck for four other agents, attention is the thing you're short of. Regenerating costs agent time, which is cheap and runs in parallel with everything else. Once I started tracking it, rescues kept taking longer than I'd guessed. Regenerated runs off a fresh main usually went green on the first try, which makes sense because the agent was working against code that actually exists now. Don't take my word for it, though. Log both times for a week and let the numbers from your own repo settle it.

Why two failures and not one? A single failed rebase is often a trivial conflict the agent can sort out by itself. A second one tells you the branch was built on assumptions that are no longer true. Patching it means fixing those assumptions one conflict at a time.

The limit: past ~4 agents, you are the merge queue

The slot contract scales to 9 slots without complaint. I don't. At around four concurrent agents, I spend more time reviewing, ordering merges and kicking off rebases than the agents spend producing work. A fifth slot just makes the queue in front of me longer.

Two things push that ceiling back. First, write task briefs with **non-overlapping file ownership**: "you own `src/billing/**` and its tests; don't edit anything else, list what you'd change outside it." Rebase failures come from overlapping ownership, and you can head off most of them in the brief. Second, stop adding slots once your merge queue sets the pace instead of agent throughput. If branches regularly wait more than one merge cycle before you look at them, you're running too many agents. Close slots until the queue drains within a cycle.

Checklist for your README

• 1. Start every agent with `wt new <task>`, never a bare `git worktree add`. Use `--deps` for at most one branch per batch.
• 2. Agents source `.env.slot` before every command. That rule lives in `CLAUDE.md` and `AGENTS.md` next to "no hardcoded ports", "never kill what you didn't start" and "never `--no-verify`".
• 3. `core.hooksPath` points at `.githooks/`, and the pre-commit hook blocks manifest, lockfile and migration changes outside the owner slot.
• 4. Task briefs name the files each agent owns, and concurrent agents never overlap.
• 5. Merge the deps or migration branch first and on its own, then leaf features, smallest diff first.
• 6. After every merge, all remaining branches rebase on main and re-run tests. Two failed rebases means `wt rm` and re-prompt.
• 7. Once a branch is merged, `wt rm <task>` drops the DB, frees the ports and releases the slot. Run `wt ls` before every new batch to make sure nothing has leaked.