Posts tagged Rate Limiting — AI × software engineering, made practical.
This post explains why exponential backoff with jitter fails when many coding agents share one API key, and how to replace it with a token-based lease coordinator, DAG-weighted queueing and checkpointed agent turns.
A rate limiter that never rejected a single request took down four services at once — because 'queue everything' and 'protect the API' turned out to be two very different jobs.
One 429 turned into a fan-out storm in under ten minutes — here's the assumption baked into every rate limiter that agentic callers blow straight through, and the three fixes that actually hold.