Your rate limiter has to work across twenty gateway instances. What does Redis do, and what breaks first?
Redis gives twenty gateways one counter mutated atomically through INCR or a Lua script, so rate limiting is fleet-wide. What breaks first is usually latency, not throughput: a slow Redis stalls every request for its timeout. Hot keys, caller clocks inside scripts and counts lost on async failover follow.
Answer
Short answer
Redis is the shared counter. Twenty gateways cannot each count locally, or a 100-a-minute limit becomes 2,000; with Redis, every instance mutates one key atomically, through INCR or a Lua script, so rate limiting holds across the fleet. What breaks first is rarely raw throughput. It is latency: a slow Redis makes every request wait for the limiter's timeout. Hot keys, gateway clocks passed into scripts, and increments lost on failover come next.
Was this answer helpful?
Likely follow-ups
- Your token-bucket script takes the current time as an argument from each gateway. One gateway's clock is two seconds fast. What does that tenant get?
- How would you split one tenant's hot limiter key across shards, and what accuracy do you give up?
- The Redis primary fails over and the replica is missing the last second of increments. Which limits does that matter for?
- How do you choose the timeout on the gateway's call to the limiter store, and what number do you measure first?
Related questions
- Your rate limiter runs in five regions with a cap of 100 requests a minute per user. One user gets 500 through. Where did the counting go wrong?hardAlso on rate-limiting and redis5 min
- You need to enforce 100 requests per minute per API key across a fleet of forty servers. Which limiting algorithm, and where does the counter live?hardAlso on rate-limiting and redis8 min
- Rate-limit an LLM gateway where one request can cost a hundred times another.hardAlso on rate-limiting5 min
- You must ingest all of English Wikipedia every day, and the source does not want to be hammered. What runs, how often, and what do you do about the run that fails at hour six?hardAlso on rate-limiting6 min