Top 4 Token bucket Interview Questions
Browse 4 real-world technical and behavioral interview questions about Token bucket. Review scenarios, edge cases, and architectural best practices.
One tenant bursts to ten times their normal traffic and every other customer's latency doubles. Your global rate limit was never hit. How would you design for fairness instead?
A global limit protects the service from overload but says nothing about how capacity is divided, so one tenant can consume most of it while staying under the ceiling. Fairness needs per-tenant accounting: a token bucket for the steady-state contract, a concurrency limit so expensive requests cannot hog workers, and round-robin queueing so a burst queues behind itself.
Design a rate limiter that allows N requests per client per minute. Write the class.
A rate limiter machine coding solution should reject fixed-window boundary bursts, usually with a token bucket. It needs no per-request history and stores two numbers per client, but only if refill is computed lazily from a timestamp rather than by running a background thread. Use this rate limiting answer to show the decision, trade-off, and evidence rather than a memorised definition.
Your rate limiter runs in five regions with a cap of 100 requests a minute per user. One user gets 500 through. Where did the counting go wrong?
Global rate limiting fails when each region owns a local counter, because five regions each allowing 100 requests creates a 500-request ceiling. Use one atomic shared counter, route each user to an owning region, or lease regional budget deliberately. It also connects distributed counters to the point an interviewer is testing.
You need to enforce 100 requests per minute per API key across a fleet of forty servers. Which limiting algorithm, and where does the counter live?
Fixed windows permit twice the limit across a boundary; the two sliding windows fix that at a memory or an accuracy cost; token bucket permits a bounded burst and leaky bucket smooths output instead. Across a fleet the counter must sit in one store mutated atomically, or each node silently enforces its own limit.