Rate Limiting & Resilience interview questions

Throttling algorithms, circuit breakers, retries with jitter, bulkheads, and graceful degradation.

5 questions in System Design.

System DesignA 24-minute read on how the topic fits together, before the questions.System design fundamentals60 short answers on one page, for revising rather than studying.

hard5

hardDesignConceptScenario

One tenant bursts to ten times their normal traffic and every other customer's latency doubles. Your global rate limit was never hit. How would you design for fairness instead?

A global limit protects the service from overload but says nothing about how capacity is divided, so one tenant can consume most of it while staying under the ceiling. Fairness needs per-tenant accounting: a token bucket for the steady-state contract, a concurrency limit so expensive requests cannot hog workers, and round-robin queueing so a burst queues behind itself.

5 minmid, senior, staff

hardScenarioDesign

A configuration change that passed review takes the entire fleet down in under a minute. What in your design should have limited the damage?

No retry policy or circuit breaker helps here, because the change was valid and the defect was already deployed. What bounds the damage is treating configuration as a rollout with stages and bake time, a kill switch whose own dependencies cannot fail with it, and a measured rollback time.

7 minsenior, staff, lead

hardScenarioDesign

How do you isolate a degraded dependency and halt a cascading failure before thread pool exhaustion takes down the entire microservice ecosystem?

Analyzing architectural resilience patterns, including circuit breakers, bulkheads, adaptive load shedding, and the mitigation of retry storms. Use this distributed systems answer to show the decision, trade-off, and evidence rather than a memorised definition. It also connects resilience to the point an interviewer is testing.

2 minsenior, staff

hardDesignScenario

Your rate limiter runs in five regions with a cap of 100 requests a minute per user. One user gets 500 through. Where did the counting go wrong?

Global rate limiting fails when each region owns a local counter, because five regions each allowing 100 requests creates a 500-request ceiling. Use one atomic shared counter, route each user to an owning region, or lease regional budget deliberately. It also connects distributed counters to the point an interviewer is testing.

5 minmid, senior, staff

hardDesignConcept

You need to enforce 100 requests per minute per API key across a fleet of forty servers. Which limiting algorithm, and where does the counter live?

Fixed windows permit twice the limit across a boundary; the two sliding windows fix that at a memory or an accuracy cost; token bucket permits a bounded burst and leaky bucket smooths output instead. Across a fleet the counter must sit in one store mutated atomically, or each node silently enforces its own limit.

8 minmid, senior, staff