Do the arithmetic before you name a fix
Take the premise at 40,000 requests a second and say the rebuild takes 200 milliseconds. Every request that arrives while the rebuild is in flight also misses, because nothing has been written yet. That is 8,000 requests in the gap, and every one of them starts its own copy of the same query. The database is not serving 8,000 different questions. It is serving one question 8,000 times.
State that number out loud in an interview. It changes the shape of the answer, because it shows the problem is duplication rather than volume. You do not need a bigger database. You need 7,999 of those requests to stop asking.
The same arithmetic tells you when this is not worth solving. If the rebuild takes two milliseconds, the gap holds 80 requests, and 80 duplicate queries is a spike your database will not notice. Cache stampede is a function of rebuild latency, not of popularity alone.
One rebuild, and everybody else waits or reads stale
Two families of fix, and the choice between them is a product decision rather than a technical one.
The first is to serialise the rebuild. The first request to miss takes a short-lived lock on the key, does the work, writes the value, releases the lock. Every other request finds the lock held and waits, then reads the freshly-written value. SET key:lock token NX EX 5 is the usual shape: create only if absent, expire after five seconds so a crashed holder cannot block the key for ever. Everybody gets fresh data. Everybody also pays the full rebuild latency, so your p99 for that second is 200 milliseconds rather than a cache hit.
The second is to serve the stale value while one request refreshes behind it. Store the value with its own logical freshness time and keep it in the cache past that point. A reader past the freshness time returns the old bytes immediately and triggers an asynchronous refresh. Latency stays flat for everyone, and readers see data up to one rebuild old. HTTP names this pattern stale-while-revalidate, and the same idea applies inside an application cache with no HTTP involved.
Say which you are choosing and why. "This is a product listing, so I serve stale for 200 milliseconds; the account balance gets the lock." That single sentence does more for a candidate than a list of every technique in existence.
The stampede you cause next week
Fixing the rebuild does not fix the pattern that produced it. If every key is written with the same lifetime by the same warm-up job, they all expire together, and the expiry itself becomes a synchronised event. Add jitter at write time so the herd disperses.
import random
BASE_TTL = 300
ttl = BASE_TTL - random.randint(0, BASE_TTL // 10)
cache.set(key, value, ex=ttl)
The better version of this refreshes early rather than expiring at all. Each reader decides, with a probability that rises as the value ages, to rebuild it in the background and return the current copy. Hot keys are read often, so a hot key gets refreshed before it ever expires, and cold keys quietly age out. Nothing has to know in advance which keys are popular; read frequency selects them.
Locks fail, and a lock nobody releases is worse than a stampede
The single-flight answer is only complete once you say what happens when the rebuilding request dies. Suppose it times out against the database. The lock is still held until its own expiry. Every waiter is blocked on a value that is never coming, and if they wait naively the stampede has been converted into a stall, which is harder to diagnose because the database looks healthy throughout.
Three things make that survivable. The lock carries a short expiry so it cannot outlive a dead holder. Waiters have their own deadline, shorter than the lock's, after which they return whatever stale copy exists or an error rather than waiting again. And the value of a failed rebuild is itself worth caching briefly, because a key that cannot be built will be requested 40,000 times a second whether or not you have decided what to do about it.
A lock also assumes both parties agree on who holds it. Delete the lock with a check that the token is still yours, not with an unconditional delete. Otherwise a slow holder whose lock has already expired will happily release the lock a second holder now owns.
Naming which of the three problems you have
Three failures wear the same costume in an incident channel, and the fixes do not overlap.
- Expiry stampede. One popular key reaches the end of its life and the herd rebuilds it. This is the question as asked, and single-flight or stale-serving is the fix.
- Eviction pressure. The working set no longer fits, so keys are evicted before they expire and the miss rate rises across the board. No amount of locking helps; the cache is too small or holding the wrong things.
- Cold start. A restarted or flushed cache has nothing in it, so everything misses at once. The fix is warming, or admitting traffic gradually, or both.
An interviewer who has run this incident is listening for the second and third to be ruled out rather than for a recital of the first. The tell that separates a strong answer is asking what the miss rate did before the spike, because a stampede is a vertical line on that graph and eviction pressure is a slope.
The cache did not fail. It did what a cache does at expiry, and the design forgot that a rebuild takes time, during which every arriving request is also a miss.
© 2026 Preptima. Originally published at preptima.com (opens in a new tab).