Top 17 Capacity planning Interview Questions
Browse 17 real-world technical and behavioral interview questions about Capacity planning. Review scenarios, edge cases, and architectural best practices.
What is the KV cache, and why does it decide how many users one GPU can serve?
The KV cache stores the attention keys and values of every token a sequence has seen, so decoding need not recompute them. It grows with tokens per sequence and with concurrent sequences, and after the weights it fills GPU memory, so tokens in flight, not requests, set how many users one GPU serves.
In annual planning, how do you size engineering headcount across product lines, and how do you defend the split?
Headcount planning starts from measured run cost per product line, not last year's shares. Fund that first, allocate the rest by each line's posture, stage bets behind gates, and plan in effective capacity, because hiring lag and attrition mean new roles add far less than their number in year one.
"Enter a new market in twelve months." How do you turn that goal into engineering bets with a sequence and kill criteria?
Define what entering means, then split the goal into a few engineering bets, each with a hypothesis, a leading indicator and a cost. Sequence the cheapest learning first, release capacity through gates, and agree kill criteria in advance that fire on early signals, not month-twelve revenue.
How would you autoscale a GPU inference service?
Scale on queue depth or in-flight concurrency rather than request rate, because a GPU replica that takes minutes to load weights cannot be added after the queue has already formed. Then decide deliberately how much warm capacity to keep, since that is what you are paying to avoid cold starts.
Your autoscaling is configured exactly as designed and the service still browns out every Monday at nine. Where is the time going?
Reactive scaling has a latency of its own — metric window, evaluation, provisioning, warm-up, health checks — and a surge that arrives faster than that latency is served entirely by the capacity you already had. The fix is a scheduled floor plus a saturation-based signal, not a more aggressive policy.
How do you decide whether to use a managed service or self-host a component, and which cloud costs catch teams out?
Managed service vs self-hosted decisions compare operational load, control, lock-in, availability and real cloud cost. The hidden costs are usually egress, cross-zone traffic, idle capacity and the team needed to run the component well. It also connects managed services to the point an interviewer is testing.
Design the home feed for a social network.
Size it first: 200M daily actives opening the feed six times a day is 42k reads/s at peak, while 100M posts fanned out to 200 followers each is 231k feed writes/s, so writes dominate. Push to ordinary accounts, pull for high-follower ones, and store ids so deletes and blocks filter on read.
Design a URL shortener.
Clarify scale first, then size it: 100M new links a day at a 100:1 read ratio gives roughly 1.2k writes/s and 116k redirects/s, needing 7-character base62 keys. Generate keys from leased counter blocks scrambled by a bijection, serve reads from cache and CDN, and redirect with 302 rather than 301.
Every stakeholder says their item is urgent. How do you decide what goes into next quarter?
A framework's job is to make the trade-off legible, not to produce a verdict: ring-fence hard-dated and reliability work first, score the discretionary remainder on one agreed model, and communicate each no as what it displaces plus the condition that would reverse it.
Before we build this, how do you estimate what the LLM feature will cost to run?
Write down requests per day, input tokens per request and expected output tokens, multiply by the provider's per-million rates, then test each assumption. Output tokens cost several times input, retries and agent loops multiply the input, and a cached prefix rewrites the arithmetic.
How do you decide how much of your team's capacity goes to roadmap work rather than keeping the lights on and paying down debt?
Measure where the time is going before setting any split, publish the allocation as a policy with a rationale rather than a percentage you invented, route interrupts through one named person, and justify debt work by the delivery cost it removes instead of asking for a refactoring sprint.
Tenants share one cluster and the largest is a thousand times the smallest. How do you place them?
Multi-tenant placement at scale should use measured resource cost, not tenant count. Keep the long tail in shared pools, move heavy tenants to reserved or dedicated capacity, and design tenant migration as a normal operation because every placement decision ages quickly.
The table is 800GB and the shard key turned out wrong. How do you reshard while writes continue?
Changing a shard key moves rows between shards, so it is a copy-and-delete across nodes rather than an UPDATE. Put a bucket-to-shard directory in front of the data, move one bucket at a time behind a freeze measured in milliseconds, verify each bucket before flipping its ownership, and keep every bucket independently reversible.
How would you design a test to find where the system breaks, rather than one that confirms it meets its target?
Ramp load steadily past the target until throughput stops rising, record the knee and the collapse point separately, and watch bounded resources such as threads, file descriptors and connection pools, because the first hard limit is often not request rate at all.
What does one prediction cost you, and how would you work it out?
Divide the all-in hourly cost of the serving capacity by the predictions it delivers in that hour at your real utilisation, then add the per-request cost of feature reads. Utilisation dominates the result, and most of what people call cost per prediction is fixed cost that one more request does not change.
Where do you put the cache, and how big does it need to be?
Choose the layer by who else can reuse the same bytes and how tolerable staleness is that far from the user, then size it from the working set rather than the dataset. Watch the miss rate, not the hit rate: origin load is proportional to misses, so 99 percent falling to 95 is a fivefold traffic increase.
Your largest customer costs more to serve than they pay you. What do you do about that architecturally?
Earn the right to the claim by attributing infrastructure cost per tenant, then separate cost caused by their usage shape from cost caused by your own design. Architecture can change the shape; the decision about who pays is commercial, and it needs the engineering evidence to happen at all.