Loading...
Loading...
Browse 14 real-world technical and behavioral interview questions about Capacity planning. Review scenarios, edge cases, and architectural best practices.
Scale on queue depth or in-flight concurrency rather than request rate, because a GPU replica that takes minutes to load weights cannot be added after the queue has already formed. Then decide deliberately how much warm capacity to keep, since that is what you are paying to avoid cold starts.
Reactive scaling has a latency of its own — metric window, evaluation, provisioning, warm-up, health checks — and a surge that arrives faster than that latency is served entirely by the capacity you already had. The fix is a scheduled floor plus a saturation-based signal, not a more aggressive policy.
Managed service vs self-hosted decisions compare operational load, control, lock-in, availability and real cloud cost. The hidden costs are usually egress, cross-zone traffic, idle capacity and the team needed to run the component well. It also connects managed services to the point an interviewer is testing.
Size it first: 200M daily actives opening the feed six times a day is 42k reads/s at peak, while 100M posts fanned out to 200 followers each is 231k feed writes/s, so writes dominate. Push to ordinary accounts, pull for high-follower ones, and store ids so deletes and blocks filter on read.
Clarify scale first, then size it: 100M new links a day at a 100:1 read ratio gives roughly 1.2k writes/s and 116k redirects/s, needing 7-character base62 keys. Generate keys from leased counter blocks scrambled by a bijection, serve reads from cache and CDN, and redirect with 302 rather than 301.
A framework's job is to make the trade-off legible, not to produce a verdict: ring-fence hard-dated and reliability work first, score the discretionary remainder on one agreed model, and communicate each no as what it displaces plus the condition that would reverse it.
Write down requests per day, input tokens per request and expected output tokens, multiply by the provider's per-million rates, then test each assumption. Output tokens cost several times input, retries and agent loops multiply the input, and a cached prefix rewrites the arithmetic.
Measure where the time is going before setting any split, publish the allocation as a policy with a rationale rather than a percentage you invented, route interrupts through one named person, and justify debt work by the delivery cost it removes instead of asking for a refactoring sprint.
Multi-tenant placement at scale should use measured resource cost, not tenant count. Keep the long tail in shared pools, move heavy tenants to reserved or dedicated capacity, and design tenant migration as a normal operation because every placement decision ages quickly.
Changing a shard key moves rows between shards, so it is a copy-and-delete across nodes rather than an UPDATE. Put a bucket-to-shard directory in front of the data, move one bucket at a time behind a freeze measured in milliseconds, verify each bucket before flipping its ownership, and keep every bucket independently reversible.
Ramp load steadily past the target until throughput stops rising, record the knee and the collapse point separately, and watch bounded resources such as threads, file descriptors and connection pools, because the first hard limit is often not request rate at all.
Divide the all-in hourly cost of the serving capacity by the predictions it delivers in that hour at your real utilisation, then add the per-request cost of feature reads. Utilisation dominates the result, and most of what people call cost per prediction is fixed cost that one more request does not change.
Choose the layer by who else can reuse the same bytes and how tolerable staleness is that far from the user, then size it from the working set rather than the dataset. Watch the miss rate, not the hit rate: origin load is proportional to misses, so 99 percent falling to 95 is a fivefold traffic increase.
Earn the right to the claim by attributing infrastructure cost per tenant, then separate cost caused by their usage shape from cost caused by your own design. Architecture can change the shape; the decision about who pays is commercial, and it needs the engineering evidence to happen at all.