Top 3 Autoscaling Interview Questions
Browse 3 real-world technical and behavioral interview questions about Autoscaling. Review scenarios, edge cases, and architectural best practices.
Design an autoscaling scheme for a self-hosted CI runner fleet. What are you actually trading off, and how do you decide the numbers?
Runner autoscaling trades idle compute cost against queue time, and the two levers that decide the trade are a warm pool sized to your usual concurrency and a scale-down delay long enough to absorb the next job without a fresh cold start.
How would you autoscale a GPU inference service?
Scale on queue depth or in-flight concurrency rather than request rate, because a GPU replica that takes minutes to load weights cannot be added after the queue has already formed. Then decide deliberately how much warm capacity to keep, since that is what you are paying to avoid cold starts.
Your autoscaling is configured exactly as designed and the service still browns out every Monday at nine. Where is the time going?
Reactive scaling has a latency of its own — metric window, evaluation, provisioning, warm-up, health checks — and a surge that arrives faster than that latency is served entirely by the capacity you already had. The fix is a scheduled floor plus a saturation-based signal, not a more aggressive policy.