hardDesignSeniorStaff
Design an autoscaling scheme for a self-hosted CI runner fleet. What are you actually trading off, and how do you decide the numbers?
4 min readUpdated 2026-08-15
Answer
Short answer
Runner autoscaling trades idle compute cost against queue time, and the two levers that decide the trade are a warm pool sized to your usual concurrency and a scale-down delay long enough to absorb the next job without a fresh cold start.
Was this answer helpful?
Likely follow-ups
- How does the design change if a large fraction of jobs need a GPU that most jobs don't?
- What happens to this scheme during a mass rebase event when three hundred PRs queue jobs within a minute of each other?
- How would you decide between scaling by queue depth versus scaling by a predicted schedule, such as expected commit volume by time of day?
- What's the failure mode if your scale-down delay is set too long, and how would you notice it happening?
Related questions
- Pods are being evicted during node pressure even though your CPU dashboards look fine. Where do you look, and what would you change so it stops?hardAlso on kubernetes4 min
- Every rolling update drops a small number of requests. Where do they go?hardAlso on kubernetes4 min
- How would you autoscale a GPU inference service?hardAlso on autoscaling6 min
- Your autoscaling is configured exactly as designed and the service still browns out every Monday at nine. Where is the time going?hardAlso on autoscaling6 min