mediumDesignConceptMidSenior
When do you scale out instead of up, and what does that force you to change?
6 min readUpdated 2026-08-15
Answer
Short answer
Scale up while one machine can still hold the workload, because it costs no distributed complexity; scale out once you need redundancy or exceed the largest instance. Scaling out only works if the tier is stateless, which means externalising session state rather than pinning users to a node.
Was this answer helpful?
Likely follow-ups
- Your app tier scales linearly but throughput plateaus. How do you find what is actually saturated?
- Where would you put a consistent-hashing balancer instead of least-connections, and why?
- How do you drain a node for deployment without dropping in-flight work or long-lived connections?
- Two of eight nodes are unhealthy and the remaining six now fail their own health checks. What went wrong?
Related questions
- How do you get a browser test into the state it needs to start, without clicking through the UI to get there?mediumAlso on session-state6 min
- Your autoscaling is configured exactly as designed and the service still browns out every Monday at nine. Where is the time going?hardAlso on load-balancing6 min
- You move an internal service to gRPC behind the same load balancer your REST services use, and one pod ends up serving most of the traffic. What is happening, and what do you change?hardAlso on load-balancing6 min
- A flash sale decrements stock from many threads. Do you go optimistic or pessimistic, and how do you prove nothing oversold?mediumSame kind of round: design5 min