Your autoscaling is configured exactly as designed and the service still browns out every Monday at nine. Where is the time going?

The policy is not the problem, the latency of the policy is

An autoscaler is a control loop, and every control loop has a response time. The mistake behind this symptom is treating "we autoscale" as equivalent to "we have capacity", when what you have is a mechanism that begins acquiring capacity some time after it is already needed. If the surge is faster than the loop, the surge is served entirely by the fleet that was running before it began, and the autoscaler's contribution arrives in time for the recovery rather than the incident.

So decompose the delay out loud, because each term is a separate lever and most of them are longer than people expect. Metrics are aggregated over a window, so a one-minute average cannot report a spike until the minute is largely over. The policy then requires the breach to persist for an evaluation period before acting, which exists to stop it reacting to noise. Provisioning begins: an instance is allocated, boots, pulls an image, and starts a process. The process initialises — connection pools, configuration fetch, dependency injection, and for a JIT-compiled runtime a period of interpreting hot paths before they are compiled. Caches are empty, so its early requests are slower and hit dependencies harder than a warm instance's. Finally it must pass enough health checks to be added to the load balancer's pool, and a conservative check adds its own interval times its threshold.

Add those honestly and a plausible total is several minutes. If Monday's traffic reaches its peak in ninety seconds, no adjustment to the policy closes that gap, because you cannot make provisioning and warm-up negative.

The binding constraint is often not the tier you are scaling

The example worth citing is Slack in January 2021, which came back from the holidays into a traffic surge that interacted badly with the scaling behaviour of AWS Transit Gateway, and found the network between its own tiers had become the constraint rather than the application. That is the general lesson for a scaling question: capacity that arrives reactively takes time to arrive, and the moment it is needed is exactly the moment it is slowest to appear.

It also points at the second failure, which is that scaling the app tier can make things worse. Sixty new instances each open a connection pool, and the database's connection limit is a fixed number that does not respond to your scaling policy at all — you have converted a queue in your app tier into connection refusals at the database, which fails every instance including the ones that were previously healthy. The same shape appears with a third-party API's rate limit, with a licence-limited component, with source-port exhaustion on a shared network gateway, and with a shared cache whose bandwidth is now split more ways.

The discipline is to enumerate what each new instance consumes from something that is not elastic, and to check that the ceiling on that thing is above your maximum instance count times its per-instance draw. Where it is not, either the pool is bounded per instance so the total stays under the limit, or a proxy is introduced to multiplex connections, or the maximum instance count is capped to what the dependency can survive. A scaling policy with no upper bound is a mechanism for finding the most fragile thing you own.

Buy the floor you need, and schedule it

Since the loop cannot be made faster than provisioning, the capacity has to already be there. For a surge that recurs on a known schedule, the answer is unembarrassing: raise the minimum instance count ahead of it. Scale on a schedule to Monday's expected floor at half past eight, and let reactive scaling handle only the difference between your forecast and reality, which is the job it is actually good at.

State the cost, because interviewers listen for whether you know you are spending money. You are paying for capacity that is idle for part of the window, and the size of that bill is the size of your forecast error plus your safety margin. That is the correct trade when the alternative is failing requests during your highest-value hour, and it is a poor trade for a service whose traffic is genuinely unpredictable, where the money is better spent on making startup faster.

Which is the other lever, and the more durable one. Every second removed from cold start is a second removed from the gap on every future surge. Pre-built images with dependencies baked in rather than fetched, lazy initialisation of anything not needed to serve the first request, a snapshot or ahead-of-time compiled runtime to cut warm-up, and pre-populating only the cache entries that matter rather than all of them. Where the platform supports keeping a small number of pre-initialised instances in reserve, that is a floor by another name and priced accordingly.

Scale on something that means saturation

CPU utilisation is the default trigger and it is a poor proxy for a service that spends its time waiting. A request handler blocked on a database call consumes almost no CPU, so a fleet can be entirely saturated — every worker occupied, requests queueing, latency climbing — at a CPU utilisation that no threshold will ever trip. The autoscaler is watching a number that is not the number.

Scale on what runs out first instead. For a request service that is in-flight requests per instance, or the depth of the accept queue, or active workers as a fraction of the pool. For a consumer, the age of the oldest unprocessed item, expressed in time rather than in count, since time is what your objective is stated in. By Little's Law, concurrency equals arrival rate times service time, so a concurrency signal responds to both a rise in arrivals and a slowdown downstream, whereas a request-rate signal is blind to the second and a CPU signal is blind to both.

Include a latency-based guardrail as well, but as an emergency trigger rather than the primary one, because it is a lagging indicator: by the time latency has moved enough to be statistically clear, the queue has already built.

When the loop makes it worse

The failure to walk through, because it turns a slow start into a repeating one, is oscillation. Traffic dips, the scale-in policy removes instances promptly to save money, traffic returns, and the fleet is now cold and short at once. Each cycle costs you the full warm-up penalty, and if the removed instances held the shared cache's working set, the remaining ones pass the miss load straight to the database.

Worse is the version where the scaling action causes the metric it reacts to. New cold instances are slower, so latency rises, so the policy scales out further, so more cold instances arrive, and each round adds load on the shared dependencies that all of them are hammering to warm up. Asymmetric behaviour is what breaks the cycle: scale out quickly and scale in slowly, with a long cooldown, on the correct grounds that being briefly over-provisioned costs money while being briefly under-provisioned costs requests. Pair that with a load balancer that ramps a new instance's share gradually rather than routing to it at full weight the moment its first health check passes, so warming up is something the fleet absorbs instead of something the new instance is punished for.

Reactive scaling has a response time made of a metric window, an evaluation period, provisioning and warm-up, and any surge faster than that sum is served by the capacity you already had — so schedule the floor for the surge you can predict, scale on a signal that means saturation rather than CPU, and check what each new instance takes from the one dependency that will not scale with you.