A launch is a transient, the test measured a steady state
Most load tests ramp over ten or twenty minutes, hold a plateau, and judge the plateau. That design deliberately discards the warm-up, because warm-up numbers are noisy. A launch is almost entirely warm-up. An email, a push notification or a press embargo lifting at 09:00 puts most of the first hour's users into the first few minutes.
Several things are cold at that moment and were warm during the plateau. Application caches are empty, so every request reaches the database. Connection pools are opening connections rather than reusing them, and TLS handshakes are at their peak. Runtimes with a JIT are still interpreting hot paths. CDN edges have not cached the new assets. The autoscaler sees CPU rise, waits for its evaluation window, and adds instances that themselves start cold, by which time the queue has already formed.
The neighbouring page on modelling a realistic workload covers cache warmth and arrival models for a known system. The launch case is harder, because each of those effects peaks together in the first minutes and the test averaged them away.
The mix was a forecast copied from a normal day
Workload models are built from production telemetry. A launch has none, so the team copies the mix of the existing product or guesses one. That mix is dominated by browsing: reads, searches and cached pages.
Launch traffic is dominated by first-time actions. Account creation, email or SMS verification, password setting, token issuance, onboarding writes, and the first load of a personalised home page that no cache holds yet. These are write-heavy, touch the authentication service and the primary database, and often call third parties. A test where 5 per cent of requests are sign-ups says almost nothing about a morning where 60 per cent are.
The measurement hid the stall
Two measurement choices flatter a system under exactly the conditions a launch creates, and any performance analysis that trusts them starts from the wrong numbers.
A closed workload uses a fixed pool of virtual users, each waiting for a response before sending the next request. When the system slows, the offered load drops with it, so the test applies less pressure precisely when production would apply more. Real launch users are an open workload: they keep arriving whether or not earlier requests have finished. Tools support both models, for example k6's arrival-rate executors and Gatling's open injection profiles, but the default scripts many teams start from are closed.
Coordinated omission is the related instrument error. A generator that waits for each response before sending the next one stops sending during a stall, so the requests that would have suffered the stall are never sent and never recorded. A ten-second pause becomes one slow sample instead of hundreds, and the p99 looks healthy. The pass criteria often make it worse: latency computed over successful requests only, or an average, turns a partial outage into a green build.
Where the generator sat matters too. A generator inside the same network, bypassing the CDN, WAF and load balancer, never exercised the components that face the public first.
Limits the test environment could not reach
Some limits only exist at production scale, and a smaller environment cannot hit them however hard it is driven.
Third-party quotas are the commonest. The payment provider, the SMS gateway, the email service and the maps API were stubbed in the test, because sandboxes are rate-limited or charge per call. At launch the real provider throttled you, and your code had no plan for a 429 except retrying it.
Fleet-size effects are the second. Each instance opens its own connection pool, so total connections scale with the number of instances. Forty pods with a pool of twenty is 800 connections, against a database configured for 500. A test environment of four pods opens 80 and never sees the limit. The same shape applies to per-account cloud API quotas and to anything that coordinates between instances.
Retries multiply the load at the worst moment
When responses slow, mobile apps retry on timeout, browsers are refreshed by impatient users, and upstream services retry their calls. Each of these multiplies the offered load at exactly the point the system has stopped coping. A closed-model test cannot produce this, because its users wait politely. Production users do not, so a system that would have recovered from a brief spike instead stays saturated by its own retry traffic.
Why a bigger test would have passed too
The tempting conclusion is that the test was too small, so the next one should run more users. A test with ten times the virtual users and the same closed model, the same stubs, the same plateau and the same smaller fleet passes again, because volume was never the gap.
The useful work is a diff. Put the test script next to the first fifteen minutes of launch telemetry and compare them line by line: arrival rate per second, endpoint mix, share of new users, distinct keys requested, calls to each third party, error codes by dependency, and which component's queue grew first. Every row that differs is a test you did not run. Turn each into a specific rehearsal with a prediction, such as "an open-model spike to the launch-minute arrival rate against cold caches will exhaust database connections before CPU", so the next result can prove you wrong.
That is the core of performance analysis after a failed launch: explain the gap between instrument and reality before you trust the instrument again.
A green load test proves the system survives the traffic you scripted. After a failed launch, the question is how the script differed from the launch, and the answer is almost never "it needed more users".
© 2026 Preptima. Originally published at preptima.com (opens in a new tab).