How do you model a realistic workload for a performance test?

The model, not the script

Most performance testing fails before a single request is sent, because the workload was invented rather than measured. "Let us run 500 users" is not a model — it does not say what those users do, how fast they do it, or against what data. A defensible model has four parts, and each should be traceable to evidence.

1. Transaction mix

Take the actual distribution of requests from production over a representative period, usually access logs or APM data, and reproduce its proportions. The common failure is testing the endpoint that is easiest to script rather than the one that dominates traffic, which produces a green result that says nothing about the peak the system will actually face.

Pick the period deliberately. A retail system's model should come from its busiest realistic hour, not from a monthly average that flattens the peak you are trying to survive.

2. Arrival pattern and concurrency

This is where the reasoning is usually tested. Concurrency and throughput are not independent — they are related through time spent per request. Little's Law makes it precise:

concurrent users = throughput × (response time + think time)

So if the requirement is 200 requests per second, response time is 0.2 s, and real users pause about 5 s between actions, you need roughly 200 × 5.2 ≈ 1040 concurrent virtual users. Quoting a user count without think time is meaningless, and it is why the same "500 user" test can generate wildly different load in two tools.

There is a second-order point worth making: open versus closed workload models. A closed model has a fixed number of virtual users who wait for a response before sending the next request, so as the system slows, offered load automatically drops and the test flatters the system. An open model injects at a fixed arrival rate regardless of response time, so a slowdown causes queues to build — which is what actually happens with real internet traffic. Choose the open model when you are testing whether the system survives a spike, and say why.

3. Data

Test data shapes results more than most people account for, and there are three specific traps.

Cardinality. A thousand virtual users hammering the same product id will sit entirely in cache and report response times the production system will never see. Parameterise from a realistic distribution, including the long tail.

Volume. Query plans change with table size. A test against a database seeded with 50,000 rows tells you nothing about behaviour at 50 million, because the optimiser may choose a different plan entirely.

Cache warmth. Decide explicitly whether you are measuring warm or cold state, and either discard a ramp-up period or report it separately. A test that silently mixes both produces a bimodal distribution that makes percentiles uninterpretable.

4. Duration and shape

Short tests hide the failures that matter. Include a ramp-up so you can see where degradation begins rather than only whether the end state passed, a steady state long enough to expose resource leaks and garbage-collection pressure — an hour minimum for a soak, considerably longer if you are hunting a slow leak — and a ramp-down. Run a separate spike test if the system faces flash traffic, because a gradual ramp will not reproduce a queue that builds in ten seconds.

Reporting it honestly

Report percentiles, never the average. An average of 200 ms is consistent with everyone getting 200 ms and with ninety percent getting 50 ms while ten percent get 1.6 s, and only one of those is an incident. Give p50, p95, and p99, and give the maximum separately since it is a single-sample outlier rather than a distribution statistic.

Always publish the error rate alongside the latency, because they trade off against each other and a latency figure alone can be actively misleading. A system that starts returning 503s quickly will show excellent response times precisely because it stopped doing the work. A result of "p99 was 400 ms" with a 12 percent error rate is a failed test, and candidates who report only the first half of that sentence are the reason performance results get mistrusted.

Finally, tie the result back to a target that was agreed before the run. "p99 under 800 ms at 200 rps with error rate under 0.1 percent" is a pass-or-fail statement. A report with no pre-agreed threshold becomes a negotiation about whether the numbers are acceptable, which is a conversation you will lose after the fact.