Skip to content
QSWEQB
mediumScenarioDesignMidSeniorLead

Your test suite takes fifty minutes and every merge waits for it. How do you fix that?

Profile before cutting: suite time concentrates in a few tests, and the cause is usually repeated setup or real waiting rather than work. Parallelise what is independent, move the slow and rarely-failing out of the merge path, and treat deleting redundant tests as legitimate.

4 min readUpdated 2026-07-26

What the interviewer is scoring

  • Whether the candidate profiles the suite before proposing changes
  • That per-test setup cost is distinguished from the work the test does
  • Does the candidate address test independence before proposing parallelisation
  • Whether moving tests out of the merge path is paired with a stated risk accepted
  • That flaky tests are treated as a throughput problem and not only a correctness one

Answer

Why fifty minutes is worse than it sounds

Before optimising, be clear about what the number costs, because it is not fifty minutes.

A developer who cannot merge for fifty minutes does not sit and wait. They start something else, and the context switch back costs more than the wait did. Reviews queue up behind it. Changes get batched together to amortise the wait, so when the pipeline does fail it is failing on three changes at once and someone has to work out which. And because the loop is long, people stop running tests locally and let CI find out, which pushes even more traffic through the slow path.

There is also a flakiness multiplier that candidates rarely mention. With a one-in-fifty per-test failure rate and a suite of a few hundred tests, a meaningful fraction of runs fail for no reason, and each of those costs another fifty minutes. Fixing the top few flaky tests can reclaim more wall-clock time than any amount of optimisation, and it is usually the cheapest available fix.

Profile first

Suite runtime is almost never spread evenly. The usual distribution has a handful of tests accounting for a large share of the total, and the reason they are slow is usually not the work they do.

Look for three things. Per-test setup: a suite that starts a container, migrates a schema, or boots the application context once per test class is paying that cost hundreds of times, and the fix is to share it — one container for the suite, transactional rollback between tests rather than a rebuild. Real waiting: sleep calls inserted to stabilise a timing-dependent test are pure cost, and replacing them with a wait on the actual condition often removes seconds each. Accidental scope: a test that boots the whole application to exercise one validation rule, usually because it was easier to copy the neighbouring test than to find the seam.

These three routinely account for most of the runtime, and none of them requires deciding which tests to sacrifice.

Parallelise, once the tests allow it

Splitting across N machines is the biggest single lever and it has one prerequisite: the tests must be independent. Most suites are not, and finding out the hard way produces failures that look like flakiness and are actually contention.

Shared mutable state is the blocker. Tests that write to the same database rows, that assume an empty table, or that depend on execution order will fail intermittently once they run concurrently. The options, roughly in order of cost: a schema or database per worker, which is clean and needs the setup to be scriptable; a transaction per test rolled back at the end, which is cheap but does not work for anything that commits; or namespacing every test's data so tests cannot see each other's rows, which is the most work and the most robust.

Sharding by historical duration rather than by file count matters more than it sounds. An even split of files gives you one worker finishing in three minutes and another in twenty, and the suite takes twenty.

Move things out of the merge path

Not every test needs to gate a merge. The question for each group is what it costs to find that failure an hour later instead of now.

The tests that stay are the ones that fail often enough to catch real mistakes and are fast: unit tests, contract tests, a thin slice of integration coverage over the paths most changes touch. The tests that can move are the ones that are slow and rarely fail — full browser journeys across every supported browser, long-running data migrations, exhaustive matrix runs across versions.

Be explicit that this is accepting risk rather than eliminating work. Something will get merged that a nightly run then catches, and the team needs an agreed response for that — who looks at the nightly result, and whether a failure reverts or rolls forward. A nightly suite nobody reads is worse than no suite, because it produces the appearance of coverage.

Deleting tests is allowed

The option teams resist. A suite that has grown for years contains tests that assert the same behaviour several times over, tests for features that no longer exist, and tests so coupled to implementation that they fail on every refactor while catching nothing.

The question is not whether a test passes but what it would tell you if it failed. A test whose failure always means "someone renamed a method" is a tax on refactoring, and removing it increases the amount of real testing the team can afford. Coverage percentage is a poor guide here — three tests through the same branch add coverage of zero and runtime of three.

Do this deliberately and in review, not unilaterally, and pair it with the profile so the argument is about a named test that costs ninety seconds and duplicates another.

What to aim for

Ten minutes is the usual target for a merge-gating pipeline, and it is chosen because it is roughly the longest a person will wait rather than switch tasks. Getting there is normally the same sequence: fix the flakiest tests, share the expensive setup, remove the sleeps, parallelise, then move the slow tail out of the path.

Whatever you achieve, track it. Pipeline duration and flake rate belong on a dashboard the team sees, because suites do not become slow in a single change — they gain forty seconds a month and nobody notices until it is fifty minutes again in a year.

Profile before you cut. The fifty minutes is usually a handful of tests paying setup costs over and over, and fixing that costs nothing in coverage.

Likely follow-ups

  • You want to parallelise but tests share a database. What are your options?
  • Which tests would you move to a nightly run, and what do you lose?
  • How does a flaky test at one failure in fifty affect a fifty-minute pipeline?
  • When is deleting a test the right answer?

Related questions

citest-performanceparallelisationfeedback-loopsflakiness