hardScenarioMidSeniorLead
Your test suite fails randomly about one run in five. How do you get it under control?
7 min readUpdated 2026-08-15
Answer
Short answer
Quarantine and measure before fixing: keep per-test pass history so you can rank offenders and restore a trustworthy signal, then work the root-cause families one at a time.
Was this answer helpful?
Likely follow-ups
- Three weeks in, the flake rate is down to one run in fifty. What do you change about the pipeline now?
- A test only fails when the suite runs in parallel with eight workers. How do you find out why?
- Who owns a quarantined test, and what stops the quarantine list becoming permanent?
- How would you tell a genuine intermittent product bug apart from a badly written test, from the failure output alone?
Related questions
- A test passes on your machine and fails in CI roughly once a week. How do you track it down?mediumAlso on flaky-tests and test-isolation5 min
- A test passes on its own and fails when the whole suite runs. How do you track that down?mediumAlso on test-isolation and flaky-tests5 min
- How do you decide what to automate and what to leave manual?mediumAlso on test-automation6 min
- Which tests should run on every commit and which should run nightly, and how do you decide?mediumAlso on test-automation7 min