mediumConceptDesignScenarioMidSeniorLead
Which tests should run on every commit and which should run nightly, and how do you decide?
7 min readUpdated 2026-08-15
Answer
Short answer
Give the blocking stage a fixed time budget and put only fast, deterministic, developer-diagnosable tests inside it; everything slow, environment-dependent or exploratory moves to a scheduled run with a named owner.
Was this answer helpful?
Likely follow-ups
- Your per-commit stage has grown to 35 minutes. What comes out first?
- The nightly suite has been red for three weeks. How do you recover it?
- How do you keep a shared staging environment usable by two teams running suites at once?
- What would make you gate a deploy on a performance test?
Related questions
- Your pipeline gates every code change. What else reaches production without passing through it?hardAlso on ci-cd and quality-gates6 min
- How do you decide what to automate and what to leave manual?mediumAlso on test-automation6 min
- Your test suite takes fifty minutes and every merge waits for it. How do you fix that?mediumAlso on parallelisation5 min
- Your API suite runs on every commit against a shared database. How do you keep the tests from corrupting each other's data?hardAlso on test-data6 min