Test impact analysis and selection
Run the tests the change could plausibly break rather than all of them. Mature tooling exists now and almost nobody in QA is using it. Enormous lever on feedback speed once a suite gets large.
Most teams run the same answer to "which tests should we run" every single time: all of them. That is safe, but it is also wasteful once a suite grows past a few thousand tests, because a one-line change to a single function almost never has the power to break every test in the repository. Test impact analysis maps a code change to the specific tests that could plausibly be affected by it, and runs only those. The tooling to do this well has existed for years. Almost nobody in QA is actually using it.
How impact analysis actually maps a change to a test
The core idea is a dependency graph between source code and tests, built from real coverage data rather than guesswork. Every time the suite runs, a coverage instrumentation tool records which lines of source code each individual test touched.
That produces a map: test A executed lines from files X and Y, test B executed lines from file Z. When a pull request changes file Y, the tool looks up which tests touched any line in that file and selects only those to run.
This is meaningfully different from simply running tests in the same directory as the changed file, which is the manual heuristic most teams fall back on without tooling. A shared utility function used across forty unrelated test files needs all forty to run when it changes.
A naive directory-based heuristic would miss most of them. A real dependency graph built from actual execution data catches that correctly, because it does not care where a test file lives, only what code path it actually exercised.
The tools that make this practical today
Several mature options exist depending on the stack:
- Bazel and other build systems with fine-grained dependency graphs: they already know which targets depend on which source files, so test selection falls out of the build graph for free.
- Microsoft's Test Impact Analysis (part of Azure DevOps): purpose-built for .NET, maps method-level coverage to test selection automatically in CI.
- Nx and Turborepo affected commands: in JavaScript monorepos, these compute which packages changed and which downstream packages depend on them, then run only the affected test suites.
- Coverage-based custom tooling: teams without one of the above can build this themselves from existing coverage reports (Istanbul, JaCoCo, coverage.py) with a script that intersects changed files against the coverage map from the last full run.
# Compare current branch against main, find files touched
CHANGED=$(git diff --name-only origin/main...HEAD)
# Look up which test files exercised any of those source paths
# using a coverage map built from the last full CI run
node scripts/impact-map.js --changed="$CHANGED" --coverage=coverage/full-run.json > affected-tests.txt
npx jest --listTests $(cat affected-tests.txt)Why almost nobody uses this despite the payoff
The honest answer is that building an accurate dependency graph is more work than running everything. The failure mode of getting it wrong (a real bug ships because the affected test was skipped) is scarier than the failure mode of leaving it alone (CI stays slow). Teams that already invested in sharding and parallelism to make the full suite fast often stop there, because parallelism is a purely additive fix: it costs money and infrastructure, not correctness risk.
Consider a logistics platform team's real experience: a suite of roughly 9,000 tests taking 38 minutes on CI even sharded across sixteen workers.
They built a coverage-based impact analysis layer on top of their existing Jest coverage reports, computing an affected-test set from a diff against the base branch.
Most pull requests dropped to under 4 minutes, since a typical change touched code covered by 200 to 400 tests rather than all 9,000.
They kept a full, unselected run on a nightly schedule and before any release branch cut. This was specifically to catch the case where the dependency graph missed an indirect dependency, which happened twice in the first month from dynamically loaded modules the coverage instrumentation could not see statically.
Where this fits in the broader speed picture
Test impact analysis and parallelism solve different problems and stack well together. Parallelism makes a fixed set of tests finish faster by spreading them across workers. Impact analysis shrinks the set of tests that need to run at all.
A team with both gets the full benefit: a smaller set of relevant tests, sharded across available workers, on every pull request, with periodic full runs as the safety net against a dependency graph that is not perfectly complete. Teams running ephemeral environments per pull request benefit even more, since a smaller affected-test set finishes before the preview is even done provisioning.
FAQ
Questions people ask
How accurate does test impact analysis need to be before it is safe to trust?
There is no universal number, but most teams validate a new setup against full suite runs for several weeks, checking that the selected subset would have caught every regression the full run caught, before letting it gate merges alone.
Does test impact analysis work for end-to-end tests, not just unit tests?
It is harder. End-to-end tests often touch broad swaths of the application through the UI, so coverage-based selection tends to select most of them anyway. It works best on unit and integration tests with narrower, more traceable coverage.
What happens when the dependency graph misses an indirect dependency?
A test that should have run gets skipped, and a regression can ship undetected until the next full run catches it. This is why most teams keep a scheduled full run as a backstop rather than relying on selection alone.
Is test impact analysis worth building for a small test suite?
Usually not. The payoff shows up once a suite is large enough that running everything takes minutes rather than seconds. A 200-test suite finishing in under a minute has little to gain from the added complexity.