Sharding, parallelism and runtime budgets

Three different things speed up a suite, and they are often confused.

Parallelism runs several tests at once on one machine, usually one worker per CPU core. Cheap, and limited by that machine.

Sharding splits the suite across several machines, each running a slice. Machine three runs shard three of five. Slower to start, and it scales as far as your budget does.

Deletion removes tests that do not earn their runtime. Free, immediate, and the option people reach for last.

A runtime budget is the number that forces the conversation: this suite must finish in ten minutes, so adding a test means making something faster or removing something else.

The terms you will hear

• Worker. One parallel process running tests on a machine.
• Shard. One slice of the suite, run on its own machine.
• Fan-out and fan-in. Splitting work across machines, then merging the reports.
• Runtime budget. An agreed maximum wall-clock duration for a stage.
• Critical path. The slowest shard, which sets the total time regardless of the others.
• Test isolation. Each test owning its own data, which is what makes any of this safe.

What each one buys

Parallelism is the first thing to turn on and it costs nothing. Four workers on a four-core runner typically cuts a suite to a third, not a quarter, because of setup overhead.

Sharding is next, and it is bounded by two things. Every shard pays the setup cost again, so twenty shards of thirty seconds each is mostly setup. And the total is set by your slowest shard, so a badly balanced split wastes most of what you paid for.

Deletion beats both on cost. Removing 200 tests that have never failed and cover a retired feature is faster than any infrastructure change, per choosing what stays.

Why it matters

Because time-to-feedback governs how the team works, and a growing suite consumes it silently.

For example, a suite of 900 tests grows by 40 a month. At some point the required stage passes twelve minutes, people start batching their merges, reviews get bigger, and diagnosing a failure gets harder. Nobody decided any of that. The suite simply grew and nothing pushed back.

A budget is what pushes back.

How to do it properly

1. Turn on parallelism first, with workers matched to cores. Measure before and after.
2. Fix the state collisions it exposes. Per-test data, no shared fixtures, no order dependence.
3. Measure per-test duration and look at the slowest twenty. There is usually a handful of tests taking a disproportionate share.
4. Shard by measured time, not by file count. Most runners support it. The difference is four balanced shards, rather than one that takes twice as long as the rest.
5. Merge the reports. A run split into five pieces needs one readable result, or nobody looks.
6. Cut the setup cost per shard. Reuse containers, cache dependencies, seed reference data once.
7. Set the budget and defend it. When the stage exceeds it, something moves later or gets deleted.

A worked budget

For example, here is one suite brought back inside its window.

The order at the bottom is the lesson. The free changes bought most of the time, and the infrastructure changes were worth doing only after that.

How to show you know it

• A time breakdown by stage. Knowing that 83 per cent sat in 38 tests is what makes the next decision obvious.
• The collisions parallelism exposed. Eleven of them, fixed with per-test data, is a genuine engineering result.
• Shards balanced by duration. Quoting the four shard times shows you understand the critical path.
• A defended budget. "Six minutes, so this new test displaces one." That sentence keeps a suite alive for years.