Running a regression cycle

A coach driver does a walk-round before pulling away. Tyres, lights, doors, mirrors, fluid levels.

The same list, every time, in the same order. Not because the coach is expected to be broken, but because the cost of finding out later is a road full of passengers.

That walk-round is a regression testing process. Somebody wrote the list down once, and now anybody can do the check and reach the same answer.

"Do a regression" is not an instruction. This is what the instruction should look like.

What a cycle actually is

Four things, and if any of them is missing you have an activity rather than a process.

• A fixed pack. The cases you re-run every release, chosen deliberately.
• A build that has stopped moving. A release candidate, deployed once, not updated mid-run.
• An order. Highest consequence first, so the news that matters arrives early.
• A record. What ran, what passed, what failed, on which build, by whom.

Why you should care about this

Because the regression cycle is where a release either gets a decision or gets a shrug.

Without a written procedure, the person running it improvises, the results live in their memory, and the answer to "was this tested last release" is a guess. That guess is the thing that fails an audit, loses an argument with support, and makes the same bug shippable twice.

There is also a personal reason. Running a clean cycle is the most visible work a tester does all month. Everyone sees the outcome, so it is worth doing in a way that stands up afterwards.

The procedure

1. Freeze the candidate. One build, deployed to the release environment, with the version recorded. Agree that nothing else ships to it until the cycle finishes.
2. Check the environment before the product. Log in, load the home page, place one throwaway order. If the basics are broken, stop and fix the environment. Ten minutes here saves a day of false failures.
3. Run the highest-consequence cases first. Payment, login, account data, anything irreversible. If one of these fails, the release conversation changes immediately and everybody wants to know now rather than at five o'clock.
4. Then run what this release touched. The areas the stories changed, plus anything sharing code with them. The test plan scope section already lists these.
5. Then the rest of the pack, in whatever order suits.
6. Record as you go, not at the end. Pass, fail, blocked, with evidence attached to the failures. Memory is not a record.
7. Triage before fixing. Group the failures. Environment, test data, a genuine defect, or a case that is simply out of date. In practice a good share of them are not product bugs at all, and your run history will tell you which ones fail regularly.
8. Re-run only what changed. After fixes, re-run the failed cases and anything the fix could plausibly touch. Not the whole pack, unless the fix was structural.
9. Write the two-line outcome. What ran, what is still open, and whether you would ship. That is the input to the release decision.

A real two-day timetable

Say the release is on Thursday and you have Tuesday and Wednesday. Here is how that looks written down, from the gift-card release at Willow Books, a small online bookshop.

Notice the shape of it. Two failures out of five were not product defects, the money cases ran first, and the whole thing produced a sentence somebody can act on. That last line about total time is what lets you argue for a smaller pack next quarter.

How to show you know it

• A written procedure. Nine steps on one page, so anybody on the team could run the cycle while you are on holiday.
• A cycle log with times. Proof of what ran and how long it took. It is also the evidence for trimming the pack.
• A triage breakdown. "Five failures: two defects, one environment, one stale case, one accepted." It shows you separate signal from noise.
• A sentence, not a spreadsheet. The recommendation at the end is the deliverable, and it belongs with the record the release rests on.