Estimating test effort

Test estimation is predicting how much effort the testing will take, usually before anyone knows quite what is being built.

Three techniques cover almost all real situations.

• Analogous estimation. Compare with something you have actually done. "The last payment feature took three days, this looks similar but simpler, so two."
• Three-point estimation. Give an optimistic, a likely and a pessimistic number, then weight them. It is honest about uncertainty instead of hiding it.
• Percentage of development effort. Testing takes a rough share of build effort, often a quarter to a third. Crude, and useful when nothing else is available.

The thing that makes an estimate useful is not the number. It is the assumption stated beside it, because that is what people can check and challenge.

The terms you will hear

• Analogous estimation. Estimating by comparison to past work. Also called comparative estimation.
• Three-point estimation. Optimistic, most likely and pessimistic, usually weighted (O + 4M + P) / 6.
• Percentage of development. Testing effort as a share of build effort.
• Buffer, or contingency. Deliberate slack for the unknown, stated openly rather than hidden inside tasks.
• Assumption. The condition your number depends on. The most valuable line in any estimate.
• Elapsed versus effort. Two days of work can take a week when you are waiting on data. Say which one you mean.

Why it matters

Because the number gets used whether it was any good or not.

For example, somebody asks in a planning meeting how long gift-card testing will take. You say a week, meaning a week of your effort, assuming the test data arrives on time. That becomes "testing is done next Friday" in a plan, the data arrives three days late, and the conversation on Friday is about testing being late.

Nothing about that is an estimation failure. It is an assumption that was never written down.

How to produce one in thirty minutes

1. List the work, not the feature. Story testing, exploratory sessions, the regression pack, re-testing fixes, the summary. These have very different sizes.
2. Size each piece against something real. The last comparable release, with the actual hours if you recorded them.
3. Use a range where you are unsure. Two to four days is honest. Three days flat, when you mean two to four, is not.
4. Add re-testing time explicitly. Fixes need re-running, and this is the line most people forget. A third of the story testing time is a reasonable start.
5. State the assumptions. Data ready by a date, environment stable, no more than a handful of defects, nobody on leave.
6. Say what would change it. "If the money maths turns out to be wrong, add two days." That sentence is what turns an estimate into a conversation.
7. Record the actuals. Even roughly. The cycle log already has the timings, and two of those make your next estimate defensible.

A worked estimate

For example, here is one for the gift-card release, produced in about half an hour.

The last block is what makes the next estimate credible. One recorded release turns estimation from a feeling into arithmetic, and the assumption that came true is the strongest argument you have for stating them.

How to show you know it

• A breakdown rather than a number. Five lines beats "about a week".
• A stated assumption that later came true. The most persuasive thing in this whole topic.
• A recorded actual against an estimate. Two of these and people start trusting your numbers.
• A range you refused to collapse. Holding "two to four days" under pressure is a professional skill, and it beats a precise number you cannot meet.