Prompting for test work: specs in, tests out
Vague prompts give plausible, useless tests. Learn to supply the spec, the constraints, the existing patterns and the definition of done — the same inputs you would give a new colleague, which is exactly why testers are good at this.
A prompt is everything you send to a model in one call. Two parts matter for test work.
The instruction is what you want done. The context is the material needed to do it: the specification, the constraints, the existing patterns, the definition of done.
Most disappointing output comes from a good instruction with no context. The model then has only its general knowledge, so it produces the average test suite for the average product, which is exactly what you did not need.
The rule of thumb is simple. Give the model what you would give a competent contractor on their first morning, and expect roughly the same quality of first draft.
The terms you will hear
- System prompt. The standing instruction that sets role and rules for every call.
- Context. The material supplied with the request. The half people forget.
- Few-shot prompting. Including one or two worked examples of the output you want.
- Zero-shot. No examples, just the instruction.
- Iteration. Refining across a few turns rather than expecting one perfect answer.
- Grounding. Requiring the output to trace back to supplied material rather than memory.
What to supply, in order
- The specification. The story, the acceptance criteria, the API schema, the policy document. Paste it, do not summarise it.
- The constraints. What must not change, what the system cannot do, which browsers matter, what is out of scope.
- An example of your house style. One existing test case or test file. This does more for output quality than any instruction about quality.
- The definition of done. What a good answer looks like. "Every case has one expected result and names its data."
- The shape you want back. A table, a list, a file. Ask for it explicitly.
- Permission to flag gaps. "Mark anything you had to assume." This turns a weakness into your most useful output, because those assumptions are usually real requirement holes.
Why it matters
Because generated tests are cheap to produce and expensive to review, so the quality of the first draft decides whether the whole exercise saves time.
For example, a weak prompt produces twenty cases that check the happy path twenty ways. Reviewing them takes an hour and yields three usable ones. A prompt with the spec, one house-style example and a request for edge cases produces twelve cases, of which eight survive review. Same model, same minute, completely different economics.
A weak prompt and a strong one
For example, here is the same task run twice, with the outputs summarised.
WEAK PROMPT
"Write test cases for a gift card feature."
what came back: 18 cases.
- 11 variations of "apply a valid gift card at checkout"
- 4 with no expected result at all ("verify it works")
- 3 for features we do not have (physical cards, expiry, multi-currency)
- 0 for partial redemption, which is the actual risk
usable after review: 3
STRONG PROMPT (same model, one minute later)
"You are helping a tester write cases for the story below.
[pasted: WB-1802 story and its 7 acceptance criteria]
Constraints: GBP only. No expiry. One card per order. Chrome,
Safari, iOS Safari, Android Chrome only. Stripe test mode.
House style, copy this shape exactly:
[pasted: our case GC-004 in full, with preconditions, data,
steps and one expected result]
Definition of done: every case names its data, has exactly one
expected result, and states its preconditions.
Do this in two steps.
1. First list anything ambiguous or unstated in the story.
2. Then write cases, grouped as: main paths, edge cases,
must-refuse cases. Mark any case that relies on an assumption."
what came back:
step 1, five ambiguities, three of which were real gaps:
- what happens if the balance exactly equals the order total
- is the card consumed before or after the discount code
- what should the error say when the code is already spent
step 2, 12 cases in our shape, 4 marked as assumption-based
usable after review: 8, plus three questions for product
TIME both prompts took under a minute to write. the second took
four minutes because of the pasting. it saved about an hour.The three ambiguities are the real prize. None of them is a test case, all of them are requirement gaps, and they came out of asking for questions before answers.
How to show you know it
- A prompt with its context. Show the pasted spec and house-style example, not just the clever instruction.
- An ambiguity list that became tickets. The strongest evidence that you used the tool as a thinking partner rather than a generator.
- A review verdict. "Twelve produced, eight kept, four rejected because they asserted nothing." That is generate freely and merge carefully in one line.
- A rejected output you can explain. Knowing why a generated case is weak is the skill, and it is the next topic.
Questions
Is prompt engineering a real skill?
Supplying the right context is. The clever phrasing matters far less than people think, and it changes with every model release. Context, constraints and examples keep working.
How long should a prompt be?
As long as the material requires. A story, its criteria, one example case and a constraints list is normal for test work, and that is several hundred words.
Should I use a system prompt?
Yes if you do this often. Put the standing rules there, such as your house style and the definition of done, and keep the per-task material in the request.
What must never go in?
Customer data, credentials or anything commercially sensitive. Use structure and fake values instead. It is the same rule as in drafting QA documents with AI.