AI Test Case Generation: Generate Freely, Merge Carefully

AI can write a test case. That's not the hard part.

Ask a model to write test cases and it will. They'll look right, use the correct fields, and read like a careful person wrote them. So the interesting question was never "can AI write a test case?" It obviously can.

The real question is what happens to the ones that look right but are wrong.

If you generate a pile of cases and drop them straight into your suite, you haven't saved the work — you've moved it. Instead of writing cases, you're now checking them. And checking a confident, wrong case is harder than writing a correct one, because you have to notice it's wrong first.

So the design that works is simple to say: generate freely, approve deliberately. Let the model write as many drafts as you like, and let a person decide which ones are real before any of them join the suite.

What AI is genuinely good at

Think of the model as a fast junior who never gets bored and never cuts a corner because it's late. That's a real strength, and it shows up in three places.

Boring permutations. Every payment method, times every country, times every account type. A person writing these by hand gets tired around row twelve and starts skipping "obvious" combinations. The model doesn't. It will happily lay out all of them in the same shape.

Consistent formatting. A good test case has the same parts every time — a stable ID, the preconditions, the steps in order, and a clear expected result. Humans drift; the model keeps the shape identical across hundreds of cases, which makes the whole suite easier to read and review.

The cases people skip when tired. Empty fields, maximum lengths, an expired card, a quantity of zero, the same action twice in a row. These negative and boundary cases are where real bugs hide, and they're exactly the ones a person leaves out at 5 p.m. The model puts them on the table.

None of this is the judgment part. It's the legwork — and the legwork is worth handing off.

Where it goes wrong: confident and wrong

Now the catch. The same model that lists every payment permutation will also, with total confidence, write a clean test case for a feature your product doesn't have.

It happens because a model fills gaps with what's plausible, not what's true. If your app looks like it should have a "resend invite" button, the model may write a careful case for one — correct fields, sensible steps, a specific expected result — for a button that was never built.

That's the dangerous failure, and it's dangerous precisely because it looks fine. An obviously broken case gets deleted on sight. A plausible, wrong one passes a quick glance, gets merged, and quietly rots your suite from the inside. Later, someone spends an afternoon working out why a "failing" test is checking something that doesn't exist.

So no — generated cases are not accurate by default, and they're not safe to merge unreviewed. They're drafts. Good, fast drafts, but drafts.

Why the review queue is the whole design

Here's the part people get backwards. They treat AI generation as the product and human review as a checkbox bolted on at the end. It's the other way around.

Writing a case is now cheap. Trusting a case is not. So the thing you build the product around is the moment of trust — the point where a person looks at a draft and says "yes, this is real, it can join the suite."

If you've used a pull request, you already know the shape. Anyone can push code to a branch; nothing reaches main until a person reviews it and merges. Test cases work the same way here. The model opens the drafts freely. A holding queue keeps them out of the trusted suite. Nothing crosses over until someone approves it.

That's what "generate freely, merge deliberately" means in practice. Generation is free and reversible, so let it run. The suite is the thing your team relies on, so guard the door to it. You can read more about why that trusted record is the thing a release rests on — the review step is what keeps it worth trusting.

And to be clear about what this doesn't do: the model isn't doing your test design or your exploratory testing. It drafts cases for behaviour you point it at. Deciding what's worth testing, and whether a case is true, is still the job. That judgment is the work, not the paperwork around it.

What a reviewer should check in under a minute

Review only works if it's fast. You are not rewriting the case — you're making a quick decision: keep it, fix it, or bin it. For most drafts that takes seconds, if you know what to look at.

Five quick checks handle almost everything:

• Is this behaviour real? Does the product actually do this? This is the one that catches the confident-but-wrong case, so check it first.
• Is the expected result specific? Could two people read it and reach the same verdict? "Error is shown" isn't good enough — which error?
• Are the preconditions right? Does the starting state make sense for the steps that follow?
• Is it a duplicate? The model may draft three versions of a case you already have.
• Does it match a real requirement? Or did the model invent the rule it's testing?

Think of it as triage, not authorship. The drafts that pass go in. The ones that need a tweak get one. The invented ones get deleted, fast — and deleting a bad draft is far cheaper than untangling a bad case six months later.

What this looks like in practice

Here are two drafts from the same batch. Both are formatted perfectly. One is real; one is not.

Keep this one — reject an expired card at checkout

• Preconditions: a logged-in user, one item in the cart, and a saved card that expired last month.
• Steps: open checkout, choose the expired card, place the order.
• Expected result: the order is not placed, an inline message says "This card has expired," and the item stays in the cart.

Run the five checks and it passes. The behaviour is real, the expected result is specific, the preconditions make sense. It goes in.

Bin this one — resend the confirmation email from the receipt page

• Preconditions: a completed order, sitting on the receipt page.
• Steps: open the receipt, click "Resend confirmation," wait for the message.
• Expected result: a message reads "Confirmation resent."

It reads just as cleanly — but there is no "Resend confirmation" button. The product never had one. The model saw a receipt page, assumed a feature most apps have, and wrote a confident case for it. The only check that catches this is the first one: is the behaviour real? A quick glance won't catch it; that question will.

Your keys, your model, your bill

There's a quieter question behind all of this: whose AI is it, and where does your data go?

With Tesbo, you bring your own AI provider key. You choose the model, the requests go out under your account, and you pay for what you use — directly, with no reseller in the middle. Tesbo doesn't ship a model of its own, and it doesn't train anything on your data. This works the same way on every plan.

That matters because of what a test case is. It describes exactly how your product behaves. Generating cases means sending that description to a model, so the account it goes through should be yours, not a vendor's. Your keys, your model, your bill — and your data stays inside your own AI account.

It also keeps you in control of cost and choice. If a better or cheaper model comes along, you switch to it. You're not locked to whatever one vendor picked for you.

Generate freely, merge deliberately

Put it together and the whole approach fits in one line. AI takes the boredom out of writing test cases — the permutations, the formatting, the edge cases nobody enjoys. It does not take the judgment out of deciding what's true.

Keep both. Let the model draft as much as it likes, cheaply. Keep a person at the door of the suite, deciding what's real. The value isn't a pile of generated cases; it's a trusted suite that grew faster without getting less trustworthy.