L1 · Running the work
L1Reference4 min read

Smoke, sanity, UAT: who runs what, and when

Terms teams use interchangeably and mean differently, which is how a release ends up signed off by nobody. What each check is for, how long it should take and who owns it — enough to settle the vocabulary with your team in a single meeting.

Three terms, used interchangeably by most teams, meaning different things to each person in the room.

Smoke testing answers one question: is this build worth testing at all? A handful of checks on the most important paths. If the application will not start or nobody can log in, you stop and save everybody an afternoon. Some teams call it a build verification test, or BVT.

Sanity testing is a narrow check after a change: does the thing that was just fixed actually work, and did the area around it survive? Smaller than a regression pack, aimed at one area.

User acceptance testing (UAT) is the customer or business representative confirming the software does what they need, in their language, before they accept it. It is not a second round of your testing, and it is not a bug hunt.

The distinction matters because each one has a different owner. When the words blur, so does the ownership, and a release ends up signed off by nobody.

The terms you will hear

  • Smoke test. A short set of checks proving the build is usable. The name comes from electronics: switch it on and see whether it smokes.
  • Build verification test (BVT). The same thing, usually automated and run by the pipeline.
  • Sanity test. A focused check after a change or a fix. Narrow and quick.
  • Regression testing. Re-running a wider pack to check that what worked still works.
  • User acceptance testing (UAT). Business or customer confirmation against their own needs.
  • Alpha and beta testing. Acceptance testing done internally, then with real users, before general release.

Why it matters

Because when the words are vague, the responsibility is too.

For example, a project manager asks whether UAT is done. The tester says yes, meaning they ran the release pack. The business sponsor has never opened the feature. Two weeks after launch, the sponsor sees it, hates the wording on the confirmation email, and asks why nobody checked. Everybody in that chain behaved reasonably, and the release still had no business sign-off.

Getting the three definitions agreed removes that whole failure mode.

How to use them properly

  1. Automate the smoke test and run it on every deploy. Five to ten checks: the app loads, login works, one core journey completes, the health endpoint answers.
  2. Run smoke before anything else in a cycle. It is step two of the regression cycle for exactly this reason.
  3. Use sanity checks after each fix. Re-test the fix, plus whatever shares code with it. Do not re-run the whole pack for a one-line change.
  4. Give UAT to the business, with support. Prepare the environment and the data, write the scenarios in their language, then let them use it. Sitting beside them and watching is the most valuable hour in the release.
  5. Never call your own testing UAT. If QA runs it, it is acceptance-style testing rather than user acceptance testing. The distinction protects you.
  6. Write down who signs what. UAT sign-off is a business signature. The QA recommendation is separate, and belongs in the summary report.

A worked set

Here is what the three look like written down for one small team, with times attached.

smoke-sanity-uat.txt
SMOKE (automated, runs on every deploy, target under 3 minutes)
  1  home page returns 200 and renders the header
  2  a customer can log in
  3  a book can be added to the basket
  4  checkout page loads with the correct total
  5  /health returns ok and reports the database as reachable
  owner: pipeline. failure blocks the deploy and pages the on-call.

SANITY (manual, after a fix, 10 to 20 minutes)
  example: after WB-1864, the 1p rounding fix
    - re-run the failing case GC-004
    - two other partial redemptions with totals ending .49 and .99
    - one full redemption, to check nothing else moved
    - the balance page for the same card
  owner: tester. not the whole pack.

UAT (business, 2 sessions of 60 minutes, before release)
  who: Sam (product), Rekha (customer support lead)
  environment: staging, seeded with 3 realistic customers, Stripe test mode
  scenarios written in their language:
    1  "a customer buys a 50 pound gift card as a birthday present and
        emails it to their sister"
    2  "the sister uses it to buy two books that cost more than 50 pounds"
    3  "a customer rings up asking how much is left on their card"
    4  "a customer wants a refund on an order they paid for with a card"
  what we did NOT give them: test case steps. they use it as a customer.
  outcome: 2 findings, neither a defect
    - the confirmation email calls it a "voucher", the site says "gift card"
    - support wants the balance page to show what was spent, not just what
      remains. Logged as a new story rather than a bug.
  sign-off: Sam, in writing, in the release ticket.

Look at the UAT findings. Neither is a defect, and both would have shipped without a business pair of eyes. The vocabulary mattered: had QA called their own pack "UAT", nobody would have run those four scenarios.

How to show you know it

  • A three-line definition your team agreed. It is a small artefact that ends a recurring argument.
  • An automated smoke test that blocks a deploy. The most useful ten checks you will ever write.
  • A UAT session you prepared but did not run. Preparing the data and scenarios, then letting the business use it, is the professional version.
  • A finding that only UAT could produce. The wording mismatch above is a perfect example, and it is why the practice exists.

Questions

What is the actual difference between smoke and sanity?

Smoke is broad and shallow, run to decide whether the build is worth testing. Sanity is narrow and slightly deeper, run after a change to check that specific area. Smoke is usually automated, sanity is usually manual.

Is UAT still needed if we have good acceptance criteria?

Yes, for anything customer-facing. Criteria capture what was agreed months ago in your words. UAT catches the gap between that and what the business actually meant, which is a validation question.

Who should run UAT if there is no separate business team?

Whoever will answer for the feature: support, sales, an account manager, the founder. The requirement is that they are not the person who built or tested it.

How many checks belong in a smoke test?

Five to ten. If it takes longer than a few minutes it is a regression pack with the wrong name, and people will start skipping it.