L6Go deeper4 min read

Coverage gap analysis with AI

Point a model at the code and the suite and ask what is untested and would matter. It is unreliable on "would matter" and quite good at "untested", and the combination still beats reading the report yourself.

Coverage gap analysis means working out what is not tested, and whether that absence matters.

Traditional code coverage answers a narrow version of the first half. It tells you which lines and branches ran during the suite. It cannot tell you that a whole scenario is missing, because a scenario nobody wrote leaves no trace.

Pointing a model at the code and the tests together gets you something different. It reads both and proposes what is missing in behaviour terms: the empty case, the permission case, the sequence nobody covered.

It is good at "untested". It is unreliable at "would matter". Knowing which half to trust is the entire skill.

The terms you will hear

Line and branch coverage. Which lines ran, and whether both sides of each decision ran.
Coverage gap. Behaviour with no test, whether or not the lines were executed.
Untested path. A route through the feature nobody exercises.
False gap. A proposal already covered by another test, often at a different level.
Risk ranking. Sorting the real gaps by consequence, which stays a human job.

Why it matters

Because a high coverage number and a serious gap coexist comfortably.

For example, a pricing module reports 84 per cent line coverage. Every line runs during the suite. Nothing tests what happens when a discount and a gift card apply to the same order, because no test does that at all. Coverage is happy. The gap is a money defect waiting to happen.

The useful question is not "which lines are uncovered" but "which behaviours have no test". A model reading your suite is a fast, cheap first pass at that question.

How to run one

Give it the code and the tests together. The module and its spec files. Without both it guesses at one side.
Ask for behaviours, not lines. "List behaviours in this module that no test in these files exercises."
Require evidence for each claim. "Name the function and say which test you expected to find." This alone removes most false gaps.
Add the context it cannot have. Which paths touch money, what broke last quarter, which customers matter. It has no way to know any of it.
Verify each surviving proposal by hand. Search the suite yourself. Roughly half of what survives step three is covered somewhere else, often at another level.
Rank the real ones by consequence. Risk over coverage applies exactly here, and it is the part no tool does.
Write two or three tests, not thirty. The output of this exercise is a short list of high-value tests, not a backlog.

A worked analysis

For example, here is one afternoon on the pricing and gift-card modules.

gap-analysis.txt

INPUT  src/checkout/*.ts (6 files) plus tests/checkout/*.spec.ts (9 files)
ASKED  behaviours with no covering test. name the function and the test
       you expected to find.

PROPOSED  38 gaps

REJECTED AFTER MY CHECK (29)
  14  covered by API tests it was not shown
   7  covered at unit level in another folder
   5  not real behaviour (misread the code)
   3  covered by a manual case in the pack
  ---
  note: 14 of the 29 disappeared simply because I had not given it the
  API tests. context, again.

REAL GAPS (9)  ranked by me, not by it
  1  discount code and gift card on the same order        HIGH, money
     nothing anywhere covers this combination
  2  refund of an order paid partly by gift card          HIGH, money
     one manual case, no automated test
  3  gift card spent concurrently from two sessions       HIGH, money
     no test. this is the WB-1867 defect
  4  order total exactly equal to the card balance        MED, boundary
  5  discount taking the total to 0.00                    MED, boundary
  6  refund after the card was voided                     MED, state
  7  admin voiding a card mid-checkout                    LOW, rare
  8  currency field null on a legacy card                 LOW, legacy
  9  CSV export with 10,000 rows                          LOW, admin only

WHAT IT RANKED FIRST
  the CSV export, because the function was long and had no test.
  length is not consequence, and that is the whole caution.

WROTE  three tests, for gaps 1, 2 and 3. Two of them found defects.
TIME   25 minutes of analysis, 2 hours of writing.

Two details are worth carrying away. Fourteen false gaps vanished once the API tests were included, and the model's own top-ranked gap was the least important one on the list.

How to show you know it

A rejected-versus-real count. "Thirty-eight proposed, nine real, three written." It shows judgement rather than tool use.
A gap that became a defect. The strongest possible evidence for doing this.
The ranking disagreement. Explaining why the CSV export was not the priority demonstrates you understand consequence.
A context correction. Noticing that fourteen false gaps came from incomplete input is the insight that makes the next run better.

Questions

Does this replace code coverage tools?

No, they answer different questions. Coverage tells you which lines ran and is exact. This proposes missing behaviours and is approximate. Use both, and trust each for what it measures.

How much code can I give it at once?

One module and its tests. Whole repositories exceed the context window and the output becomes vague, which is the context window limit in practice.

Should I write every real gap it finds?

No. Rank them and write the top few. A long list of new tests has a maintenance cost, and choosing what stays in the pack applies to new cases as much as old ones.

Can it write the missing tests too?

Yes, and review them properly. Generated tests for a gap it identified tend to assert weakly, so the six failure modes still apply.