Guardrails, refusals and over-refusal

A smoke alarm that never goes off is dangerous.

A smoke alarm that shrieks every time you make toast is also dangerous, because within a week somebody takes the battery out.

Both are failures of the same device, and only one of them gets reported. Nobody rings the manufacturer to say "your alarm went off when there was no fire" — they just disable it and stop thinking about it.

AI guardrails fail in exactly these two directions, and the toast direction is nearly always invisible.

Two failures, one device

Under-blocking is the one everybody tests. The feature says something harmful, gives advice it should not, or helps with something out of bounds. It is visible, embarrassing and gets fixed quickly.

Over-blocking is the toast. A nurse asking a legitimate clinical question gets refused. A customer asking about a medical product gets a lecture. A developer asking about a security header gets treated as an attacker. Every one of those users concludes the feature is useless, and none of them opens a ticket.

The second failure is where the money leaks, and it is almost never measured.

Why you should care about this

Because a feature that refuses too much is abandoned quietly, and nobody can point at the moment it happened.

There is a version of this that is worse. A team ships a fix for a real safety finding, tightens the filter, and never re-tests the ordinary cases. Two weeks later the feature refuses a whole category of legitimate questions, support tickets climb, and the connection to the safety fix is invisible because nobody kept a set of things that must not be refused.

Keeping that second set is a small amount of work and it is nearly unique to testers. Everyone remembers to test that the alarm goes off. Somebody has to remember the toast.

How you test both directions

1. Build two sets. A must-refuse set — genuinely out of bounds requests, ten or so. And a must-answer set — legitimate requests near the boundary, twenty or so. The second one is the artefact nobody else will build.

2. Fill the must-answer set with the awkward neighbours. Medical questions from clinicians. Security questions from engineers. Legal questions from lawyers. Blunt language from frustrated but reasonable customers. Anything containing a word that looks scary out of context.

3. Run both sets on every guardrail change. This is the whole discipline. A tightening that fixes the must-refuse set and breaks the must-answer set is not a fix, it is a trade nobody agreed to.

4. Judge the refusals, not just count them. A good refusal explains and offers the next step. "I cannot help with that" with no route forward is a poor refusal even when refusing was right.

5. Measure the rate in production. Refusal rate per week, by category if you can. A jump after a release is the clearest signal you will get that the alarm is now shouting at toast — one of the proxy numbers from online evals.

6. Watch for over-refusal in other languages. Add the language cases to your golden dataset so they are not tested once and forgotten. Guardrails behave differently outside English, in both directions. Test at least one other language your users actually use.

7. Keep both sets in the eval suite. Must-refuse as a hard gate, must-answer as a quality number — the split from evals as a marking scheme.

Try this today

Write twenty must-answer cases before you write any must-refuse ones. That inversion is deliberate — the must-refuse list writes itself, and the other one only exists if you make it.

That table is the argument. One genuine hole closed, four legitimate categories broken, and a production number that moved eight points — none of which would have been visible without a must-answer set written in advance.

How to show you know it

A must-answer set. Twenty legitimate near-boundary cases. Almost nobody has one, and it is the whole point of this topic.

A before-and-after across a guardrail change. Both directions, in one table. It reframes safety work as a trade-off rather than a one-way ratchet.

A refusal-rate chart with a release marker. Cheap to produce and immediately understood by non-technical stakeholders.

A poor refusal, rewritten. "I cannot help with that" versus "I cannot give medical dosing advice, but here is our clinical contact." Quality of refusal is a real finding, not a nitpick.