AI governance: the EU AI Act, ISO 42001, NIST AI RMF
What you may need to document, and by when. The EU timeline has already shifted once, so learn the shape of the obligations rather than memorising dates — and note that most of the evidence they ask for is testing evidence.
A restaurant kitchen has a hygiene rating stuck on the door.
The inspector who awarded it did not taste the food. They checked whether there were procedures, whether the fridge temperatures were being recorded, whether the records were real and current, and whether somebody was clearly responsible.
The food matters. But the rating is about evidence that somebody is systematically checking the food.
AI governance works exactly like that. And here is the part nobody tells testers: most of the evidence a regulator or an auditor asks for is testing evidence. You are already producing it. It is just not written down in a form anybody could hand over.
The shape of the obligations
Different frameworks, remarkably similar asks. The EU AI Act is law with staged deadlines. ISO 42001 is a management-system standard you can be certified against. The NIST AI Risk Management Framework is voluntary guidance widely used as a structure. Sector rules — medical, financial, employment — sit on top.
Strip the vocabulary away and they want four things.
Know what you built and how risky it is. A description of the system, its purpose, and an assessment of what could go wrong for the people affected.
Show that you tested it. Evidence of evaluation against the risks you identified, with results, dates and versions.
Show a human is in the loop. Who reviews, who can override, who is accountable when it goes wrong.
Tell people what they are dealing with. Transparency to users that they are interacting with AI, and about what it does with their data.
Why you should care about this
Because "show us your testing evidence" is a request that lands on the QA function, and the honest answer is often "it is in Slack".
Everything on this layer produces exactly what these frameworks ask for. A golden dataset is your evaluation set. Eval runs with dates and model versions are your evaluation records. A blast-radius table is a risk assessment. A red-team report is adversarial testing evidence. Approval steps are human oversight.
The work exists. What is usually missing is the discipline of keeping it dated, versioned and findable — and that turns a three-week scramble before an audit into an afternoon of collecting links.
There is also a plain career argument. Testers who can produce compliance-grade evidence become the person who unblocks enterprise deals, because the security questionnaire and the audit both ask the same question: prove it.
Most of what an auditor wants is what you already did. The gap is almost never the testing. It is that nobody wrote it down with a date on it.
How you get ready
1. Find out whether anything applies. Ask legal or compliance. The answer depends on where users are, what the feature decides, and which sector you are in. Do not guess either way.
2. Write the one-page system description. What it does, what it is not for, which model, which data, which tools, who owns it. This single page is asked for by every framework and rarely exists.
3. Turn the blast-radius table into the risk register. Same content, one extra column: who is affected and how badly. You already have the hard part.
4. Date and version every result. Model version, dataset version, date, who ran it. This is the cheapest habit here and the one that makes everything else usable as evidence.
5. Write down the human oversight. Which actions need approval, who reviews the failures, who can turn it off. If the answer is "nobody", that is a finding worth more than any bug.
6. Keep the records somewhere durable. Not a laptop, not a chat thread — the same place your other release evidence lives, where a release record belongs.
7. Re-run the evidence pack on model upgrades. An evaluation from two model versions ago is a historical document, not evidence.
Try this today
Write the evidence index. Not the evidence — the index. One page listing what exists, where it lives, and when it was last refreshed.
FEATURE support assistant (customer-facing, EU + UK users)
OWNER R. Mehta (eng), S. Okafor (product)
CHECKED 2026-08-17 applicability confirmed with: legal (pending)
WHAT AN AUDITOR ASKS FOR WHAT WE HAVE WHERE DATED
system description one-pager missing -- <--
risk assessment blast-radius table, 7 tools wiki 08-14
evaluation set golden dataset, 24 cases repo 08-12
evaluation results eval runs, model + date stamped repo/CI 08-16
adversarial testing red-team report, 22 attempts wiki 08-17
security list review OWASP charter, 10 rows wiki 08-17
human oversight refunds >100 need approval code -- <-- unwritten
transparency to users "AI assistant" label in UI product -- <-- unverified
data handling in test records transcripts redacted, 30 days ops 08-01
model version pinning floating alias config -- <-- FINDING
TOP THREE GAPS
1 model version is not pinned, so behaviour can change with no record
2 human oversight exists in code but is written down nowhere
3 no system descriptionNine of twelve rows already exist because the testing was done. The three gaps are an afternoon's work each, and the last two are the kind of thing that stalls a deal for a month if nobody noticed until the questionnaire arrived.
How to show you know it
The evidence index. One page, dated, with gaps marked. It is the most senior-looking artefact on this whole layer and it takes an hour.
Dated, versioned results. Showing that every eval result carries a model version and a date proves you understand what makes a result evidence rather than a memory.
A named oversight gap. "Approval exists in code but no policy says who reviews it." Governance findings are usually gaps in writing, not in software.
Restraint about the law. "This is the shape of what is asked; the current dates are a question for legal." Knowing where your authority ends is part of doing this well.
Questions
Is this not the legal team's job?
They own the interpretation. They cannot produce the evidence — it comes from testing, engineering and product. The most useful thing you can do is hand them a dated index of what exists.
We are a small team with no compliance function. Where do we start?
The one-page system description and the evidence index. Both are cheap, both are asked for by every framework and every enterprise questionnaire, and neither needs a lawyer to begin.
Do these rules apply to internal tools?
Often more than people expect, particularly anything touching employment, performance or access to services. Ask rather than assume, and note the answer with the date you asked.
How is this different from ordinary release documentation?
Mostly it is not, which is the good news. The differences are that an outsider must be able to follow it, and that risk assessment and human oversight are explicit rather than implied.