What a Customer's Security Questionnaire Is Actually Testing

A security questionnaire lands in the inbox of a head of quality on a Tuesday, usually attached to a deal that's already six weeks into procurement. Sales wants answers by Friday. The questionnaire itself is forty to eighty questions long, and a chunk of them are about testing: do you test before release, how do you track defects, can you show evidence of QA sign off. The team that answers in a day and the team that spends three weeks assembling screenshots are not different in how rigorous their testing is. They differ in one thing: whether the evidence already existed or had to be built after the fact. This post is about what these questions actually ask for, why "we test everything" isn't an answer, and what to have in place before the next one arrives.

What the questions actually say

Strip the legal phrasing off a typical enterprise questionnaire and the testing section usually reduces to a short list:

• Do you have a documented process for testing changes before release
• Can you show that a specific release was tested, and by whom
• How do you track and resolve defects found during testing
• Is there a record linking a requirement or ticket to the test that verified it
• How long do you retain testing records

None of these ask whether your test suite is good. They ask whether you can produce a record. A reviewer reading "we have a rigorous QA process" learns nothing, because every vendor writes that sentence. A reviewer who gets a screenshot of a test run tied to a release, with a timestamp and a named reviewer, learns something specific: that this happened, for this release, and someone can point to it again next year.

Having controls versus being able to evidence them

This is where most deals actually slow down, and it's a distinction worth stating plainly: having a control and being able to evidence it are two different capabilities. A team can run a genuinely thorough regression pass before every release and still fail this part of a review, because the proof of that pass lived in a Slack thread, a spreadsheet someone updated inconsistently, or a tester's memory.

Take a mid size SaaS team shipping every two weeks. They test thoroughly. Nobody on that team would say QA is an afterthought. But when the questionnaire asks for evidence tied to their last three releases, what they have is a mix of closed Jira tickets, a QA channel with pass and fail messages that scroll away, and one spreadsheet that stopped being updated four months ago. The testing happened. The evidence didn't survive it.

The three week scramble, honestly

Here's what actually happens on the team that doesn't have this ready. Someone gets assigned to "pull together the evidence," usually a QA lead who now has this stacked on top of their regular sprint work. They go back through Jira looking for test related tickets. They message engineers from three releases ago asking if they remember testing a particular feature. They screenshot whatever they can find in a test runner or spreadsheet, paste it into a slide deck, and write captions explaining what each screenshot is supposed to prove.

The result, after three weeks of someone's time, is weaker evidence than the team actually had while the work was happening. A screenshot pasted into a deck after the fact has no independent timestamp a reviewer can trust, no link back to the original ticket, and depends on someone's memory of what happened months earlier. The team spent real hours producing something less convincing than a live record would have been for free. That's the actual cost of not having this set up: not that the deal is lost, but that weeks go to reconstructing something that should have existed already.

What to put in place before the questionnaire arrives

None of this requires predicting exactly what a future buyer will ask. It requires a small set of habits that make evidence a side effect of testing rather than a separate task:

• Keep test cases written down somewhere durable, not in a person's head or a chat thread
• Record pass and fail results against a specific release, not just "it works"
• Link each test back to the requirement or ticket it verifies
• Keep a history you can pull up later, not just the latest state
• Decide who owns test sign off for a release, and make that visible, not implied

None of this is exotic. It's the difference between testing that leaves a trail and testing that leaves a memory. The trail is what a reviewer is actually asking for when they write "can you demonstrate your QA process."

We've written more on this specific problem in how an audit trail turns testing into evidence and on the broader discipline behind it in our guide to test case management.