Vendor lock-in in test management: it's not the licence

A QA lead at a mid sized fintech spends three weeks every autumn preparing the audit evidence pack. Screenshots of test runs, spreadsheets that reconstruct who approved what, and a folder of PDFs exported "just in case" the tool changes hands. She isn't preparing for an audit failure.

She's preparing for the fact that if her team ever needs to switch test management tools, that audit trail is the thing she is most afraid of losing. This post is for her, and for anyone who has wondered whether the tool they picked two years ago is now a wall instead of a room.

Most people evaluate vendor lock-in in test management by asking about the contract. Is it a monthly plan. Is there an exit clause. Can we get a refund. Those questions matter for procurement. They miss the part that actually traps teams: the shape of the data, and whether it survives the trip out. Vendor lock-in in test management is real, but it rarely looks like the thing people prepare for.

Lock-in is a data-shape problem, not a licence problem

Here's the reframe worth sitting with. A tool can be open source, free, and fully yours to run, and still lock you in harder than a commercial tool with a closed licence. The licence tells you who owns the software. It says nothing about whether your test history comes with you when you leave.

What actually decides whether you're stuck is simple. Does the export format carry everything you put into the tool. Or does it carry only the parts that were easy to build an export button for.

The four layers, and how badly each one travels

Not everything you store in a test management tool is equally portable. There are four layers, and they get harder to move in this order.

• Case text: the steps, the expected result, the preconditions. This almost always exports cleanly, because it is structured text.
• Case structure and folders: the hierarchy, the suites, the tags. This usually survives too, though deep nesting sometimes flattens.
• Execution history: every pass, fail, and skip, tied to a date, a build, and a tester. This is where exports start to thin out.
• Links between a case and the requirement or issue it covers: the traceability that ties a test back to a Jira ticket or a compliance requirement. This is the layer most exports drop entirely.

The first two layers get all the attention in a sales demo. The last two are the ones an auditor actually asks about. They are also the ones almost nobody checks before signing up.

Why history is the layer that traps you

Case text can be retyped. It is tedious, but a team of four people can retype 400 cases in a week if they have to.

Execution history cannot be retyped, because it isn't a description of something. It is a record of something that already happened. Nobody can reconstruct that the login test failed on build 4.12 on a Tuesday in March. Nobody can reconstruct that it passed after a fix shipped that same afternoon. Once that record is gone, it stays gone.

This is also precisely the evidence a regulator or an internal auditor wants to see. Not "here is our test plan" but "here is proof this was run, on this date, by this person, with this result." A tool that loses execution history on export doesn't just cost convenience. It costs the one thing you cannot manufacture after the fact.

A worked example: the 1,400 case suite

Picture a team running a 1,400 case regression suite for a healthcare scheduling product. They decide to move to a different test management tool. The migration goes smoothly on paper. The export tool runs, a progress bar fills up, and 1,400 cases land in the new system with their steps and folders intact.

What doesn't come across is eleven months of pass and fail results. The export brought the definitions of the tests. It did not bring the record of what happened when they actually ran.

The team is left with a clean, empty shell of a test suite. There is also a spreadsheet graveyard of screenshots taken "just in case" that nobody can search or filter. If a customer now asks for proof the checkout flow was tested before the last three releases, the answer is a manual search through old spreadsheets instead of a filtered report. That search takes an afternoon. A filtered report would have taken thirty seconds.

The uncomfortable case: open source can lock you in tighter than closed

This is the part that surprises people. An open source test management tool with no licence fee at all can still trap a team, if its export format only carries case text and drops execution history and requirement links. Being able to read the source code, or even self host the database, does not automatically mean your data comes out in a usable shape.

A closed commercial tool with a documented API and a complete export of cases, runs, and links gives you a real way out, even though you never see a line of its source. The licence answers who owns the software. It does not answer whether you can leave with what you put in. Those are different questions, and conflating them is how teams end up surprised two years into a contract, usually right when a migration is already underway and there is no time left to be surprised.

How to test for lock-in before you adopt, not after

The fix here is almost embarrassingly simple, and almost nobody does it. On day one of a trial, before entering a single real test case, create a handful of sample cases. Run them a few times with different results. Link one to a fake ticket. Then hit export. Open the file. Read it.

• Are the execution results in the export, with dates and outcomes, or just the case definitions
• Are the links to requirements or issues present, or has the export dropped anything that isn't a plain text field
• Is the format something you could actually import elsewhere, or a proprietary blob you would need the vendor's help to read

If the answer to any of those is disappointing, you know it on day one of a free trial. That is the only point at which this check is cheap. Two years into a contract, with 1,400 cases and a compliance deadline, it is not a check anymore. It is a crisis, and crises during audits tend to involve a lot of unpaid weekend work.

What good looks like

A test management tool that treats your data as genuinely yours has three properties that are easy to check for.

• A documented API, so you can pull data out programmatically rather than relying on a single export button that might change or break
• A full export that includes executions and attachments, not just case text and folder structure
• The ability to take a database dump you own outright, if the tool is self hosted, so the worst case is "I have a backup" rather than "I have to ask for one"

None of these guarantee a team will never want to switch tools. They guarantee that if a switch happens, it costs a migration project instead of a rebuild from memory. A regulator does not accept "we used to have that" as an answer. Neither should the team that has to say it out loud in a review meeting, in front of the person who signed off on the original contract.

Tesbo manages and drafts documented test cases with a structure built to keep case text, folders, execution history, and requirement links together. History is not treated as an afterthought bolted onto an export button after the fact. The same thinking applies to how a team plans a migration and how it thinks about metrics once cases and runs are in one place, both of which are worth reading before the next tool decision gets made.