Run a Migration Dry Run Before You Pick a Cutover Date

Somebody in a planning meeting asks when the team can be off the old test case tool, and a QA lead gives an answer that sounds reasonable. Three weeks, maybe four. That number gets written into a roadmap slide before a single case has actually moved. Six weeks later the team is still exporting, because the suite with 40 custom fields did not survive the trip. Nobody found that out until the real cutover was already underway. This happens constantly, and it is avoidable. A migration dry run is a few hours of unglamorous work that replaces the guess with a measured number. It can be run by a QA lead alone, without pulling an engineer off their sprint. This post is that procedure, laid out so you can run it this week.

What a dry run actually is

A dry run is a full import of a representative sample of your test cases into a scratch instance of the new tool. It is not a spot check where someone grabs three cases that look easy and confirms they came through fine. That kind of check tells you the happy path works and nothing else, and the happy path was never the risk.

The risk is the suite nobody has opened in eight months. It is the set of cases linked to requirements in a way the export format does not actually preserve. A proper dry run answers one question with evidence. If we moved everything today, what would break, and how long would it take. Everything after that is detail.

Choosing a sample that is actually representative

The sample decides whether the dry run tells you anything. A convenient sample, like the newest folder or the one the QA lead happens to remember best, tells you almost nothing about the rest of the suite. Pick the sample instead by picking for risk:

• The largest single suite in the repository, because it stresses import performance and exposes pagination or timeout limits
• The suite with the most custom fields, since field mapping is where migrations quietly lose data
• A suite with attachments, screenshots, log files, anything binary, because attachment handling is inconsistent across tools
• Any suite with requirement links or traceability references, because those links are often the first thing dropped in an export

If your repository has 1,400 cases, a sample in the 100 to 150 range is usually enough to surface the real problems. Build it from these four categories rather than picking at random.

Consider the release held on a Thursday because the migration was rushed the week before. A rushed migration and a rushed release share the same root cause: nobody measured the risk first.

The checklist to run against the result

Once the sample has been exported and imported into the scratch instance, walk it line by line against a checklist rather than eyeballing it. Each check catches a specific class of failure:

• Case count in equals case count out, or the difference is explained
• Every custom field maps to something, not a generic notes catch all
• Attachments open and are not silently dropped
• Requirement links point at the right requirement, not a broken reference
• Run history, if you need it, actually carried over rather than starting fresh
• Formatting inside steps, such as numbered lists, code blocks and images, survived the round trip
• Owners and authors mapped to real users rather than a shared import account

Anything that fails goes on a list. That list is the actual scope of the migration, not the vague sense that it should be fine.

Timing it properly

The number a QA lead actually needs from a dry run is wall clock time, not a vibe. Time three separate things:

• How long the export takes from the old tool, including any manual steps like generating a file and downloading it
• How long the import takes into the new tool, including field mapping setup that only has to happen once
• What has to be frozen while this runs, and whether testers can keep logging results in the old tool during export

That last point matters more than it looks. If the export requires a freeze, the real migration window is not how long the import takes. It is how long the team can afford to not log results, which is a very different number and usually a much smaller one.

What the dry run tells you that nothing else will

The honest output of a dry run is a decision, not just a report. It tells you whether the cutover is an afternoon of work or a two week project. That answer determines whether it happens this quarter at all.

A team that discovers the dry run took 40 minutes for 120 representative cases can reasonably plan a Friday afternoon cutover. A team that discovers two field mappings do not exist, and attachments need to be handled by hand, needs to either fix the mapping problem first or budget a real project, not a weekend.

Neither outcome is a failure. The failure is committing to a date before you know which of these two situations you are in.

Writing it down

The dry run is only useful to the team if someone writes down what it found. A one page record, kept somewhere the whole team can see, should state what moved cleanly and what did not. It should also state what the team explicitly agreed it was willing to lose.

If run history older than a year is not coming across and everyone agrees that is fine, write that down as a decision. Do not let it become a surprise discovered during the real cutover. This record also becomes the thing you point to later if someone asks why a particular case looks different after migration.

A worked example

A QA lead at a mid sized product team had 1,400 cases in an aging tool and a roadmap deadline six weeks out. Rather than committing to that date, they pulled a sample of 120 cases. The sample covered four groups.

The largest suite ran to 340 cases. The suite with the most custom fields had 12 fields, including two dropdowns with team specific values. A folder held 30 attachments. A small set of cases linked to requirements in a separate tracker.

The export took 12 minutes. The import took 35 minutes, most of it spent on field mapping setup that would not need repeating for the full migration. Two field mappings turned out not to exist at all in the new tool. Both were custom dropdowns with values specific to that team, and both would have needed manual recreation.

Attachments came through intact. Requirement links did not map automatically and needed a follow up script. With that information the QA lead reported back a real estimate: two days of engineering time to solve the field mapping and requirement link gaps, then a half day cutover once those were fixed. That is a schedule a team can actually commit to, and it came from an afternoon of dry run work rather than a guess.