The Calm Way to Upgrade a Self-Hosted Test Tool

Somewhere on your team there is a self-hosted test management tool that nobody has upgraded since it was installed. Self-hosted test tool upgrades tend to work like this: it runs fine, so it gets ignored, until a plugin breaks or a security advisory lands and someone has to touch it under pressure. Handled well, an upgrade is an hour of attention every month or two. Handled badly, it eats a Friday afternoon, and the person who inherited it spends the evening restoring a backup instead of going home. This post lays out the ordered procedure that keeps upgrades boring, and makes the case that version pinning is the one feature that decides which of those two Fridays you get.

The ordered procedure that keeps an upgrade boring

There is no trick to a safe upgrade. It is a sequence, and skipping a step is what turns a routine task into an incident.

• Read the release notes for the version you are moving to, specifically the migration and breaking changes sections
• Take a fresh backup immediately before you touch anything, not a backup from last week
• Run the migration against a copy of that backup first, on a throwaway instance
• Only then upgrade the real instance, in a maintenance window where rollback is still possible

Each step earns its place. Skip the release notes and you find out about a breaking config change from an error log at 6pm on a Thursday.

Skip the fresh backup and your rollback point is a week old. That means reconstructed test runs and case edits that nobody remembers making, because a week of normal work sits between the backup and the failed upgrade.

Skip the dry run and the schema migration becomes a live experiment on production data, with real test cases and real run history as the test subjects.

None of these four steps takes long on its own. Reading release notes for a point release is usually 10 minutes. A fresh backup on a database of a few gigabytes is a couple of minutes.

The dry run against a copy, on a spare VM or a local container, is 15 to 20 minutes including the wait for the migration to finish. The total is closer to an hour than a day, which is exactly why skipping any of it to save 20 minutes is a bad trade.

Why the copy step is not optional

A backup protects you if the upgrade fails cleanly and you can just restore. It does not protect you from the failure mode that actually causes the bad Fridays: a schema migration that fails halfway through.

Picture a migration that adds a column, backfills it across 40,000 test case records, then errors out on record 31,000 because of a data shape nobody anticipated. The database is now in a state that is neither the old schema nor the new one.

Restoring the backup works, but only if you noticed the failure before anyone wrote new data against the half migrated schema. If a teammate logged a bug report against a test case ten minutes after the failed migration, restoring the backup throws that report away too.

Run that same migration against a copy first, and the failure happens on a throwaway instance where the only cost is your time, not your Friday and not a teammate's bug report.

Version pinning, and what its absence actually means

This is the part that turns a routine restart into a schema gamble. If your deployment pulls a rolling tag, meaning the image tag always points at whatever is currently latest, you do not control which version you are running.

You control when you pull, and upstream controls what you get. That sounds like a minor distinction until you need to answer a specific question: what version was running last Tuesday when a test run's status changed unexpectedly.

With a rolling tag, a routine container restart can silently pull a new image, including one that carries a schema migration you never planned for. You cannot tell, after the fact, whether last week's restart was a no-op or a version jump, because the tag never changed even though the image behind it did.

Kiwi TCMS is a concrete example of this pattern. The Community Edition is rolling release with no version-tagged images, and version tags are part of the paid Self Support tier (kiwitcms.org, "Community Edition Explained", checked 18 February 2026). That is a real, disclosed product decision, not a hidden catch, but it is worth knowing before you build your upgrade calendar around a free tier.

Version pinning fixes this by letting you choose exactly when a schema change happens. You pull v16.3 on purpose, on a Tuesday morning, with a backup and a dry run already done. You do not pull it by accident because a container orchestrator restarted a pod overnight.

A well run test management practice treats the tool itself the same way it treats the product it tests: changes go through a deliberate process, not an accident of timing.

The auditability consequence

This matters more in test management than in most other tooling, because the whole point of the system is to be the record of what happened and when. If you cannot say which version of the tool was running in March, you cannot fully explain the run history from March.

Think about an auditor or a new engineering lead asking why a batch of test runs from March shows results in a format that later runs do not use. "We're not sure which version was live that week" is not a great answer to give during a compliance review or a postmortem.

Pinned versions turn that into a one line answer: check the deployment manifest, see the tag, done.

The same logic applies to smaller, everyday questions. When a field on a test case starts behaving oddly, the first useful question is usually whether anything changed recently. A pinned version history answers that in seconds, while a rolling tag turns it into a guessing game about what upstream might have shipped in the last few weeks.

Making it routine instead of an emergency

An upgrade only stays calm if someone owns it on purpose. Three things turn "we should upgrade at some point" into a task that actually happens on schedule.

• A named owner, one person whose job it is to check for new releases, not a shared team responsibility that nobody picks up
• A calendar entry, a recurring reminder every 6 to 8 weeks to read the release notes even if nothing looks urgent
• A written rollback step that someone has actually performed once, not just described in a wiki page nobody has tested

That last one matters most. A rollback procedure that has only ever been written down, never run, tends to have a missing step that only shows up when you need it under pressure. Run it once on a test instance when nothing is on fire, and it becomes a five minute task instead of a stressful one.

A named owner does not need to be a senior engineer. It just needs to be one specific person, listed somewhere the team can find it, so the task does not silently fall through the cracks of shared ownership.

Put the calendar reminder on their calendar directly, not on a team calendar nobody checks. The teams that never lose a Friday to an upgrade are not the ones with the most sophisticated tooling. They are the ones who treat it as a small, scheduled, owned task instead of something that only happens when a security advisory forces it.