GitHub CI Meets Self-Hosted Test Management

An SDET on a mid sized platform team spends a Tuesday afternoon wiring up GitHub Actions self-hosted test management reporting for her team. The API works fine from her laptop. It works fine from a colleague's machine on the office VPN. It does not work from GitHub's own runner, and no amount of retrying the request fixes it, because the request never had a route to take. This is not a rare misconfiguration. It's the default state of the world for any team that runs test management on infrastructure it controls, and it quietly breaks more CI integrations than every authentication bug combined. This post covers what causes it and the three practical ways to get past it.

The problem, stated plainly

A GitHub hosted runner lives on GitHub's network. Your self-hosted test management instance lives on yours. It's probably behind a firewall, a VPN, or a private subnet with no public ingress at all. By default, these two networks have no path between them. It doesn't matter how well the API is designed. It doesn't matter how correct the authentication token is. A workflow step that tries to POST results to https://tms.internal.example.com from a GitHub hosted runner will simply time out, because the packet has nowhere to go.

Teams usually discover this the same way. The integration works in every test except the one that matters, which is the actual CI run. It's easy to mistake this for a bug in the workflow YAML, a DNS issue, or a broken token. Teams can burn an afternoon debugging things that were never wrong. The actual fix isn't a debugging exercise. It's a network design decision, and there are three of them worth knowing.

Three ways to close the gap

There is no single correct answer here. Each option trades effort for exposure differently. The right one depends on how sensitive your test management data is and how much infrastructure your team is willing to run.

• A self-hosted GitHub Actions runner inside your network. You install the runner agent on a machine (or container) that already has network access to your test management instance. The workflow executes there instead of on GitHub's infrastructure, so the request to your instance is just an internal call. Nothing has to cross the public internet.
• An authenticated ingress exposed carefully. You put your test management instance, or just its results endpoint, behind a reverse proxy with a public address. Lock it down with strong authentication, IP allowlisting for GitHub's published IP ranges, and TLS. The hosted runner reaches out to a public URL that happens to forward to a private service.
• A store and forward job. The hosted runner writes results to somewhere it can reach, like an artifact, a queue, or a bucket. A separate process that lives inside your network, running on a schedule or triggered by that write, picks the results up later and posts them to your instance.

Each of these solves the same problem. None of them is free.

Why the self-hosted runner usually wins

Of the three, the self-hosted runner is the most work up front. You have to provision the machine. You have to keep the runner software patched. You have to make sure it has enough capacity to actually run your suite. That's a real, ongoing cost, and it's the reason teams try to avoid it first.

It's also usually the right call, for one reason. It keeps your test management instance completely unexposed. Nothing about it changes to accommodate CI: no new port, no new public endpoint, no new surface for someone to probe.

When a security reviewer looks at the architecture diagram, "the CI job runs on a machine that's already inside the trusted network" is a sentence they can approve without a follow up meeting. "We opened an authenticated endpoint on our internal test management server" is a sentence that starts one.

Secrets handling is where teams get it wrong

Whichever path you choose, the workflow needs a credential to authenticate to your test management instance's API. This is the step where things go sideways most often, and it's worth being deliberate about it.

The credential stored in your GitHub Actions secrets should be an API token scoped specifically to submitting results, and nothing more. It should not be an admin account. It should not be a personal login. It should not be a broadly scoped service account that happens to also be able to delete projects or manage users.

If that token ever leaks, and secrets in CI systems do leak, the blast radius should be small. Someone should only be able to post fake test results, not gain full administrative control of your test management instance.

Rotate the token on a schedule. Store it only in the workflow's secrets store. Never echo it in logs, even for debugging. A curl -v with the token in a header, left in a workflow's debug output, is a common way for a scoped token to end up somewhere it shouldn't be.

What to submit, and when

Once the network problem is solved, the next mistake is submitting results at the wrong point in the pipeline, or not submitting them at all when it matters most.

Results should be submitted on every run, pass or fail, not just on green builds. A results feed that only shows successes tells you nothing useful. The runs that fail are exactly the ones a test manager or lead needs visibility into.

The detail almost everyone forgets: the step that posts results has to run even when the test step before it fails. In GitHub Actions, that means using if: always() on the reporting step, or an equivalent construct. Without it, a workflow with a default configuration skips every step after a failure. That means the one run where results mattered most is the one run that never gets recorded.

A worked example

Consider a suite of 900 test cases that takes 50 minutes to run in CI. A naive integration might try to post a result the moment each test finishes. That means 900 individual API calls fired off over the course of the run.

That's slow and chatty, and it puts unnecessary load on your instance. If the network connection to your test management server has any intermittent hiccups, you risk losing individual results with no clean way to tell which ones went missing.

A better pattern collects all 900 results in memory or in a local file as the suite runs. It posts a single payload at the end, after the always() step confirms the run has actually finished, success or failure. That's one API call instead of 900. It's easier to retry as a unit if the request fails. It also gives you one clear timestamp for when a run's results landed, which matters later when someone is trying to reconstruct what happened on a particular Thursday's release.