Cloud basics: environments, config, secrets
Where the thing runs, how it gets its configuration, and how credentials reach it without being committed. Enough to debug an environment-only failure and to avoid being the person who pasted a production key into a test fixture.
Most testers can run a test suite long before they understand why it passed on staging and failed in production. The gap is almost never the test itself. It is the environment the test ran against: a different config value, a missing secret, a service pointed at the wrong host. This post covers what an environment actually is, and how configuration reaches an app without being baked into the code. It also covers how credentials get there without ending up in a git commit. It is enough to debug an environment-only failure, and to avoid being the person who pastes a production key into a test fixture.
What "environment" actually means
An environment is a specific combination of code, configuration, and infrastructure running together. Dev, staging, and production usually run the same code but different configuration: different database, different API keys, sometimes a different feature flag set entirely. When a bug only reproduces in one environment, the first question is never "is the code broken." It is "what is different about this environment's configuration."
A team shipping a checkout flow might see a discount code test pass in staging and fail in production. The code is identical. The difference turns out to be a feature flag that ships the new pricing engine to staging only, because production traffic is still on the old one. This is a case where the test oracle shifted under the test without the assertion changing. That is a configuration difference wearing a code-bug costume, and it is one of the most common shapes a flaky-looking failure takes.
How configuration reaches the app
Most modern apps follow a version of twelve-factor config. Settings that vary between environments live outside the codebase, injected as environment variables at runtime. A DATABASE_URL, an API_BASE, a FEATURE_FLAGS_ENDPOINT. The code reads process.env.DATABASE_URL and never hardcodes which database that actually is.
Config typically arrives through one of three routes. It might be a .env file loaded at startup. It might be environment variables injected by the CI runner or container orchestrator. Or it might be a remote config service like AWS Parameter Store or a feature flag platform. Reading a container's environment block often tells you exactly which route an app uses before you ever open its code.
Secrets are configuration that must not leak
A secret, an API key, a database password, a signing token, is configuration with consequences if it leaks. The rule that matters for testers is simple: a secret should never live as a literal string in a test fixture, a committed .env file, or a hardcoded constant in test code. It should be injected at run time from a vault or a CI secret store, scoped as narrowly as possible, and rotated if it ever escapes.
# Confirm which environment a running service actually points at
curl -s https://staging.example.com/health | jq '.env, .apiBase'
echo $DATABASE_URL | sed -E 's/(:\/\/[^:]+:)[^@]+(@)/\1***\2/'This matters for QA specifically because test fixtures look disposable and are not. A JSON fixture copied from a real staging response, or a recorded HTTP cassette, can carry a genuine credential. The fix is generating fixtures with clearly fake values instead of copying live data, the same discipline that matters for test data management. The secrets management post covers scanning tools and vaulting patterns in more depth.
Debugging an environment-only failure
When something only fails in one environment, work through a short checklist before touching the test code. This overlaps heavily with the discipline behind flakiness triage. An environment mismatch is one of the most common root causes hiding behind a flaky label:
- Diff the environment variables between the passing and failing environment, not just the ones you expect to matter.
- Check whether a feature flag or remote config value differs, since these change behavior without a deploy.
- Confirm the service is actually pointed at the database or API host you think it is, not a stale value from a cached container image.
- Look for a secret that expired or was rotated in one environment but not the other, which produces auth failures that look like flakiness.
Consider a QA engineer on a mid-size SaaS team who spent a full day chasing an intermittent login failure that only happened in the CI pipeline. The suite worked locally every time. The eventual cause was that CI injected a SESSION_SECRET that rotated on every deploy. A cached auth token from a previous run was still being reused by a fixture. The fix was regenerating the token as part of test setup instead of storing it as a static fixture value. That change also removed a secret that had been sitting in the fixtures folder for months.
FAQ
Questions people ask
What is the fastest way to tell if a failing test is a config problem or a real bug?
Run the same test against a different environment with the same code deployed. If it passes there, the code is fine and the difference is configuration, a feature flag, or a secret.
Should test fixtures ever contain real secrets, even for a quick local test?
No. Use clearly fake values generated the same way as any other test data. A real secret in a fixture folder outlives the person who put it there.
Where do environment variables usually come from in a CI pipeline?
A secrets manager like Vault or AWS Secrets Manager, injected by the CI runner at job start, or a repository-level secret store like GitHub Actions secrets.
How is staging supposed to differ from production if they run the same code?
Only in configuration: database, API keys, feature flags, and sometimes traffic volume. If the code diverges too, staging stops being a reliable signal.