L4Go deeper4 min read

SLOs and error budgets

The grown-up version of "is it good enough": an explicit reliability target and an agreed allowance for falling short. Gives quality arguments a number instead of an opinion, which is how they start being won.

Every team eventually has the same argument: engineering wants to ship a risky change, QA wants more testing time, and there is no shared number to settle it. Service level objectives and error budgets exist to end that argument by turning "how reliable do we need to be" into an explicit, agreed number that both sides can point at instead of arguing from gut feeling.

SLIs, SLOs, and SLAs are not the same thing

A service level indicator (SLI) is a measurement: the percentage of requests that succeed, or the percentage that complete under 300ms. A service level objective (SLO) is a target for that measurement over a time window, such as "99.9% of requests succeed over a rolling 30 days." A service level agreement (SLA) is a contractual promise to a customer, usually looser than the internal SLO, with financial penalties attached if missed.

Teams often skip straight to picking an SLA number without ever defining the SLI it depends on, which produces targets nobody can actually measure or defend. Start with the SLI: pick a metric that reflects what the user experiences, not what is easiest to query from your existing dashboards.

The error budget: your permission slip to take risks

An error budget is simply 100% minus your SLO, expressed as an amount of allowed failure. A 99.9% availability SLO over 30 days allows about 43 minutes of downtime in that window. That 43 minutes is the error budget, and it exists to be spent, not hoarded.

This reframing changes how release decisions get made. Instead of QA arguing against every risky deployment on principle, the question becomes concrete: how much of the remaining error budget would this change plausibly consume, and do we have room for it this month? A team that has already burned through its budget from an earlier incident has a real, numeric reason to slow down, not just a vague sense of caution. This is the same discipline behind tracking change failure rate: both turn a subjective risk call into a number someone can defend.

A worked example: setting an SLO for a checkout API

A subscription box company set out to define an SLO for their checkout API after a string of complaints that checkout "felt slow sometimes" without a clear definition of what "sometimes" meant. They picked two SLIs: the percentage of checkout requests completing successfully, and the percentage completing under 800ms.

Looking at three months of historical data, actual performance sat around 99.95% success and 96% under 800ms. Setting the SLO at their current performance would leave no error budget to spend on anything. Instead they set the availability SLO at 99.9% and the latency SLO at 95% under 800ms, giving themselves deliberate room to take some risk on upcoming changes without immediately breaching the target.

error-budget-calc.txt

SLO: 99.9% success over 30 days
Total requests in window: 12,000,000
Allowed failures: 12,000,000 x 0.001 = 12,000
Failures so far this month: 7,400
Remaining error budget: 4,600 failures (about 38% of budget left)

Where QA fits into an SLO practice

QA's role is not to set the SLO alone, since that number needs input from product on what customers tolerate and from engineering on what is achievable. QA's role is to catch changes likely to burn the error budget before they ship, by weighting test effort toward the SLIs the business actually tracks rather than toward areas that are easy to test but do not move the number.

This also changes how test failures get triaged. A flaky test on a feature with no SLO attached is an annoyance. A flaky test on the exact endpoint your error budget tracks is a signal worth chasing down immediately, because it is measuring the same thing production will measure once the change ships.

Map your test suite's critical-path tests to the SLIs they protect.
Flag any SLO-relevant endpoint that lacks dedicated test coverage.
Treat a burn-rate alert (budget being consumed faster than the window allows) as a release-blocking signal, not just an ops concern.

FAQ

Questions people ask

What happens if we blow through the error budget every month?

That means your SLO is set below what the system can actually deliver given current investment. Either the target needs to loosen, or the team needs to invest in reliability work before taking on more feature risk.

Should every service have an SLO?

No. Reserve SLOs for services where reliability directly affects users or revenue. An internal admin tool used by three people rarely needs the same rigor as a public checkout API.

How does an error budget relate to [incident response](/insights/roadmap/incident-response-and-rca)?

A single bad incident can consume a month's entire error budget at once. That is intentional: it forces a real conversation about whether the incident's root cause needs fixing before more risk is taken on.

Is a 99.99% SLO always better than 99.9%?

Not automatically. Each additional nine costs disproportionately more engineering effort to achieve, and if users cannot perceive the difference, that effort is better spent elsewhere.