SLOs and Error Budgets: A Practical Guide for Engineering Teams

SLOs and error budgets turn reliability into a number engineering and product agree on. A service level objective sets a target for how often a service must work well, such as 99.9% of requests succeeding over 30 days. The error budget is the remaining 0.1%, which you can spend on releases and experiments before slowing down to fix things.

The idea is simple, but teams often get stuck in the details. Which metrics count? How strict should the target be? What happens when the budget runs out? This guide answers those questions in the order you will meet them.

SLIs, SLOs and SLAs: the differences

These three terms get mixed up constantly. They are related, but each has a different job.

A useful rule: your SLO should be stricter than any SLA you sign. If the SLA promises 99.5%, an internal SLO of 99.9% gives you warning well before you owe anyone a credit.

Choosing SLIs that reflect user experience

A good SLI measures what users feel, not what your servers feel. CPU usage and memory are useful for diagnosis, but a user does not care that a node is at 90% CPU if their page loads quickly.

The most common SLI types are:

• Availability: the proportion of requests that succeed (for example, not a 5xx response).
• Latency: the proportion of requests faster than a threshold, such as 300 ms for an API call.
• Quality or correctness: the proportion of responses that return complete, correct data, which matters for search, recommendations and data pipelines.
• Freshness: for data products and batch jobs, how recently data was updated.
• Throughput or completion: for pipelines and queues, the proportion of jobs finished on time.

Express SLIs as a ratio of good events to valid events. "Good requests divided by total requests" is easy to reason about, easy to aggregate and maps directly to an error budget. Measure as close to the user as practical: load balancer logs or synthetic checks often tell a truer story than application metrics alone.

Start with a small number. Two or three SLIs per critical user journey, such as login, search and checkout, are enough. Teams that define twenty SLIs for one service rarely look at any of them.

Setting SLO targets

The temptation is to pick a number with lots of nines. Resist it. Every extra nine costs real engineering effort, and users often cannot tell the difference beyond a certain point because their own network or device is less reliable than your service.

A practical way to set targets:

1. Look at history. Measure the SLI for the past few months. If you have delivered 99.7% without heroics, that is a sensible starting point.
2. Ask what users notice. Talk to product and support. At what point do complaints rise? That threshold is closer to the right target than an arbitrary figure.
3. Consider dependencies. A service cannot be more reliable than the dependencies it needs on every request, unless you design for failure with caching or fallbacks.
4. Pick a window. Rolling 28 or 30 day windows are common. They smooth out single bad hours without hiding a bad month.
5. Review quarterly. Tighten the SLO if users expect more and you can deliver it; loosen it if you are spending effort on reliability nobody notices.

To make the numbers concrete: over 30 days, a 99.9% availability SLO allows roughly 43 minutes of full downtime, and 99.5% allows about 3.6 hours. Error budgets are usually counted in failed requests rather than minutes, but the time view helps people understand the trade-off.

Writing an error budget policy

An error budget without a policy is just a dashboard. The policy says, in advance, what the team will do as the budget is spent. Agreeing it before an incident avoids arguments in the middle of one.

A simple policy might say:

• Budget healthy: ship as normal, run experiments, take calculated risks.
• Budget more than half spent early in the window: review recent changes, add tests or canary steps for risky releases.
• Budget exhausted: pause feature releases for that service, except urgent fixes and security patches, until the SLI recovers. Engineering time moves to reliability work identified in recent post-mortems.
• Repeated exhaustion: escalate to engineering and product leadership to agree a reliability plan or revisit the SLO.

The policy needs sign-off from product as well as engineering. Its whole value is that both sides agreed the rules when nobody was under pressure.

Alerting on burn rate

Alerting on every failed request or brief latency spike produces noise. Alerting only when the monthly SLO is breached is too late. Burn rate alerting sits in between.

Burn rate is how fast you are consuming the error budget relative to the window. A burn rate of 1 means you will use exactly the whole budget by the end of the window. A burn rate of 10 means you will use it in a tenth of the time.

A common pattern uses two kinds of alert:

• Fast burn: a high burn rate over a short period (for example, an hour), which pages someone immediately because users are being hurt now.
• Slow burn: a lower burn rate sustained over a longer period (for example, several hours or a day), which raises a ticket for the next working day.

Pair each long window with a shorter one, so alerts clear quickly once the problem is fixed. Tune the thresholds to your traffic; low-traffic services need care, because a handful of failures can look like a huge burn rate.

On-call and incident severity

SLOs give on-call engineers a clear test: is the error budget burning fast enough that users are affected? That replaces many alerts based on raw infrastructure metrics, which cuts noise and makes pages more meaningful.

Severity levels then set the response. Our managed services work to these target response and resolution times:

Map severities to SLO impact where you can. A fast-burn alert on a critical journey is usually P1 or P2; a slow burn is usually P3. Keep rotations humane: enough people that nobody is paged every other week, clear escalation paths and handovers at the end of every shift.

Blameless post-mortems

Every significant incident, and every exhausted error budget, should produce a post-mortem. Blameless does not mean nobody is accountable. It means the review focuses on how the system, process and tooling allowed the failure, because people make reasonable decisions with the information they have.

A good post-mortem covers:

1. A timeline of detection, response and recovery.
2. User impact, expressed in SLI and error budget terms.
3. Contributing factors, not a single root cause.
4. What went well, so you keep doing it.
5. Action items with owners and dates, tracked to completion.

Review action items in a regular reliability meeting. Post-mortems that produce actions nobody completes teach the team that the process does not matter.

How a managed SRE team runs SLOs and error budgets 24x7

Running SLOs well across time zones takes more than a good dashboard. When we run SRE as part of our managed services for a client, onboarding follows four stages:

1. Assess: map critical user journeys, review existing monitoring and incidents, and draft SLIs and SLOs with your product and engineering leads.
2. Transition: build SLO dashboards and burn rate alerts, write runbooks, shadow your on-call and take over rotations gradually.
3. Stabilise: reduce alert noise, fix the recurring issues behind most pages and agree the error budget policy.
4. Optimise: review SLOs quarterly, automate common fixes and feed post-mortem actions into the roadmap.

Staggered shifts give follow-the-sun coverage without overnight heroics, and handovers include current budget status for every critical service. Your team keeps ownership of priorities and architecture. If you are weighing this against building your own rotation, our comparison of managed DevOps and in-house teams sets out the trade-offs.

Frequently asked questions

What is the difference between an SLO and an SLA?

An SLO is an internal reliability target your team aims for, such as 99.9% of requests succeeding over 30 days. An SLA is a contract with customers that sets consequences, often service credits, if a lower threshold is missed. Keep the SLO stricter than the SLA so you have warning before contractual impact.

How do you calculate an error budget?

Subtract the SLO from 100%. A 99.9% SLO gives a 0.1% error budget, so if you serve ten million requests in the window, around ten thousand can fail before the budget is gone. Track consumption continuously and compare it with time elapsed in the window.

What happens when the error budget runs out?

That depends on your error budget policy, which should be agreed in advance. Typically, feature releases for the affected service pause, except urgent and security fixes, and engineering effort moves to reliability work until the SLI recovers.

What is a good SLO target to start with?

Start from what you already deliver, measured over the past few months, and check it against what users notice. Many teams begin around 99.5% to 99.9% for critical user-facing services, then adjust each quarter. Avoid targets stricter than your dependencies can support.

Do small teams need SLOs?

Yes, perhaps more than large ones. A few well-chosen SLOs replace dozens of noisy alerts, help a small team decide when to ship and when to fix, and make conversations with product about reliability far more concrete.

How Sunday Labs can help

Sunday Labs runs SRE, monitoring and 24x7 support as a managed service, alongside your own engineering team. Senior engineers define SLIs with you, build burn rate alerting, run on-call against clear severity targets and lead blameless post-mortems, with founders reviewing the work every week. If you want reliability you can measure rather than argue about, start a conversation and we will help you set your first SLOs.