[DevOps Architecture] SRE Practices

About this article

As the thirteenth installment of the “DevOps Architecture” category in the series “Architecture Crash Course for the Generative-AI Era,” this article explains SRE practices.

SRE is engineering that eliminates operations - reducing manual work is the main job. This article handles toil reduction, prioritization via SLO, error-budget operations, chaos engineering, on-call rotation, and team topology (Embedded SRE / Platform SRE) - design that operationally translates Google-originated operational engineering.

What are SRE practices in the first place

Picture factory production management. A well-run factory doesn’t have workers manually checking quality every time — it has automated inspection, early defect detection, line-halt criteria, and improvement cycles built into the system. Maintaining quality through systems rather than individual skill is the essence of production management.

SRE practices are production management for system operations. They are the concrete methods Google systematized — toil reduction, prioritization via SLO, error-budget operations, chaos engineering — maintaining and improving operational quality without depending on specific individuals.

Without SRE practices, operations become an endless repetition of manual work. Engineers get consumed by alert response and can’t invest time in essential improvements.

Why SRE is needed

First, because cloud and microservices have complicated operations. The number of moving parts has grown past what manual operation absorbs. Managing a modern system running hundreds of services by hand is impossible, which makes operating through code mandatory. Second, because development speed and reliability have to be balanced, and the only way to balance them is numerically. Traditional operations teams tended to prefer “stop the changes and stabilise” and ended up opposed to development; SRE mediates between the two with a number, the error budget. Third, because the operational burden on engineers has to come down; an organisation that spends its engineering time on toil never improves. Manual on-call and alert handling wears engineers down; automation cuts the load and lets them focus on solving the real problems.

The main SRE practices

The main 8 practices systematized in Google’s SRE book. Combine these to build operational culture.

Reducing toil — the real job of SRE

Repeated manual work and automatable operational work is called Toil at Google, with the explicit goal of keeping it under 50% of SRE time. Beyond 50%, SREs can’t develop and the org can’t generate value.

Toil isn’t evil but is work that doesn’t grow you. SREs writing code to eliminate it is the main job, with the ideal being investing 20%+ monthly in automation.

Error-budget operation is the other core practice.

The core of SRE is disciplining operations by error budget against SLO. Explicit switching - accelerate development while error budget remains, redirect to reliability investment on exhaustion.

Error budget remaining 70% --> Ship new features aggressively
Error budget remaining 30% --> Cautious, careful
Error budget remaining 5%  --> Freeze releases, stabilize
Error budget remaining 0%  --> Stop new features, focus on quality

This mechanism dissolves dev-vs-ops conflict. Decision can be made on the objective fact “budget exhausted” rather than “stop releases.”

Chaos engineering and the production readiness review

The technique of intentionally causing failures in production to verify system fault tolerance. Originated with Netflix’s Chaos Monkey, “randomly killing instances in production” forces designs that constantly tolerate failures.

The premise is system architecture “on the premise that failures happen,” and periodically practicing breaking builds an org that doesn’t panic during real-world failures.

Spreading the SLO culture is what makes either of them stick.

Just installing SLOs doesn’t function. Settling as organizational culture is needed - the following habits matter.

The core of cultural transformation is discarding “100% uptime = ideal.” SLO is a symbol of “don’t aim for perfection, aim for sufficient.”

Capacity planning sits alongside them.

Plans for future traffic growth. Falling behind links directly to incidents and missed opportunities, so it’s an important area SRE continuously works on.

The safer strategy for services expecting rapid growth is holding more margin.

The production readiness review is the gate before launch.

The mechanism of auditing before putting new services into production. Called PRR at Google, where the SRE team evaluates dev-team services.

The gate of don’t put services failing PRR into production guarantees quality.

Its relationship with DevOps is worth stating plainly.

SRE is positioned as the concrete implementation of DevOps. If DevOps is philosophy / culture, SRE is the practical pattern.

It’s also said that “DevOps is the ideal, SRE is the implementation pattern.”

Platform engineering — where SRE develops next

Developing SRE thinking, the dedicated team improving in-house developer experience is Platform Engineering. Provides an Internal Developer Platform (IDP), preparing environments where dev teams can autonomously and safely deploy / operate.

Backstage (OSS by Spotify) is the representative IDP, adopted by many companies.

Three scenarios

If you are building solo or on a small team

Developers double as SREs and phases 1 and 2 are enough. Getting a metrics platform in place with something like CloudWatch and measuring SLIs is a good result. SLO operation, error budgets and production readiness reviews can wait until the organisation matures. On toil, finding “any manual task taking more than two hours a month” and automating it is enough.

Personal / Startup: Ship in One Month Is Correcten.senkohome.com/arch-intro-case-startup/

If you are a small or mid-size SaaS

One or two engineers doubling as SREs, trialling SLOs and growing a toil-reduction culture. Push through to phase 3, make release decisions on the error budget, and manage runbooks in Git. Once you can form a dedicated team of three or more, production readiness reviews, monthly chaos exercises and an internal developer platform on Backstage start to raise development speed.

Small-Mid SaaS - Lean on Managed and Run with Few Peopleen.senkohome.com/arch-intro-case-saas/

If you are a large or regulated enterprise

A two-tier structure of a central SRE function and business-unit SREs, combined with AIOps. The centre supplies the company-wide golden path while business units run their own SLOs. Automating routine response with tools such as Datadog Bits AI or Resolve AI leaves people free to concentrate on strategy and improvement design.

Large-Enterprise Core: Design That Holds Up for Yearsen.senkohome.com/arch-intro-case-enterprise/

Phased SRE-maturity roadmap

SRE is a cultural transformation that doesn’t realize overnight. Phased introduction compliant with Google SRE Workbook is realistic.

The Toil target line is under 50% of SRE time - Google’s official guideline. Over 50% means SREs can’t develop and the org can’t generate value. Allocating 20% monthly to automation investment is the empirical rule keeping Toil at sustainable levels.

SRE is not signs but how time is spent. Toil over 50% is sign-only, not real SRE.

AI decision axes — AIOps is moving into its third stage

AIOps is being adopted in 3 stages in practice

As of 2026, practical AIOps adoption is progressing in the following 3 stages:

• Stage 1: Anomaly-detection automation (AI detects metric baseline deviations → alerts humans)
• Stage 2: Root-cause estimation (AI cross-analyzes logs, traces, and metrics for RCA → notifies via Slack)
• Stage 3: Auto-recovery (AI executes recovery operations per Runbook → humans confirm after the fact)

Most organizations are at Stage 1-2, but advancing to Stage 3 requires codified, trustworthy Runbooks and a design that strictly controls AI’s operational permissions via IAM.

Toil-automation ROI has changed with AI

Previously, Toil automation was judged by “development cost of automation scripts vs. repeated cost of manual work.” In configurations where AI reads Runbooks and auto-executes, there’s no need to write automation scripts from scratch - cases are increasing where just maintaining Markdown procedure documents lets AI execute them. The ROI of automation investment has improved dramatically.

Pitfalls and forbidden moves

Here are the six most dangerous ways adopting SRE goes wrong. Every one has the structure of changing the sign over the door and nothing else.

Author’s note - cases visualizing the gap between “SRE sign” and “real SRE”

Whether SRE is a surface sign or real cultural transformation greatly divides the org’s fate.

After Google launched its SRE team in 2003 and the practice was disclosed in the 2016 book “Site Reliability Engineering,” companies worldwide followed. But centered on Japanese companies, cases of just changing existing ops teams’ business cards to “SRE”, with reality remaining manual night on-call and phone response, came one after another. Without time or authority to write code, after a year Toil rate at 95%, SLO defined only formally with no one watching - non-laughing field cases are repeatedly told even today.

In contrast, Netflix is famous for thoroughgoing SRE thinking. Netflix has continuously killed production instances randomly with Chaos Monkey since the 2010s, with design on the premise of breaking in production becoming standard. As a result, even on AWS partial outages, Netflix alone calmly continues service - cases observed multiple times. What’s visible here is the fact that “SRE isn’t signs but a problem of how time is spent and culture.”

It’s not “calling yourself SRE makes you SRE” - only whether you can have time confronting Toil-reduction with code decides SRE’s essence.

What to decide - what is your project’s answer?

For each of the following, try to articulate your project’s answer in 1-2 sentences. Starting work with these vague always invites later questions like “why did we decide this again?”

• SRE-org placement (centralized / distributed)
• SLO management process (setup, review frequency)
• Error-budget operational rules (freeze criteria)
• Toil-reduction target (under 50%, automation investment rate)
• Chaos engineering (frequency, scope)
• PRR process (new-service introduction audit)
• AI tool adoption (AIOps, auto-diagnosis)

Related Articles

Summary

This article covered SRE practices, including main practices, Toil reduction, error-budget operations, chaos engineering, PRR, Platform Engineering, and AIOps collaboration.

Phase introduction, keep Toil under 50%, mediate balance via error budget, delegate routine to AIOps. That is the practical answer for SRE practices in 2026.

Next time we’ll cover documentation (README, ADR, Runbook).

Back to series TOC -> ‘Architecture Crash Course for the Generative-AI Era’: How to Read This Book

I hope you’ll read the next article as well.