DevOps Architecture

[DevOps Architecture] Deploy Strategy

[DevOps Architecture] Deploy Strategy

About this article

As the eighth installment of the “DevOps Architecture” category in the series “Architecture Crash Course for the Generative-AI Era,” this article explains deploy strategy.

The mainstream culture today is deploying frequently, small, safely. Deploy frequency, failure rate, and recovery speed are the core of DORA4 metrics. This article covers strategies like Rolling / Blue-Green / Canary / Feature Flag / Shadow Deployment / Dark Launch, rollback design, and AI-era auto Canary judgment.

Before you read this

This article is mostly about the flow of building, testing, releasing and monitoring a service. If IT vocabulary is unfamiliar, reading the primer "From Development to Operations" first makes it far easier to follow. You can also look anything up in the glossary as you read.

What is deploy strategy

CD Environment Promotion Flow

Deploy strategy is “the playbook for how you deliver a new version of software to the production environment.”

Think of road construction. You could shut down every lane at once and do the work (all-at-once deploy), or you could close one lane at a time and keep the rest open (rolling update). For extra caution, you could repave a side road first to check for problems before moving to the main road (canary release). Software is the same — deciding “to what scope, in what order, and with how much safety margin” you roll out a new version ahead of time is deploy strategy.

Why deploy strategy is needed

First, because a failed deployment is the single largest source of incidents. A large share of production outages happen at the moment of release, and how you deploy is therefore a reliability decision. Second, because deploying frequently raises quality. Small, frequent changes shrink the blast radius of each one and shorten the time to identify a cause. Third, because it underwrites the speed of the business. Being able to ship a feature the day it is ready, rather than waiting for a monthly window, is a competitive property.

The main strategies

There are multiple patterns in deploy strategies. Each differs in risk, cost, and complexity, so use according to system.

Deployment Strategy Pattern Comparison Like road construction. Close all lanes, one lane at a time, or test on a side road? Rolling Update Rolling Update K8s default No extra infra needed / simple Slow rollback / old-new coexistence Risk: Medium Blue-Green Blue Green Switching Switch 2 environments instantly Instant switch & rollback 2x infrastructure cost Risk: Low Canary 5%→20%→100% Gradual rollout Minimize impact on anomalies Auto-rollback capable Risk: Lowest Feature Flag Code is in production, feature is OFF Separate deployment from release A/B testing, staged release, emergency stop LaunchDarkly / Unleash / Flagsmith Risk: Lowest Recreate Full stop → start new version Simplest but causes downtime Maintenance notice required For internal tools / batch processing Risk: High Shadow Run new version in parallel with production Responses are discarded (for verification) Verify new version with production traffic Large-scale replacement / ML model comparison Risk: Low Canary + Feature Flag is the standard Deploy small & often — each incident is smaller and rollback is easier
StrategyContentRisk
Rolling UpdateSequential replacementMid
Blue-GreenSwitch between 2 environmentsLow, 2x cost
CanaryOnly some users on new versionLowest
Feature FlagCode in production, feature OFFLowest
RecreateStop all → start newHigh
ShadowRun new in parallel with productionLow, verification-oriented

Among them, Canary matters because it can catch problems visible only in production early. Expose the new version a little to real user behavior, real data distribution, and real traffic patterns unreproducible in test env, and on anomaly, retreat with minimum affected users. Cases of breaking only in production despite no problems in staging aren’t rare, and Canary is the last safety net.

But Canary has corresponding implementation costs. Traffic routing splitting requests by ratio (Service Mesh or Ingress weighting), and metric auto-judgment comparing new and old error rates and latencies (Flagger, Argo Rollouts, etc.) are needed - starting from Canary at the stage without monitoring foundation is hard. That’s why “Canary + Feature Flag” is told as the modern standard, adopted as the equipment set for fine control and minimizing problem impact.

Rolling update and blue-green

The method of sequentially replacing old version with new. Kubernetes’s default strategy, replacing some pods to new version. Simple with no additional infrastructure, but the downside of slow rollback on problems.

ProsCons
No additional infrastructureSlow rollback
SimpleLong mixed old-new state
K8s standardTrouble on schema changes

Suited for minor updates but unsuited for schema changes or incompatible changes.

Blue-green is the other half of that pair.

The method of preparing 2 production environments (Blue, Green) and switching with load balancer. Blue is current, Green is new - if no problems, switch traffic to Green; on problems, just return to Blue.

Blue-Green Deployment Switching Flow Prepare 2 production environments and switch instantly via load balancer User Traffic Load Balancer Switch Blue (Current) App v1.2.0 DB (shared) Standby (instant rollback if issues) Green (New) ACTIVE App v1.3.0 DB (shared) Active (traffic goes here) Instant switch & rollback is the biggest strength. 2x infrastructure cost is the trade-off
ProsCons
Instant switch / rollback2x infrastructure cost
Can handle DB schema changesCare needed with shared DB
Usable as verification envData integrity issues

Canary — catching early what only production shows

The method of phased deployment of new version to some users (5% → 20% → 50% → 100%). Problems can be detected early, minimizing affected users. As the safest deploy strategy, standardized in SRE organizations.

PhaseRatioObservation period
Phase 11-5%15 min
Phase 220%30 min
Phase 350%1 hour
Phase 4100%-

Auto-monitor error rate and latency at each phase, ideally auto-rolling back on anomaly. Tools like Argo Rollouts, Flagger, Spinnaker support this.

Feature flags — separating deployment from release

The method of deploying code to production while dynamically controlling feature ON/OFF. Separates deploy and release, creating the state of “deployed but invisible to anyone.” Usable for A/B tests, phased releases, emergency stops too.

ToolCharacteristics
LaunchDarklyEnterprise standard
FlagsmithOSS version available
UnleashOSS, GitLab-integrated
PostHogIntegrated with analytics
Custom implementationConfig files or DB

Separating “code deploy” and “feature release” is the modern thinking, dramatically lowering risk.

Database migration and automatic rollback

Deploys with DB schema changes are the most difficult area. Special design is needed to maintain code-DB integrity while changing without downtime. Expand and Contract pattern is standard.

StepContent
ExpandExtend schema to dual-support old and new
Code updateStart using new column
BackfillFill new column with old data
ContractDrop old column

Trying to do everything in one release stops production. Phased migration over multiple releases is the modern best practice.

Automatic rollback is the other half.

Mechanism to auto-revert to previous version on detecting anomaly after deploy. No waiting for human judgment, minimizing damage. Auto-judges by monitoring metrics (error rate, latency, SLO).

TriggerExample
Sudden error-rate spike5xx rate doubles
Latency degradationP95 exceeds threshold
SLO violationBurn rate exceeds 10x
Manual judgmentRoll back with 1 button

Not depending on human operators is modern - creating a 24-hour auto-protected state.

Three scenarios

If you are building solo or on a small web service

GitHub Actions with a rolling update is enough. A feature flag built yourself from a config file or a database column is fine, and no extra infrastructure is needed. Start from deploying automatically to production on a green CI run and reverting to the previous commit by hand if something goes wrong.

Personal / Startup: Ship in One Month Is Correcten.senkohome.com/arch-intro-case-startup/

If you are a small or mid-size SaaS

This is the stage for GitHub Actions with a canary (Argo Rollouts) and feature flags (Unleash, PostHog). Roll out in three steps of 5, 20 and 100 percent, monitoring error rate and P95 latency automatically and rolling back automatically on an anomaly. Databases migrate without downtime through expand and contract. As microservices multiply, moving to ArgoCD with GitOps and Flagger for independent per-service deployment makes hundreds of deployments a month realistic.

Small-Mid SaaS - Lean on Managed and Run with Few Peopleen.senkohome.com/arch-intro-case-saas/

If you are a large enterprise or in finance

Blue-green with a pre-approval workflow and a change advisory board. Fix release times outside business hours, apply database schema changes in a separate release beforehand, and retain audit logs permanently. Where the cost of an incident is extremely high, being certain you can retreat matters more than speed.

Large-Enterprise Core: Design That Holds Up for Yearsen.senkohome.com/arch-intro-case-enterprise/

DORA 4-metric numerical gates

Note: Industry baseline values as of April 2026. Will become outdated as technology and the talent market shift, so requires periodic updates.

The world standard for measuring deploy-strategy quality is the DORA 4 metrics.

DORA metricEliteHighMediumLow
Deploy frequencyMultiple/dayWeekly-dailyMonthly-weeklyLess than monthly
Lead time for changes (commit → prod)Within 1 hour1 day-1 week1 week-1 month1-6 months
Change failure rate0-15%16-30%16-30%46-60%
Recovery time (MTTR)Within 1 hourWithin 1 day1 day-1 week1 week-1 month

Canary deployment numerical gates: Phase 1: 1-5% / 15 min observation, Phase 2: 20% / 30 min, Phase 3: 50% / 1 hour, Phase 4: 100%. Auto-rollback on error rate exceeding old version + 0.5% or P99 > old version x 1.2. This is the Argo Rollouts / Flagger standard judgment criterion.

To aim for Elite level, multiple daily deploys + within-1-hour recovery. Canary + Feature Flag is the premise.

AI decision axes — AI assists the deployment decision

AI assists deploy judgment

The judgment logic for SLI (error rate, latency) during Canary deployment is an area AI can analyze and propose. Rules like “auto-rollback if P99 latency exceeds 300ms on 5% Canary traffic” are defined in code, and AI validates threshold reasonableness from past deploy history.

Additionally, automated analysis flows where AI detects anomalies from post-deploy logs and metrics, estimates “what’s causing the diff from last deploy,” and notifies via Slack are becoming widespread.

Feature Flag and AI-generated code compatibility

When delegating code generation to AI, having Feature Flags as a premise enables safe releases. Place AI-written code under a Feature Flag, first expose to internal users only → gradually roll out if no issues - this flow minimizes quality risk from AI-generated code.

Pitfalls and forbidden moves

Here are the six most dangerous ways a deployment goes wrong. Every one of them carries enough force to tip a company over.

Forbidden moveWhy it is bad → what to do instead
Deploying the new version to every server at oncethe Knight Capital pattern → require a canary and staged rollout
Running the database migration and the code deploy togetheryou cannot roll back when they disagree → separate them with expand and contract
Treating a green CI run as licence for 100 percent rolloutdefects that only appear in production get missed → passing CI is a necessary condition, not sufficient; stage the rollout
No automatic rollbackwaiting for a human decision enlarges the damage → configure an SLI-based automatic judgement
The “large and rare” big-bang release once a montheach change grows and the impact of a failure becomes enormous → small and frequent, as a rule
Leaving feature flags in place after usetwo hundred accumulate and nobody knows which are live → make flag removal a task once the rollout completes

Setting the deployment time during peak business hours, and never rehearsing a rollback, are the operational slack that breeds incidents too. Once a quarter is the guideline for a rehearsal.

Author’s note - “just one deploy” that erased a company

Cases of breaking down by underestimating deploy strategy are repeatedly carved into industry history.

The August 2012 Knight Capital incident is the extreme north of these lessons. Knight Capital, a major US market maker, on releasing new auto-trading features, put production live with old code remaining on 1 of 8 trading servers. The old code mistook a new flag for a different feature, triggering massive unintended auto-trades, losing about $440M in just 45 minutes (exceeding then-equity), with the company effectively bankrupt. A symbolic case where one human error of “missing just 1 server in deploy steps” erased a public company.

Another famous one is the June 2021 Fastly global outage. At major CDN vendor Fastly, one customer config change tripped a latent bug, simultaneously dropping major worldwide sites - Reddit, Amazon, UK government sites, NYT, CNN - for about 1 hour. A case told to show modern deploy-risk depth where “even protecting your own production, one upstream config change stops the world.”

Both have “deploy-strategy laxness” as the lethal blow, slapping home that without Canary, Feature Flag, and auto-rollback equipment, human errors directly link to corporate life-or-death.

Recording decision rationale

Deploy-strategy selection directly impacts incident risk and release speed, so recording why you chose that strategy as an ADR is important.

ItemContent
TitleAdopt Canary Release as deploy strategy
StatusApproved
ContextAn EC site with 500K monthly active users experienced 2 full-deploy incidents over the past 6 months (total revenue impact: ~$60K). Want to limit incident blast radius while raising release frequency from weekly to daily
DecisionUse Canary Release (initial traffic 5% → phased expansion) as the standard deploy strategy
Rationale- Incident impact limited to 5% of users, minimizing revenue loss
- Monitor error rate and latency SLIs, auto-rollback on threshold breach
- ~Half the infra cost vs Blue-Green (no need to duplicate entire environment)
Rejected alternativesBlue-Green → Maintaining 2 full production environments adds ~$60K/year. Rolling Update → Risk of propagating to all nodes on failure is unacceptable at 500K-user scale
OutcomeIntroduce Argo Rollouts for Canary phase control. SLI dashboard and auto-rollback rule setup are prerequisite tasks

Store ADRs in docs/adr/ as Markdown, with a rule to always file a new ADR when changing deploy strategy - this keeps decision history traceable. The greatest value of ADRs is that when you look back later, “why we made this choice” is immediately clear.

What to decide - what is your project’s answer?

For each of the following, try to articulate your project’s answer in 1-2 sentences. Starting work with these vague always invites later questions like “why did we decide this again?”

  • Deploy strategy (Rolling / Blue-Green / Canary)
  • CI/CD tool (GitHub Actions etc.)
  • Feature Flag adoption (LaunchDarkly etc. or custom)
  • Phased-release ratio (5→20→50→100)
  • Auto-rollback criteria (SLI threshold)
  • DB-migration strategy (Expand / Contract)
  • Deploy-frequency target (daily / weekly / monthly)

Summary

This article covered deploy strategy, including Rolling, Blue-Green, Canary, Feature Flag, Progressive Delivery, auto-rollback, and zero-downtime DB migration.

Deploy small and frequently, make Canary+Feature Flag standard, auto-rollback by SLI, non-stop DB changes via Expand and Contract. That is the practical answer for deploy strategy in 2026.

Next time we’ll cover monitoring and observability (metrics, traces, log integration).

Back to series TOC -> ‘Architecture Crash Course for the Generative-AI Era’: How to Read This Book

I hope you’ll read the next article as well.

📚 Series: Architecture Crash Course for the Generative-AI Era (67/95)