About this article
This article is the sixth deep dive in the âSoftware Architectureâ category of the Architecture Crash Course for the Generative-AI Era series, covering transaction design.
The mechanism that guarantees âall-succeed-or-all-undoâ â the design core that prevents money-disappearing or money-appearing accidents like bank transfers. The article covers ACID properties, isolation levels, distributed transactions, eventual consistency, Saga / Outbox patterns, the CAP theorem, and idempotency, with axes for sorting consistency level per business need.
Before you read this
This article is mostly about development: programs and APIs. If IT vocabulary is unfamiliar, reading the primer "Programs and APIs" first makes it far easier to follow. You can also look anything up in the glossary as you read.
What is a transaction in the first place
A transaction is, in a nutshell, âa mechanism that makes a series of operations either all succeed or all roll back.â
Imagine an ATM bank transfer. When A sends $100 to B, âdeduct from Aâs accountâ and âadd to Bâs accountâ must succeed as a pair. If only one succeeds, $100 either vanishes or appears out of thin air. The mechanism guaranteeing âall succeed or all roll backâ is a transaction.
Why transaction design matters
What happens if transaction design is left vague? Money-disappearing or money-appearing accidents happen for real. Some data â bank-account balances â needs strict consistency; some â SNS âlike countsâ â can drift slightly without harm. Applying maximum consistency to everything makes the system too heavy; sorting precision per business need is the design core.
Transaction design must be considered in tandem with overall structure (especially microservices). If 1 DB suffices, ACID alone is enough â whether you can tilt toward a configuration that avoids distributed transactions directly affects operational cost.
What keeps âmoney from disappearingâ
Transaction design decides âwhich data needs what granularity of consistencyâ in the system. Some data â bank-account balances â needs strict consistency; some â SNS âlike countsâ â can drift slightly without harm. Applying maximum consistency to everything makes the system too heavy; sorting precision per business need is the design core.
It must be considered together with overall structure (especially microservices). If 1 DB is enough, ACID alone suffices; whether you can lean on a configuration that avoids distributed transactions ties directly to operational cost.
Consider it together with overall structure. If 1 DB is enough, ACID alone is enough.
ACID properties
The classic transaction guarantee RDBMSes have offered for decades is ACID â four properties whose initials guarantee data integrity strongly.
ACID matters because applicationsâ biggest losses come from âmoney / inventory / order data inconsistency.â Once data drifts, root-cause investigation, correction, and customer-facing costs balloon, shaking trust in the entire business.
ACID established with RDB development from the 1970s, designed assuming âstrong integrity guaranteed within one DB.â It worked perfectly when servers and DBs were one machine each. In the cloud-era distributed environment, maintaining ACID across multiple DBs and services requires locking all nodes; one network outage stops everything, making it operationally impractical. Thatâs why eventual consistency and Saga (covered later) emerged.
| Property | Substance |
|---|---|
| Atomicity | All-or-nothing; no partial state |
| Consistency | Always fail on constraint violations |
| Isolation | Concurrent execution doesnât interfere |
| Durability | Committed data survives failures |
Within a single DB, ACID is solidly sufficient â the default for finance, accounting, inventory, and other strong-consistency-required businesses.
Isolation levels
In ACID, the realistic trade-off with performance is âIâ (Isolation). The level you pick determines how strictly other transactions are kept apart. Stricter raises consistency but lowers concurrent performance.
| Level | Allowed phenomena | Performance |
|---|---|---|
| READ UNCOMMITTED | See othersâ uncommitted values | Fastest |
| READ COMMITTED | Same query may produce different results (non-repeatable read) | Fast |
| REPEATABLE READ | New rows may appear mid-transaction (phantom read) | Mid |
| SERIALIZABLE | All prevented (strictest) | Slow |
PostgreSQL defaults to READ COMMITTED, MySQL (InnoDB) to REPEATABLE READ. Higher isolation costs performance, so strengthen only the parts that need it.
The difficulty of distributed transactions
In microservices and multi-DB configurations, transactions across multiple DBs sometimes become necessary. Traditionally 2PC (Two-Phase Commit â ask all DBs âprepared?â then commit together) solved this, but it has fatal flaws in the cloud era.
2PC forces a two-phase exchange of âall nodes prepared -> all nodes commitâ, so any one node going down blocks everything. In cloud environments where network failures are routine, running 2PC in production frequently stops the whole service â âeffectively non-functionalâ in practice.
The cloud-native principle is to avoid 2PC. Use eventual consistency instead.
Eventual Consistency
Eventual consistency allows a relaxed guarantee: ânot consistent immediately, but consistent eventually.â Writes succeed instantly; propagation to other replicas and services happens asynchronously. So users may temporarily see stale data on screen.
Bank accounts canât drift even momentarily, but âlike counts,â âfollower counts,â âreview countsâ on e-commerce sites work fine even if 1 second stale. Treating âdata where slight drift is fineâ as eventually consistent secures the systemâs overall availability and scalability.
Examples: Amazon inventory counts, Twitter follower counts, Instagram likes, YouTube view counts.
Per data type, judge from business requirements whether strong consistency is required or eventual is sufficient.
Saga pattern
Saga is a pattern that implements distributed transactions as âa chain of small local transactions + compensation.â In a hotel-booking flow, âreservation creation,â âpayment,â âinventory reservation,â ânotificationâ run on different services; failure mid-flow triggers compensation to undo earlier steps.
Unlike 2PC, this doesnât lock all services; each step completes locally, so it works in cloud environments. But incomplete compensation design risks leaving half-done state.
Pseudocode for orchestration-style Saga: each step completes in its own local TX; on failure, compensation runs in reverse order.
async function bookHotel(input: BookingInput) {
const completed: Array<() => Promise<void>> = [];
try {
const reservation = await reservationSvc.create(input);
completed.push(() => reservationSvc.cancel(reservation.id));
const payment = await paymentSvc.charge(input.amount);
completed.push(() => paymentSvc.refund(payment.id));
await inventorySvc.reserve(input.roomId);
completed.push(() => inventorySvc.release(input.roomId));
await notificationSvc.notify(input.userId);
return { ok: true, reservationId: reservation.id };
} catch (err) {
// Compensate in reverse order (idempotent assumed)
for (const compensate of completed.reverse()) {
await compensate().catch(logCompensationFailure);
}
throw err;
}
}
Compensation must be idempotent. If retries running double donât produce stable results, the saga itself becomes a new source of consistency incidents. In production, combine with the Outbox pattern to reliably synchronize message sending and DB updates â the standard approach.
Two Saga forms
Saga has two implementation styles, differing in where flow control sits.
| Style | Trait | Pros | Cons |
|---|---|---|---|
| Orchestration | Central orchestrator controls order | Visible flow, easy debugging | Central-aggregation risk |
| Choreography | Event-driven, services act autonomously | Loose coupling, scales | Hard to grasp the whole |
When business is complex and order matters, Orchestration. When simple and services are independent, Choreography.
Default is starting from Orchestration. Systems where the whole isnât visible become operational hell.
Outbox pattern
Outbox harmonizes âDB writeâ and âmessage-queue sendâ in a single DB transaction. It prevents the recurring microservices trouble âDB succeeded but Kafka send failedâ (Kafka: distributed message-queue infrastructure) â the standard pattern.
- Record the message-to-send into the
outboxtable together with business data. - A separate process (Relay) reads
outboxand sends to the message queue. - Update the sent flag.
This prevents the accident of only one of âDB commitâ or âsendâ succeeding. For event-driven microservice architecture, near-mandatory.
Saga + Outbox is the default; near-mandatory for microservices.
CAP theorem
The foundation of distributed-system architecture is the CAP theorem: Consistency, Availability, and Partition tolerance canât be simultaneously satisfied â thatâs the constraint.
Network partitions happen in reality, so âPâ is essentially required. Practical choice becomes âCP (consistency-priority)â vs âAP (availability-priority).â Business requirements determine whether âerr out but be accurateâ or âkeep running with stale data.â
| Choice | Representative systems | Business |
|---|---|---|
| CP (consistency) | Traditional RDB, MongoDB (configurable) | Banking, payments, inventory |
| AP (availability) | DynamoDB, Cassandra, Redis | SNS, e-commerce browsing, analytics |
Reverse from business requirements. Picking distributed DBs without CAP awareness produces unexpected behavior.
Idempotency
In distributed systems, âthe same request arriving multiple timesâ due to network failures and timeouts is routine. The property guaranteeing the result is the same regardless of repeat execution is idempotency.
Without idempotent âpayment APIs,â retries cause double charges. Idempotency is achieved by clients attaching a request ID and servers deduplicating by that ID â the default. By HTTP method, GET / PUT / DELETE are spec-defined idempotent; POST is not, requiring care.
In microservices, event-driven, and retry processing, idempotency is required; bolting it on later is hard, so design from the start.
Decision criteria: choosing consistency level
The core of transaction design is identifying the consistency level needed per business. Demanding strong consistency for all data crashes performance; making everything eventual breaks the business.
The decision axis: âif this data is even briefly inconsistent, will there be financial, legal, or reputational damage?â Money-moving processing, inventory allocation, records subject to taxation or audit canât tolerate even one cent / one unit drift, so strong consistency is mandatory; compromising here shakes the business itself.
Conversely, âdata that works business-wise even with slight driftâ like like counts, view counts, recommendations, log aggregations is fine with eventual consistency; applying strong consistency would sacrifice performance and scalability. Asking the business âhow would it hurt if this data were a few seconds stale?â and reasoning back is most reliable.
| Business | Required consistency | Reason |
|---|---|---|
| Bank transfers / payments | Strong (ACID) | Drift = financial damage |
| Inventory | Strong or Saga | Overselling = refunds and trust loss |
| Order history | Strong | Tax / audit requirements |
| Likes, view counts | Eventual | Slight drift is harmless |
| Recommendations | Eventual | Stale is OK if working |
| Logs / analytics | Eventual | Scale over real-time |
Strong consistency is a high-cost requirement; limit it to where itâs truly needed.
Data-type Ă consistency-level ladder
Note: industry rates as of April 2026. Periodic refresh required.
âAll strong consistencyâ is excess and âall eventualâ is dangerous; sorting consistency level per data type is the operational core. Typical industry split:
| Data type | Consistency level | Reason | Implementation |
|---|---|---|---|
| Bank balance / payments | Strong (ACID, SERIALIZABLE) | 1-cent drift = damage | 1-DB ACID |
| Orders / inventory allocation | Strong | Overselling = refunds and trust loss | 1-DB ACID or Saga |
| Member registration / auth | Strong | Direct security risk | 1-DB ACID |
| Tax / audit records | Strong + tamper protection | Legal requirement | ACID + WORM |
| Cart info | Mid (READ COMMITTED) | Temporary drift OK | 1-DB TX |
| Likes / view counts | Eventual | Seconds-stale OK | Async aggregation |
| Recommendations | Eventual | Stale is OK if working | Batch recompute |
| Logs / analytics | Eventual | No real-time need | Kafka + DWH |
The isolation-level numeric gate: PostgreSQLâs default READ COMMITTED as the basis, with âonly inventory allocation and financial transactions individually elevated to SERIALIZABLEâ as the rule. Applying SERIALIZABLE to all tables drops concurrent performance several-fold; limit it to the necessary parts.
Strong consistency is high-cost; limit it to genuinely necessary business.
Distributed-transaction traps
Common ways microservice-crossing / multi-DB-crossing transactions fail. All produce production data inconsistency.
| Forbidden move | Why |
|---|---|
| 2PC (Two-Phase Commit) in cloud production | Network failure blocks all services; effectively nonfunctional |
| Retries without idempotency keys | Network failures cause double payments, double inventory decrements, multiple notifications |
| Saga without compensation design | Half-done state (charged but no inventory) remains; manual repair required |
| DB write and message-queue send in separate TX | Without Outbox, âDB success / Kafka send failâ breaks consistency |
SERIALIZABLE on all data | Concurrent performance drops several-fold; limit to necessary places |
| Concurrent updates without optimistic lock | Lost-update accident; one update vanishes |
| Treating timeout = failure and retrying | Server might have succeeded; idempotency key required |
| Reading from DB replica then immediately writing | Replication lag (ms-seconds) overwrites with stale value |
| DIY distributed transactions | Building without knowing Saga / Outbox always breaks; lean on libraries (Temporal, etc.) |
| Assuming âACID means safeâ | ACID applies only within 1 DB; across multiple services, different design (Saga + Outbox) is required |
| Implementing retries without idempotency carelessly | Timeout != failure; retries without idempotency keys are the direct cause of double processing |
Strictly speaking, the Knight Capital 2012 incident wasnât a distributed-TX accident, but the scenario of âone machine left with old code + retry runawayâ producing 45 minutes / $440M loss / company dissolution shows the horror of neglecting idempotency and consistency design (full details in Appendix: Major Incident Catalog).
âDistributed + retry + no idempotencyâ is the triple-landmine set for double processing.
AI decision axes â Staying inside one database is the AIâs safe zone
Typical holes in AI-generated transaction code
When you instruct AI to âwrite order processing,â it writes single-DB ACID transactions accurately. The BEGINâINSERTâUPDATEâCOMMIT basic pattern exists abundantly in training data.
Problems emerge in cross-service scenarios. For example, if you instruct âcharge the payment service â decrement inventory service â write to orders tableâ as 3 steps, AI writes code calling each step sequentially but often omits compensation (refund) for step 1 when step 2 fails.
Specific areas AI tends to miss:
- Refund logic when inventory decrement fails after payment API success
- Atomicity between message-queue sends and DB writes (scenes requiring the Outbox pattern)
- Idempotency-key design to prevent double-processing on retries
- Recovery when server-side processing succeeded after a timeout
These are typical âworks on the happy path but breaks on error pathsâ code, undetectable by unit tests.
Why 1-DB-complete architecture is safest in the AI era
Within the scope where ACID works on a single PostgreSQL instance, AI writes transaction control correctly. INSERT/UPDATE within BEGIN~COMMIT, data integrity via foreign-key constraints, optimistic locking with WHERE version = ? â all are classic patterns where AI generation accuracy is high.
In other words, deciding at the architecture level to âkeep things 1-DB-complete as much as possibleâ maximizes the range where AI can write safely. Distributed transactions are an area where even humans struggle to implement correctly, and maintaining quality when delegating to AI is even harder.
Caveats when delegating idempotency-key design to AI
When you instruct AI to âadd idempotency to this API,â it writes an implementation retrieving Idempotency-Key from the request header and storing it in Redis. Due to Stripe API influence, many examples exist in training data, and the basic pattern is accurate.
However, the following design decisions are not made automatically by AI, so they need to be decided in advance:
- Key expiration period (24 hours or 7 days)
- Behavior when the same key arrives with a different request body (return 409 or ignore)
- Key storage destination (Redis / DB / both)
- Key format (fixed UUID v4, or user-ID + operation-type combination)
These depend on business requirements, so the realistic approach is to codify them in design documents or ADRs before delegating implementation to AI.
âThe day we treated timeout = failureâ (industry case)
A project added 3-retry to the client side as timeout protection for the payment API; the result was duplicate charges to a small number of users. Server-side processing had succeeded; only the response was delayed, and all retries went through.
Few developers havenât watched a similar failure happen nearby. Retry implementation tends to come in optimistically, and the âprobably failed, letâs resendâ mindset is the textbook path to production double-charges.
Without an idempotency key, âtimeout = failureâ is almost never a safe assumption. The âdonât know if it succeededâ state across the network is routine in distributed systems. Design idempotency keys before adding retries â thatâs the rule.
Distributed + retry + no idempotency = double-processing landmine. Triple-set is the safety condition.
What you must decide â whatâs your projectâs answer?
Articulate your projectâs answer in 1-2 sentences for each:
- Per-data consistency requirement (strong / eventual)
- Isolation level (the lowest line business tolerates)
- Distributed-transaction handling (Saga / Outbox / avoid)
- Retry policy and idempotency
- Locking strategy (optimistic / pessimistic)
- CAP choice (CP / AP)
Write your answers down as an ADR. A concrete guide to writing them is here.
Related Articles
Summary
This article covered transaction design â ACID, isolation levels, Saga, Outbox, the CAP theorem, idempotency.
Sort consistency level per data, prioritize 1-DB completion, and use the Saga + Outbox + idempotency triple-set when distributed is needed. The 2026 realistic answer including AI era.
The next article is the Software Architecture categoryâs final installment: authentication and sessions (server session / JWT / OAuth).
I hope youâll read the next article as well.
Also popular with readers
đ Series: Architecture Crash Course for the Generative-AI Era (29/95)