SystemDesign.website
Track 6 ยท Module 6.2 ยท Consistency, Transactions & Distributed Consensus

Isolation Levels & Distributed Sagas

How distributed systems maintain data integrity without locking the world. Master ACID Isolation Levels (Dirty Reads to Serializable), 2-Phase Commit (2PC) locking bottlenecks, and the Distributed Saga Pattern (Choreography vs Orchestration).

โฑ๏ธ Reading Time: ~28 min
๐ŸŽฏ Architecture: ACID MVCC, 2PC, Saga Orchestrator, Compensating Transactions
๐ŸŽฌ Format: Vacation Booking Analogy + Live 4-Step Saga Simulator

โœˆ๏ธ 1. The Core Intuition: The Vacation Booking Dilemma

๐ŸŒด The Analogy: Booking Flight + Hotel + Rental Car

Imagine booking a holiday package touching 3 independent companies:
1. Airline Service: Charges $500 for the flight.
2. Hotel Service: Charges $400 for the resort room.
3. Rental Car Service: Fails because all cars are sold out!

In a monolithic SQL database, you would simply type ROLLBACK; and everything resets instantly. But in microservices, the Airline and Hotel are separate third-party services with their own databases! What do you do?

โŒ Two-Phase Commit (2PC / XA)

Lock the airline seat, hotel room, and car simultaneously. If the car company's network stalls, all seats and rooms stay locked forever!

๐Ÿšจ The Flaw: Blocking synchronous locks kill performance and cause cascading timeouts.
โœ… The Distributed Saga Pattern

Execute each local transaction sequentially. When the car rental fails, automatically trigger Compensating Transactions ($C_2$: Refund Hotel, $C_1$: Refund Airline) in reverse!

๐Ÿ›ก๏ธ The Advantage: Non-blocking, highly available, and resilient across distributed microservices.

๐Ÿ”’ 2. Single-Node ACID Isolation Levels & Concurrency Anomalies

Before solving distributed transactions, we must understand the 4 ANSI SQL isolation levels on a single database:

Isolation LevelDirty ReadNon-Repeatable ReadPhantom ReadMechanism
Read Uncommittedโš ๏ธ Allowedโš ๏ธ Allowedโš ๏ธ AllowedNo read locks. Reads uncommitted writes that may abort.
Read Committedโœ… Preventedโš ๏ธ Allowedโš ๏ธ AllowedReads only committed data (PostgreSQL/Oracle default).
Repeatable Readโœ… Preventedโœ… Preventedโš ๏ธ AllowedSnapshot Isolation using Multi-Version Concurrency (MVCC) (MySQL default).
Serializableโœ… Preventedโœ… Preventedโœ… PreventedTwo-Phase Locking (2PL) or Serializable Snapshot Isolation (SSI).

๐ŸŽญ 3. Saga Architecture: Choreography vs Orchestration

1. Choreography (Event-Driven / Pub-Sub)

Microservices emit and listen to Kafka events without a central coordinator. Service A emits OrderCreated $\implies$ Service B reacts and emits PaymentAuthorized.

โ€ข Pros: Simple and decentralized for short 2-3 step workflows.
โ€ข Cons: High complexity for 5+ steps; difficult to trace or monitor rollbacks.
2. Orchestration (Centralized State Machine)

A dedicated Saga Orchestrator (e.g. Temporal.io, AWS Step Functions) coordinates every step via RPC, tracking execution state in a durable database.

โ€ข Pros: Single source of truth, easy visualization, handles retries and timeouts cleanly.
โ€ข Cons: Requires running orchestrator infrastructure (e.g. Temporal workers).

๐Ÿ’ป 4. Production Saga Orchestrator Implementation (Multi-Language)

Here is the production implementation of a resilient Saga Orchestrator with automated compensating rollback across 5 languages:

Select Language:
# Python Async Saga Orchestrator with Compensations
class OrderSagaOrchestrator:
    def __init__(self):
        self.compensations = []

    async def execute(self, order_id: str, amount: float):
        try:
            # Step 1: Create Order
            await order_service.create(order_id, amount)
            self.compensations.append(lambda: order_service.cancel(order_id))

            # Step 2: Authorize Payment
            payment_id = await payment_service.charge(order_id, amount)
            self.compensations.append(lambda: payment_service.refund(payment_id))

            # Step 3: Reserve Inventory (May Fail!)
            await inventory_service.reserve(order_id)
            self.compensations.append(lambda: inventory_service.release(order_id))

            return "SAGA_COMPLETED"
        except Exception as e:
            # Rollback in reverse order
            for compensate in reversed(self.compensations):
                await compensate()
            raise Exception(f"Saga Failed, rolled back: {e}")

๐Ÿงช 5. Interactive Lab: Live 4-Step Distributed Saga Simulator

โšก Real-Time Distributed Workflow Engine

E-Commerce Saga Orchestrator Lab

Inject failure at any step and watch automated compensating rollbacks!
1. Injected Failure Point:
2. Orchestrator Engine:Temporal State Machine
3. Trigger Workflow:
STEP 1
Create Order
โšช Pending
STEP 2
Charge Payment
โšช Pending
STEP 3
Reserve Inventory
โšช Pending
STEP 4
Book Shipping
โšช Pending
๐Ÿš€ Saga Orchestrator Ready: Click "๐Ÿš€ Run Checkout Saga" to observe the forward execution and automated rollback!

๐ŸŽฏ 6. Test Your Understanding

Scenario 1: Complex 8-Step Loan Approval Workflow

You are architecting an 8-step banking loan approval system with complex business rules, human approvals, and 24-hour timeouts. Should you use Saga Choreography (Kafka events) or Saga Orchestration (Temporal)?

Scenario 2: The Two-Phase Commit Bottleneck

Why do high-throughput cloud microservices avoid 2-Phase Commit (2PC) in favor of Sagas?

โ—‹

Completed this module?

Mark as finished to update your curriculum progress tracker.