Saga Pattern & Distributed Transactions

Why distributed transactions are a real problem#

Inside a monolith, "place an order" is usually one database transaction: debit inventory, charge the payment, create the order row, all inside a single BEGIN/COMMIT, with the database itself guaranteeing that either all of it happens or none of it does. Once OrdersService, PaymentService, and InventoryService are separate processes with separate databases (the whole point of splitting them apart, per this track's introduction), there is no single transaction spanning all three anymore — each service can only commit its own local database work.

The classic textbook answer, two-phase commit (2PC), has a coordinator ask every participant to "prepare" (lock resources, confirm it could commit), then tells everyone to actually commit only once all have agreed. It works, but it's a poor fit for microservices in practice: every participant has to hold locks for the entire duration of the coordinator's round trip, a coordinator crash mid-protocol can leave participants blocked indefinitely, and it requires every participant to support the same distributed-transaction protocol — which most modern datastores and third-party APIs (a payment gateway, in particular) simply don't. 2PC shows up occasionally within a single database cluster, but essentially never across independent microservices.

The Saga pattern solves the same problem differently: instead of one atomic transaction, a saga is a sequence of local transactions, each committed independently and immediately, where every step that changes state also defines a compensating action — an explicit undo — to run if a later step fails. There's no locking across services and no coordinator holding everyone hostage; instead, the system accepts that it will pass through temporarily inconsistent states, and guarantees it always reaches a consistent one eventually, either "fully succeeded" or "fully compensated back to the start."

The two ways to implement a saga#

Neither is strictly better — choreography keeps each service simpler and more decoupled at the cost of the overall flow being implicit; orchestration makes the flow explicit and easy to reason about at the cost of a new component (the orchestrator) that every step now depends on.

A complete worked example: placing an order#

The same saga — reserve stock, charge payment, ship the order — built both ways. If any step fails, everything already done has to be undone in reverse order.

Happy path:
  1. OrdersService:     create order (status = PENDING)
  2. InventoryService:  reserve stock
  3. PaymentService:    charge card
  4. ShippingService:   schedule shipment
  5. OrdersService:     mark order CONFIRMED

Failure at step 3 (payment declined) triggers compensations, in reverse:
  3'. PaymentService:   (nothing charged — no compensation needed)
  2'. InventoryService: release the reserved stock
  1'. OrdersService:    mark order CANCELLED

Choreography: each service reacts to the previous one's event#

No central coordinator — every service publishes an event when it finishes its own step, and listens for the events that mean it's now its turn (or that something upstream failed):

// OrdersService — starts the saga, and reacts to both success and failure events later
async function placeOrder(orderRequest) {
  const order = await db.orders.insert({ ...orderRequest, status: 'PENDING' });
  await eventBus.publish('order.created', { orderId: order.id, items: order.items });
  return order;
}

eventBus.on('payment.failed', async (event) => {
  await db.orders.update(event.orderId, { status: 'CANCELLED' });
  // no compensating action needed here — this service's own step never happened successfully
});

eventBus.on('shipment.scheduled', async (event) => {
  await db.orders.update(event.orderId, { status: 'CONFIRMED' });
});

// InventoryService — reacts to order.created, and to a failure further down the chain
eventBus.on('order.created', async (event) => {
  const reserved = await inventory.reserve(event.items);

  if (!reserved) {
    await eventBus.publish('inventory.reservation-failed', { orderId: event.orderId });
    return;
  }
  await eventBus.publish('inventory.reserved', { orderId: event.orderId, items: event.items });
});

// Compensating action — undoes step 2 if a later step (payment) fails
eventBus.on('payment.failed', async (event) => {
  await inventory.release(event.orderId);
});

// PaymentService — the step that fails in this walkthrough
eventBus.on('inventory.reserved', async (event) => {
  const charged = await paymentGateway.charge(event.orderId, event.amount);

  if (!charged) {
    await eventBus.publish('payment.failed', { orderId: event.orderId });
    return;
  }
  await eventBus.publish('payment.charged', { orderId: event.orderId });
});

Every service only knows two things: what event starts its own step, and what event(s) mean it needs to compensate. PaymentService never had to know ShippingService exists at all — the saga's overall shape emerges from how the events happen to be wired together, which is exactly choreography's strength and its weakness at once.

Orchestration: one coordinator drives every step#

The same saga, with a single OrderSagaOrchestrator issuing a command for each step and deciding, from the result, what happens next:

class OrderSagaOrchestrator {
  async run(orderRequest) {
    const order = await orderService.create(orderRequest);

    try {
      await inventoryService.reserve(order.id, order.items);
    } catch (err) {
      await orderService.cancel(order.id);
      throw new SagaFailedError('inventory-reservation', order.id);
    }

    try {
      await paymentService.charge(order.id, order.total);
    } catch (err) {
      // undo step 2, then step 1 — reverse order, exactly like unwinding a call stack
      await inventoryService.release(order.id);
      await orderService.cancel(order.id);
      throw new SagaFailedError('payment', order.id);
    }

    try {
      await shippingService.schedule(order.id);
    } catch (err) {
      await paymentService.refund(order.id);
      await inventoryService.release(order.id);
      await orderService.cancel(order.id);
      throw new SagaFailedError('shipping', order.id);
    }

    await orderService.confirm(order.id);
    return order;
  }
}

The orchestrator calls each service directly (synchronously here, though it could just as easily issue commands over a queue) and explicitly runs the correct compensations in reverse order the moment any step throws. Reading this one method top to bottom tells you the entire saga — there's no need to go hunting through four separate services' event handlers to reconstruct what happens on a payment failure.

Designing compensating actions#

A compensating action is not a rollback in the database sense — it's a new, forward-moving operation that semantically undoes a previous one. "Release the stock reservation" and "refund the payment" are compensations; there is no ROLLBACK statement that spans services. Two properties every compensating action needs:

• Idempotency — a compensation might run more than once (a retry after a timeout, a duplicate event), so releasing stock that's already released, or refunding a payment that's already refunded, must be a safe no-op, not an error or a double-refund.
• It must actually be possible. Some steps can't be perfectly undone — an email already sent, a shipment already physically dispatched. In those cases the compensation is often "best-effort" (send a follow-up "please disregard" email) rather than a true reversal, which is exactly why the step order in a saga matters: put the truly irreversible step (physically shipping the package) last, after everything reversible has already succeeded.

Common mistakes#

• Reaching for two-phase commit across independently owned microservice databases — it requires every participant to hold locks for the coordinator's entire round trip and doesn't work at all against third-party APIs like a payment gateway, which is exactly the situation sagas are built for.
• Writing a compensating action that isn't idempotent, so a retried compensation (after a timeout, or a duplicate failure event) issues a second refund or double-releases stock that's already been released.
• Ordering saga steps so the truly irreversible one (physically shipping a package, sending a non-retractable email) happens before steps that are easy to compensate — put irreversible steps last, once everything reversible has already succeeded.
• In choreography, letting the "shape" of the saga exist only implicitly across many services' event handlers with no documentation anywhere — six months later, no one can answer "what exactly happens if payment fails?" without tracing events through every service's code.