--- title: 'Compensating Transactions' source: 'https://academia.sh/en/courses/resilience-patterns/compensating-transactions' course: 'Resilience and Reliability' language: en updated: '2026-08-23T07:01:29+00:00' license: 'CC BY-SA 4.0' --- # Compensating Transactions Balancing steps that cannot be rolled back: only two of twenty-four orderings satisfying the dependencies, the case where an unrollbackable step cannot be moved to the end, a failing compensation call leaving 2.979 jobs a day stuck, and the intermediate state window coming from stuck work rather than from rollback. The previous lesson stopped duplication: the idempotency key kept the same work from running a second time. The drill's second open problem still stands. When the end-of-day billing workflow fails at its third step, the first two steps' effects stay final; those are not duplicated effects but **half-finished** ones, and the idempotency key says nothing about them. A **compensating transaction** is a step that balances a completed step's effect with an inverse operation — the same concept as The Data Access Layer and Business Logic course's **compensating step**, which was built there inside a saga arrangement, where the possibility of the compensation itself failing was already measured; the mechanics are not repeated here. Do not let the similar name mislead: the Caching, Queues and Asynchronous Processing course's **catch-up policy** is a separate decision about how a lagging consumer catches up. This lesson has three questions: where an uncompensatable step can be placed in the ordering, how many jobs stay stuck when a compensation call itself fails, and how long the intermediate state stays visible in the rollback window. ## The Ordering Is Not Free End-of-day billing has four steps: closing the period, writing the invoice line, opening the collection entry, and sending the seller notification. Two of them cannot be rolled back. **Closing the period** cannot be rolled back, because reopening a closed period drops the carrier events that arrived in between out of scope and makes the same day billable twice. **The seller notification** cannot be rolled back, because a notification already sent can only be corrected by a second notification. The rule is well known: an uncompensatable step goes at the end of the ordering. The problem is that the steps carry **dependencies**. The invoice line cannot be written before the period closes, the collection entry cannot open without the invoice line, and the notification cannot be sent before the invoice line. How many orderings both satisfy the dependencies and honor the rule is a countable question. ```js // compensating/flow.mjs — the in-process model of the end-of-day billing workflow. // There is no service, network, or store: steps are plain objects, rollback is a loop. // A compensating transaction is the same thing as M16/K04's compensating step; its mechanics were built there. export const STEP = [ { name: "close-period", compensate: null, requires: [] }, // cannot be rolled back { name: "invoice-line", compensate: "cancel-line", requires: ["close-period"] }, { name: "collection-entry", compensate: "revert-entry", requires: ["invoice-line"] }, { name: "seller-notification", compensate: null, requires: ["invoice-line"] }, // cannot be rolled back ]; export function* permutations(steps) { // every ordering if (steps.length === 0) { yield []; return; } for (let i = 0; i < steps.length; i += 1) for (const k of permutations(steps.filter((_, j) => j !== i))) yield [steps[i], ...k]; } export const valid = (d) => d.every((a, i) => a.requires.every((o) => d.slice(0, i).some((x) => x.name === o))); // One run: the failure occurs at step k; completed steps are compensated in reverse order. // compensationFailure: the compensation call's failure probability; attempts: compensation attempt count. export function run(ordering, k, { compensationFailure, attempts, rnd }) { const completed = ordering.slice(0, k); const r = { uncompensated: 0, compensationCalls: 0, stuck: 0, round: 0 }; for (const a of [...completed].reverse()) { r.round += 1; if (a.compensate === null) { r.uncompensated += 1; continue; } let succeeded = false; for (let d = 0; d < attempts && !succeeded; d += 1) { r.compensationCalls += 1; if (d > 0) r.round += 1; succeeded = rnd() >= compensationFailure; } if (!succeeded) r.stuck += 1; } return r; } export function generator(seed) { // 32-bit linear congruential generator, seed exposed let s = seed >>> 0; return () => { s = (Math.imul(s, 1103515245) + 12345) >>> 0; return s / 4294967296; }; } ``` **DD5 — the compensation call's failure probability is 0.05.** Rationale: compensation is itself a remote call and runs right after a failure; the target service may still be broken at that moment, so it carries a higher error margin than an ordinary call. Its sensitivity is given by sweeping the attempt count. The number is an **assumption** and is not added to K01's table. ```js // compensating/measure.mjs — the uncompensated step and rollback rounds across valid orderings import { STEP, permutations, valid, run, generator } from "./flow.mjs"; const ALL = [...permutations(STEP)]; const VALID = ALL.filter(valid); console.log(`${ALL.length} orderings, ${VALID.length} of them satisfy the dependencies`); console.log(`cannot be rolled back: ${STEP.filter((a) => a.compensate === null).map((a) => a.name).join(", ")}`); const noFailure = () => 1; // failure-free compensation run: rnd() = 1 >= failure rate console.log(`\n${"ordering".padEnd(66)}${"uncompensated".padStart(15)}${"rounds".padStart(8)}`); const summary = []; for (const d of VALID) { let uncompensated = 0, rounds = 0; for (let k = 1; k <= STEP.length; k += 1) { // failure at step k const r = run(d, k - 1, { compensationFailure: 0, attempts: 1, rnd: noFailure }); uncompensated += r.uncompensated; rounds += r.round; } summary.push([d.map((a) => a.name).join(" > "), uncompensated, rounds]); console.log(`${summary.at(-1)[0].padEnd(66)}${String(uncompensated).padStart(15)}${String(rounds).padStart(8)}`); } const chosen = VALID[summary.findIndex((o) => o[1] === Math.min(...summary.map((x) => x[1])))]; console.log(`\nchosen ordering: ${chosen.map((a) => a.name).join(" > ")}`); console.log(`${"step that fails".padEnd(28)}${"completed".padStart(12)}${"compensated".padStart(14)}` + `${"uncompensated".padStart(16)}${"rollback rounds".padStart(18)}`); for (let k = 1; k <= STEP.length; k += 1) { const r = run(chosen, k - 1, { compensationFailure: 0, attempts: 1, rnd: noFailure }); console.log(`${`${k}. (${chosen[k - 1].name})`.padEnd(28)}${String(k - 1).padStart(12)}` + `${String(r.compensationCalls).padStart(14)}${String(r.uncompensated).padStart(16)}${String(r.round).padStart(18)}`); } // DD5: the compensation call's failure probability is 0.05. Attempt count is swept. const N = 100_000, FAILURE_RATE = 0.05; console.log(`\n${N} runs, DD5 compensation failure rate ${FAILURE_RATE}; the failing step is chosen with an even distribution`); console.log(`${"attempts".padStart(9)}${"stuck jobs".padStart(13)}${"ratio".padStart(10)}` + `${"calls/job".padStart(12)}${"avg rollback rounds".padStart(22)}`); for (const attempts of [1, 2, 3]) { const rnd = generator(20260730); let stuck = 0, calls = 0, rounds = 0; for (let i = 0; i < N; i += 1) { const k = i % STEP.length; // failing step, even distribution const r = run(chosen, k, { compensationFailure: FAILURE_RATE, attempts, rnd }); stuck += r.stuck > 0 ? 1 : 0; calls += r.compensationCalls; rounds += r.round; } console.log(`${String(attempts).padStart(9)}${String(stuck).padStart(13)}` + `${(stuck / N).toFixed(5).padStart(10)}${(calls / N).toFixed(3).padStart(12)}` + `${(rounds / N).toFixed(3).padStart(22)}`); } ``` ``` 24 orderings, 2 of them satisfy the dependencies cannot be rolled back: close-period, seller-notification ordering uncompensated rounds close-period > invoice-line > collection-entry > seller-notification 3 6 close-period > invoice-line > seller-notification > collection-entry 4 6 chosen ordering: close-period > invoice-line > collection-entry > seller-notification step that fails completed compensated uncompensated rollback rounds 1. (close-period) 0 0 0 0 2. (invoice-line) 1 0 1 1 3. (collection-entry) 2 1 1 2 4. (seller-notification) 3 2 1 3 100000 runs, DD5 compensation failure rate 0.05; the failing step is chosen with an even distribution attempts stuck jobs ratio calls/job avg rollback rounds 1 3724 0.03724 0.750 1.500 2 214 0.00214 0.788 1.538 3 8 0.00008 0.790 1.540 ``` These numbers are in the **measurement** class: they come from a process-internal model and a generator seeded at 20260730, and they reproduce under the same seed. ## The Rule Only Applies as Far as the Dependencies Allow Twenty-two of the twenty-four orderings violate a dependency; two orderings remain, and both start with `close-period`. This is the rule's limit: **one of the two uncompensatable steps cannot be moved to the end, because the other three steps depend on it.** The notification can be moved to the end; closing the period cannot. The table below shows the cost step by step. If the failure occurs at the first step, nothing has completed; the uncompensated step count is 0. If it occurs at the second, third, or fourth step, the period is closed and stays closed: the uncompensated step count is 1 in all three cases. Changing the ordering does not lower this number, it only moves the position of the second uncompensatable step — when the notification is moved to third place, the total uncompensated step count climbs from 3 to 4. The design rule that follows from this has nothing to do with ordering. If an uncompensatable step cannot be moved to the end, **making it compensatable** is the only way out: if closing the period is defined as writing a flag rather than a deletion, it has a compensation. The ordering rule is a placement tool; it does not substitute for step design. ## Stuck Work and the Intermediate State Window The block below counts compensation's own failure. At one attempt, 3.724 percent of jobs stay **stuck**: the compensation call failed, the step's effect stands, and the job is neither completed nor rolled back. A second attempt brings the ratio down to 0.00214, a third to 0.00008. In exchange, compensation calls per job climb from 0.750 to 0.790, and rollback rounds from 1.500 to 1.540 — a second attempt cuts stuck work **seventeenfold** for 5 percent more calls. The intermediate state window is the invoice line appearing "issued" while the rollback is still underway. This window's length is measured in rounds; giving a round a duration requires a conversion factor. **DD6 — one rollback round is 0.25 seconds.** Rationale: a round is a remote call plus its confirmation. Its sensitivity is given at four times that value. **DD7 — stuck work is closed within 4 hours**, the same as K01's V10 batch window; that is the billing job's window. ```js // compensating/cost.mjs — converting the measured ratios into daily numbers at K01 scale const INVOICE_LINES = 4000; // K01 computed value: daily invoice lines const KK2 = 0.02; // M19/K03 choreography lesson's assumption: workflow failure rate const ROUND_SEC = 0.25, STUCK_HOURS = 4; // DD6 round duration, DD7 time to close stuck work const failed = INVOICE_LINES * KK2; console.log(`daily workflows ${INVOICE_LINES}, KK2 = ${KK2} -> failed ${failed}/day`); console.log(`\n${"ordering".padEnd(22)}${"uncompensated steps/day".padStart(26)}${"a year".padStart(9)}`); for (const [name, ratio] of [["notification last", 3 / 4], ["notification third", 4 / 4]]) console.log(`${name.padEnd(22)}${(failed * ratio).toFixed(1).padStart(26)}` + `${(failed * ratio * 365).toFixed(0).padStart(9)}`); // measure.mjs measurements: stuck work ratio and average rollback rounds, by compensation attempts const MEASURED = [[1, 0.03724, 1.500], [2, 0.00214, 1.538], [3, 0.00008, 1.540]]; console.log(`\n${"attempts".padStart(9)}${"stuck work/day".padStart(16)}${"a year".padStart(8)}` + `${"rollback s".padStart(12)}${"intermediate s/day".padStart(20)}${"h/day from stuck".padStart(19)}`); for (const [attempts, ratio, rounds] of MEASURED) { const stuck = failed * ratio, rollback = rounds * ROUND_SEC; console.log(`${String(attempts).padStart(9)}${stuck.toFixed(3).padStart(16)}` + `${(stuck * 365).toFixed(0).padStart(8)}${rollback.toFixed(3).padStart(12)}` + `${(failed * rollback).toFixed(1).padStart(20)}${(stuck * STUCK_HOURS).toFixed(3).padStart(19)}`); } const [, ratio1, rounds1] = MEASURED[0]; const factor = (t) => (failed * ratio1 * STUCK_HOURS * 3600) / (failed * rounds1 * t); console.log(`\nat one attempt, from stuck work / from rollback = ${factor(ROUND_SEC).toFixed(0)}x`); console.log(`DD6 sensitivity: at a round duration of ${ROUND_SEC * 4} s instead of ${ROUND_SEC} s, intermediate state is ` + `${(failed * rounds1 * ROUND_SEC * 4).toFixed(1)} s/day, ratio ${factor(ROUND_SEC * 4).toFixed(0)}x`); console.log(`\nfailure-free day's cost: 2 of the 4 steps carry a compensation function, ` + `${(INVOICE_LINES * 2).toLocaleString("en-US")} written compensation paths a day; compensation calls run ` + `${(failed * 0.75).toFixed(0)}/day, ${(0.75 * KK2).toFixed(4)} per workflow`); ``` ``` daily workflows 4000, KK2 = 0.02 -> failed 80/day ordering uncompensated steps/day a year notification last 60.0 21900 notification third 80.0 29200 attempts stuck work/day a year rollback s intermediate s/day h/day from stuck 1 2.979 1087 0.375 30.0 11.917 2 0.171 62 0.385 30.8 0.685 3 0.006 2 0.385 30.8 0.026 at one attempt, from stuck work / from rollback = 1430x DD6 sensitivity: at a round duration of 1 s instead of 0.25 s, intermediate state is 120.0 s/day, ratio 358x failure-free day's cost: 2 of the 4 steps carry a compensation function, 8,000 written compensation paths a day; compensation calls run 60/day, 0.0150 per workflow ``` ## The Numbers for Two Days **The failure-free day's cost sits in the code path.** Two of the four steps must carry a compensation function, and those functions sit in every workflow definition: 8000 written compensation paths a day for 4000 workflows. How often they run is very small — 0.0150 per workflow, 60 calls a day. The cost is not a runtime load, it is **untested code**: alongside a path that runs 21,900 times a year, the codebase carries a path that runs with a probability of 1.5 percent. Closing the period being uncompensatable also produces a constraint on the failure-free day: no correction can be made on a closed period. **The failure day's gain is two numbers.** At KK2 = 0.02, 80 workflows fail a day. With the ordering rule applied, the uncompensated step count is 60 a day, 21,900 a year; if the notification is moved to third place, it is 80 and 29,200. Ordering alone makes a difference of 7300 uncompensated steps a year. Adding one compensation attempt brings stuck work down from 2.979 to 0.171 a day, from 1087 to 62 a year. The intermediate state window has two sources, and their sizes are not comparable. Rollback itself produces a total of **30.0 seconds** of intermediate state a day. Stuck work produces **11.917 hours** a day — 1430 times as much. Even at four times DD6, the ratio stays at 358x, so the result is not sensitive to the round-duration assumption. **The intermediate state window is set by compensation's own failure, not by rollback's speed**; a team that wants to shorten the window should tune the compensation attempt count, not the round duration. ## Summary - Only two of the twenty-four orderings satisfy the dependencies, and both start with the uncompensatable `close-period` step: the move-to-the-end rule cannot be applied to this step. - When the failure occurs at the second, third, or fourth step, the uncompensated step count is 1 in every case; what saves a step that cannot be moved to the end is not ordering but designing the step to be compensatable. - The notification's position makes a difference of 7300 uncompensated steps a year: 60/day and 21,900/year at the end, 80/day and 29,200/year at third place. - Compensation carries its own failure: at DD5 = 0.05, one attempt leaves 3.724 percent of jobs stuck; a second attempt brings the ratio down to 0.00214 and raises calls per job from 0.750 to 0.790. - The overwhelming share of the intermediate state window comes from stuck work, not rollback: 30.0 seconds a day against 11.917 hours, 1430 times; even at four times DD6, still 358 times. - The failure-free day's cost is 8000 written compensation paths, and only 60 of them run a day; what the codebase carries is not runtime load but rarely run code. ## Next Step Compensation kept the workflow's **steps** correct. The **data** the same workflow carries was never in question. When the carrier reports a delivery, it attaches proof of delivery alongside it: the recipient's signature or a photo of the delivery point. This addition turns the state event's record into something far larger than K01's 220 bytes, and that something passes through the queue, gets copied to every consumer, and is carried again on every redelivery. The next lesson takes on pulling the large payload out of the message: the message's size, the bytes crossing the queue, how long the payload lives in a separate store, and how many times the reference left behind after the payload is deleted dangles.