--- title: 'Choreography and Orchestration' source: 'https://academia.sh/en/courses/application-layer/choreography-and-orchestration' course: 'The Application Layer and Service Interaction' language: en updated: '2026-08-23T07:01:22+00:00' license: 'CC BY-SA 4.0' --- # Choreography and Orchestration Building the end-of-day billing job — four steps spread across four services — in two models: choreography, where steps trigger each other with events, and orchestration, where a single orchestrator calls the steps; showing that the shared transaction boundary is lost, building the compensating step, and measuring both modes in messages, compensated steps, uncompensated steps, and half-finished jobs. Every event in the previous lesson was single-step: a state is written and the job is done. End-of-day billing is not like that. For a seller's one day, a tariff is calculated, a discount is applied, an invoice line is produced, and a notification is sent. Four steps wait on each other, and each step runs in a separate service — tariff and invoice in the billing context, discount in the contract context, notification on the delivery operations side. There are two models for building this job. In the **choreography** arrangement, there is no orchestrator in the middle: when a step finishes, it publishes an event, and the next step's service is subscribed to that event. In the **orchestration** arrangement, a single orchestrator calls the steps in sequence and keeps track of where the job stands itself. Choreography is the workflow form of the event-driven architecture covered in the Architectural Styles course; the style itself and the publish-subscribe mechanics were established there and in the Caching, Queues and Asynchronous Processing course, and are not repeated here. What is measured is the numbers the two modes leave behind when they run the same job through the same failures. ## No Shared Transaction Boundary The Data Access Layer and Business Logic course established the unit of work and the transaction boundary: the writes inside one boundary either commit together or none of them commit. When four steps run in four separate services, this foundation is gone. Each service opens and closes its own boundary; when the third step fails, the writes from the first and second steps **have already committed**, and there is no call that rolls them back together. What replaces it is a **compensating step**: a second step is defined for each step that undoes its effect in business terms, and when an error occurs, the completed steps are compensated in reverse order. Compensation is not rollback; the write stays visible for a while, then is closed with an opposite write. The fourth step — notification — cannot be compensated, because a notification that has been sent cannot be taken back; that is why it is placed at the end of the sequence. This ordering rule is the same for both modes, and in the model it is written as the `compensation` field being `null`. ```js // workflow/job.mjs — the in-process model of the end-of-day billing workflow. Four steps // run in four separate services; each service closes its own transaction boundary, and // there is no shared boundary. export const STEP = [ { name: "tariff", service: "billing", compensation: "tariff-cancel" }, { name: "discount", service: "contract", compensation: "discount-cancel" }, { name: "invoice", service: "invoicing", compensation: "invoice-cancel" }, { name: "notification", service: "notification", compensation: null }, // cannot be undone, so it is last ]; export const JOBS = ["S1", "S2", "S3", "S4", "S5", "S6", "S7", "S8"]; export const FAILURE = { S3: "invoice", S7: "discount" }; // deterministic failure points // mode: "choreography" triggers steps with events, "orchestration" calls them with a single // orchestrator. missingSubscriber: a service in choreography that has not subscribed to the // cancellation event (null means the chain is complete). export function run({ mode, missingSubscriber = null }) { const s = { completed: 0, steps: 0, messages: 0, compensatedSteps: 0, uncompensatedSteps: 0, halfFinishedJobs: 0 }; const committed = new Map(STEP.map((a) => [a.name, new Set()])); // committed writes const trace = []; for (const job of JOBS) { if (mode === "choreography") s.messages += 1; // the event that starts the workflow const done = []; let failure = null; for (const a of STEP) { if (mode === "orchestration") s.messages += 1; // orchestrator -> service command if (FAILURE[job] === a.name) { failure = a.name; s.messages += 1; break; } // error reply or event s.steps += 1; committed.get(a.name).add(job); // step committed in its own boundary done.push(a); s.messages += 1; // choreography: the event the step publishes; orchestration: the reply } if (failure === null) { s.completed += 1; continue; } let open = 0; for (const a of [...done].reverse()) { if (a.compensation === null) continue; const subscribed = mode === "orchestration" || a.service !== missingSubscriber; s.messages += mode === "orchestration" ? 2 : (subscribed ? 1 : 0); // compensation command+reply / cancel event if (subscribed) { committed.get(a.name).delete(job); s.compensatedSteps += 1; } else { s.uncompensatedSteps += 1; open += 1; } // no one listened, the write stayed } if (open > 0) s.halfFinishedJobs += 1; trace.push(`${job}: stopped at the ${failure} step, committed steps ${done.length}, left open ${open}`); } return { ...s, trace, remainingWrites: STEP.map((a) => `${a.name}=${committed.get(a.name).size}`).join(" ") }; } // Structural measures: fixed once the mode is chosen, independent of the run. export function structure(mode) { const n = STEP.length, compensable = STEP.filter((a) => a.compensation !== null).length; const o = mode === "orchestration"; return { "units carrying the workflow definition": o ? 1 : n, "units holding workflow state": o ? 1 : 0, "units queried to read status": o ? 1 : n, "units responsible for compensation": o ? 1 : compensable, "units changed when a step is appended": o ? 2 : 1, "transaction boundaries": n, "atomically reversible steps": 0, }; } ``` The third run tests a question: in choreography, compensation is also triggered by an event, meaning every service must be subscribed to the cancellation event. `missingSubscriber` produces the case where one service has not set up that subscription. ```js // workflow/measure.mjs — two modes on the same job set: steps, messages, compensation, half-finished jobs, and structure import { run, structure, JOBS, STEP, FAILURE } from "./job.mjs"; const K = [["choreography", run({ mode: "choreography" })], ["chor+missing sub", run({ mode: "choreography", missingSubscriber: "billing" })], ["orchestration", run({ mode: "orchestration" })]]; const MEASURE = [["completed", "completed job"], ["steps", "steps run"], ["messages", "message"], ["compensatedSteps", "compensated step"], ["uncompensatedSteps", "uncompensated step"], ["halfFinishedJobs", "half-finished job"]]; console.log(`${JOBS.length} seller-days, ${STEP.length} steps, ` + `${Object.keys(FAILURE).length} jobs fail: ${Object.entries(FAILURE).map(([j, a]) => `${j}@${a}`).join(", ")}`); console.log(); console.log(`${"measure".padEnd(22)}${K.map(([a]) => a.padStart(18)).join("")}`); for (const [k, label] of MEASURE) console.log(`${label.padEnd(22)}${K.map(([, r]) => String(r[k]).padStart(18)).join("")}`); console.log(`\nfailure trace (same in all three runs):`); for (const line of K[2][1].trace) console.log(` ${line}`); console.log(`committed remaining writes:`); for (const [label, r] of K) console.log(` ${label.padEnd(18)}${r.remainingWrites}`); const STRUCT = Object.keys(structure("orchestration")); console.log(`\n${"structural measure".padEnd(40)}${"choreography".padStart(13)}${"orchestration".padStart(14)}`); for (const a of STRUCT) console.log(`${a.padEnd(40)}${String(structure("choreography")[a]).padStart(13)}` + `${String(structure("orchestration")[a]).padStart(14)}`); const [chor, , orch] = K.map(([, r]) => r); console.log(`\nmessage ratio orchestration/choreography = ${(orch.messages / chor.messages).toFixed(3)}`); console.log(`messages per job: choreography ${(chor.messages / JOBS.length).toFixed(2)}, ` + `orchestration ${(orch.messages / JOBS.length).toFixed(2)}`); ``` ``` 8 seller-days, 4 steps, 2 jobs fail: S3@invoice, S7@discount measure choreography chor+missing sub orchestration completed job 6 6 6 steps run 27 27 27 message 40 38 64 compensated step 3 1 3 uncompensated step 0 2 0 half-finished job 0 2 0 failure trace (same in all three runs): S3: stopped at the invoice step, committed steps 2, left open 0 S7: stopped at the discount step, committed steps 1, left open 0 committed remaining writes: choreography tariff=6 discount=6 invoice=6 notification=6 chor+missing sub tariff=8 discount=6 invoice=6 notification=6 orchestration tariff=6 discount=6 invoice=6 notification=6 structural measure choreography orchestration units carrying the workflow definition 4 1 units holding workflow state 0 1 units queried to read status 4 1 units responsible for compensation 3 1 units changed when a step is appended 1 2 transaction boundaries 4 4 atomically reversible steps 0 0 message ratio orchestration/choreography = 1.600 messages per job: choreography 5.00, orchestration 8.00 ``` ## Reading the Numbers Both modes do the same job: 6 of 8 seller-days completed, 27 steps ran, compensated steps 3 in both. There is no difference in the job outcome. The difference collects in two other places. The first is message count: choreography 40, orchestration 64 — a ratio of 1.600. Five per job against eight. The reason is structural: in choreography, a completed step publishes a single event and the next step picks it up itself; in orchestration, every step needs a command and a reply, and so does compensation. The orchestrator's knowledge of where the job stands is not free; it charges itself for the messages that carry it. The second is in the third column, and this is where the real decision sits. In choreography, compensation is also triggered by an event; when the billing service is not subscribed to the cancellation event, compensated steps fall from 3 to 1, uncompensated steps rise from 0 to 2, and 2 jobs are left half-finished. The committed write in the `tariff` row rises from 6 to 8: the tariff calculation for two sellers keeps sitting in the system, even though their invoices were never produced. These two writes produce no error message; they only show up when month-end reconciliation does not balance. The missed subscription has no counterpart in orchestration, because the compensation list lives in the orchestrator's code. The structural table names this: compensation responsibility is spread across 3 units in choreography, gathered into 1 unit in orchestration. The same pattern holds for the workflow definition and its state. In choreography, the workflow's definition is not written down as a whole anywhere; reading it means looking at four services' subscription lists, and you cannot ask a single unit "where is this job now." One row in the table runs the other way: when a step is appended at the end, 1 unit changes in choreography (the new service subscribes to the last event), 2 in orchestration (the new service and the orchestrator). Choreography's gain is concrete here, and it must be read together with the 1.600 multiplier in messages. ## Back to the Estimate The model's 8 jobs and 2 failures are not a volume; what is measured is a ratio per job. Volume comes from K01: 4,000 invoice lines a day is a **computed value**, and since each line is one seller's one day, 4,000 workflows run per day; the end-of-day window of 4 hours is K01's V10 **assumption**. **KK2 — end-of-day workflow failure rate: 0.02.** Its rationale is missing contract records, tariff changes that arrive after the window has closed, and shipments not yet closed producing a steady baseline. The model's 2/8 ratio is not realistic, and it is not presented as such or added to K01's table. Its sensitivity is calculated together with 0.04. ```js // workflow/cost.mjs — applies the model's ratios to K01's end-of-day job volume import { run, structure, JOBS, STEP } from "./job.mjs"; const INVOICE_LINES = 4000; // K01: daily invoice lines (computed value) — one workflow per seller const WINDOW = 4 * 3600; // K01 V10: end-of-day window 4 hours (assumption) const chor = run({ mode: "choreography" }), orch = run({ mode: "orchestration" }); const gap = run({ mode: "choreography", missingSubscriber: "billing" }); const messagesPerJob = { choreography: chor.messages / JOBS.length, orchestration: orch.messages / JOBS.length }; const failingJobs = 2; // number of jobs that fail in the model const compensatedPerFailure = chor.compensatedSteps / failingJobs; const uncompensatedPerFailure = gap.uncompensatedSteps / failingJobs; console.log(`workflow ${INVOICE_LINES}/day, window ${WINDOW} s -> ` + `${(INVOICE_LINES / WINDOW).toFixed(4)} jobs/s, ${(INVOICE_LINES * STEP.length / WINDOW).toFixed(4)} steps/s`); for (const [label, n] of Object.entries(messagesPerJob)) console.log(` ${label.padEnd(13)} ${n.toFixed(2)} messages/job -> ${(INVOICE_LINES * n / WINDOW).toFixed(4)} messages/s`); console.log(`\n${"KK2".padStart(5)}${"failed jobs/day".padStart(17)}${"comp steps/day".padStart(16)}` + `${"uncomp steps/day".padStart(18)}${"half-finished/day".padStart(20)}`); for (const KK2 of [0.02, 0.04]) { // KK2: end-of-day workflow failure rate const failed = INVOICE_LINES * KK2; console.log(`${KK2.toFixed(2).padStart(5)}${failed.toFixed(0).padStart(17)}` + `${(failed * compensatedPerFailure).toFixed(0).padStart(16)}` + `${(failed * uncompensatedPerFailure).toFixed(0).padStart(18)}${failed.toFixed(0).padStart(20)}`); } console.log(`\nratios from the model: compensated steps per failure = ${compensatedPerFailure.toFixed(2)}, ` + `uncompensated remaining with a missing subscriber = ${uncompensatedPerFailure.toFixed(2)}`); console.log(`comparison with K01's peak write of 97.22 req/s: orchestration message flow ` + `${(100 * (INVOICE_LINES * messagesPerJob.orchestration / WINDOW) / 97.22).toFixed(2)}%`); console.log(`units to open to read the workflow definition: choreography ` + `${structure("choreography")["units carrying the workflow definition"]}, orchestration ` + `${structure("orchestration")["units carrying the workflow definition"]}`); ``` ``` workflow 4000/day, window 14400 s -> 0.2778 jobs/s, 1.1111 steps/s choreography 5.00 messages/job -> 1.3889 messages/s orchestration 8.00 messages/job -> 2.2222 messages/s KK2 failed jobs/day comp steps/day uncomp steps/day half-finished/day 0.02 80 120 80 80 0.04 160 240 160 160 ratios from the model: compensated steps per failure = 1.50, uncompensated remaining with a missing subscriber = 1.00 comparison with K01's peak write of 97.22 req/s: orchestration message flow 2.29% units to open to read the workflow definition: choreography 4, orchestration 1 ``` The message difference shrinks when it is scaled up. Over the four-hour window, workflow rate is 0.2778 per second, step rate is 1.1111; choreography produces 1.3889 messages per second, orchestration 2.2222. Orchestration's excess is 0.83 messages per second, about 2.29 percent of K01's peak write flow. The 1.600 multiplier in message count is not a capacity problem system-wide. The other column behaves the opposite way under the same scaling. With KK2 = 0.02, 80 workflows fail per day; at the model's ratio, 120 compensating steps run per day. A single missing subscription in the compensation chain means 80 uncompensated steps and 80 half-finished jobs per day. If KK2 doubles, it becomes 160. These numbers produce no outage and raise no error counter; they only surface at month-end reconciliation. The order-of-magnitude gap between the two measured costs settles the decision: choreography's message gain is about two percent of the peak flow, while the cost of one gap in the compensation chain is 80 wrong records a day. For workflows with more than three steps that need compensation, orchestration is the defensible choice, because it gathers compensation responsibility into a single unit; for notification-style flows with no compensation, expected to grow steps that know nothing of each other, choreography is the right fit. ## Summary - When four steps run in four services, there is no shared transaction boundary: transaction boundaries number 4, atomically reversible steps 0; when an error occurs, the earlier steps stay committed. - A compensating step replaces it; the step that cannot be compensated (notification) is placed at the end of the sequence. - Both modes produce the same result — 6 completed jobs, 27 steps, 3 compensated — but choreography spends 40 messages against orchestration's 64 (a ratio of 1.600; 5 per job against 8). - In choreography, compensation responsibility is spread across 3 units: a single missing subscription dropped compensated steps from 3 to 1, left 2 steps uncompensated, and raised `tariff` writes from 6 to 8. - The structural difference collects in the workflow's definition: in choreography, the definition is spread across 4 units and the unit holding state is 0; in orchestration, both are 1. The one row that runs the other way is extension — appending a step changes 1 unit in choreography, 2 in orchestration. - Back to K01: 4,000 workflows and a 4-hour window give 0.2778 jobs/s, 1.1111 steps/s; the message gap is 0.83 per second (2.29 percent of the peak write), while a missing subscription with KK2 = 0.02 leaves 80 uncompensated steps and 80 half-finished jobs per day. ## Next Step Every failure in this lesson announced itself clearly: a step returned an error, and the compensation chain either worked or did not. Most failures in a real workflow do not speak up like that. The orchestrator crashes after sending a command, a step hangs without replying, a compensation call is tried once and dropped. If no one asks about such a job's status, it appears to be "running" forever, and the window closes. The next lesson builds the arrangement that closes this gap: a scheduler that durably records the job and starts its steps, an agent that runs a step remotely, and a supervisor that finds steps whose time has expired and either re-drives or compensates them — and it measures what the supervisor's scan frequency does to the count of half-finished jobs.