---
title: 'Service Architectures'
source: 'https://academia.sh/en/courses/service-architectures'
language: en
updated: '2026-08-23T07:00:29+00:00'
license: 'CC BY-SA 4.0'
lessons: 18
outcomes:
    - 'Makes a criteria-based choice among monolithic, modular, and distributed architectures'
    - 'Draws service boundaries according to the domain model'
    - 'Applies inter-service communication and consistency patterns'
---

# Service Architectures

Splitting the single deployment unit: a criteria-based comparison of the monolith, modular monolith, service-oriented, microservice, and serverless options, incremental decomposition strategies, drawing the boundary by coupling, data ownership, and rate of change, the chained latency and failure propagation of synchronous calls, event-based loose coupling, gathering edge responsibilities, separating network concerns from the application, address resolution, and, in distributed consistency, the cost of two-phase commit, compensating workflows, the outbox, consumer-side idempotency, and contract evolution.

## Architectural Options

Building and measuring the same application in five forms: the monolithic application's strengths and scaling limits, the modular monolith where boundaries are protected inside the code, enterprise service thinking, the microservices where independent deployment brings an operational cost, the constraints of the event-triggered function model, and migration strategies built on the strangler fig pattern through gradual decomposition.

### Monolithic Application

Source: https://academia.sh/en/courses/service-architectures/monolithic-application

Building and measuring a working monolithic loan service: seven modules, ten import edges, one process, zero network hops; the cheapness of a local call, the ease of change that a single deployment unit brings, and the scaling limit that appears when that same unit is replicated.

The previous course, Caching, Queues and Asynchronous Processing, closed by pointing at a
single assumption: every queue, delivery guarantee, and backpressure mechanism it built was
built **inside a single deployment unit**. This course takes on splitting that unit apart.
Before anything can be split, what is being split has to stand on its own.

This lesson builds and measures a working **monolith** implementation of the library loan
service. What it builds is the base for the next five lessons: each one splits it by one
step, or counts the cost of splitting over these same measurements. The definition of
architectural styles was covered in the Architectural Styles course; the work here is not
definition, it is implementation and measurement.

#### The Boundary of the System

**AO1.** State is kept in a single JSON file. In a real deployment this would be a shared
data store; the property the model carries is that state lives **outside the process, in one
place**.

**AO2.** Modules talk to each other in the same process, through a local function call.
Network latency, serialization, and partial failure do not exist in this model; they are
measured with real processes in the fourth lesson.

#### The System: A Monolithic Loan Service

The system carries five business domains: catalog, membership, loans, notification, and
pricing. There is a shared module that holds the state.

```js
// store.mjs — state for all modules; single file, single process
import { existsSync, readFileSync, writeFileSync } from "node:fs";
const PATH = "data.json";
const INITIAL = {
  book: { 1: { title: "Blindness", status: "on_shelf" }, 2: { title: "The Book of Sand", status: "on_shelf" } },
  member: { 4: { name: "Derek", open: 1, limit: 3, balance: 0 },
         5: { name: "Grace", open: 3, limit: 3, balance: 0 } },
  loans: [],
};
export function read() {
  if (existsSync(PATH) === false) writeFileSync(PATH, JSON.stringify(INITIAL));
  return JSON.parse(readFileSync(PATH, "utf8"));
}
export function write(d) { writeFileSync(PATH, JSON.stringify(d)); }
```

Three business modules read from and write to this store; the notification module carries no
state.

```js
// catalog.mjs — book records
import { read, write } from "./store.mjs";
export function status(bookId) { return read().book[bookId].status; }
export function markOnLoan(bookId) {
  const d = read();
  if (d.book[bookId].status !== "on_shelf") throw new Error("book not on shelf");
  d.book[bookId].status = "on_loan";
  write(d);
}
```

```js
// membership.mjs — member records and loan limit
import { read, write } from "./store.mjs";
export function isEligible(memberId) { const m = read().member[memberId]; return m.open < m.limit; }
export function addOpenLoan(memberId) { const d = read(); d.member[memberId].open += 1; write(d); }
```

```js
// pricing.mjs — late fee
import { read, write } from "./store.mjs";
export const DAILY_RATE = 2;
export function fee(daysLate) { return daysLate > 0 ? daysLate * DAILY_RATE : 0; }
export function addToBalance(memberId, amount) { const d = read(); d.member[memberId].balance += amount; write(d); }
```

```js
// notification.mjs — notification to member
export function send(memberId, text) { console.log(`  notification -> member ${memberId}: ${text}`); }
```

The module that runs the workflow calls all four of them in order.

```js
// loan.mjs — loan-issuing workflow; calls the four modules in sequence
import { read, write } from "./store.mjs";
import * as catalog from "./catalog.mjs";
import * as membership from "./membership.mjs";
import * as pricing from "./pricing.mjs";
import * as notification from "./notification.mjs";

export function issueLoan(bookId, memberId, daysLate = 0) {
  if (membership.isEligible(memberId) === false) throw new Error("loan limit exceeded");
  catalog.markOnLoan(bookId);
  membership.addOpenLoan(memberId);
  const amount = pricing.fee(daysLate);
  if (amount > 0) pricing.addToBalance(memberId, amount);
  const d = read();
  d.loans.push({ bookId, memberId });
  write(d);
  notification.send(memberId, `book ${bookId} loan issued`);
  return { bookId, memberId, amount };
}
```

```js
// app.mjs — single entry point; runs the workflow start to finish
import { issueLoan } from "./loan.mjs";
import { status } from "./catalog.mjs";
const [bookId = 1, memberId = 4, daysLate = 0] = process.argv.slice(2).map(Number);
try {
  const s = issueLoan(bookId, memberId, daysLate);
  console.log(`loan issued: book ${s.bookId} -> member ${s.memberId}, fee ${s.amount}`);
} catch (h) {
  console.log("failed:", h.message);
}
console.log(`  catalog: book ${bookId} = ${status(bookId)}`);
```

#### Running the Workflow

Four requests test four paths: a loan that succeeds, a second request for a book already on
loan, a member at their limit, and a record that produces a late fee.

```sh
rm -f data.json
node app.mjs 1 4
echo "--- same book a second time ---"
node app.mjs 1 4
echo "--- member at the limit ---"
node app.mjs 2 5
echo "--- record with a late fee ---"
node app.mjs 2 4 3
```

```
  notification -> member 4: book 1 loan issued
loan issued: book 1 -> member 4, fee 0
  catalog: book 1 = on_loan
--- same book a second time ---
failed: book not on shelf
  catalog: book 1 = on_loan
--- member at the limit ---
failed: loan limit exceeded
  catalog: book 2 = on_shelf
--- record with a late fee ---
  notification -> member 4: book 2 loan issued
loan issued: book 2 -> member 4, fee 6
  catalog: book 2 = on_loan
```

What stands out in the third run is that the membership check ran **before** the catalog
check: because the limit was exceeded, the book was never marked at all. What keeps this
ordering in place is that both checks sit in the same process, in the same `issueLoan` body.
After the fifth lesson this ordering will turn into a contract; for now it is a single `if`
line.

#### Measurement

The numbers the architectural discussion will carry come from reading the source files.

```js
// measure.mjs — module, import edge and cross-module call count in the workflow
import { readdirSync, readFileSync } from "node:fs";
const EDGE = /from "\.\/([\w-]+\.mjs)"/g;
const files = readdirSync(".").filter((a) => a.endsWith(".mjs") && a !== "measure.mjs").sort();
let total = 0;
for (const f of files) {
  const target = [...readFileSync(f, "utf8").matchAll(EDGE)].map((m) => m[1].replace(".mjs", ""));
  total += target.length;
  console.log(`${f.padEnd(18)} -> ${target.join(", ") || "(none)"}`);
}
const flow = [...readFileSync("loan.mjs", "utf8")
  .matchAll(/\b(catalog|membership|pricing|notification)\.\w+\(/g)];
console.log(`module: ${files.length}   import edge: ${total}`);
console.log(`cross-module call in the workflow: ${flow.length}   network hop: 0`);
console.log(`process: 1   deployment unit: 1   startup step: 1`);
```

```sh
node measure.mjs
```

```
app.mjs            -> loan, catalog
catalog.mjs        -> store
loan.mjs           -> store, catalog, membership, pricing, notification
membership.mjs     -> store
notification.mjs   -> (none)
pricing.mjs        -> store
store.mjs          -> (none)
module: 7   import edge: 10
cross-module call in the workflow: 6   network hop: 0
process: 1   deployment unit: 1   startup step: 1
```

Seven modules, ten import edges, six cross-module calls. All six of these calls are local:
they pass parameters on the stack, receive the return value directly, and let errors
propagate through `throw`. The network hop count is zero. Bringing the system up is a single
step; running the loan workflow end to end does not require waiting for any other process to
be ready.

These three numbers — six local calls, zero hops, one deployment unit — say on their own what
the monolith cheapens. There is no such thing as a call getting lost halfway through: a
function either returns or throws. Consistency between catalog and membership does not
require building any mechanism, because both write to the same `data.json` file from the same
process. A change that spans all five domains is tested in a single build and ships in a
single deployment.

#### The Scaling Limit

The cost shows up in the fact that this same unit cannot be split. When load on the loan path
rises, the only thing that can be done is to replicate the entire unit.

```sh
# scale.sh — three copies of the same monolith; each copy carries every module
rm -rf copy1 copy2 copy3
for k in copy1 copy2 copy3; do
  mkdir "$k"
  for m in store catalog membership pricing notification loan app; do cp "$m.mjs" "$k/"; done
  ( cd "$k" && node app.mjs 1 4 | grep '^loan' | sed "s|^|$k: |" )
done
echo "distributed module file: $(ls copy*/*.mjs | wc -l | tr -d ' ')"
```

```
copy1: loan issued: book 1 -> member 4, fee 0
copy2: loan issued: book 1 -> member 4, fee 0
copy3: loan issued: book 1 -> member 4, fee 0
distributed module file: 21
```

Two numbers should be read here. First: even though only the loan path needed extra capacity,
the number of distributed module files went from 7 to 21. Three copies of the notification and
pricing code stand ready even though nothing needed them. The unit of scaling is **the entire
codebase**; knowing which module is the bottleneck does not shrink this unit.

Second, the run itself: all three copies loaned out the same book. Because each copy carries
its own `data.json` file, all three saw the book as on the shelf. In a real deployment, state
would move to a shared store and this tripling would disappear — but then all copies would
load onto a single store. This is where the monolith's scaling limit sits: the process can be
replicated, the state cannot.

There is a limit on the change side too. Because there is a single deployment unit, even a
one-line change to the fee rate republishes the catalog and membership modules along with it.
In measurement terms: one file touched, seven modules redeployed.

#### Three Columns

Every lesson in this course compares the same job across two implementations and fills in
three columns.

| | Monolith |
|---|---|
| Cheapens | Local calls, a shared transaction boundary, a single build, a single deployment step |
| Makes expensive | The scaling unit is the entire codebase; a one-line change redeploys seven modules |
| Failure mode created | A memory leak or infinite loop in one module stops every domain together |

The third row was not measured, because in a single process the only way to measure it is to
crash the process. It is left as a record: the same failure is tried in separate processes in
the fourth lesson, for comparison.

#### Summary

- The library loan service was built as a monolith: 7 modules, 10 import edges, 1 process, 1
  deployment unit; the loan workflow runs start to finish.
- All 6 cross-module calls in the workflow are local; network hops are zero, and the startup
  step count is one.
- A single transaction boundary and local calls keep catalog and membership consistent
  without any extra mechanism.
- The scaling limit shows up under replication: adding capacity to only the loan path turned
  7 modules into 21 files, because the unit of scaling is the entire codebase.
- A single deployment unit means even a one-line fee change redeploys every module.

#### Next Step

One of the measured numbers has not been questioned yet: `loan.mjs` imports all four modules
directly, but nothing stops `notification.mjs` from importing `catalog.mjs` too, or catalog
reaching into membership. The monolith has no boundary inside its code; the import graph grows
toward a complete graph if nobody watches it. The next lesson measures this graph, counts the
**boundary violations** where one module reaches into another, and builds the arrangement
where the boundary is protected by a rule inside the code: the process is still one, the
deployment unit is still one, but import edges are no longer free.

### Modular Monolith

Source: https://academia.sh/en/courses/service-architectures/modular-monolith

Protecting boundaries inside the code: measuring the import graph across domains, counting the boundary violations where one module reaches into another, the files touched and the public surface that grows when the rule is put in place; process and deployment unit stay a single one throughout.

The previous lesson measured the monolith: seven modules, ten import edges, one process, one
deployment unit. What the measurement left open was **which rule** those ten edges were drawn
by. The answer was: no rule at all. A module could import any file of any other module;
nothing stopped it.

This lesson closes that gap. The same loan workflow is rebuilt with its domains split into
directories; first the state where boundaries erode on their own is measured, then the
boundary is protected by a rule. This arrangement, where boundaries are protected inside the
code, is called a **modular monolith**. The process count, the deployment unit count, and the
workflow's output **do not change at all**; the only thing that changes is the import graph.
This is the one place where the architecture can be measured independently of deployment.

The `monolith` here carries the same word as **monolithic persistence** in the Performance
Anti-Patterns and Monitoring course, but it is a separate thing: there, the single item is the
data store; here, the single item is the deployment unit.

#### The Arrangement

**AO3.** State is kept in memory because the system runs within a single execution. This
simplification, which replaces the first lesson's file store, does not affect the
measurement: what is measured is the import graph.

Each of the five domains is a directory. Each directory has an `index.mjs` that is exposed to
the outside, along with the domain's internal files.

```sh
# setup.sh — same workflow as 01, domains became directories; state is kept in memory for a single run
rm -rf broken && mkdir -p broken/catalog broken/membership broken/pricing \
  broken/notification broken/loan && cd broken
cat > store.mjs <<'JS'
export const data = { book: { 1: { status: "on_shelf" } },
                      member: { 4: { name: "Derek", open: 1, limit: 3, balance: 0 } } };
JS
cat > catalog/record.mjs <<'JS'
import { data } from "../store.mjs";
export const book = (id) => data.book[id];
JS
cat > catalog/index.mjs <<'JS'
import { book } from "./record.mjs";
export const status = (id) => book(id).status;
JS
cat > membership/record.mjs <<'JS'
import { data } from "../store.mjs";
export const member = (id) => data.member[id];
JS
cat > membership/index.mjs <<'JS'
import { member } from "./record.mjs";
export const isEligible = (id) => member(id).open < member(id).limit;
export const addOpenLoan = (id) => { member(id).open += 1; };
JS
cat > pricing/index.mjs <<'JS'
import { member } from "../membership/record.mjs";
export const fee = (days) => (days > 0 ? days * 2 : 0);
export const addToBalance = (id, amount) => { member(id).balance += amount; };
JS
cat > notification/index.mjs <<'JS'
import { member } from "../membership/record.mjs";
export const send = (id, text) => console.log(`  notification -> ${member(id).name}: ${text}`);
JS
cat > loan/index.mjs <<'JS'
import { book } from "../catalog/record.mjs";
import * as membership from "../membership/index.mjs";
import * as pricing from "../pricing/index.mjs";
import * as notification from "../notification/index.mjs";
export function issueLoan(bookId, memberId, daysLate = 0) {
  if (membership.isEligible(memberId) === false) throw new Error("loan limit exceeded");
  if (book(bookId).status !== "on_shelf") throw new Error("book not on shelf");
  book(bookId).status = "on_loan";
  membership.addOpenLoan(memberId);
  const amount = pricing.fee(daysLate);
  if (amount > 0) pricing.addToBalance(memberId, amount);
  notification.send(memberId, `book ${bookId} loan issued`);
  return amount;
}
JS
cat > app.mjs <<'JS'
import { issueLoan } from "./loan/index.mjs";
import { status } from "./catalog/index.mjs";
const amount = issueLoan(1, 4, 3);
console.log(`loan issued: book 1 -> member 4, fee ${amount}`);
console.log(`  catalog: book 1 = ${status(1)}`);
JS
find . -name "*.mjs" | sort
node app.mjs
```

```
./app.mjs
./catalog/index.mjs
./catalog/record.mjs
./loan/index.mjs
./membership/index.mjs
./membership/record.mjs
./notification/index.mjs
./pricing/index.mjs
./store.mjs
  notification -> Derek: book 1 loan issued
loan issued: book 1 -> member 4, fee 6
  catalog: book 1 = on_loan
```

The listing is relative to the `broken/` directory. The workflow runs, the fee is
calculated, the notification goes out. The directory split looks tidy on inspection.

#### Measuring the Boundary Violation

The rule to be measured is this: a domain may import only the `index.mjs` file of another
domain. Every import outside of that is a **boundary violation** — one module reaching into
another.

```js
// boundary.mjs — domain boundary rule: only index.mjs may be imported from another domain
import { readdirSync, readFileSync, statSync } from "node:fs";
const root = process.argv[2] ?? ".";
const domains = readdirSync(root).filter((a) => statSync(`${root}/${a}`).isDirectory()).sort();
let edge = 0, violation = 0, surface = 0;
for (const domain of domains) {
  for (const file of readdirSync(`${root}/${domain}`).sort()) {
    if (file.endsWith(".mjs") === false) continue;
    const text = readFileSync(`${root}/${domain}/${file}`, "utf8");
    if (file === "index.mjs") surface += [...text.matchAll(/^export /gm)].length;
    for (const [, target] of text.matchAll(/from "(\.[^"]+)"/g)) {
      edge += 1;
      const parts = target.split("/");
      if (parts[0] === ".." && parts.length === 3 && parts[2] !== "index.mjs") {
        violation += 1;
        console.log(`VIOLATION ${domain}/${file} -> ${target}`);
      }
    }
  }
}
console.log(`domain: ${domains.length}  import edge: ${edge}  public surface: ${surface}  boundary violation: ${violation}`);
```

```sh
node boundary.mjs broken
```

```
VIOLATION loan/index.mjs -> ../catalog/record.mjs
VIOLATION notification/index.mjs -> ../membership/record.mjs
VIOLATION pricing/index.mjs -> ../membership/record.mjs
domain: 5  import edge: 10  public surface: 7  boundary violation: 3
```

Three violations. All three are single-line imports added later, none of which bothered
anyone. Pricing reached straight into the member record to raise the member's balance;
notification did the same to read the member's name; loan opened the catalog record to read
the book's status. None of them produced an error, because everything is reachable within the
same process.

The cost of this shows up not in the violation count, but in how a change spreads. If the
record format inside `membership/record.mjs` changes, two foreign domains have to be fixed
along with membership's own `index.mjs`.

#### Putting the Rule in Place

The fix turns the three outward-reaching imports into the domain's public surface: the
missing operations are added to the relevant domain's `index.mjs`, and the three violating
files are rewritten to go through this new surface.

```sh
# fix.sh — the rule is applied: each domain is only reachable through its index.mjs
rm -rf modular && cp -r broken modular && cd modular
cat > catalog/index.mjs <<'JS'
import { book } from "./record.mjs";
export const status = (id) => book(id).status;
export const isOnShelf = (id) => book(id).status === "on_shelf";
export const markOnLoan = (id) => { book(id).status = "on_loan"; };
JS
cat > membership/index.mjs <<'JS'
import { member } from "./record.mjs";
export const isEligible = (id) => member(id).open < member(id).limit;
export const addOpenLoan = (id) => { member(id).open += 1; };
export const name = (id) => member(id).name;
export const addToBalance = (id, amount) => { member(id).balance += amount; };
JS
cat > pricing/index.mjs <<'JS'
import { addToBalance as addToMemberBalance } from "../membership/index.mjs";
export const fee = (days) => (days > 0 ? days * 2 : 0);
export const addToBalance = (id, amount) => addToMemberBalance(id, amount);
JS
cat > notification/index.mjs <<'JS'
import { name } from "../membership/index.mjs";
export const send = (id, text) => console.log(`  notification -> ${name(id)}: ${text}`);
JS
cat > loan/index.mjs <<'JS'
import * as catalog from "../catalog/index.mjs";
import * as membership from "../membership/index.mjs";
import * as pricing from "../pricing/index.mjs";
import * as notification from "../notification/index.mjs";
export function issueLoan(bookId, memberId, daysLate = 0) {
  if (membership.isEligible(memberId) === false) throw new Error("loan limit exceeded");
  if (catalog.isOnShelf(bookId) === false) throw new Error("book not on shelf");
  catalog.markOnLoan(bookId);
  membership.addOpenLoan(memberId);
  const amount = pricing.fee(daysLate);
  if (amount > 0) pricing.addToBalance(memberId, amount);
  notification.send(memberId, `book ${bookId} loan issued`);
  return amount;
}
JS
echo "file rewritten: 5"
node app.mjs
```

```
file rewritten: 5
  notification -> Derek: book 1 loan issued
loan issued: book 1 -> member 4, fee 6
  catalog: book 1 = on_loan
```

The output is identical to the first run. Nothing changed on the caller's side.

```sh
node boundary.mjs modular
echo "external file reaching into membership's internal file -> broken: $(grep -l 'membership/record.mjs' broken/*/*.mjs | wc -l | tr -d ' ')  modular: $(grep -l 'membership/record.mjs' modular/*/*.mjs | wc -l | tr -d ' ')"
```

```
domain: 5  import edge: 10  public surface: 11  boundary violation: 0
external file reaching into membership's internal file -> broken: 2  modular: 0
```

Three numbers should be read. Violations dropped from 3 to 0. The import edge count **stayed
at 10**: the rule does not reduce the edge count, it changes what the edges point to; three
edges moved from a domain's internal file to that domain's public surface. The public
surface went from 7 to 11 — four newly exported names.

The last line gives the real payoff. The number of files reaching into membership's internal
file from outside dropped from 2 to 0. When the record format inside `membership/record.mjs`
changes, the files that need touching are 3 in the broken version (its own `index.mjs` plus
two foreign domains) and 1 in the modular version.

`addToBalance` in the pricing domain turned into a wrapper that calls membership's function of
the same name. This is the direct cost of the rule: every new cross-domain need adds a layer
of indirection and widens the public surface.

#### The Rule Staying in Place

This arrangement has a weakness: the rule belongs to the checker, not the compiler.

```sh
# leak.sh — a one-line import is added to the modular version; the program keeps running
sed -i.bak '1i\
import { member } from "../membership/record.mjs";
' modular/notification/index.mjs
rm -f modular/notification/index.mjs.bak
( cd modular && node app.mjs | head -n 1 )
node boundary.mjs modular | tail -n 2
```

```
  notification -> Derek: book 1 loan issued
VIOLATION notification/index.mjs -> ../membership/record.mjs
domain: 5  import edge: 11  public surface: 11  boundary violation: 1
```

The program kept running; no test broke. Only the checker spoke up. The violation count went
from 0 to 1, the import edge count from 10 to 11. A boundary violation is not a runtime error,
it is a measurement: left unmeasured, it grows freely. The `-i` option used for in-place
editing behaves differently between the GNU and BSD versions, so a backup extension is given
and then removed here; it produces the same result on both.

#### Three Columns

| | Modular monolith |
|---|---|
| Cheapens | Files touched when a domain's internal structure changes dropped from 3 to 1; the boundary became a countable measure |
| Makes expensive | Public surface went from 7 to 11; every cross-domain need adds a layer of indirection |
| Failure mode created | The rule only holds while a checker runs; a one-line import silently rolled back the boundary without breaking anything |

The numbers that did not change are worth noting too: process 1, deployment unit 1, network
hop 0. A modular monolith is not a deployment decision, it is a code organization decision.
All domains still ship together, and an infinite loop in one domain still stops all of them
together.

#### Summary

- Even after domains were split into directories, the import graph eroded on its own: three
  domains had reached into a neighbor's internal file, and none of them produced an error.
- A boundary violation is a measurable unit: every line that imports a file other than
  `index.mjs` from another domain is a violation.
- When the rule was put in place, 5 files were rewritten, violations dropped from 3 to 0, and
  import edges stayed at 10; the rule does not reduce edges, it moves their endpoints to the
  public surface.
- The cost is the public surface widening from 7 to 11, and a wrapper being added for every
  cross-domain need.
- The rule depends on the checker; a one-line import brought the violation back without
  breaking the program.

#### Next Step

The modular monolith protected the boundary inside the code, but it never questioned one
thing: `store.mjs` keeps holding the shared data structure for every domain. Catalog and
membership both write to different branches of the same object, so the boundary exists in the
import graph but not in the data. The next lesson grows that shared ground: what happens when
domains become separate units even while they talk through a shared contract and a common
data model — it measures how many units a single change in the shared layer touches.

### Service-Oriented Architecture

Source: https://academia.sh/en/courses/service-architectures/service-oriented-architecture

Measuring enterprise service thinking: units talking through a shared contract and a canonical data model, counting how many units are coupled to the shared layer, how many units a canonical field rename touches, and the silent wrong behavior born when the owner is updated but the consumer is not.

The previous lesson built the boundary into the import graph, but `store.mjs` kept holding the
shared data structure for every domain. The boundary existed in the code, not in the data.
This lesson grows that shared ground: domains turn into separate units that carry their own
state, talk through a shared path, and let an **enterprise shared model** decide the shape of
that conversation.

This arrangement is called **service-oriented architecture**. The definition of the style was
covered in the Architectural Styles course; what is measured here is not the definition, it
is the coupling the shared layer brings into the code.

**AO4.** The units are not separate processes; they all talk within the same process, over a
message bus. What is measured is contract coupling, not the network. Splitting into processes
happens in the next lesson.

#### Shared Layer

The shared layer defines two things: the envelope every message carries, and the fields of
the canonical member record.

```js
// shared/contract.mjs — enterprise shared model: message envelope and canonical member fields
export const ENVELOPE = ["source", "operation", "body"];
export const MEMBER_FIELDS = ["memberNo", "name", "openLoans", "limit", "balance"];
export function validate(message) {
  for (const a of ENVELOPE) if (a in message === false) throw new Error(`envelope field missing: ${a}`);
  return message;
}
```

```js
// bus.mjs — enterprise service bus; every message is validated against the shared contract
import { validate } from "./shared/contract.mjs";
const units = new Map();
export const counter = { message: 0 };
export function register(operation, fn) { units.set(operation, fn); }
export function send(message) {
  counter.message += 1;
  return units.get(validate(message).operation)(message.body);
}
```

Each unit carries its own state and registers what it does with the bus. There is no direct
import; units know each other not by name, but by **operation name**.

```js
// catalog.mjs — book unit
import { register } from "./bus.mjs";
const books = { 1: { bookNo: 1, title: "Blindness", status: "on_shelf" },
                 2: { bookNo: 2, title: "The Book of Sand", status: "on_shelf" } };
register("book.status", ({ bookNo }) => books[bookNo].status);
register("book.markOnLoan", ({ bookNo }) => { books[bookNo].status = "on_loan"; });
```

```js
// membership.mjs — unit holding the canonical member record
import { register } from "./bus.mjs";
import { MEMBER_FIELDS } from "./shared/contract.mjs";
const members = {
  4: { memberNo: 4, name: "Derek", openLoans: 1, limit: 3, balance: 0 },
  5: { memberNo: 5, name: "Grace", openLoans: 3, limit: 3, balance: 0 },
};
export const missingFields = (m) => MEMBER_FIELDS.filter((a) => a in m === false);
register("member.get", ({ memberNo }) => members[memberNo]);
register("member.addOpenLoan", ({ memberNo }) => { members[memberNo].openLoans += 1; });
register("member.addToBalance", ({ memberNo, amount }) => { members[memberNo].balance += amount; });
```

```js
// pricing.mjs — late fee unit
import { register } from "./bus.mjs";
register("fee.calculate", ({ daysLate }) => (daysLate > 0 ? daysLate * 2 : 0));
```

```js
// notification.mjs — notification unit; reads the name field from the canonical record
import { register, send } from "./bus.mjs";
register("notification.send", ({ memberNo, text }) => {
  const member = send({ source: "notification", operation: "member.get", body: { memberNo } });
  console.log(`  notification -> ${member.name}: ${text}`);
});
```

The loan unit runs the workflow. The place to notice is the limit check: it takes the
canonical record and applies the rule **itself**.

```js
// loan.mjs — workflow; reads the canonical member record and applies the rule itself
import { send, counter } from "./bus.mjs";
export function issueLoan(bookNo, memberNo, daysLate = 0) {
  const call = (operation, body) => send({ source: "loan", operation, body });
  const member = call("member.get", { memberNo });
  if (member.openLoans >= member.limit) throw new Error("loan limit exceeded");
  if (call("book.status", { bookNo }) !== "on_shelf") throw new Error("book not on shelf");
  call("book.markOnLoan", { bookNo });
  call("member.addOpenLoan", { memberNo });
  const amount = call("fee.calculate", { daysLate });
  if (amount > 0) call("member.addToBalance", { memberNo, amount });
  call("notification.send", { memberNo, text: `book ${bookNo} loan issued` });
  return { amount, message: counter.message };
}
```

```js
// app.mjs — loads every unit and runs the workflow
import "./catalog.mjs"; import "./membership.mjs"; import "./pricing.mjs";
import "./notification.mjs";
import { issueLoan } from "./loan.mjs";
for (const [bookNo, memberNo, days] of [[1, 4, 3], [2, 5, 0]]) {
  try {
    const s = issueLoan(bookNo, memberNo, days);
    console.log(`loan issued: book ${bookNo} -> member ${memberNo}, fee ${s.amount}, message ${s.message}`);
  } catch (h) { console.log(`failed (member ${memberNo}):`, h.message); }
}
```

```sh
node app.mjs
```

```
  notification -> Derek: book 1 loan issued
loan issued: book 1 -> member 4, fee 6, message 8
failed (member 5): loan limit exceeded
```

The first measurement is the message count: one loan workflow produced **8 messages**. In the
first lesson the same workflow ran with 6 local calls. The difference comes from two places.
The notification unit now sends a separate message to get the member's name; the loan unit
also pulls the canonical record and applies the rule itself, instead of calling a behavior
like `isEligible`. The shared model shifted the conversation **from behavior to data**.

#### Unit Coupled to the Shared Model

The measure of coupling is how many units reference the canonical field names.

```js
// coupling.mjs — which unit references which canonical field name
import { readdirSync, readFileSync } from "node:fs";
import { MEMBER_FIELDS } from "./shared/contract.mjs";
const excluded = ["coupling.mjs", "app.mjs", "bus.mjs"];
const units = readdirSync(".").filter((a) => a.endsWith(".mjs") && excluded.includes(a) === false);
let coupled = 0;
for (const u of units.sort()) {
  const text = readFileSync(u, "utf8");
  const found = MEMBER_FIELDS.filter((a) => new RegExp(`\\b${a}\\b`).test(text));
  if (found.length > 0) coupled += 1;
  console.log(`${u.padEnd(20)} canonical field reference: ${found.join(", ") || "(none)"}`);
}
console.log(`unit: ${units.length}   unit coupled to shared model: ${coupled}`);
```

```sh
node coupling.mjs
```

```
catalog.mjs          canonical field reference: (none)
loan.mjs             canonical field reference: memberNo, openLoans, limit
membership.mjs       canonical field reference: memberNo, name, openLoans, limit, balance
notification.mjs     canonical field reference: memberNo, name
pricing.mjs          canonical field reference: (none)
unit: 5   unit coupled to shared model: 3
```

Three of the five units know the field names of the canonical member record. In the previous
lesson this number was one: only the membership domain recognized the `open` field, and loan
asked it through `isEligible`. The shared model spread the record's internal shape to three
units.

The distinction here is this: `notification.mjs` and `loan.mjs` do not **own** the member
record, but they are coupled to its shape. Data ownership sits with the membership unit; data
knowledge sits in three places.

#### The Shared Layer Changing

Say a field of the canonical model is renamed. Making the change in the shared layer and in
the field's owner looks like it should be enough.

```sh
# rename.sh — the canonical field name changes; only the shared layer and the field's owner are updated
sed -i.bak 's/openLoans/openLoanCount/g' shared/contract.mjs membership.mjs
rm -f shared/contract.mjs.bak membership.mjs.bak
echo "file touched: 2"
node app.mjs
```

```
file touched: 2
  notification -> Derek: book 1 loan issued
loan issued: book 1 -> member 4, fee 6, message 8
  notification -> Grace: book 2 loan issued
loan issued: book 2 -> member 5, fee 0, message 15
```

The second request went through. Member 5 was at the limit and had been rejected in the
previous run; now they got a loan. No error was printed, no warning appeared anywhere.

The cause is a single line in the loan unit: `member.openLoans` now returns `undefined`, and
the comparison `undefined >= 3` returns `false`, so the limit check always passes. A name
change in the shared model produced **not an error but wrong behavior** at the consumer.

The fix is carrying the same change to the consumer unit as well.

```sh
# fix.sh — the same change is also carried to the consumer unit
sed -i.bak 's/openLoans/openLoanCount/g' loan.mjs
rm -f loan.mjs.bak
echo "file touched: 3 (shared/contract.mjs, membership.mjs, loan.mjs)"
node app.mjs
```

```
file touched: 3 (shared/contract.mjs, membership.mjs, loan.mjs)
  notification -> Derek: book 1 loan issued
loan issued: book 1 -> member 4, fee 6, message 8
failed (member 5): loan limit exceeded
```

The measure is this: a field name change in the shared layer touched **3 files**, and these
three files have to ship together. Even though units were registered separately, the
canonical model binds them to a single release step. The `-i` option behaves differently
between the GNU and BSD versions when editing in place, so a backup extension is given and
then removed here.

#### Three Columns

| | Service-oriented architecture |
|---|---|
| Cheapens | Units do not import each other; adding a new unit only means registering an operation name, none of the existing units change |
| Makes expensive | Units coupled to the shared model went from 1 to 3; a canonical field name change forces 3 files to release together; the workflow produces 8 messages instead of 6 calls |
| Failure mode created | No error is raised when the shared model's owner is updated but its consumer is not; the limit check silently stops working |

The third column is this lesson's real payoff. The shared layer is a coupling point, and the
defects it produces mostly show up not as a crash, but as **silent wrong behavior**. The loan
unit ran, returned a response, wrote a record; it just failed to apply a rule it was supposed
to.

#### Summary

- In the service-oriented arrangement units do not import each other; they talk by operation
  name over a shared path, and adding a new unit does not change the existing ones.
- The shared canonical model shifted the conversation from behavior to data: the workflow
  produced 8 messages instead of 6 local calls, because consumers pull the record and apply
  the rule themselves.
- The number of units that know the canonical field names went from 1 to 3; data ownership
  stayed with a single unit while data knowledge spread to three.
- A field name change in the shared layer touched 3 files and bound all three to a single
  release step.
- When the owner was updated but the consumer was not, the system raised no error; a member
  at the limit was given a loan. The shared model produces defects as silent wrong behavior.

#### Next Step

In this arrangement units were split apart, but the cost of splitting still has not been
paid: everything runs in the same process, `send` is a function call, a message is an object.
If one unit crashes, all of them crash; if one unit redeploys, all of them redeploy. The next
lesson splits the same loan workflow into **genuinely separate processes**: each unit listens
on its own port, messages travel over the network. What gets measured is process count,
endpoint count, network hops per request, chained latency, the deployment units a change
touches, and the steps needed to bring the system up.

### Microservices

Source: https://academia.sh/en/courses/service-architectures/microservices

Splitting the same loan workflow into genuinely separate processes: five processes, eight endpoints, nine network hops per request, chained latency; one deployment unit redeploying without stopping the others, and the unfinished workflow born when one process goes down.

The previous lesson split the units apart but kept them in the same process: `send` was a
function call, a message was an object. This lesson splits the same loan workflow into
**genuinely separate processes**. Each unit listens on its own port, holds its own data,
starts on its own, and stops on its own. This arrangement, where each unit has its own
deployment unit, is called a **microservice**; the definition of the style was covered in the
Architectural Styles course, what is measured here is the cost the split produces in the run
and in the code.

**AO5.** Five processes talk to each other on the same machine, over the loopback interface.
The latency, packet loss, and fragmentation of a real network do not exist in this
arrangement. The properties the model carries are: a call **crosses a process boundary**, it
gets serialized, the target process can go down independently and can be restarted
independently.

#### Five Processes

Each service takes its port from the command line and reads its neighbors' addresses from an
environment variable. State-changing endpoints are called with `POST`.

```js
// catalog-service.mjs — book unit; its own process, its own data
import { createServer } from "node:http";
const port = Number(process.argv[2]);
if (!port) { console.log("usage: node catalog-service.mjs <port>"); process.exit(0); }
const book = { 1: { status: "on_shelf" }, 2: { status: "on_shelf" }, 3: { status: "on_shelf" } };
createServer((request, response) => {
  const u = new URL(request.url, "http://y");
  const no = u.searchParams.get("bookNo");
  if (u.pathname === "/status") return response.end(JSON.stringify({ status: book[no].status }));
  if (u.pathname === "/mark-on-loan") { book[no].status = "on_loan"; return response.end("{}"); }
  response.statusCode = 404;
  response.end("{}");
}).listen(port, () => console.log(`catalog ${port}`));
```

```js
// membership-service.mjs — member unit; its own process, its own data
import { createServer } from "node:http";
const port = Number(process.argv[2]);
if (!port) { console.log("usage: node membership-service.mjs <port>"); process.exit(0); }
const member = { 4: { name: "Derek", openLoans: 1, limit: 5, balance: 0 },
              5: { name: "Grace", openLoans: 3, limit: 3, balance: 0 } };
createServer((request, response) => {
  const u = new URL(request.url, "http://y");
  const m = member[u.searchParams.get("memberNo")];
  if (m && u.pathname === "/get") return response.end(JSON.stringify(m));
  if (m && u.pathname === "/add-open-loan") { m.openLoans += 1; return response.end("{}"); }
  if (m && u.pathname === "/add-balance") {
    m.balance += Number(u.searchParams.get("amount"));
    return response.end("{}");
  }
  response.statusCode = 404;
  response.end("{}");
}).listen(port, () => console.log(`membership ${port}`));
```

```js
// fee-service.mjs — late fee unit; daily rate lives here
import { createServer } from "node:http";
const port = Number(process.argv[2]);
if (!port) { console.log("usage: node fee-service.mjs <port>"); process.exit(0); }
const DAILY_RATE = 2;
createServer((request, response) => {
  const u = new URL(request.url, "http://y");
  if (u.pathname !== "/calculate") { response.statusCode = 404; return response.end("{}"); }
  const days = Number(u.searchParams.get("days"));
  response.end(JSON.stringify({ fee: days > 0 ? days * DAILY_RATE : 0 }));
}).listen(port, () => console.log(`fee ${port}`));
```

```js
// notification-service.mjs — notification unit; asks the membership process for the member's name
import { createServer } from "node:http";
const port = Number(process.argv[2]);
if (!port) { console.log("usage: node notification-service.mjs <port>"); process.exit(0); }
createServer(async (request, response) => {
  const u = new URL(request.url, "http://y");
  if (u.pathname !== "/send") { response.statusCode = 404; return response.end("{}"); }
  const y = await fetch(`http://127.0.0.1:${process.env.MEMBERSHIP}/get?memberNo=${u.searchParams.get("memberNo")}`);
  console.log(`notification -> ${(await y.json()).name}: ${u.searchParams.get("text")}`);
  response.end("{}");
}).listen(port, () => console.log(`notification ${port}`));
```

The loan service runs the same sequence as `loan.mjs` in the third lesson. The only
difference is that each step is now a network call. It counts the hops it makes and writes
the duration of the chain to its own log.

```js
// loan-service.mjs — the process running the workflow; calls the others over the network
import { createServer } from "node:http";
const port = Number(process.argv[2]);
if (!port) { console.log("usage: node loan-service.mjs <port>"); process.exit(0); }
const A = { catalog: process.env.CATALOG, membership: process.env.MEMBERSHIP,
            fee: process.env.FEE, notification: process.env.NOTIFICATION };
let hop = 0;
const call = async (target, path, method = "GET") => {
  hop += 1;
  return (await fetch(`http://127.0.0.1:${A[target]}${path}`, { method })).json();
};
createServer(async (request, response) => {
  const u = new URL(request.url, "http://y");
  if (u.pathname !== "/loan") { response.statusCode = 404; return response.end("{}"); }
  const memberNo = u.searchParams.get("memberNo"), bookNo = u.searchParams.get("bookNo");
  const days = u.searchParams.get("days") ?? 0;
  hop = 0;
  const t0 = performance.now();
  try {
    const member = await call("membership", `/get?memberNo=${memberNo}`);
    if (member.openLoans >= member.limit) throw new Error("loan limit exceeded");
    if ((await call("catalog", `/status?bookNo=${bookNo}`)).status !== "on_shelf")
      throw new Error("book not on shelf");
    await call("catalog", `/mark-on-loan?bookNo=${bookNo}`, "POST");
    await call("membership", `/add-open-loan?memberNo=${memberNo}`, "POST");
    const { fee } = await call("fee", `/calculate?days=${days}`);
    if (fee > 0) await call("membership", `/add-balance?memberNo=${memberNo}&amount=${fee}`, "POST");
    await call("notification", `/send?memberNo=${memberNo}&text=book+${bookNo}`, "POST");
    console.log(`loan ${bookNo}/${memberNo}: hop ${hop}, duration ${Math.round(performance.now() - t0)} ms`);
    response.end(JSON.stringify({ status: "issued", fee, hop }));
  } catch (h) {
    console.log(`loan ${bookNo}/${memberNo}: ERROR ${h.message}, hop ${hop}`);
    response.statusCode = 400;
    response.end(JSON.stringify({ status: "failed", error: h.message, hop }));
  }
}).listen(port, () => console.log(`loan ${port}`));
```

#### Bringing It Up

```sh
# start.sh — five processes come up; each process's output is written to its own log
rm -f pid.txt
node catalog-service.mjs 8792 >catalog.log 2>&1 & echo "catalog $!" >>pid.txt
node membership-service.mjs 8793 >membership.log 2>&1 & echo "membership $!" >>pid.txt
node fee-service.mjs 8794 >fee.log 2>&1 & echo "fee $!" >>pid.txt
MEMBERSHIP=8793 node notification-service.mjs 8795 >notification.log 2>&1 & echo "notification $!" >>pid.txt
CATALOG=8792 MEMBERSHIP=8793 FEE=8794 NOTIFICATION=8795 \
  node loan-service.mjs 8791 >loan.log 2>&1 & echo "loan $!" >>pid.txt
for p in 8791 8792 8793 8794 8795; do
  for i in $(seq 40); do curl -s -o /dev/null "http://127.0.0.1:$p/" && break; sleep 0.2; done
done
echo "process brought up: $(wc -l <pid.txt | tr -d ' ')   startup step: 5 background + 1 readiness wait"
echo "endpoint: $(grep -oh '"/[a-z-]*"' ./*.mjs | sort -u | wc -l | tr -d ' ')   deployment unit: 5"
grep -oh '"/[a-z-]*"' ./*.mjs | sort -u | tr -d '"' | tr '\n' ' '
echo
```

```
process brought up: 5   startup step: 5 background + 1 readiness wait
endpoint: 8   deployment unit: 5
/add-balance /add-open-loan /calculate /get /loan /mark-on-loan /send /status
```

In the first lesson this number was one: `node app.mjs`. Here there are five background
processes and one readiness wait; ports and neighbor addresses have to be supplied from
outside.

#### Hops Per Request

```sh
curl -s "http://127.0.0.1:8791/loan?bookNo=1&memberNo=4&days=3"
echo
curl -s "http://127.0.0.1:8791/loan?bookNo=2&memberNo=5"
echo
```

```
{"status":"issued","fee":6,"hop":7}
{"status":"failed","error":"loan limit exceeded","hop":1}
```

The loan service made 7 calls. The notification service made one more call to get the
member's name; together with the request the client sent to the loan service, that comes to
**9 network hops per request** in total. In the third lesson the same workflow produced 8
messages — the message count did not change, every message just came to cross a process
boundary.

The chain proceeds in sequence: each call waits for the previous one's return, so the
latencies add up. The loan service writes one line to `loan.log` for every request; the first
request's line came out as `loan 1/4: hop 7, duration 21 ms` in this run.
The duration field depends on the machine, the core, and the load at that moment — the value
above was read in this run and changes on every run. The quantity that does not depend on the
run is **7 sequential round trips**, and this number only drops if the chain is shortened.
The second request stopped at the membership check, so it finished with a single hop: when an
error surfaces early, the rest of the chain never gets built.

#### Independent Deployment

The real payoff is measured here: let the fee rate change.

```sh
# restart.sh — only the fee process changes and restarts; the others never stop
kill "$(awk '$1 == "fee" { print $2 }' pid.txt)"
sed -i.bak 's/const DAILY_RATE = 2;/const DAILY_RATE = 5;/' fee-service.mjs
rm -f fee-service.mjs.bak
node fee-service.mjs 8794 >>fee.log 2>&1 & echo "fee $!" >>pid.txt
for i in $(seq 40); do curl -s -o /dev/null "http://127.0.0.1:8794/" && break; sleep 0.2; done
echo "process restarted: 1 / 5   deployment unit touched: 1"
curl -s "http://127.0.0.1:8791/loan?bookNo=2&memberNo=4&days=3"
echo
```

```
process restarted: 1 / 5   deployment unit touched: 1
{"status":"issued","fee":15,"hop":7}
```

The new rate is in effect. The catalog, membership, notification, and loan processes never
stopped; they kept their in-memory state. In the first lesson the same one-line change
redeployed seven modules together; here, one deployment unit restarted.

This payoff is not unlimited. The canonical `openLoans` field appears in two deployment
units — membership and loan. Renaming it was a change touching 3 files within a single unit
in the third lesson; here it requires **two separate deployment units to release in a
coordinated order**. Independent deployment is independent only for changes that do not touch
the contract.

#### Partial Failure

```sh
# fault.sh — the notification process stops; the same request breaks at the end of its path
kill "$(awk '$1 == "notification" { print $2 }' pid.txt)"
sleep 0.5
curl -s "http://127.0.0.1:8791/loan?bookNo=3&memberNo=4"
echo
curl -s "http://127.0.0.1:8792/status?bookNo=3"
echo
curl -s "http://127.0.0.1:8793/get?memberNo=4"
echo
```

```
{"status":"failed","error":"fetch failed","hop":6}
{"status":"on_loan"}
{"name":"Derek","openLoans":4,"limit":5,"balance":21}
```

The request came back failed, but the book is on loan and the member's open loan count went
up. The chain that broke on the sixth hop did not undo the effect of the first five hops. This
did not happen in the monolith: an exception left the whole body, and no module was left
having finalized half a job.

The second line of the same output says something else: the catalog service is still
responding. When the notification process went down, the system as a whole did not stop —
paths that do not touch notification keep working. The failure is now **partial**; this is
both a gain and a new problem, because the state of the system now sits somewhere between
"up" and "down".

```sh
# stop.sh — stops every process
while read -r name p; do kill "$p" 2>/dev/null; done <pid.txt
echo "stopped"
```

```
stopped
```

#### Three Columns

| | Microservices |
|---|---|
| Cheapens | A rate change restarted 1 of 5 deployment units; each unit owns its own data, the scaling unit is a single service |
| Makes expensive | 9 network hops and 7 sequential round trips per request; startup went from 1 step to 6 steps; addresses have to be supplied from outside |
| Failure mode created | A request that breaks mid-chain leaves an unfinished state behind; the failure is partial, the system is neither fully up nor fully down |

#### Summary

- The same loan workflow was split into 5 separate processes: 8 endpoints, 5 deployment
  units, 9 network hops per request, 7 sequential round trips.
- The message count did not change from the third lesson; what changed is that every message
  now crosses a process boundary, and sequential round trips add up their latencies.
- Bringing the system up went from 1 step to 6; every process's port and neighbor addresses
  have to be supplied from outside.
- A fee rate change restarted only 1 deployment unit; the other four processes never stopped.
  A change that touches the contract forces two units to release in a coordinated order.
- When the notification process went down, the request came back failed, but the book was on
  loan and the member's counter had already gone up: a workflow broken mid-chain leaves an
  unfinished state behind.

#### Next Step

These five processes are long-lived: they stay up whether a request arrives or not, keep
their state in memory, and are started once and left running. The next lesson shrinks the
deployment unit by one more step and removes long-livedness: the unit becomes a function that
exists only when an event triggers it. What gets measured are the extra calls that come from
being unable to hold anything in memory between two calls, a per-call time limit splitting the
workflow, and the last local calls still left inside a process turning into network calls too.

### Serverless Approach

Source: https://academia.sh/en/courses/service-architectures/serverless-approach

Measuring the constraints of the event-triggered function model: the extra store calls that come from being unable to hold anything in memory between two calls, a per-call time limit splitting the workflow, and the last local calls left inside a process turning into outside calls.

The previous lesson's five processes were long-lived: started once, staying up whether a
request arrived or not, holding member records in their own memory. This lesson shrinks the
deployment unit by one more step and removes long-livedness. The unit is no longer a process,
it is **a function that exists only when an event triggers it**.

This arrangement is called **serverless**. The definition of the style and its initial setup
cost were covered in the Architectural Styles course; what is measured here is the function
model's **runtime constraints**: nothing staying in memory between two calls, a time limit on
every call, and the last local calls left inside a process going outside too.

**AO6.** The function runner is a local model: for every event it starts a new process with
`node` and applies a fixed time limit to that process. The model carries three properties —
memory is not shared between calls, an exceeded time limit cuts the work off mid-way, and
everything outside the function is reached over the network.

#### Runner and Store

State has to sit somewhere. Since functions cannot hold it, the single long-lived process is
a key-value store.

```js
// store-service.mjs — single long-lived process holding state between functions
import { createServer } from "node:http";
const port = Number(process.argv[2]);
if (!port) { console.log("usage: node store-service.mjs <port>"); process.exit(0); }
const box = { "book:1": { status: "on_shelf" },
               "member:4": { name: "Derek", openLoans: 1, limit: 5, balance: 0 },
               overdue: ["o1", "o2", "o3"], processed: [] };
let calls = 0;
const counts = {};
createServer(async (request, response) => {
  const u = new URL(request.url, "http://y");
  const a = u.searchParams.get("a");
  const track = () => { calls += 1; counts[a] = (counts[a] ?? 0) + 1; };
  if (u.pathname === "/get") { track(); return response.end(JSON.stringify(box[a] ?? null)); }
  if (u.pathname === "/put") {
    track();
    let g = "";
    for await (const p of request) g += p;
    box[a] = JSON.parse(g);
    return response.end("{}");
  }
  if (u.pathname === "/counter") {
    const y = { calls, member: counts["member:4"] ?? 0 };
    if (u.searchParams.get("reset")) { calls = 0; for (const k in counts) delete counts[k]; }
    return response.end(JSON.stringify(y));
  }
  response.statusCode = 404;
  response.end("{}");
}).listen(port, () => console.log(`store ${port}`));
```

```js
// runner.mjs — function invoker: a new process for each event, a time limit per call
import { spawnSync } from "node:child_process";
export const TIMEOUT_MS = 400;
export function call(fn, ...event) {
  const s = spawnSync("node", [fn, ...event], { encoding: "utf8", timeout: TIMEOUT_MS });
  return { cut: s.signal !== null, output: s.stdout.trim() };
}
```

#### Four Functions

Each function takes its event from the command line, does its job, prints the result, and
dies. None of them can leave anything for the next call.

```js
// function-eligibility.mjs — event-triggered function; starts with an empty memory every call
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const member = await (await fetch(`http://127.0.0.1:${S}/get?a=member:${process.argv[2]}`)).json();
console.log(JSON.stringify({ eligible: member.openLoans < member.limit }));
```

```js
// function-mark.mjs — marks the book on loan, increases the member's counter
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const [bookNo, memberNo] = process.argv.slice(2);
const write = (a, d) => fetch(`http://127.0.0.1:${S}/put?a=${a}`, { method: "POST", body: JSON.stringify(d) });
const book = await (await fetch(`http://127.0.0.1:${S}/get?a=book:${bookNo}`)).json();
if (book.status !== "on_shelf") { console.log(JSON.stringify({ error: "book not on shelf" })); process.exit(0); }
await write(`book:${bookNo}`, { status: "on_loan" });
const member = await (await fetch(`http://127.0.0.1:${S}/get?a=member:${memberNo}`)).json();
member.openLoans += 1;
await write(`member:${memberNo}`, member);
console.log(JSON.stringify({ status: "on_loan" }));
```

```js
// function-fee.mjs — calculates the late fee and writes it to the member's balance
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const [memberNo, days] = process.argv.slice(2).map(Number);
const fee = days > 0 ? days * 2 : 0;
if (fee > 0) {
  const member = await (await fetch(`http://127.0.0.1:${S}/get?a=member:${memberNo}`)).json();
  member.balance += fee;
  await fetch(`http://127.0.0.1:${S}/put?a=member:${memberNo}`, { method: "POST", body: JSON.stringify(member) });
}
console.log(JSON.stringify({ fee }));
```

```js
// function-notify.mjs — reads the member's name from the store and prints the notification
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const [memberNo, text] = process.argv.slice(2);
const member = await (await fetch(`http://127.0.0.1:${S}/get?a=member:${memberNo}`)).json();
console.log(`notification -> ${member.name}: ${text}`);
```

```js
// flow.mjs — loan workflow split into four function calls; each call a new process
import { call } from "./runner.mjs";
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const counter = async () => (await (await fetch(`http://127.0.0.1:${S}/counter?reset=1`)).json());
await counter();
const [bookNo, memberNo, days] = ["1", "4", "3"];
let fn = 0;
const c = (name, ...event) => { fn += 1; return JSON.parse(call(name, ...event).output || "null"); };
const e = c("function-eligibility.mjs", memberNo);
if (e.eligible) {
  c("function-mark.mjs", bookNo, memberNo);
  const { fee } = c("function-fee.mjs", memberNo, days);
  fn += 1;
  console.log(call("function-notify.mjs", memberNo, `book ${bookNo}`).output);
  console.log(`loan issued: book ${bookNo} -> member ${memberNo}, fee ${fee}`);
}
const s = await counter();
console.log(`function call: ${fn}   new process: ${fn}   store call: ${s.calls}   member:4 access: ${s.member}`);
```

```sh
# start.sh — the single long-lived process holding state comes up, then the workflow runs
node store-service.mjs 8796 >store.log 2>&1 & echo $! >store.pid
for i in $(seq 40); do curl -s -o /dev/null "http://127.0.0.1:8796/" && break; sleep 0.2; done
echo "long-lived process: 1 (store)"
STORE=8796 node flow.mjs
```

```
long-lived process: 1 (store)
notification -> Derek: book 1
loan issued: book 1 -> member 4, fee 6
function call: 4   new process: 4   store call: 8   member:4 access: 6
```

#### The Cost of Not Holding State

The last line carries three numbers.

**New process: 4.** In the previous lesson a single loan request started no new process; five
processes were already up and serving the request. Here every request spawns four processes,
and all four end. This is the direct consequence of the deployment unit existing only for the
duration of the call.

**Store call: 8.** In the microservice arrangement the member record lived in the membership
process's memory; reaching the record was an in-process object read. Here no function can
carry the record forward, so each function has to pull it from the store on its own call.

**`member:4` access: 6.** The same member record crossed the network six times inside a
single loan workflow: eligibility read it, mark read and wrote it, fee read and wrote it,
notify read it. In the previous lesson the calls made to the membership service were three,
and all three worked on the same object in memory. The difference is the function model's
most concrete cost: **anything that cannot be held in memory crosses the network.**

There is a second consequence tied to this. In the previous lesson `fee-service` kept the
daily rate in its own memory and did the calculation in-process; here the calculation is
still done in-process, but writing the result requires pulling the member record and writing
it back. The last local steps left inside a process turned into outside calls too.

#### Time Limit

The second constraint is the per-call time limit. A job that produces notifications for three
overdue books takes 300 ms per record; the limit is 400 ms.

```js
// function-overdue.mjs — processes overdue notifications sequentially in a single call
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const records = await (await fetch(`http://127.0.0.1:${S}/get?a=overdue`)).json();
const done = [];
for (const r of records) {
  await new Promise((c) => setTimeout(c, 300));
  done.push(r);
  await fetch(`http://127.0.0.1:${S}/put?a=processed`, { method: "POST", body: JSON.stringify(done) });
}
console.log(JSON.stringify({ processed: done.length }));
```

```js
// function-overdue-single.mjs — processes one record; state is carried between calls in the store
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const records = await (await fetch(`http://127.0.0.1:${S}/get?a=overdue`)).json();
await new Promise((c) => setTimeout(c, 300));
const done = await (await fetch(`http://127.0.0.1:${S}/get?a=processed`)).json();
done.push(records[Number(process.argv[2])]);
await fetch(`http://127.0.0.1:${S}/put?a=processed`, { method: "POST", body: JSON.stringify(done) });
console.log(JSON.stringify({ processed: done.length }));
```

```js
// duration.mjs — the same work runs first in a single call, then split across calls
import { call, TIMEOUT_MS } from "./runner.mjs";
const S = process.env.STORE;
if (!S) { console.log("STORE environment variable required"); process.exit(0); }
const get = (path) => fetch(`http://127.0.0.1:${S}${path}`);
const reset = () => fetch(`http://127.0.0.1:${S}/put?a=processed`, { method: "POST", body: "[]" });
const track = async () => (await (await get("/counter?reset=1")).json()).calls;
const processed = async () => (await (await get("/get?a=processed")).json()).length;

await reset(); await track();
const single = call("function-overdue.mjs");
console.log(`single run: cut=${single.cut}  store call=${await track()}  processed=${await processed()}/3  limit=${TIMEOUT_MS} ms`);

await reset(); await track();
let cut = 0;
for (const i of ["0", "1", "2"]) if (call("function-overdue-single.mjs", i).cut) cut += 1;
console.log(`split run : call=3  cut=${cut}  store call=${await track()}  processed=${await processed()}/3`);
```

```sh
STORE=8796 node duration.mjs
```

```
single run: cut=true  store call=2  processed=1/3  limit=400 ms
split run : call=3  cut=0  store call=9  processed=3/3
```

In the single run the job was left half-finished: one of three records was processed and the
process was cut off mid-way. The cut is not an error, it is the runner's rule — the code has
no fault, the work is incomplete. What stands out here is that the cut call **left a partial
effect**: one record had already been written to the store.

In the split version the same job spread across three calls and completed. The cost shows up
in the store call count: it went from 2 to 9. The reason is that the work's state can no
longer be carried between calls — every call has to reread both the input list and what has
been processed so far. When the time limit split the workflow, carrying state paid the cost
of the split.

These numbers are relatively resilient to the environment for two reasons: the work per record
was chosen at 300 ms, the limit at 400 ms; the quantities independent of the run are the cut
call count, the processed record count, and the store call count.

```sh
# stop.sh — stops the store process
kill "$(cat store.pid)" 2>/dev/null
echo "stopped"
```

```
stopped
```

#### Three Columns

| | Serverless |
|---|---|
| Cheapens | The deployment unit shrank to a single file; the standing process count dropped from 5 to 1, a function does not exist outside of a call |
| Makes expensive | 4 new processes per request; 6 network accesses to the same member record; store calls went from 2 to 9 once the work was split |
| Failure mode created | The time limit cuts work off mid-way: the code has no fault, the result is incomplete, and a partial effect is left in the store |

#### Summary

- In the function model the deployment unit exists only for the duration of the call: a loan
  workflow produced 4 function calls and 4 new processes; in the previous lesson this number
  was zero.
- Because nothing stayed in memory between two calls, state moved to the store: the workflow
  made 8 store calls and reached a single member record over the network 6 times.
- The last local steps left inside a process turned into outside calls too; even though the
  calculation still runs in-process, writing the result requires a read and a write.
- The per-call time limit cut the work off mid-way: one of three records was processed, and
  the cut call left a partial effect behind.
- Splitting the work into calls kept it under the limit, but it created a state-carrying cost:
  store calls went from 2 to 9.

#### Next Step

At this point the same loan workflow has been built in five forms, and each one has written
its own three columns. One question remains: given a working monolith in hand, **how** does a
move to one of these forms happen. The next lesson takes on the migration itself: the gradual
decomposition of the path, the strangler fig pattern letting the old and new paths run
together for a while, and the cost of that period on the code side — shared logic held in two
codebases, double maintenance cost, and reversibility.

### Architectural Migration Strategies

Source: https://academia.sh/en/courses/service-architectures/architectural-migration-strategies

Moving from a working monolith to a gradual decomposition: the code-side measure of the strangler fig pattern — shared logic held in two codebases, a single rate change touching two files, paths diverging, and single-step reversibility through a routing switch.

Five forms were built, and each one wrote its own three columns. What was never taken on was
how to move from a working monolith to one of these forms. This lesson measures the migration
itself.

Moving the entire system to a new form in one shot means taking on every measured cost at
once. Instead, the path is decomposed one route at a time: a single step of a workflow moves
to a new unit, the rest stays in the monolith, and a switch decides which path traffic goes
down. As more routes move, the old body shrinks. This arrangement is called the **strangler
fig** migration; the Case Studies course measured the traffic side of this pattern, what gets
measured here is the **code side**.

**AO7.** The new route carries a single step: the late fee calculation. The rest of the loan
workflow stays in the monolith. What is measured is not traffic share, it is the cost of
holding the same logic in two codebases.

#### The Switch

The old route is the fee rule itself, from the first lesson.

```js
// monolith/fee.mjs — the fee rule on the old path
// RULE START
export const DAILY_RATE = 2;
export const fee = (days) => (days > 0 ? days * DAILY_RATE : 0);
// RULE END
```

The router sits between the workflow and the fee step. While the switch is off, the call
stays local; while it is on, it goes to the new unit over the network.

```js
// monolith/router.mjs — strangler fig switch; the step goes to the old or new path
import { fee as oldFee } from "./fee.mjs";
export async function calculateFee(days) {
  const p = process.env.NEW_FEE;
  if (!p) return { fee: oldFee(days), route: "old" };
  return (await fetch(`http://127.0.0.1:${p}/calculate?days=${days}`)).json();
}
```

```js
// monolith/app.mjs — loan workflow; only the fee step goes through the router
import { calculateFee } from "./router.mjs";
const state = { book: "on_shelf", openLoans: 1, limit: 5 };
const days = Number(process.argv[2] ?? 3);
if (state.openLoans >= state.limit) { console.log("loan limit exceeded"); process.exit(0); }
state.book = "on_loan";
state.openLoans += 1;
const { fee, route } = await calculateFee(days);
console.log(`loan issued: book 1 -> member 4, ${days} days late, fee ${fee} (fee route: ${route})`);
```

The new unit carries the rule in its own codebase. It has to carry it: being a separate
deployment unit means it cannot import the monolith's files.

```js
// fee-service.mjs — new path; the same rule in a second codebase
import { createServer } from "node:http";
const port = Number(process.argv[2]);
if (!port) { console.log("usage: node fee-service.mjs <port>"); process.exit(0); }
// RULE START
export const DAILY_RATE = 2;
export const fee = (days) => (days > 0 ? days * DAILY_RATE : 0);
// RULE END
createServer((request, response) => {
  const days = Number(new URL(request.url, "http://y").searchParams.get("days"));
  response.end(JSON.stringify({ fee: fee(days), route: "new" }));
}).listen(port, () => console.log(`fee ${port}`));
```

```sh
node monolith/app.mjs 3
```

```
loan issued: book 1 -> member 4, 3 days late, fee 6 (fee route: old)
```

```sh
# start.sh — the new fee process comes up, the same request runs with the switch on
node fee-service.mjs 8797 >fee.log 2>&1 & echo $! >fee.pid
for i in $(seq 40); do curl -s -o /dev/null "http://127.0.0.1:8797/" && break; sleep 0.2; done
NEW_FEE=8797 node monolith/app.mjs 3
```

```
loan issued: book 1 -> member 4, 3 days late, fee 6 (fee route: new)
```

Both paths gave the same result. This is the desired state of the migration: when the new
path opens, nothing looks different from outside. The only difference is in the `fee route`
field.

#### Two Codebases

What gets measured is not that these two runs are equal, but the cost of keeping that
equality up.

```js
// duplication.mjs — measures the same business rule being held in two codebases
import { readFileSync } from "node:fs";
const rule = (path) => {
  const s = readFileSync(path, "utf8").split("\n");
  return s.slice(s.indexOf("// RULE START") + 1, s.indexOf("// RULE END"))
    .map((x) => x.trim()).filter(Boolean);
};
const old = rule("monolith/fee.mjs"), current = rule("fee-service.mjs");
const same = old.filter((s) => current.includes(s));
console.log(`rule line: old ${old.length}, new ${current.length}, identical ${same.length}`);
console.log(`codebase holding the same logic: 2   file touched by a rate change: 2`);
console.log(`routing switch: NEW_FEE   rollback step: 1   redeploy: 0`);
```

```sh
node duplication.mjs
```

```
rule line: old 2, new 2, identical 2
codebase holding the same logic: 2   file touched by a rate change: 2
routing switch: NEW_FEE   rollback step: 1   redeploy: 0
```

Both lines of the rule sit identical in the two codebases. For the duration of the migration
this is not a choice, it is a necessity: as long as the old route still takes traffic it has
to carry the logic, and the new route has to carry its own copy too, because it cannot reach
the other's files.

#### Double Maintenance

The cost surfaces when the rule changes. Say the daily rate is updated in the new codebase
only.

```sh
# diverge.sh — the rate changes only in the new codebase; the two paths diverge
kill "$(cat fee.pid)"
sleep 0.3
sed -i.bak 's/export const DAILY_RATE = 2;/export const DAILY_RATE = 5;/' fee-service.mjs
rm -f fee-service.mjs.bak
node fee-service.mjs 8797 >>fee.log 2>&1 & echo $! >fee.pid
for i in $(seq 40); do curl -s -o /dev/null "http://127.0.0.1:8797/" && break; sleep 0.2; done
NEW_FEE=8797 node monolith/app.mjs 3
node monolith/app.mjs 3
```

```
loan issued: book 1 -> member 4, 3 days late, fee 15 (fee route: new)
loan issued: book 1 -> member 4, 3 days late, fee 6 (fee route: old)
```

Same member, same book, same delay — two different fees. Which amount applies is decided not
by the business rule, but by which path the request happened to land on. This is a failure
mode specific to the migration period, and it needs no process to crash: both codebases are
running, and each one is correct on its own terms.

The measure is this: for the duration of the migration, every change touching the fee rule
has to be written to **two files**. This cost is not multiplied by the number of routes
moved, it is multiplied by how long the move takes. A migration left half-finished means
permanent double maintenance.

#### Reversibility

The pattern's real payoff sits in the same place.

```sh
# rollback.sh — the routing switch is turned off; the new service is not stopped
echo "switch off, no redeploy"
node monolith/app.mjs 3
echo "new service still up: $(curl -s 'http://127.0.0.1:8797/calculate?days=3')"
```

```
switch off, no redeploy
loan issued: book 1 -> member 4, 3 days late, fee 6 (fee route: old)
new service still up: {"fee":15,"route":"new"}
```

The old behavior came back in a single step: an environment variable was not supplied, that
is all. The new service was not stopped, nothing was redeployed, no file changed. Had the old
code been deleted, the equivalent of this step would have been redeploying the old codebase.

The cost of reversibility is that the old route is **not deleted**. Counting the migration as
finished happens by permanently turning the switch on and deleting the old rule; until then,
double maintenance continues. The two costs are two faces of the same coin.

```sh
# reconcile.sh — the same change is also carried to the old codebase; the two paths match again
sed -i.bak 's/export const DAILY_RATE = 2;/export const DAILY_RATE = 5;/' monolith/fee.mjs
rm -f monolith/fee.mjs.bak
echo "file touched: 2 (monolith/fee.mjs, fee-service.mjs)"
NEW_FEE=8797 node monolith/app.mjs 3
node monolith/app.mjs 3
kill "$(cat fee.pid)" 2>/dev/null
```

```
file touched: 2 (monolith/fee.mjs, fee-service.mjs)
loan issued: book 1 -> member 4, 3 days late, fee 15 (fee route: new)
loan issued: book 1 -> member 4, 3 days late, fee 15 (fee route: old)
```

#### Three Columns

| | Gradual migration |
|---|---|
| Cheapens | Rollback is 1 step, redeploy 0; the cost of the new form is tested on a single route, the whole system does not move |
| Makes expensive | The same rule sits in 2 codebases; every change is written to 2 files; the router is an extra layer of indirection |
| Failure mode created | The same request gives two different results when only one codebase is updated; no process crashes, and each path is correct on its own terms |

#### Summary

- The strangler fig migration moves a single step of a workflow to a new unit, leaves the
  rest in place, and ties traffic to a routing switch.
- The new unit is a separate deployment unit, so it has to carry the rule in its own
  codebase; the measurement shows the same two lines sitting identical in two codebases.
- The measure of double maintenance is that a rate change has to be written to 2 files; when
  it was written to only one, the same request gave two different results, 15 and 6.
- Rollback is 1 step and requires no redeploy, because the old route is still standing; the
  cost of this payoff is that double maintenance continues.
- The migration ends when the switch is turned on permanently and the old rule is deleted; a
  migration left half-finished means permanent double maintenance.

#### Next Step

Across this topic the same loan workflow was built in five forms — monolith, modular
monolith, service-oriented arrangement, microservices, serverless functions — and the sixth
lesson measured the cost of moving to one of them. Every measurement changed some number:
import edges, boundary violations, units coupled to the shared model, processes, endpoints,
network hops, store calls, files touched.

All of these measurements shared one assumption that was never questioned: **where the
boundary runs** was taken as given. Catalog, membership, loans, notification, and pricing
were drawn as five separate modules as early as the first lesson; the next five lessons
inherited that line as it was and only argued over which form it would take. But the real
decision is not the form, it is where the line sits. Why is pricing not inside membership?
Why is notification separate from loans?

This question has not been tied to any criterion so far. The next topic takes it on directly:
what evidence, visible in the code and in the run, decides where a boundary should run.

## Service Boundaries and Communication

Where the boundary should pass and how to talk across it: boundary-drawing criteria based on coupling, data ownership, and change coupling; chained latency and error propagation in synchronous calls; what event-based loose coupling gains and what it costs; edge responsibilities collected at the gateway; a client-specific aggregation layer; network concerns separated from the application; and address resolution joining health information in service discovery.

### Boundary-Drawing Criteria

Source: https://academia.sh/en/courses/service-architectures/boundary-drawing-criteria

Measuring from the code where the boundary should pass: counting the inter-module import edge as a coupling measure, determining each table's owner and finding non-owner writes, change coupling extracted from the change log, and comparing three candidate boundary drawings by the same measures.

The previous topic built the options spanning from the monolith to the serverless approach, and
counted the cost of each in deployment units, processes, and deployment steps. All the options
shared one common assumption: where the boundary passes was taken as given. Yet the most
expensive decision on that list is not the name of the chosen architectural option — it is the
location of the boundary. A boundary drawn in the wrong place touches every change on both sides,
no matter which architecture is chosen.

This lesson ties the boundary's location to a measure, and the measure lives inside the code.
Domain modeling, bounded contexts, and the context map were built in the Domain-Driven Design
course; the domain model is not taught here. The question here is narrower: given a working
modular monolith, which number decides where the boundary is drawn. The through-line is the
library loan system, and it has five domain modules: catalog, membership, loan, notification,
billing.

#### The Application to Measure

The block below writes the application to disk and runs the workflow for issuing a book loan.
The module files are generated from a map; each module carries in its own source which modules
it imports, which tables it writes to, and which table it owns.

```js
// setup.mjs — writes the library loan application as module files and runs the workflow
import { mkdirSync, writeFileSync } from "node:fs";

const MODULE = {                       // imports: domain modules | writes: tables | owns: tables
  catalog:      { imports: [],                  writes: ["book", "copy"], owns: ["book", "copy"] },
  membership:   { imports: [],                  writes: ["member"],       owns: ["member"] },
  loan:         { imports: ["catalog", "membership"], writes: ["loan", "copy"], owns: ["loan"] },
  billing:      { imports: ["loan"],            writes: ["fee", "loan"],  owns: ["fee"] },
  notification: { imports: ["membership", "loan"], writes: ["notification"], owns: ["notification"] },
};

mkdirSync("app", { recursive: true });
writeFileSync("app/store.mjs",
  "export const records = [];\n" +
  "export const write = (table, row) => records.push({ table, ...row });\n");

for (const [name, m] of Object.entries(MODULE)) {
  writeFileSync(`app/${name}.mjs`,
    `// owns: ${m.owns.join(", ")}\n` +
    'import { write, records } from "./store.mjs";\n' +
    m.imports.map((i) => `import { ${i}Read } from "./${i}.mjs";\n`).join("") +
    `export const ${name}Read = () => records.filter((k) => k.table === "${m.owns[0]}");\n` +
    `export function ${name}Process(data) {\n` +
    m.imports.map((i) => `  ${i}Read();\n`).join("") +
    m.writes.map((t) => `  write("${t}", { step: "${name}", ...data });\n`).join("") +
    "}\n");
}
writeFileSync("app/app.mjs",
  'import { records } from "./store.mjs";\n' +
  'import { membershipProcess } from "./membership.mjs";\n' +
  'import { catalogProcess } from "./catalog.mjs";\n' +
  'import { loanProcess } from "./loan.mjs";\n' +
  'import { notificationProcess } from "./notification.mjs";\n' +
  'import { billingProcess } from "./billing.mjs";\n' +
  'const request = { member: 7, book: 412 };\n' +
  "membershipProcess(request); catalogProcess(request); loanProcess(request);\n" +
  "notificationProcess(request); billingProcess(request);\n" +
  'export const written = records.map((k) => `${k.step}->${k.table}`);\n');

const { written } = await import("./app/app.mjs");
console.log(`module files: ${Object.keys(MODULE).length} domain modules + store + app entry`);
console.log(`workflow (issuing a loan for a book) produced ${written.length} writes:`);
console.log(written.join("  "));
```

```
module files: 5 domain modules + store + app entry
workflow (issuing a loan for a book) produced 8 writes:
membership->member  catalog->book  catalog->copy  loan->loan  loan->copy  notification->notification  billing->fee  billing->loan
```

This is the starting point: one deployment unit, one process, eight writes. In this form the
system runs and there is no boundary at all. The boundary-drawing decision comes after this.

#### Two Measures Read From the Code

The first two measures are extracted directly from the source files. **Coupling** is reduced
here to a single number: an import a module makes from another domain module is one edge. **Data
ownership** is the second measure: every table has exactly one owner module, and a non-owner
module writing to that table is counted separately.

```js
// measure.mjs — reads the app/ directory setup.mjs wrote; the measure is extracted from the code
import { readFileSync, readdirSync } from "node:fs";

const files = readdirSync("app").filter((d) => d !== "store.mjs" && d !== "app.mjs");
const M = {};
for (const d of files) {
  const t = readFileSync(`app/${d}`, "utf8");
  M[d.replace(".mjs", "")] = {
    imports: [...t.matchAll(/from "\.\/(\w+)\.mjs"/g)].map((m) => m[1]).filter((a) => a !== "store"),
    writes: [...t.matchAll(/write\("(\w+)"/g)].map((m) => m[1]),
    owns: t.match(/^\/\/ owns: (.+)$/m)[1].split(", "),
  };
}
const names = Object.keys(M);
const ownerOf = (table) => names.find((a) => M[a].owns.includes(table));
const edges = names.flatMap((a) => M[a].imports.map((b) => [a, b]));
const foreignWrites = names.flatMap((a) => M[a].writes.filter((t) => ownerOf(t) !== a).map((t) => [a, t]));

console.log(`${"module".padEnd(15)}${"imports".padEnd(20)}${"writes".padEnd(15)}owns`);
for (const a of names) {
  console.log(`${a.padEnd(15)}${(M[a].imports.join(",") || "-").padEnd(20)}` +
    `${M[a].writes.join(",").padEnd(15)}${M[a].owns.join(",")}`);
}
console.log(`total: ${names.length} modules, ${edges.length} import edges, ` +
  `${names.reduce((s, a) => s + M[a].writes.length, 0)} writes, ` +
  `${foreignWrites.length} non-owner writes (${foreignWrites.map(([a, t]) => `${a}->${t}`).join(", ")})`);
```

```
module         imports             writes         owns
billing        loan                fee,loan       fee
catalog        -                   book,copy      book,copy
loan           catalog,membership  loan,copy      loan
membership     -                   member         member
notification   membership,loan     notification   notification
total: 5 modules, 5 import edges, 8 writes, 2 non-owner writes (billing->loan, loan->copy)
```

Five edges and two non-owner writes. Within a single deployment unit these two writes are merely
a layout flaw: the loan module changes copy status directly, and the billing module puts an
overdue marker on the loan record. Because both are in the same process, inside the same
transaction, they cost nothing today. The moment the boundary is drawn, these turn into
**boundary violations**: a service writing to another service's table now means a dependency on
either an endpoint or a shared table.

#### The Third Measure: Change Coupling

The third measure is not visible in the source files; it lives in the codebase's history.
**Change coupling** is how often two modules change together in the same change. The measure is
this: a boundary that separates two frequently co-changing modules spreads every change across
two deployment units.

**SB1 — the last quarter's change log is twenty lines, and each line gives the modules a change
touched.** Rationale: every change in the version history can be reduced from the files it
touched to a set of modules; this lesson fixes the result of that reduction as input. If the
count and the distribution change, the ranking below changes with them; the measure itself does
not.

```js
// candidates.mjs — reads the same directory, compares three candidate boundary drawings by the same measures
import { readFileSync, readdirSync } from "node:fs";

const M = {};
for (const d of readdirSync("app").filter((x) => x !== "store.mjs" && x !== "app.mjs")) {
  const t = readFileSync(`app/${d}`, "utf8");
  M[d.replace(".mjs", "")] = {
    imports: [...t.matchAll(/from "\.\/(\w+)\.mjs"/g)].map((m) => m[1]).filter((a) => a !== "store"),
    writes: [...t.matchAll(/write\("(\w+)"/g)].map((m) => m[1]),
    owns: t.match(/^\/\/ owns: (.+)$/m)[1].split(", "),
  };
}
const names = Object.keys(M);
const ownerOf = (table) => names.find((a) => M[a].owns.includes(table));
const edges = names.flatMap((a) => M[a].imports.map((b) => [a, b]));
const foreignWrites = names.flatMap((a) => M[a].writes.filter((t) => ownerOf(t) !== a).map((t) => [a, t]));

// SB1: last quarter's change log — each line is one change and the modules it touched
const CHANGES = [
  "late fee rate: loan billing", "loan extension: loan billing",
  "fee exemption: billing", "reminder text: notification",
  "contact preference: membership notification", "membership suspension: membership loan",
  "genre tag: catalog", "search field: catalog",
  "copy status: catalog loan", "reservation queue: catalog loan",
  "penalty threshold: loan billing", "member address field: membership",
  "notification channel: notification", "renewal reminder: membership notification",
  "overdue days calculation: loan billing", "lost copy: catalog loan billing",
  "cover image: catalog", "authentication: membership",
  "return flow: loan", "fee refund: billing",
].map((s) => [s.split(": ")[0], s.split(": ")[1].split(" ")]);

const pairs = {};
for (const [, ms] of CHANGES) {
  for (let i = 0; i < ms.length; i += 1) {
    for (let j = i + 1; j < ms.length; j += 1) {
      const c = [ms[i], ms[j]].sort().join(" + ");
      pairs[c] = (pairs[c] ?? 0) + 1;
    }
  }
}
console.log(`${CHANGES.length} changes with pairs that changed together (SB1):`);
for (const [c, n] of Object.entries(pairs).sort((a, b) => b[1] - a[1])) {
  console.log(`  ${c.padEnd(30)}${n}`);
}

const CANDIDATE = {
  "A reference/transaction": [["catalog", "membership"], ["loan", "billing", "notification"]],
  "B per-entity": [["catalog"], ["membership"], ["loan"], ["billing"], ["notification"]],
  "C change coupling": [["loan", "billing"], ["catalog"], ["membership", "notification"]],
};
console.log(`\n${"candidate".padEnd(26)}${"units".padStart(6)}${"cross-boundary imports".padStart(24)}` +
  `${"ownership violations".padStart(22)}${"kept in one unit".padStart(19)}${"deployments".padStart(13)}`);
for (const [a, split] of Object.entries(CANDIDATE)) {
  const unitOf = (m) => split.findIndex((b) => b.includes(m));
  const touched = CHANGES.map(([, ms]) => new Set(ms.map(unitOf)).size);
  const total = touched.reduce((s, n) => s + n, 0);
  console.log(`${a.padEnd(26)}${String(split.length).padStart(6)}` +
    `${String(edges.filter(([x, y]) => unitOf(x) !== unitOf(y)).length).padStart(24)}` +
    `${String(foreignWrites.filter(([x, t]) => unitOf(x) !== unitOf(ownerOf(t))).length).padStart(22)}` +
    `${`${touched.filter((n) => n === 1).length}/${CHANGES.length}`.padStart(19)}` +
    `${String(total).padStart(13)}`);
}

// Same change, four implementations: SB1's first line
const [dName, dMods] = CHANGES[0];
console.log(`\n"${dName}" change (${dMods.join(", ")}) across four implementations:`);
console.log(`${"implementation".padEnd(26)}${"files touched".padStart(16)}${"deployment units".padStart(18)}` +
  `${"imports->network".padStart(19)}${"writes crossing boundary".padStart(27)}`);
for (const [g, split] of Object.entries({ "modular monolith": [names], ...CANDIDATE })) {
  const unitOf = (m) => split.findIndex((b) => b.includes(m));
  console.log(`${g.padEnd(26)}${String(dMods.length).padStart(16)}` +
    `${String(new Set(dMods.map(unitOf)).size).padStart(18)}` +
    `${String(edges.filter(([x, y]) => dMods.includes(x) && dMods.includes(y) && unitOf(x) !== unitOf(y)).length).padStart(19)}` +
    `${String(foreignWrites.filter(([x, t]) => dMods.includes(x) && unitOf(x) !== unitOf(ownerOf(t))).length).padStart(27)}`);
}
```

```
20 changes with pairs that changed together (SB1):
  billing + loan                5
  catalog + loan                3
  membership + notification     2
  loan + membership             1
  billing + catalog             1

candidate                  units  cross-boundary imports  ownership violations   kept in one unit  deployments
A reference/transaction        2                       3                     1              14/20           26
B per-entity                   5                       5                     2              10/20           31
C change coupling              3                       3                     1              16/20           24

"late fee rate" change (loan, billing) across four implementations:
implementation               files touched  deployment units   imports->network   writes crossing boundary
modular monolith                         2                 1                  0                          0
A reference/transaction                  2                 1                  0                          1
B per-entity                             2                 2                  1                          2
C change coupling                        2                 1                  0                          1
```

#### Reading the Three Candidates

The most frequently co-changing pair is loan and billing: five of the twenty changes touch both
at once. Catalog and loan change together three times, membership and notification twice.

**Candidate B, one service per entity, is the cleanest-looking drawing and the most expensive by
measure.** All five of the five import edges cross the boundary, both of the two ownership
violations turn into boundary violations, and only ten of the twenty changes stay within a single
unit. The total deployment count across the twenty changes is 31.

**Candidates A and C are equal on coupling** — both give three edges and one violation. The
distinction shows up in the third measure: A keeps fourteen of the changes within a single unit,
C keeps sixteen; total deployments are 26 against 24. The difference comes from A separating
membership and notification — that pair changes together twice, and each time A touches two
units.

The rule that emerges from this: **coupling and ownership narrow the field of candidates; change
coupling makes the decision.** The first two measures read the code's current state, the third
reads how the code changes; the boundary is drawn by the third, because the boundary's cost is
paid at every change.

#### The Same Change, Four Implementations

The last table follows a single change: the late fee rate, which touches loan and billing. The
number of files touched is two in all four implementations; drawing a boundary does not reduce a
change's code volume. What changes is where those two files belong.

**What it made cheaper.** In the modular monolith and in candidate C, the change stays within a
single deployment unit: one verification, one deployment, one rollback. C does this with no
difference from the monolith, while it has still split the system into three independent units —
catalog changes never touch the loan unit at all.

**What it made more expensive.** In candidate B, the same change touches two deployment units,
one import edge turns into a network call, and two writes cross the boundary. Billing writing to
the loan table is no longer writing to its own data — it is writing to another unit's data;
either that unit opens an endpoint, or the table is shared between two units. Either way means a
new contract.

**What new failure mode it created.** In the monolith, both files are released together and there
is never an inconsistent moment between them. Split across two deployment units, a window opens
between the two releases: the new rate is already in effect in billing, but loan is still sending
the old field. This is a failure mode that does not exist in the monolith, and it is born from the
boundary itself. The one ownership violation left in A and C belongs to the same class: the loan
module writing copy status will, the moment catalog moves to a separate unit, either turn into a
call or lead to two units writing the same row.

#### Summary

- The boundary's location is tested with three measures, all three read from the code: import
  edges (coupling), owner module per table (data ownership), and the module set of changes
  (change coupling).
- The measured application has 5 domain modules, 5 import edges, and 8 writes; two of the writes
  come from a non-owner module, and while they cost nothing inside a single deployment unit, they
  turn into boundary violations once the boundary is drawn.
- The one-service-per-entity drawing is the worst on all three measures: 5 of 5 edges cross the
  boundary, both of the 2 violations remain, only 10 of 20 changes stay within a single unit, and
  the total deployment count is 31.
- Two candidates equal on coupling are separated by change coupling: 14/20 against 16/20, and 26
  against 24 deployments; a boundary that separates a frequently co-changing pair touches two
  units at every change.
- The same change touches 2 files in all four implementations; the boundary does not change the
  file count — it changes the deployment unit count, the network call count, and the number of
  writes that cross the boundary.

#### Next Step

The boundary has been tied to a measure, but the moment it is drawn, a number that sat quietly in
the table starts to work: an import turning into a network call. An import edge between modules is
a function call within the same process, and its cost is measured on the stack. The same edge
between two deployment units is a network call, and its cost is measured in rounds. The next
lesson builds this conversion with real processes: the same loan-issuing workflow runs first in a
single process, then in separate processes, and as the chain grows longer, how the end-to-end
round count grows is measured. The same run asks a second question — when one of these processes
is down, what share of incoming requests goes unanswered.

### Synchronous Service Calls

Source: https://academia.sh/en/courses/service-architectures/synchronous-service-calls

Measuring the call crossing the boundary with real processes: the end-to-end round count growing with chain length, a distributed call finishing the same work in fewer rounds, the call amplification that gives the number of internal requests one external request turns into, and the share of entry points left unanswered when a process closes.

The previous lesson tied the boundary to a measure and left a number sitting quietly in the
table: an import turning into a network call. An import edge within the same process is a
function call; the same edge between two deployment units is a network call. This lesson builds
that conversion not with a model but with real, separate operating-system processes, and counts
the difference in rounds.

Timeouts, retries, and distributing the end-to-end budget across a chain were built and measured
in The Application Layer and Service Interaction course; that arithmetic is not repeated here.
The three measures here are: the end-to-end round count by chain length, the number of internal
requests one external request spawns, and the share of requests left unanswered when a process
goes down. A synchronous call is one where the caller waits for the response and folds the result
into its own response.

#### The Same Work, Two Layouts

The loan-issuing workflow has four steps: recording the loan, checking membership, checking for
unpaid fees, and allocating a copy. The service below is a single source file; which role it
takes on comes from its command-line arguments. Every step spends the same duration, so the
measured time can be divided by the number of work steps.

```js
// service.mjs — <name> <port> <mode: parallel|sequential> <step> [successor-port...]; long-lived process
import { createServer } from "node:http";
const [name, port, mode, step, ...successors] = process.argv.slice(2);
if (!port) { console.log("usage: node service.mjs <name> <port> <mode> <step> [successors...]"); process.exit(1); }
const WORK = 25;                      // duration of one work step (ms); measured time is divided by this unit
let processed = 0;
const call = async (p) => {           // the successor's status code is checked: an error crosses the boundary
  const y = await fetch(`http://127.0.0.1:${p}/work`);
  if (y.status !== 200) throw new Error(`successor ${p} status ${y.status}`);
  return y.json();
};

createServer(async (request, response) => {
  if (request.url === "/count") { response.end(JSON.stringify({ name, processed })); return; }
  if (request.url === "/reset") { processed = 0; response.end("reset"); return; }
  processed += 1;
  for (let i = 0; i < Number(step); i += 1) await new Promise((c) => setTimeout(c, WORK));
  try {
    let children = [];
    if (mode === "sequential") for (const p of successors) children.push(await call(p));
    else children = await Promise.all(successors.map(call));
    response.end(JSON.stringify({ name, step: Number(step), children }));
  } catch (e) {
    response.statusCode = 502;
    response.end(JSON.stringify({ name, error: e.message }));
  }
}).listen(Number(port));
```

The same four steps will run in two layouts. In the split layout there are four processes: the
loan service calls the membership and catalog services, and the membership service calls the
billing service. In the monolithic layout, all four steps run sequentially in a single process;
the loan endpoint's `step` value is four and there is no successor call at all. A third layout
exists for comparison: the loan service calls its two successors in sequence instead of in
parallel.

The line that checks the status code on a successor call determines the rest of the lesson.
Within a process, a step's failure propagates upward as an exception; across the boundary it is
only a status code. If the caller does not check that code, the failure does not propagate — it
is swallowed: the response looks successful, but the membership check inside it was never done.

#### Measurement

The measurement tool runs in three modes. Latency mode sends three requests to each entry point
and finds the round count by dividing the median by the work-step duration; the duration
measurement depends on the environment, the round count does not. Multiplier mode resets the
counters, sends five requests to a single entry point, and sums the requests processed by the
processes. Failure mode sends one request to each entry point and counts how many go unanswered.

**SB2 — external requests are distributed equally across the four entry points.** Rationale: the
affected request share is the traffic weight of the entry points that depend on the service that
goes down; when the weights are unknown, an equal distribution makes the share measurable. When
the real distribution is known, the same calculation is done with weights.

```js
// measure.mjs — <mode: latency|multiplier|failure> [name]; runs while service.mjs processes are up
const WORK = 25;                                 // same work step duration as service.mjs
const UNIT = { billing: 8941, membership: 8942, catalog: 8943, loan: 8944,
  "sequential-loan": 8945, "single-process": 8946 };
const FAILURE_ENTRY = ["billing", "membership", "catalog", "loan"];
const requestUnit = async (p) => {
  const y = await fetch(`http://127.0.0.1:${p}/work`);
  if (y.status !== 200) throw new Error(`status ${y.status}`);
  return y.json();
};
const nodeCount = (a) => 1 + (a.children ?? []).reduce((s, x) => s + nodeCount(x), 0);
const stepCount = (a) => a.step + (a.children ?? []).reduce((s, x) => s + stepCount(x), 0);
const [mode, target] = process.argv.slice(2);

if (mode === "latency") {
  console.log(`${"entry".padEnd(18)}${"processes".padStart(11)}${"network hops".padStart(14)}` +
    `${"work steps".padStart(12)}${"measured rounds".padStart(17)}`);
  let maxOverhead = 0;
  for (const name of Object.keys(UNIT)) {
    const durations = [];
    let tree;
    for (let i = 0; i < 3; i += 1) {
      const t = performance.now();
      tree = await requestUnit(UNIT[name]);
      durations.push(performance.now() - t);
    }
    const median = durations.sort((a, b) => a - b)[1];
    const nodes = nodeCount(tree), rounds = Math.floor(median / WORK);
    maxOverhead = Math.max(maxOverhead, median - rounds * WORK);
    console.log(`${name.padEnd(18)}${String(nodes).padStart(11)}${String(nodes - 1).padStart(14)}` +
      `${String(stepCount(tree)).padStart(12)}${String(rounds).padStart(17)}`);
  }
  console.log(`measured duration = rounds x ${WORK} ms + overhead; overhead in this run stayed ` +
    `under ${Math.ceil(maxOverhead / WORK) * WORK} ms (less than one work step)`);
} else if (mode === "multiplier") {
  for (const name of Object.keys(UNIT)) await fetch(`http://127.0.0.1:${UNIT[name]}/reset`);
  const N = 5;
  for (let i = 0; i < N; i += 1) await requestUnit(UNIT.loan);
  let total = 0;
  for (const name of Object.keys(UNIT)) {
    const { processed } = await (await fetch(`http://127.0.0.1:${UNIT[name]}/count`)).json();
    total += processed;
    if (processed > 0) console.log(`  ${name.padEnd(18)}${String(processed).padStart(8)} requests`);
  }
  console.log(`${N} external requests to the loan entry point -> ${total} processed requests total, ` +
    `call amplification x${total / N} (${total / N - 1} internal requests per external request)`);
} else if (mode === "failure") {
  const dropped = [];
  for (const name of FAILURE_ENTRY) {
    try { await requestUnit(UNIT[name]); } catch { dropped.push(name); }
  }
  console.log(`${target.padEnd(16)}${(dropped.join(",") || "-").padEnd(36)}` +
    `${dropped.length}/${FAILURE_ENTRY.length}`);
}
```

The shell block below brings up six processes, runs the three measurements, then closes three
services one by one, running failure mode after each one and restarting the closed service
before moving on.

```bash
start() {
  case $1 in
    billing)    node service.mjs billing 8941 parallel 1 & ;;
    membership) node service.mjs membership 8942 parallel 1 8941 & ;;
    catalog)    node service.mjs catalog 8943 parallel 1 & ;;
    loan)       node service.mjs loan 8944 parallel 1 8942 8943 & ;;
  esac
}
for s in billing membership catalog loan; do start $s; done
node service.mjs sequential-loan 8945 sequential 1 8942 8943 &
node service.mjs single-process 8946 parallel 4 &
sleep 1

node measure.mjs latency
echo
echo "call amplification:"
node measure.mjs multiplier
echo
printf '%-16s%-36s%s\n' "closed service" "unanswered entries" "affected request share"
for s in billing catalog loan; do
  pkill -f "service.mjs $s" ; sleep 0.3
  node measure.mjs failure $s
  start $s ; sleep 0.5
done
pkill -f "node service.mjs"
```

```
entry               processes  network hops  work steps  measured rounds
billing                     1             0           1                1
membership                  2             1           2                2
catalog                     1             0           1                1
loan                        4             3           4                3
sequential-loan             4             3           4                4
single-process              1             0           4                4
measured duration = rounds x 25 ms + overhead; overhead in this run stayed under 25 ms (less than one work step)

call amplification:
  billing                  5 requests
  membership               5 requests
  catalog                  5 requests
  loan                     5 requests
5 external requests to the loan entry point -> 20 processed requests total, call amplification x4 (3 internal requests per external request)

closed service  unanswered entries                  affected request share
billing         billing,membership,loan             3/4
catalog         catalog,loan                        2/4
loan            loan                                1/4
```

#### Chain Length Determines the Round Count

The first three rows read as the chain getting longer. A request to the billing entry point
finishes in one process: zero hops, one round. The membership entry point calls billing: two
processes, one hop, two rounds. The loan entry point runs four processes at once: three hops,
four work steps. **Chained latency** becomes visible here in rounds: the end-to-end round count
is the chain's sequential step count, and the measurement returns exactly that.

The fourth and sixth rows are the lesson's real comparison. The loan entry point finishes its
four work steps in **three** rounds; a single process finishes the same four steps in **four**.
The gain does not come from the split itself — it comes from the opportunity the split opens up:
because the membership branch and the catalog branch do not have to wait on each other, they are
called in parallel, and the end-to-end duration is the larger of the two branches, not their sum.
The fifth row confirms this — the same four processes, the same three hops, but a loan service
that calls its successors in sequence finishes in four rounds. **A distributed call wins back a
round; a sequential call does not.**

The duration measurement depends on the environment, and the cost of a hop over loopback is
small; the entire overhead stayed under one work step. On a real network, the same three hops are
not this cheap, but the round count does not change: the round is a quantity independent of the
run, and the latency budget is multiplied by rounds.

#### Multiplier and Affected Share

The call amplification is the second measure. Five external requests to the loan entry point make
the processes run twenty requests in total: a multiplier of four, meaning three internal
requests per external request. This number is a direct result of the boundary decision; the
boundary-crossing import edges the previous lesson measured are exactly the edges that turn into
internal requests at run time. As the number of boundary-crossing edges grows, the multiplier
grows, and every service has to size its capacity not for its own traffic but for traffic
multiplied by that factor.

The last table turns the new failure mode into a number. When the billing process goes down, not
only its own entry point but the membership and loan entry points that depend on it also go
unanswered: three of the four entry points, seventy-five percent of requests under SB2. When
catalog goes down, two entry points are affected; when loan goes down, one. **The affected share
is the dependency closure of the service that goes down**: the service at the bottom of the chain
is the most expensive, because every entry point above it depends on it. In the monolithic layout
this table is a single row — if the process goes down, all requests drop, but there is no such
thing as partial failure.

The triple resolves like this. **What got cheaper:** the four work steps finish in three rounds,
and every service can scale on its own; one service going down does not take every entry point
with it — when lending, at the top of the chain, goes down, three of the four entry points stay
up. **What got more expensive:** three network hops per request, four processes,
four times the processed-request volume, and a startup arrangement split into four. **The failure
mode that was born:** partial failure. The system is no longer either fully up or fully down;
there is a response rate that varies by entry point, and if the successor's status code is not
checked, that rate never shows up — it turns invisibly into successful responses that did
incomplete work.

#### Summary

- The end-to-end round count is the chain's sequential step count: the measurement gave 1, 2, and
  3 rounds for the zero-, one-, and three-hop entry points respectively; overhead stayed under
  one work step.
- The same four work steps finish in 4 rounds in a single process and in 3 rounds across four
  processes; the gain is not the split itself but the parallel call the split enables — the
  layout that calls its successors in sequence finishes in 4 rounds.
- The call amplification is the run-time counterpart of the boundary-crossing edges: 5 external
  requests turned into 20 processed requests, 3 internal requests per external request.
- The affected request share when a process goes down is that service's dependency closure: 3/4
  for billing, 2/4 for catalog, 1/4 for loan.
- The boundary turns a failure from an exception into a status code; if the caller does not check
  the status code, the failure does not propagate — it is swallowed, and a response that looks
  successful contains incomplete work.

#### Next Step

All of these measurements lived inside a single assumption: the caller waits until the
successor's work is done. The multiplier, the round count, and the affected share are all born
from that wait — because the loan service waits for the notification to be sent, the notification
service going down drops the loan request. Yet not every step in a workflow has to finish for the
response to be given. The next lesson builds the same workflow a second time: one of the steps on
the response path stops being a call and turns into an event, processed by its consumer on its
own schedule. What will be measured is also clear — the work step that drops off the response
path, the contract burden that appears once the publisher and the consumer are separate
deployment units, and the consistency window between an event being written and its effect
becoming visible.

### Asynchronous Integration

Source: https://academia.sh/en/courses/service-architectures/asynchronous-integration

Measuring two implementations of the same workflow, synchronous and event-based: the work step and unit count that drop off the response path, the contract burden created because the publisher does not know its consumers, the deployment units bound to each field in the event body, and the consistency window between the response returning and the effect becoming visible.

Every number in the previous lesson was born from a single assumption: the caller waits until the
successor's work is done. Remove the wait, and both the round count and the dependency closure
change. This lesson builds the same loan-issuing flow a second time; this time the notification
and billing steps are pulled off the response path and turned into consumers of an event, and the
two implementations are compared by the same measures.

Queue mechanics, delivery guarantees, back pressure, and competing consumers were built and
measured in the Caching, Queues and Asynchronous Processing course; they are not repeated here.
What is measured is not the carrier itself, but how the coupling on the two sides of the boundary
changes.

**SB3 — the carrier is modeled as an append log with fixed-interval polling.** Rationale: since
delivery and ordering guarantees are measured in a separate course, the carrier's plainest form is
enough here; the three quantities measured (units on the response path, units bound per field,
the consistency window) are independent of the carrier's type. The log is a real file, the polling
runs in a real process; only the carrier's plainness is a model.

#### Two Implementations, One Source

The loan service runs in two modes. In synchronous mode, it calls both its successors and waits
for both responses. In event mode, it writes the same body to the log and returns. The difference
is one line, but that line also changes the service's configuration: in event mode the successor
address list is empty.

```js
// loan.mjs — <mode: synchronous|event> <port> <notification-port> <fee-port>; long-lived process
import { createServer } from "node:http";
import { appendFileSync } from "node:fs";
const [mode, port, pn, pf] = process.argv.slice(2);
if (!port) { console.log("usage: node loan.mjs <mode> <port> <notification-port> <fee-port>"); process.exit(1); }
const WORK = 25;                                   // duration of writing the loan record (ms)
const SUCCESSOR = mode === "synchronous" ? [pn, pf] : [];   // in event mode the publisher carries no successor address
let sequence = 0;

createServer(async (request, response) => {
  if (request.url === "/successors") { response.end(JSON.stringify({ successors: SUCCESSOR.length })); return; }
  await new Promise((c) => setTimeout(c, WORK));
  sequence += 1;
  const event = { type: "loan_issued", sequence: sequence, memberId: 7, copyId: 400 + sequence,
    overdueDays: 0, title: "Book loan issued" };
  const body = JSON.stringify(event);
  if (mode === "event") appendFileSync("events.log", `${body}\n`);
  else await Promise.all(SUCCESSOR.map((p) =>
    fetch(`http://127.0.0.1:${p}/work`, { method: "POST", body })));
  response.end(body);
}).listen(Number(port));
```

The notification and billing units also run from a single source and take on two roles at once:
called as endpoints, they take the body from the request; run as consumers, they read the same
body from the log. The work itself is the same function in both cases; only where the body comes
from changes.

```js
// unit.mjs — <name: notification|fee> <port> <poll: 0|1>; runs both as an endpoint and as a consumer
import { createServer } from "node:http";
import { appendFileSync, existsSync, readFileSync } from "node:fs";
const [name, port, poll] = process.argv.slice(2);
if (!port) { console.log("usage: node unit.mjs <name> <port> <poll>"); process.exit(1); }
const WORK = 25, INTERVAL = 200;                     // the unit's work and polling interval (ms)
const CONTRACT = {                               // the fields each unit reads from the event body
  notification: (o) => `notification "${o.title}" sent to member ${o.memberId}`,
  fee: (o) => `member ${o.memberId}, copy ${o.copyId}, ${o.overdueDays} day(s) overdue`,
};
const handle = async (o) => {
  await new Promise((c) => setTimeout(c, WORK));
  appendFileSync(`effect-${name}.log`, `${o.sequence} ${CONTRACT[name](o)}\n`);
};

createServer(async (request, response) => {
  let s = "";
  for await (const p of request) s += p;
  await handle(JSON.parse(s));
  response.end("done");
}).listen(Number(port));

let read = 0, busy = false;
if (poll === "1") setInterval(async () => {
  if (busy || !existsSync("events.log")) return;
  busy = true;
  const lines = readFileSync("events.log", "utf8").split("\n").filter(Boolean);
  while (read < lines.length) { await handle(JSON.parse(lines[read])); read += 1; }
  busy = false;
}, INTERVAL);
```

#### The Measurement Tool

The tool does two jobs. In run mode, it sends three requests to find the round count, then waits
for effects to settle and sends one more request to see whether the new effect is visible the
moment the response returns. It repeats the same observation 600 milliseconds later. In contract
mode no process is up at all; only two source files are read, and how many deployment units each
field in the event body binds is counted.

```js
// measure.mjs — <mode: synchronous|event|contract>; the first two modes run while the processes are up
import { existsSync, readFileSync } from "node:fs";
const WORK = 25, UNIT_PORT = { loan: 8951 }, UNIT = ["notification", "fee"];
const wait = (ms) => new Promise((c) => setTimeout(c, ms));
const effect = () => UNIT.reduce((s, a) => s + (existsSync(`effect-${a}.log`)
  ? readFileSync(`effect-${a}.log`, "utf8").split("\n").filter(Boolean).length : 0), 0);
const requestLoan = () => fetch(`http://127.0.0.1:${UNIT_PORT.loan}/work`).then((y) => y.text());
const [mode] = process.argv.slice(2);

if (mode === "synchronous" || mode === "event") {
  const { successors } = await (await fetch(`http://127.0.0.1:${UNIT_PORT.loan}/successors`)).json();
  const durations = [];
  for (let i = 0; i < 3; i += 1) {
    const t = performance.now();
    await requestLoan();
    durations.push(performance.now() - t);
  }
  const rounds = Math.floor(durations.sort((a, b) => a - b)[1] / WORK);
  await wait(600);
  const before = effect();
  await requestLoan();
  const immediate = effect() > before;                   // is the new effect visible the moment the response returns
  await wait(600);
  const after = effect() > before;
  console.log(`${mode.padEnd(14)}${String(rounds).padStart(18)}${String(1 + successors).padStart(24)}` +
    `${String(successors).padStart(28)}${(immediate ? "yes" : "no").padStart(20)}` +
    `${(after ? "yes" : "no").padStart(14)}${`${effect()}/8`.padStart(14)}`);
} else if (mode === "contract") {
  const eventBlock = readFileSync("loan.mjs", "utf8").match(/const event = \{([\s\S]*?)\};/)[1];
  const fields = [...eventBlock.matchAll(/(\w+):/g)].map((m) => m[1]);
  const source = readFileSync("unit.mjs", "utf8");
  const consumer = {};
  for (const m of source.matchAll(/^ {2}(\w+): \(o\) => (.*)$/gm)) {
    consumer[m[1]] = [...m[2].matchAll(/o\.(\w+)/g)].map((x) => x[1]);
  }
  const shared = [...source.replace(/^ {2}\w+: \(o\) => .*$/gm, "").matchAll(/o\.(\w+)/g)]
    .map((x) => x[1]);
  const consumerNames = Object.keys(consumer);
  console.log(`${"event field".padEnd(14)}${"publisher".padStart(11)}${"reading consumers".padStart(19)}` +
    `${"shared code".padStart(13)}${"bound deployment units".padStart(24)}`);
  let total = 0;
  for (const a of fields) {
    const reading = consumerNames.filter((t) => consumer[t].includes(a));
    const o = shared.includes(a) ? 1 : 0;
    const bound = 1 + (o ? consumerNames.length : reading.length);
    total += bound;
    console.log(`${a.padEnd(14)}${"1".padStart(11)}${String(reading.length).padStart(19)}` +
      `${String(o).padStart(13)}${String(bound).padStart(24)}`);
  }
  console.log(`${fields.length}-field event body, ${consumerNames.length} consumer units; ` +
    `average bound units per field ${(total / fields.length).toFixed(2)}`);
  console.log(`field no consumer reads: ` +
    `${fields.filter((a) => consumerNames.every((t) => !consumer[t].includes(a)) && !shared.includes(a)).join(", ")}`);
}
```

```bash
rm -f events.log effect-notification.log effect-fee.log
printf '%-14s%18s%24s%28s%20s%14s%14s\n' mode "response rounds" \
  "units on response path" "successors publisher knows" "effect at response" "600 ms later" "effect lines"
node unit.mjs notification 8952 0 &
node unit.mjs fee 8953 0 &
node loan.mjs synchronous 8951 8952 8953 &
sleep 1
node measure.mjs synchronous
pkill -f "node unit.mjs" ; pkill -f "node loan.mjs" ; sleep 0.3
rm -f effect-notification.log effect-fee.log

node unit.mjs notification 8952 1 &
node unit.mjs fee 8953 1 &
node loan.mjs event 8951 8952 8953 &
sleep 1
node measure.mjs event
pkill -f "node unit.mjs" ; pkill -f "node loan.mjs" ; sleep 0.3
echo
node measure.mjs contract
```

```
mode             response rounds  units on response path  successors publisher knows  effect at response  600 ms later  effect lines
synchronous                    2                       3                           2                 yes           yes           8/8
event                          1                       1                           0                  no           yes           8/8

event field     publisher  reading consumers  shared code  bound deployment units
type                    1                  0            0                       1
sequence                1                  0            1                       3
memberId                1                  2            0                       3
copyId                  1                  1            0                       2
overdueDays             1                  1            0                       2
title                   1                  1            0                       2
6-field event body, 2 consumer units; average bound units per field 2.17
field no consumer reads: type
```

#### Work That Drops Off the Response Path

The first two columns give the gain. In the synchronous implementation, the response takes two
rounds: writing the loan record and the successors' work. In the event-based implementation, the
same request returns in one round, because all that is left on the response path is appending one
line to the log. The last column shows that the work is not lost: in both implementations, four
requests produce eight effect lines.

The third column gives the real size of the gain. The number of deployment units on the response
path drops from three to one. Read against the previous lesson's measure, the meaning is clear:
the notification and billing units have left the loan entry point's dependency closure. In the
synchronous implementation, either of these two units going down drops the loan request; in the
event-based implementation, that outage never shows up on the response path at all.

#### The Contract Spreads

The fourth column gives the first half of the cost. In the synchronous implementation, the
publisher carries two successor addresses: it knows whom it calls, and that is readable at the
call site. In event mode this number is zero. The publisher no longer knows its consumers; this
is exactly the loose coupling that was wanted, and exactly the source of the contract burden.

The second table turns this burden into a number. The event body has six fields, and each field
binds an average of 2.17 deployment units. Both consumers read the member identifier: when the
field's name or meaning changes, three units at once must be touched. The sequence number does
not appear in any consumer's contract line, but it is read in the shared handling code, so it,
too, binds three units.

The last row is the characteristic problem of asynchronous integration: no consumer reads the
`type` field. The publisher has to defend every field it puts in the body forever, because it
cannot see which fields are actually read. In a synchronous call, this information was readable
at the call site; on the other side of a log, it is not. Contract evolution and consumer
protection are addressed in this course's final topic.

#### The Consistency Window

The fifth and sixth columns show the new failure mode. In the synchronous implementation, the
effect is already in place the moment the response returns: the caller returns knowing the fee
row has been written. In the event-based implementation, there is no effect the moment the
response returns, and there is one 600 milliseconds later. The gap between the two is the
**consistency window**, and in this setup its upper bound is known: the polling interval plus the
consumer's work duration, that is, 225 milliseconds. The measurement confirms this bound from
both ends — no effect at the start of the window, an effect at the end.

Inside the window, the system is inconsistent, and the inconsistency is observable: a request
reading the member's account during that interval sees the loan record but not the fee row. This
is a state that does not exist in the synchronous implementation, and it cannot be removed — only
narrowed and accommodated in the interface. The second new failure mode comes from the same
place: while a consumer is down, events pile up in the log, response time is never disturbed, and
the failure never shows up on the response path. The problem is noticed only if someone is
watching the accumulating events.

The triple resolves like this. **What got cheaper:** the response round dropped from 2 to 1, and
the units on the response path dropped from 3 to 1; two consumer units left the dependency
closure. **What got more expensive:** the contract spread across three deployment units, an
average of 2.17 units are bound per field, and because the publisher's known successor count fell
to zero, which field is actually read is no longer visible from the code. **The failure mode that
was born:** the consistency window, and a consumer failure that is invisible on the response path.

#### Summary

- When the same workflow is built as event-based, the response round drops from 2 to 1 and the
  deployment units on the response path drop from 3 to 1; four requests produce 8 effect lines in
  both implementations.
- The publisher's known successor count drops from 2 to 0; this number is both the source of
  loose coupling and the source of the contract burden.
- In the six-field event body, an average of 2.17 deployment units are bound per field; the
  member identifier binds three units at once, and no consumer reads the `type` field.
- The consistency window was confirmed by measurement from both ends: no effect at response time,
  an effect 600 ms later; the upper bound is the polling interval plus the consumer's work.
- Asynchronous integration removes consumer failure from the response path; the failure no longer
  shows up in response time — it shows up in accumulating events.

#### Next Step

Over two lessons, both sides of the boundary have been worked: where the boundary should pass,
how to call across it, and turning some calls into events instead of making them at all. One side
has never been addressed in any of these arrangements — the outside. The client still knows every
service's address individually, each service carries its own identity check in its own code, and
the same rate-limit rule sits written in more than one place. The next lesson counts this
repetition: how many lines in how many services one edge responsibility is repeated as, how many
files and endpoints change when these responsibilities move to a single place, and how many hops
are added per request.

### API Gateway

Source: https://academia.sh/en/courses/service-architectures/api-gateway

Measuring the collection of edge responsibilities on the application side: how many lines in how many services identity and rate-limit code is repeated as, the files, externally exposed endpoints, and client calls that change when the responsibility moves to the gateway, merging's effect on rounds, and the hop added per request.

The previous two lessons worked both sides of the boundary: where the boundary should pass, how
to call across it, which calls should never be made at all and turn into an event instead. One
side has never been addressed — the outside. The client knows every service's address
individually, each service carries its own identity check in its own code, and the same
rate-limit rule sits written in more than one place.

The API gateway, backend for frontend, and gatekeeper patterns were built and measured on the
traffic side in The Traffic Layer course; that measurement is not repeated here. The question
here is on the application side: how many lines in how many services does the same edge
responsibility sit as, how many files and endpoints change when it moves to a single place, how
many hops are added per request.

The through-line is the loan page: rendering one page needs data from all three of the catalog,
membership, and loan services.

#### The Same Responsibility, Two Layouts

The block below writes the same three services in two forms. The lines carrying edge
responsibility are marked in the source; the lean form is produced by stripping those lines from
the same files, and the marked lines are written once, in a single gateway process. Both forms
are real, running source.

```js
// setup.mjs — writes the same endpoint service in two forms: edge responsibility in the service and in the gateway
import { mkdirSync, writeFileSync } from "node:fs";
const SERVICE = { catalog: 8961, membership: 8962, loan: 8963 }, GATEWAY = 8960;

const service = (name, port) => [
  `// service-${name}.mjs — service producing one part of the loan page`,
  'import { createServer } from "node:http";',
  `const PORT = ${port}, WORK = 25;`,
  "let processed = 0, reset = () => {};",
  "const COUNTER = new Map(), LIMIT = 3; reset = () => COUNTER.clear();        // edge: rate limit",
  "createServer(async (request, response) => {",
  '  if (request.url === "/count") { response.end(String(processed)); return; }',
  '  if (request.url === "/reset") { processed = 0; reset(); response.end("r"); return; }',
  '  const member = request.headers["x-member"];                              // edge: identity',
  '  if (!member) { response.statusCode = 401; response.end("no identity"); return; } // edge: identity',
  "  const n = (COUNTER.get(member) ?? 0) + 1; COUNTER.set(member, n);        // edge: rate limit",
  '  if (n > LIMIT) { response.statusCode = 429; response.end("rate limited"); return; } // edge: rate limit',
  "  processed += 1;",
  "  await new Promise((c) => setTimeout(c, WORK));",
  `  response.end(JSON.stringify({ name: "${name}" }));`,
  "}).listen(PORT);",
];
const gateway = [
  "// gateway.mjs — process collecting the edge responsibilities and merging the responses",
  'import { createServer } from "node:http";',
  `const PORT = ${GATEWAY}, SUCCESSOR = [${Object.values(SERVICE).join(", ")}];`,
  "let reset = () => {};",
  "const COUNTER = new Map(), LIMIT = 3; reset = () => COUNTER.clear();        // edge: rate limit",
  "createServer(async (request, response) => {",
  '  if (request.url === "/reset") { reset(); response.end("r"); return; }',
  '  const member = request.headers["x-member"];                              // edge: identity',
  '  if (!member) { response.statusCode = 401; response.end("no identity"); return; } // edge: identity',
  "  const n = (COUNTER.get(member) ?? 0) + 1; COUNTER.set(member, n);        // edge: rate limit",
  '  if (n > LIMIT) { response.statusCode = 429; response.end("rate limited"); return; } // edge: rate limit',
  "  const parts = await Promise.all(SUCCESSOR.map((p) =>",
  '    fetch(`http://127.0.0.1:${p}/data`).then((y) => y.json())));           // edge: merge',
  "  response.end(JSON.stringify({ merged: parts }));",
  "}).listen(PORT);",
];
const edgeLines = (lines) => lines.filter((s) => s.includes("// edge:")).length;
const write = (dir, file, lines) => {
  mkdirSync(dir, { recursive: true });
  writeFileSync(`${dir}/${file}`, `${lines.join("\n")}\n`);
  return lines.length;
};

let withEdgeLines = 0, withEdgeMarks = 0, leanLines = 0, leanMarks = 0;
for (const [name, port] of Object.entries(SERVICE)) {
  const full = service(name, port), plain = full.filter((s) => !s.includes("// edge:"));
  withEdgeLines += write("with-edge", `service-${name}.mjs`, full);
  withEdgeMarks += edgeLines(full);
  leanLines += write("lean", `service-${name}.mjs`, plain);
  leanMarks += edgeLines(plain);
}
const gatewayLines = write("lean", "gateway.mjs", gateway);
const n = Object.keys(SERVICE).length;
console.log(`${"layout".padEnd(22)}${"files".padStart(7)}${"lines".padStart(7)}` +
  `${"edge lines".padStart(14)}${"touched when rule changes".padStart(28)}${"externally exposed endpoints".padStart(31)}`);
console.log(`${"edge in services".padEnd(22)}${String(n).padStart(7)}${String(withEdgeLines).padStart(7)}` +
  `${String(withEdgeMarks).padStart(14)}${String(n).padStart(28)}${String(n).padStart(31)}`);
console.log(`${"edge in gateway".padEnd(22)}${String(n + 1).padStart(7)}` +
  `${String(leanLines + gatewayLines).padStart(7)}${String(leanMarks + edgeLines(gateway)).padStart(14)}` +
  `${"1".padStart(28)}${"1".padStart(31)}`);
console.log(`edge responsibility had been repeated as ${withEdgeMarks / n} lines across ${n} services; ` +
  `in the gateway it is a single copy in ${edgeLines(gateway)} lines`);
```

```
layout                  files  lines    edge lines   touched when rule changes   externally exposed endpoints
edge in services            3     48            15                           3                              3
edge in gateway             4     48             6                           1                              1
edge responsibility had been repeated as 5 lines across 3 services; in the gateway it is a single copy in 6 lines
```

The total line count is the same in both forms; the moved code did not disappear, it changed
location. Three numbers change: the repeated edge lines drop from 15 to 6, the number of files
touched when the identity or rate-limit rule changes drops from 3 to 1, the number of externally
exposed endpoints drops from 3 to 1. The gateway's sixth marked line is a responsibility that does
not exist in the services at all: merging.

#### The Difference in the Run

**SB4 — the rate limit is three requests per member.** Rationale: the value is not a real
threshold — it is a measure-sized quantity chosen so that the limit becomes visible within twelve
requests. What is measured is not the limit's value but how many places it is enforced in, and how
many requests the same rule grants outward.

```js
// measure.mjs — <measure: round|limit> <mode: direct|gateway>; runs while the processes are up
const WORK = 25, SERVICE = [8961, 8962, 8963], GATEWAY = 8960;
const HEADER = { headers: { "x-member": "7" } };
let calls = 0, serviceCalls = 0;
const endpoints = new Set();
const request = async (p, path) => {
  calls += 1; endpoints.add(p);
  if (SERVICE.includes(p)) serviceCalls += 1;
  const y = await fetch(`http://127.0.0.1:${p}${path}`, HEADER);
  await y.text();
  return y.status;
};
const noIdentity = async (p, path) => {         // request without the x-member header
  const y = await fetch(`http://127.0.0.1:${p}${path}`);
  await y.text();
  return y.status;
};
const reset = async () => {
  for (const p of [...SERVICE, GATEWAY]) await fetch(`http://127.0.0.1:${p}/reset`).catch(() => {});
};
const reached = async () => {
  let t = 0;
  for (const p of SERVICE) t += Number(await (await fetch(`http://127.0.0.1:${p}/count`)).text());
  return t;
};
const [measure, mode] = process.argv.slice(2);
const PATH = {
  "direct-sequential": async () => { for (const p of SERVICE) await request(p, "/data"); },
  "direct-parallel": () => Promise.all(SERVICE.map((p) => request(p, "/data"))),
  gateway: () => request(GATEWAY, "/page"),
};

if (measure === "round") {
  console.log(`${"client path".padEnd(18)}${"client calls".padStart(16)}${"network hops".padStart(13)}` +
    `${"touched endpoints".padStart(19)}${"reached services".padStart(18)}${"measured rounds".padStart(17)}`);
  for (const name of Object.keys(PATH).filter((a) => a.startsWith(mode))) {
    const durations = [];
    for (let i = 0; i < 3; i += 1) {
      await reset();
      const t = performance.now();
      await PATH[name]();
      durations.push(performance.now() - t);
    }
    await reset();
    calls = 0; serviceCalls = 0; endpoints.clear();
    await PATH[name]();
    const u = await reached();
    console.log(`${name.padEnd(18)}${String(calls).padStart(16)}` +
      `${String(calls + u - serviceCalls).padStart(13)}${String(endpoints.size).padStart(19)}` +
      `${String(u).padStart(18)}${String(Math.floor(durations.sort((a, b) => a - b)[1] / WORK)).padStart(17)}`);
  }
} else if (measure === "limit") {
  await reset();
  let accepted = 0, rejected = 0;
  for (let i = 0; i < 12; i += 1) {
    const status = mode === "gateway" ? await request(GATEWAY, "/page") : await request(SERVICE[i % SERVICE.length], "/data");
    if (status === 200) accepted += 1; else rejected += 1;
  }
  console.log(`rate limit (3 per member): 12 external requests -> ${accepted} accepted, ${rejected} rejected, ` +
    `reached services ${await reached()}`);
  await reset();
  const status = mode === "gateway" ? [await noIdentity(GATEWAY, "/page")]
    : await Promise.all(SERVICE.map((p) => noIdentity(p, "/data")));
  console.log(`request without identity: ${status.length} endpoints checked -> ` +
    `${status.filter((d) => d === 401).length} times 401, reached services ${await reached()}`);
}
```

```bash
echo "-- edge responsibility in each service --"
for a in catalog membership loan; do node with-edge/service-$a.mjs & done
sleep 1
node measure.mjs round direct
node measure.mjs limit direct
pkill -f "with-edge/service" ; sleep 0.3

echo
echo "-- edge responsibility in the gateway --"
for a in catalog membership loan; do node lean/service-$a.mjs & done
node lean/gateway.mjs &
sleep 1
node measure.mjs round gateway
node measure.mjs limit gateway
pkill -f "lean/service" ; pkill -f "lean/gateway"
```

```
-- edge responsibility in each service --
client path           client calls network hops  touched endpoints  reached services  measured rounds
direct-sequential                3            3                  3                 3                3
direct-parallel                  3            3                  3                 3                1
rate limit (3 per member): 12 external requests -> 9 accepted, 3 rejected, reached services 9
request without identity: 3 endpoints checked -> 3 times 401, reached services 0

-- edge responsibility in the gateway --
client path           client calls network hops  touched endpoints  reached services  measured rounds
gateway                          1            4                  1                 3                1
rate limit (3 per member): 12 external requests -> 3 accepted, 9 rejected, reached services 9
request without identity: 1 endpoints checked -> 1 times 401, reached services 0
```

#### Merging and Rounds

The first two rows are the case where the client assembles the loan page itself. The client makes
three calls, knows three separate endpoints, and reaches three services. If it makes the calls in
sequence, the page renders in three rounds; if in parallel, in one round. The difference is in the
client's hands and cannot be controlled from the server side: the same page's latency depends on
how the client is written.

The third row renders the same page through the gateway. The client makes one call, knows one
endpoint, three requests still reach the services, and the page returns in one round. When
merging moves to the server side, the worst case is leveled to the best case: the client does not
need to know to make the parallel call, the gateway already does. **What is gained is not average
latency — it is control over latency.**

The fourth column gives the cost: network hops per request go from three to four. In the direct
layout, the client's three calls went directly to the services; in the gateway layout there is one
outer hop and three inner hops. Because the gateway does no work of its own, this hop did not add
a round, but it did take a share of the latency budget.

#### The Same Rule, Two Different Allowances

The rate-limit rows produce the lesson's sharpest number. The rule is byte-for-byte identical in
both layouts: three requests per member. When twelve requests are sent, the direct layout accepts
nine of them, the gateway layout accepts three.

The reason is the limit's **unit**. A limit that counts per service grants a member nine requests,
not three, in a three-service system; as long as the member spreads their requests across the
services, they never hit a limit. A limit that counts at the gateway counts by page request. The
last column levels the service load of the two layouts: in both cases, nine requests reach the
services. So the same written rule, under the same service load, grants three times as much
allowance outward. **An edge rule cannot be read without knowing where it is enforced.**

The identity row shows the same thing on the verification side. A request without identity is
rejected three times, at three separate endpoints, in the direct layout, and once in the gateway
layout. Three rejections are three separate copies of the same rule, and it is only a matter of
time before the copies drift apart from each other.

#### The Triple

**What it made cheaper.** Repeated edge lines dropped from 15 to 6, files touched when the rule
changes dropped from 3 to 1, externally exposed endpoints dropped from 3 to 1, client calls
dropped from 3 to 1; for a client running sequentially, the page dropped from 3 rounds to 1.

**What it made more expensive.** Hops per request rose from 3 to 4, and a new deployment unit and
a new process were added to the system. The gateway carries the three services' endpoints in its
own source: when a service's endpoint path or response shape changes, the number of files touched
rises back to two. The single file gained on edge responsibility is given back on the contract
side.

**What new failure mode it created.** All outside traffic now passes through a single process. In
the previous lesson's measure, the gateway's dependency closure is every entry point: when it goes
down, the affected request share is one. In the direct layout, when catalog went down the page
rendered incomplete; in the gateway layout, when the gateway goes down the page does not render at
all. The second failure mode is quieter: because the rate limit and identity check no longer live
in the services, a call that bypasses the gateway passes through no check at all. Lean services do
not defend themselves; the defense depends on a single place being configured correctly.

#### Summary

- Edge responsibility had been repeated as 5 lines each across three services (15 total); in the
  gateway it became a single copy in 6 lines, and the total line count did not change.
- Files touched when the identity or rate-limit rule changes dropped from 3 to 1, externally
  exposed endpoints from 3 to 1, client calls from 3 to 1.
- When merging moved to the server, the sequential client's 3 rounds dropped to 1; in exchange,
  network hops per request rose from 3 to 4 and a new deployment unit was added.
- The same "3 requests per member" rule accepts 9 of 12 requests when counted per service and 3
  when counted at the gateway; requests reaching the services are 9 in both cases.
- The gateway is the dependency closure of all outside traffic, and lean services do not defend
  themselves; verification is tied to the correctness of a single place.

#### Next Step

The gateway assembled a single page, and that page was assumed to have a single recipient. The
library's loan page, though, is rendered in more than one place: the touchscreen terminal by the
shelf wants only copy status, a handheld client wants a small summary, and the staff screen wants
the member's entire history. When all three pass through the same gateway, its merging code has to
carry three separate shapes at once, and every client change touches a shared file. The next
lesson counts this cost, and measures, on the code side, what building a separate aggregation
layer per client — instead of one general gateway — makes cheaper and what it makes more
expensive.

### Backend for Frontend

Source: https://academia.sh/en/courses/service-architectures/backend-for-frontend

Measuring a client-specific aggregation layer: the excess data and request count carried by two clients fed from one general endpoint, the effect of separate aggregation layers on those same two numbers, and the repeated logic lines paid for it along with the silent-divergence risk.

The previous lesson collected edge responsibilities into a single place: verification, rate
limiting, and routing no longer live inside every service — they sit in the one layer in front.
That layer's silent assumption is this: whatever the services behind the boundary return goes
straight to the client. But clients connecting to the same endpoint do not want the same
response. The library loan system's web list shows a member's entire loan history along with the
book title, author, and shelf code; the mobile summary screen shows only two numbers and a title.
Both are fed from the same endpoint.

This lesson's job is to measure that situation and compare it against a **backend for frontend**
layout. The pattern's definition was built in The Traffic Layer course and measured there on the
traffic side; what is measured here is the application side: the bytes the client carries, the
requests it makes, and the repetition the aggregation layer costs in code.

The setup's assumptions: **SB10** there are two clients (the web list, the mobile summary) and no
third client. **SB11** catalog, membership, and loan are three separate processes; a catalog
record carries ten fields. **SB12** the measurement does not measure duration; it counts the
request count, the bytes carried, and the field count on the client's network.

#### Three Downstream Services and One General Endpoint

Three services run as three separate processes from a single file. Each keeps its own request and
byte counter; these counters will be used to measure the load the downstream side sees.

```js
// bff/service.mjs — three downstream services, one file: node bff/service.mjs catalog 8741
import { createServer } from "node:http";

const [name, port] = [process.argv[2], Number(process.argv[3])];
const BOOK = {
  "978-0": { isbn: "978-0", title: "In Search of Lost Time", author: "M. Proust", translator: "C. Scott Moncrieff",
    publisher: "Chatto and Windus", yearPublished: 1913, pages: 542, genre: "novel", language: "en", shelf: "PQ-2631" },
  "978-1": { isbn: "978-1", title: "Silent House", author: "O. Pamuk", translator: "R. Finn",
    publisher: "Knopf", yearPublished: 1983, pages: 320, genre: "novel", language: "en", shelf: "PL-248" },
  "978-2": { isbn: "978-2", title: "Introduction to Algorithms", author: "T. Cormen", translator: null,
    publisher: "MIT", yearPublished: 2009, pages: 1312, genre: "textbook", language: "en", shelf: "QA-76" },
  "978-3": { isbn: "978-3", title: "Steppenwolf", author: "H. Hesse", translator: "B. Creighton",
    publisher: "Picador", yearPublished: 1927, pages: 224, genre: "novel", language: "en", shelf: "PT-2617" },
};
const MEMBER = { memberNo: "U-41", firstName: "Elena", lastName: "Cross", email: "e.cross@example.test",
  phone: "+90-555-0000", address: "Kordon Cad. 12, Izmir", memberSince: "2019-04-02",
  memberType: "researcher", penaltyBalance: 0, cardStatus: "active" };
const LOAN = [
  { isbn: "978-0", borrowed: "2026-02-10", dueDate: "2026-03-10", extensions: 1 },
  { isbn: "978-1", borrowed: "2026-02-24", dueDate: "2026-03-24", extensions: 0 },
  { isbn: "978-2", borrowed: "2026-02-14", dueDate: "2026-03-14", extensions: 2 },
  { isbn: "978-3", borrowed: "2026-03-02", dueDate: "2026-04-01", extensions: 0 },
];

let requests = 0, bytes = 0;                        // requests and body bytes sent, per service
const SERVE = { catalog: (u) => BOOK[u.split("/")[2]], membership: () => MEMBER, loan: () => LOAN };

createServer((req, res) => {
  if (req.url === "/count") {
    res.writeHead(200, { "Content-Type": "application/json" });
    return res.end(JSON.stringify({ name, requests, bytes }));
  }
  const body = JSON.stringify(SERVE[name](req.url));
  requests += 1;
  bytes += Buffer.byteLength(body);
  res.writeHead(200, { "Content-Type": "application/json" });
  res.end(body);
}).listen(port, "127.0.0.1", () => console.log(`${name} 127.0.0.1:${port}`));
```

The general endpoint merges the membership and loan records and returns them as they are. It does
not add catalog data, because the catalog sits behind another boundary and this endpoint was
written before that one. The result is this: any client that wants to show a book title has to go
to the catalog service itself after receiving the list.

```js
// bff/general-endpoint.mjs — single general endpoint 8744: both clients call here, get the same body
import { createServer } from "node:http";

const GET = async (u) => (await fetch(u)).json();

createServer(async (req, res) => {
  const memberNo = req.url.split("/")[2];
  const member = await GET("http://127.0.0.1:8742/member/" + memberNo);
  const loan = await GET(`http://127.0.0.1:8743/member/${memberNo}/loans`);
  const body = JSON.stringify({ member, loan });
  res.writeHead(200, { "Content-Type": "application/json" });
  res.end(body);
}).listen(8744, "127.0.0.1", () => console.log("general-endpoint 127.0.0.1:8744"));
```

#### Two Clients, Two Aggregation Layers

In the second implementation, every client has its own aggregation layer. The web layer assembles
the list and computes the overdue days; the mobile layer extracts two numbers and a title from
the same data. Both are separate deployment units, and neither uses the other's code.

```js
// bff/bff-web.mjs — aggregation layer for the web list, 8745
import { createServer } from "node:http";

const TODAY = "2026-03-16";
const CATALOG = "http://127.0.0.1:8741/book/";
const MEMBERSHIP = "http://127.0.0.1:8742/member/";
const LOAN = "http://127.0.0.1:8743/member/";
const GET = async (u) => (await fetch(u)).json();
const dayDiff = (a, b) => Math.round((Date.parse(a) - Date.parse(b)) / 86400000);

createServer(async (req, res) => {
  const memberNo = req.url.split("/")[2];
  const member = await GET(MEMBERSHIP + memberNo);
  const loans = await GET(`${LOAN}${memberNo}/loans`);
  const books = await Promise.all(loans.map((o) => GET(CATALOG + o.isbn)));
  const rows = loans.map((o, i) => ({
    title: books[i].title, author: books[i].author, shelf: books[i].shelf,
    due: o.dueDate, overdueDays: Math.max(0, dayDiff(TODAY, o.dueDate)),
  }));
  const body = JSON.stringify({ reader: `${member.firstName} ${member.lastName}`, rows });
  res.writeHead(200, { "Content-Type": "application/json" });
  res.end(body);
}).listen(8745, "127.0.0.1", () => console.log("bff-web 127.0.0.1:8745"));
```

```js
// bff/bff-mobile.mjs — aggregation layer for the mobile summary, 8746
import { createServer } from "node:http";

const TODAY = "2026-03-16";
const CATALOG = "http://127.0.0.1:8741/book/";
const MEMBERSHIP = "http://127.0.0.1:8742/member/";
const LOAN = "http://127.0.0.1:8743/member/";
const GET = async (u) => (await fetch(u)).json();
const dayDiff = (a, b) => Math.round((Date.parse(a) - Date.parse(b)) / 86400000);

createServer(async (req, res) => {
  const memberNo = req.url.split("/")[2];
  const member = await GET(MEMBERSHIP + memberNo);
  const loans = await GET(`${LOAN}${memberNo}/loans`);
  const overdue = loans.filter((o) => dayDiff(TODAY, o.dueDate) > 0).length;
  const nearest = loans.slice().sort((a, b) => Date.parse(a.dueDate) - Date.parse(b.dueDate))[0];
  const book = await GET(CATALOG + nearest.isbn);
  const body = JSON.stringify({ reader: member.firstName, overdue,
    nearest: { title: book.title, daysLeft: dayDiff(nearest.dueDate, TODAY) } });
  res.writeHead(200, { "Content-Type": "application/json" });
  res.end(body);
}).listen(8746, "127.0.0.1", () => console.log("bff-mobile 127.0.0.1:8746"));
```

#### Measurement

The measurement script runs four cases: two clients, two implementations. In each case it
collects the number of requests the client makes, the bytes it receives, and the number of leaf
fields in the body it receives; it prunes the body down to the names the client actually reads and
works out the unused bytes. The last column is the leak measure: how many of the names the client
reads are the downstream services' own field names.

```js
// bff/measure.mjs — two clients x two implementations: requests, bytes, excess data, repeated lines
import { readFileSync } from "node:fs";

const get = async (u) => {
  const m = await (await fetch(u)).text();
  return { bytes: Buffer.byteLength(m), data: JSON.parse(m) };
};
const COUNTER = [8741, 8742, 8743].map((p) => `http://127.0.0.1:${p}/count`);
const total = async () => (await Promise.all(COUNTER.map(get)))
  .reduce((s, x) => s + x.data.requests, 0);

// Field names the clients actually read to put on screen
const FIELD = {
  "web/general endpoint": ["firstName", "lastName", "isbn", "dueDate", "author", "shelf"],
  "mobile/general endpoint": ["firstName", "isbn", "dueDate"],
  "web/bff": ["reader", "rows"],
  "mobile/bff": ["reader", "overdue", "nearest"],
};
const SCENARIO = {
  "web/general endpoint": async () => {                    // general endpoint + one catalog call per book
    const g = await get("http://127.0.0.1:8744/member/U-41/summary");
    const books = await Promise.all(g.data.loan.map((o) => get(`http://127.0.0.1:8741/book/${o.isbn}`)));
    return { requests: 1 + books.length, bytes: g.bytes + books.reduce((s, x) => s + x.bytes, 0),
      body: { ...g.data, books: books.map((x) => x.data) } };
  },
  "mobile/general endpoint": async () => {                  // same general endpoint + only the nearest book
    const g = await get("http://127.0.0.1:8744/member/U-41/summary");
    const y = g.data.loan.slice().sort((a, b) => Date.parse(a.dueDate) - Date.parse(b.dueDate))[0];
    const books = await get(`http://127.0.0.1:8741/book/${y.isbn}`);
    return { requests: 2, bytes: g.bytes + books.bytes, body: { ...g.data, books: [books.data] } };
  },
  "web/bff": async () => {
    const g = await get("http://127.0.0.1:8745/member/U-41/list");
    return { requests: 1, bytes: g.bytes, body: g.data };
  },
  "mobile/bff": async () => {
    const g = await get("http://127.0.0.1:8746/member/U-41/summary");
    return { requests: 1, bytes: g.bytes, body: g.data };
  },
};

const leaf = (v) => Array.isArray(v) ? v.reduce((s, e) => s + leaf(e), 0)
  : v && typeof v === "object" ? Object.values(v).reduce((s, e) => s + leaf(e), 0) : 1;
const prune = (v, k) => {                           // keep only the names the client reads
  if (Array.isArray(v)) return v.map((e) => prune(e, k));
  if (v && typeof v === "object") {
    const o = {};
    for (const [a, d] of Object.entries(v)) {
      if (k.has(a)) o[a] = d;
      else { const sub = prune(d, k); if (sub && Object.keys(sub).length) o[a] = sub; }
    }
    return o;
  }
  return null;
};
const meaningful = (file) => readFileSync(file, "utf8").split("\n")
  .map((s) => s.trim()).filter((s) => s && !s.startsWith("//") && /[A-Za-z]/.test(s));

// Field names the downstream services produce: the basis of the leak measure
const raw = await Promise.all(["8742/member/U-41", "8743/member/U-41/loans", "8741/book/978-0"]
  .map((y) => get(`http://127.0.0.1:${y}`)));
const DOWNSTREAM = new Set(raw.flatMap((h) => Object.keys(Array.isArray(h.data) ? h.data[0] : h.data)));

const rows = [];
for (const [name, run] of Object.entries(SCENARIO)) {
  const before = await total();
  const s = await run();
  const after = await total();
  const fields = new Set(FIELD[name]);
  rows.push({ name, body: s.body, requests: s.requests, internal: after - before, bytes: s.bytes,
    incoming: leaf(s.body), used: leaf(prune(s.body, fields)),
    excess: s.bytes - Buffer.byteLength(JSON.stringify(prune(s.body, fields))),
    leaked: FIELD[name].filter((a) => DOWNSTREAM.has(a)).length });
}

const B = ["client requests", "downstream requests", "received B", "incoming leaves", "used", "excess B", "leaked names"];
console.log(`${"client/implementation".padEnd(26)}${B.map((b) => b.padStart(22)).join("")}`);
for (const s of rows) {
  console.log(`${s.name.padEnd(26)}${[s.requests, s.internal, s.bytes, s.incoming, s.used, s.excess, s.leaked]
    .map((v) => String(v).padStart(22)).join("")}`);
}

console.log(`\nmobile/bff body: ${JSON.stringify(rows.at(-1).body)}`);
const w = meaningful("bff/bff-web.mjs"), m = meaningful("bff/bff-mobile.mjs");
const shared = w.filter((s) => m.includes(s));
console.log(`\nbff-web meaningful lines = ${w.length}, bff-mobile = ${m.length}, ` +
  `identical in both = ${shared.length}`);
console.log(`general endpoint file = ${meaningful("bff/general-endpoint.mjs").length} lines, ` +
  `aggregation layer files = ${w.length + m.length} lines`);
console.log(`downstream service field names = ${DOWNSTREAM.size}; aggregation layer = 2 deployment units`);
```

```bash
for s in "service.mjs catalog 8741" "service.mjs membership 8742" "service.mjs loan 8743" \
         "general-endpoint.mjs" "bff-web.mjs" "bff-mobile.mjs"; do
  node bff/$s > /dev/null &
done
for p in 8741 8742 8743 8744 8745 8746; do
  curl -s --retry 30 --retry-connrefused --retry-delay 0 -o /dev/null http://127.0.0.1:$p/count
done
node bff/measure.mjs
kill $(jobs -p)
```

```
client/implementation            client requests   downstream requests            received B       incoming leaves                  used              excess B          leaked names
web/general endpoint                           5                     6                  1365                    66                    22                   916                     6
mobile/general endpoint                        2                     3                   795                    36                    10                   567                     3
web/bff                                        1                     6                   447                    21                    21                     0                     0
mobile/bff                                     1                     3                    89                     4                     4                     0                     0

mobile/bff body: {"reader":"Elena","overdue":2,"nearest":{"title":"In Search of Lost Time","daysLeft":-6}}

bff-web meaningful lines = 19, bff-mobile = 19, identical in both = 13
general endpoint file = 10 lines, aggregation layer files = 38 lines
downstream service field names = 23; aggregation layer = 2 deployment units
```

#### Reading the Numbers

The first two rows give the cost of the single general endpoint. Building the web list screen
takes five requests and receives 1365 bytes; 916 of those bytes, that is, sixty-seven percent,
never reach the screen at all. The mobile summary screen makes two requests to show two numbers
and a title, receives 795 bytes, and 567 of those go unused. On the field side the ratio is
sharper still: the mobile client receives thirty-six leaf values and reads ten of them.

The bottom two rows show the aggregation layer's effect. Client requests drop from 5 and 2 to 1.
Received bytes drop from 1365 to 447 on the web side, and from 795 to 89 on the mobile side — the
data carried for the mobile screen shrinks to about a ninth. Excess bytes are 0 in both cases,
because the layer builds the body exactly in the shape the screen wants.

The downstream request column, though, does not change: 6 in both implementations on the web
side, 3 on the mobile side. This is the pattern's most commonly misunderstood point. The
aggregation layer does not reduce the load on the downstream services; it does the same work, it
just changes **where** that work happens. Five round trips move from the client's network to the
internal network. The gain is on the client's network, and there a round trip costs far more than
an in-server call.

The leaked-names column measures coupling. In the general-endpoint layout, all six of the names
the web client reads are the downstream services' own field names; the `dueDate` name comes from
the loan service's vocabulary and reaches all the way down into the client's code. In the
aggregation-layer layout, this number is 0: the client sees the screen's own vocabulary — `due`,
`overdueDays`, `nearest`. When the loan service renames its field, the client code in the first
layout has to be fixed and republished; in the second layout, the fix stays inside the layer.

#### Repeated Logic and Silent Divergence

The cost is in the last three rows. The general-endpoint file is ten meaningful lines; the two
aggregation layers are thirty-eight lines together. Thirteen of those are identical, word for
word, in both files: the three service addresses, the `GET` helper, `dayDiff`, the `TODAY`
constant, and the member and loan calls. So setting up one layer per client means copying the
same logic as many times as there are clients. When a third client arrives, thirteen lines will
be written a third time.

The failure mode this repetition creates is not the copy itself — it is only one of the copies
getting updated. The loan service renames its `dueDate` field to `deadline`, and only the web
layer is updated.

```bash
sed 's/dueDate/deadline/g' bff/service.mjs > bff/service-v2.mjs   # loan field renamed
sed 's/dueDate/deadline/g' bff/bff-web.mjs > bff/bff-web-v2.mjs  # only the web layer was updated
node bff/service.mjs catalog 8741 > /dev/null &
node bff/service.mjs membership 8742 > /dev/null &
node bff/service-v2.mjs loan 8743 > /dev/null &
node bff/bff-web-v2.mjs > /dev/null &
node bff/bff-mobile.mjs > /dev/null &
for p in 8741 8742 8743 8745 8746; do
  curl -s --retry 30 --retry-connrefused --retry-delay 0 -o /dev/null http://127.0.0.1:$p/count
done
echo "web    $(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8745/member/U-41/list) $(curl -s http://127.0.0.1:8745/member/U-41/list | head -c 96)"
echo "mobile $(curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:8746/member/U-41/summary) $(curl -s http://127.0.0.1:8746/member/U-41/summary)"
kill $(jobs -p)
```

```
web    200 {"reader":"Elena Cross","rows":[{"title":"In Search of Lost Time","author":"M. Proust","shelf":"
mobile 200 {"reader":"Elena","overdue":0,"nearest":{"title":"In Search of Lost Time","daysLeft":null}}
```

Both layers return 200. The web list is correct; the mobile summary gives `overdue` as 0 instead
of 2, and `daysLeft` as `null` instead of -6. Because the unread field is `undefined`, the date
comparison silently produced a wrong result, and no error was logged anywhere. In the
single-general-endpoint layout, the same change would have broken both clients at once, and that
is a failure that is easier to notice. In the aggregation-layer layout, the failure shows up as
**divergence between clients**: one screen is correct, the other is silently wrong.

The rule that follows concerns the pattern's ownership. The aggregation layer is a deployment unit
belonging to the team that writes the screen, not to the downstream service; when the downstream
service's contract changes, a fix is needed once per layer, and the only thing that shows all
those fixes were made is testing the layers against the contract.

#### Summary

- In the single-general-endpoint layout, the web list carries 5 requests and 1365 bytes, the
  mobile summary 2 requests and 795 bytes; 916 and 567 of those bytes never reach the screen.
- The client-specific aggregation layer brings the request count down to 1, received bytes down
  to 447 and 89; excess bytes become 0 for both clients.
- Requests reaching the downstream services do not change (6 and 3): the layer does not reduce
  the load, it moves the round trips from the client's network to the internal network.
- Coupling measure: at the general endpoint, all 6 of the names the client reads are downstream
  field names; at the aggregation layer, 0.
- Code paid: the two layers are 38 meaningful lines, 13 of them identical; plus 2 new deployment
  units.
- Failure mode born: when a downstream field is renamed and only one layer is updated, the two
  clients silently diverge — web returns 200 and correct, mobile returns 200 and `overdue: 0`.

#### Next Step

The aggregation layer, and the gateway in front of it, are the same kind of solution: taking a
responsibility out of the service code and putting it in a separate place. What both of them took
on had to do with **shape** — which field goes to whom, how the body is built. But another pile
of things that has nothing to do with shape still sits inside the service code: the retry loop,
the timeout duration, the transport's security, and the counters that collect each call's
metrics. In every one of the three layers written through this lesson, the `fetch` call is bare;
there is no timeout, no retry, no counter. The next lesson takes on the layout that pulls this
work out of the service code and moves it into a separate process, and builds the measure on the
code side: how many lines got deleted from the application code, what responsibility is left
behind, and what running in a separate process cost.

### Service Mesh

Source: https://academia.sh/en/courses/service-architectures/service-mesh

Pulling network concerns out of the application code: counting the lines deleted by moving retry, timeout, secure transport, and metric collection into a sidecar, naming the responsibility that stays behind, and measuring the process, hop, and field-knowledge cost of running in a separate process.

Where the previous lesson left off, there were three aggregation layers, and all three had a bare
outbound call: no timeout, no retry, no counter. In a real system, these three jobs would have to
be rewritten in every service's code. In the library loan system, the loan service talks to the
fee service; the trip goes over a network, the transport is secure, and the other side
occasionally returns a temporary error.

This lesson's question is: if retry, timeout, secure transport, and metric collection are pulled
out of the application code and handed to a separate process running alongside the main one, how
many lines get deleted from the application code, what is left behind, and what is the cost. The
sidecar and the ambassador were defined in The Traffic Layer course and measured there at the
deployment-unit level; here the measure is on the code side. When a process like this runs next
to every service, and all of them are given policy from a single place, the layout that results
is called a **service mesh**, and the place the policy is given from is called the **control
plane**.

Assumptions: **SB13** the fee service talks over secure transport, and the certificate is
generated at measurement time. **SB14** the fee service rejects the first two read requests with
a temporary error; the payment endpoint processes the record, but its first response is treated
as lost. **SB15** the measurement does not measure duration; it counts lines, processes, hops,
and collections.

#### The Fee Service

The fee service's work endpoint runs over secure transport. A plain management endpoint sits
alongside it; that endpoint is the measurement rig — it reads and resets the counters.

```js
// mesh/fee.mjs — fee service: work endpoint 8751 over secure transport, management endpoint 8757 plain
import { createServer as tls } from "node:https";
import { createServer } from "node:http";
import { readFileSync } from "node:fs";

const counter = { read: 0, payment: 0, collected: 0 };
const reset = () => Object.keys(counter).forEach((k) => (counter[k] = 0));

tls({ key: readFileSync("mesh/key.pem"), cert: readFileSync("mesh/cert.pem") },
  (q, y) => {
    if (q.url === "/penalty/pay") {
      counter.payment += 1;
      counter.collected += 1;                       // the record is processed; the first response is treated as lost in transit
      const code = counter.payment === 1 ? 503 : 200;
      y.writeHead(code, { "Content-Type": "application/json" });
      return y.end(JSON.stringify({ collected: counter.collected, amount: 12.5 }));
    }
    counter.read += 1;
    const code = counter.read <= 2 ? 503 : 200;  // the first two read requests are temporarily rejected
    y.writeHead(code, { "Content-Type": "application/json" });
    y.end(JSON.stringify(code === 200 ? { balance: 12.5 } : { error: "temporary" }));
  }).listen(8751, "127.0.0.1", () => console.log("fee 127.0.0.1:8751"));

createServer((q, y) => {
  if (q.url === "/reset") reset();
  y.writeHead(200, { "Content-Type": "application/json" });
  y.end(JSON.stringify(counter));
}).listen(8757, "127.0.0.1");
```

#### Network Concerns in the Application Code

In the first implementation, the loan service does everything itself. The lines carrying a
network concern are marked; the measurement will count that mark.

```js
// mesh/loan-thick.mjs — every network concern lives in the application code, 8752
import { createServer } from "node:http";
import { request } from "node:https";                                   // [tls]
import { readFileSync } from "node:fs";                                 // [tls]

const CA = readFileSync("mesh/cert.pem");                                    // [tls]
const TIMEOUT = 400;                                                    // [timeout]
const RETRY = 3;                                                        // [retry]
const metric = {};                                                      // [metric]

const call = (path, method) => new Promise((resolve, reject) => {       // [tls]
  const q = request({ host: "127.0.0.1", port: 8751, path,              // [tls]
    method, ca: CA }, (y) => {                                          // [tls]
    let g = "";
    y.on("data", (p) => (g += p));
    y.on("end", () => resolve({ code: y.statusCode, body: g }));
  });
  q.setTimeout(TIMEOUT, () => q.destroy(new Error("timeout")));         // [timeout]
  q.on("error", reject);                                                // [timeout]
  q.end();
});

const safe = (method) => method === "GET";        // only an idempotent call is retried [retry]
const attempt = async (path, method, action) => {
  const o = (metric[action] ??= { requests: 0, tries: 0, errors: 0 });   // [metric]
  o.requests += 1;                                                      // [metric]
  let last;
  for (let i = 0; i < (safe(method) ? RETRY : 1); i += 1) {             // [retry]
    o.tries += 1;                                                       // [metric]
    last = await call(path, method);
    if (last.code < 500) break;                                         // [retry]
  }
  if (last.code >= 500) o.errors += 1;                                  // [metric]
  return last;
};

const json = (y, code, g) => {
  y.writeHead(code, { "Content-Type": "application/json" });
  y.end(JSON.stringify(g));
};

createServer(async (q, y) => {
  const [root, member] = q.url.split("/").filter(Boolean);
  if (root === "metric") return json(y, 200, metric);                    // [metric]
  const job = root === "pay" ? { path: "/penalty/pay", method: "POST" }
    : { path: `/penalty/${member}`, method: "GET" };
  const s = await attempt(job.path, job.method, root);
  json(y, s.code, { action: root, code: s.code, body: s.body });
}).listen(8752, "127.0.0.1", () => console.log("loan-thick 127.0.0.1:8752"));
```

#### The Same Job, a Thin Application, and a Sidecar

In the second implementation, the business logic stays line-for-line the same; the only thing
that changes is the body of the `attempt` function. The application now makes a plain call to a
local address.

```js
// mesh/loan-thin.mjs — network concerns live outside the application, 8753: plain call to a local address
import { createServer } from "node:http";

const attempt = (path, method) => fetch(`http://127.0.0.1:8754${path}`, { method })
  .then(async (y) => ({ code: y.status, body: await y.text() }));

const json = (y, code, g) => {
  y.writeHead(code, { "Content-Type": "application/json" });
  y.end(JSON.stringify(g));
};

createServer(async (q, y) => {
  const [root, member] = q.url.split("/").filter(Boolean);
  const job = root === "pay" ? { path: "/penalty/pay", method: "POST" }
    : { path: `/penalty/${member}`, method: "GET" };
  const s = await attempt(job.path, job.method, root);
  json(y, s.code, { action: root, code: s.code, body: s.body });
}).listen(8753, "127.0.0.1", () => console.log("loan-thin 127.0.0.1:8753"));
```

Every deleted line moves to the sidecar. What the sidecar sees is a path, a method, and a status
code; it does not know what job is being done.

```js
// mesh/sidecar.mjs — sidecar 8754: timeout, retry, secure transport, metrics
import { createServer } from "node:http";
import { request } from "node:https";
import { readFileSync } from "node:fs";

const CA = readFileSync("mesh/cert.pem");
const TIMEOUT = 400;
const RETRY = 3;
const RETRIED_CODE = [502, 503, 504];      // the policy is a status code; it does not know what the job is
const metric = {};

const call = (path, method) => new Promise((resolve, reject) => {
  const q = request({ host: "127.0.0.1", port: 8751, path, method, ca: CA }, (y) => {
    let g = "";
    y.on("data", (p) => (g += p));
    y.on("end", () => resolve({ code: y.statusCode, body: g }));
  });
  q.setTimeout(TIMEOUT, () => q.destroy(new Error("timeout")));
  q.on("error", reject);
  q.end();
});

createServer(async (q, y) => {
  if (q.url === "/metric" || q.url === "/reset") {
    const g = JSON.stringify(metric);
    if (q.url === "/reset") Object.keys(metric).forEach((a) => delete metric[a]);
    y.writeHead(200, { "Content-Type": "application/json" });
    return y.end(g);
  }
  const o = (metric[q.url] ??= { requests: 0, tries: 0, errors: 0 });
  o.requests += 1;
  let last;
  for (let i = 0; i < RETRY; i += 1) {
    o.tries += 1;
    last = await call(q.url, q.method);
    if (!RETRIED_CODE.includes(last.code)) break;
  }
  if (last.code >= 500) o.errors += 1;
  y.writeHead(last.code, { "Content-Type": "application/json" });
  y.end(last.body);
}).listen(8754, "127.0.0.1", () => console.log("sidecar 127.0.0.1:8754"));
```

#### Measurement

The measurement runs the same two jobs in both layouts: a balance query and a penalty payment. It
then reads the fee service's counters, the metrics on both sides, and the code's lines.

```js
// mesh/measure.mjs — two layouts run the same job: outcome, retries, collected, deleted lines
import { readFileSync } from "node:fs";

const get = async (u) => JSON.parse(await (await fetch(u)).text());
const code = async (u) => (await fetch(u)).status;
const meaningful = (d) => readFileSync(d, "utf8").split("\n").map((s) => s.trim())
  .filter((s) => s && !s.startsWith("//") && /[A-Za-z]/.test(s));
const netConcern = (d) => readFileSync(d, "utf8").split("\n")
  .filter((s) => /\/\/ \[/.test(s)).length;                 // line marked as a network concern

const LAYOUT = [["thick application", 8752, null], ["thin + sidecar", 8753, 8754]];
const outcome = [];
for (const [name, app, tool] of LAYOUT) {
  await get("http://127.0.0.1:8757/reset");            // fee counters
  if (tool) await get(`http://127.0.0.1:${tool}/reset`);
  const balance = await code(`http://127.0.0.1:${app}/balance/U-41`);
  const pay = await code(`http://127.0.0.1:${app}/pay/U-41`);
  const counter = await get("http://127.0.0.1:8757/count");
  const metric = await get(`http://127.0.0.1:${tool ?? app}/metric`);
  outcome.push({ name, balance, pay, counter, metric, processes: tool ? 3 : 2, hops: tool ? 2 : 1 });
}

const B = ["balance code", "payment code", "reached fee", "collected", "processes", "local hops"];
console.log(`${"layout".padEnd(18)}${B.map((b) => b.padStart(14)).join("")}`);
for (const s of outcome) {
  console.log(`${s.name.padEnd(18)}${[s.balance, s.pay, s.counter.read + s.counter.payment,
    s.counter.collected, s.processes, s.hops].map((v) => String(v).padStart(14)).join("")}`);
}
console.log("");
for (const s of outcome) {
  const d = Object.entries(s.metric).map(([a, o]) => `${a} ${o.requests}/${o.tries}/${o.errors}`);
  console.log(`${s.name.padEnd(18)} metric (requests/tries/errors): ${d.join("  ")}`);
}

const SERVICE = 5;                        // catalog, membership, loan, fee, notification
const k = meaningful("mesh/loan-thick.mjs"), i = meaningful("mesh/loan-thin.mjs");
const y = meaningful("mesh/sidecar.mjs");
const deleted = k.filter((s) => !i.includes(s));
console.log(`\nloan-thick = ${k.length} lines, loan-thin = ${i.length} lines, ` +
  `deleted = ${deleted.length}, added = ${i.filter((s) => !k.includes(s)).length}`);
console.log(`lines marked as network concerns: application code ${netConcern("mesh/loan-thick.mjs")} -> 0`);
console.log(`network-concern lines for ${SERVICE} services: application side ${SERVICE * deleted.length}, ` +
  `sidecar side ${y.length} (single file)`);
```

```bash
openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj "/CN=localhost" \
  -addext "subjectAltName=IP:127.0.0.1" -keyout mesh/key.pem -out mesh/cert.pem 2>/dev/null
node mesh/fee.mjs > /dev/null &
node mesh/loan-thick.mjs > /dev/null &
node mesh/loan-thin.mjs > /dev/null &
node mesh/sidecar.mjs > /dev/null &
for p in 8752 8753 8754 8757; do
  curl -s --retry 30 --retry-connrefused --retry-delay 0 -o /dev/null http://127.0.0.1:$p/metric
done
node mesh/measure.mjs
kill $(jobs -p)
```

```
layout              balance code  payment code   reached fee     collected     processes    local hops
thick application            200           503             4             1             2             1
thin + sidecar               200           200             5             2             3             2

thick application  metric (requests/tries/errors): balance 1/3/0  pay 1/1/1
thin + sidecar     metric (requests/tries/errors): /penalty/U-41 1/3/0  /penalty/pay 1/2/0

loan-thick = 38 lines, loan-thin = 13 lines, deleted = 28, added = 3
lines marked as network concerns: application code 18 -> 0
network-concern lines for 5 services: application side 140, sidecar side 33 (single file)
```

#### Deleted Lines and the Responsibility Left Behind

The application code drops from 38 meaningful lines to 13; 28 lines are deleted, and 3 come in
their place. Eighteen of the deleted lines were marked as network concerns, and none of that mark
remains in the application code at all. The measure's real meaning is in the multiplier: because
the same 28 lines would be written in each of five services, the total on the application side is
140 lines; on the sidecar side there is a single 33-line file, and its policy is given from the
outside.

The responsibility left behind has three items, and none of them can be handed off to the
sidecar. First, which path is reached with which method: the knowledge that the `/penalty/pay`
endpoint is a `POST` lives in the application. Second, what the returned status code means in
business terms — what will the application that sees a 503 show the user. Third, and most
important, whether retrying a call is safe. In the thick implementation this knowledge is one
line, in the `safe` function; in the thin implementation, that line was deleted and nothing was
put in its place.

The cost shows up in two columns. The process count goes from 2 to 3, and the local hops a
request touches go from 1 to 2. Requests reaching the fee service go from 4 to 5. This is the
code-side counterpart of The Traffic Layer course's process-per-deployment-unit measure: the same
job, with one extra hop.

The metric rows show the loss. In the thick layout, the counters are kept as `balance` and
`pay` — that is, by the job's name; in the thin layout, as `/penalty/U-41` and `/penalty/pay` —
that is, by the fee service's path vocabulary. The sidecar can only guess the difference between
"a balance query" and "a penalty payment" by looking at the path. Because the member number is
part of the path, every member produces a separate metric key, and aggregating those keys without
domain knowledge also turns into a job outside the application.

#### What the Retry Does Not Know

The most expensive column in the table is the `collected` column. Both layouts made the same two
requests; in the thick layout the fee service processed one collection, in the thin layout it
processed two.

The reason is where the policy sits. The fee service's payment endpoint processes the record and
returns its first response as 503 — imitating the case where the response is lost in transit. In
the thick application, the `safe` function had closed the `POST` call off from retrying: the
payment was attempted once, a 503 returned to the application, and collected became 1. The
sidecar, though, only looks at the status code; it saw 503, retried the same request, got 200 on
the second attempt, and returned 200 to the application. The loan service believes the operation
succeeded; the member's penalty has been collected twice, and there is no error logged on either
side.

This is the failure mode that is born: **a retry decision made at the network layer replays a
call that the business layer had closed off from replay.** This failure mode did not exist in the
thick layout, because the code making the decision had the knowledge of what the call was.

The fix is to call some of the responsibility back. The sidecar's policy can be narrowed by
method, or the application can put an idempotency key on the request and let the fee service drop
a repeated collection. Neither is free: the first puts domain knowledge inside the network
policy, the second adds a bookkeeping job to the fee service. What the measurement shows is this:
network concerns can be pulled out of the application, but the knowledge of **which call can be
safely repeated** cannot be pulled out — it only changes location.

#### Summary

- The same job was run in two layouts: application code dropped from 38 lines to 13, 28 lines
  were deleted and 3 were added; none of the 18 lines marked as network concerns remained in the
  application.
- In a five-service system, the same concern holds 140 lines on the application side; on the
  sidecar side there is a single 33-line file, and its policy is given from the outside.
- Responsibility left in the application: choosing the path and method, the status code's
  business meaning, and whether repeating a call is safe.
- Cost paid: processes went from 2 to 3, local hops from 1 to 2, requests reaching the fee
  service from 4 to 5; metric keys fell from the job's name to the path vocabulary.
- Failure mode born: a status-code-based retry replayed a payment call that was closed off from
  repetition; collections became 2 instead of 1, and no error was logged on either side.

#### Next Step

Both implementations shared a silent assumption: the fee service's address was written into the
code as `127.0.0.1:8751`. In the thick application this address sits in the application file, in
the thin application in the sidecar file; either way, it is a single constant. But a service may
not have a single copy — copies can come and go, and a copy that has left can go on looking valid
for a while. This topic's final lesson takes on resolving the address from outside the code: it
builds a registry process, gives records a lease, ties health information to routing, and
compares this against fixed configuration through two numbers — the file touched in deployment
when a copy is added, and the requests sent to a dead address when a copy dies.

### Service Discovery

Source: https://academia.sh/en/courses/service-architectures/service-discovery

Building address resolution as an implementation: a registry process with leased records, copies registering themselves, and health information tied to routing; compared against fixed configuration through the file touched in deployment and the requests sent to a dead address.

Both of the previous two lessons shared a silent assumption: the other side's address is a fixed
string inside the code. The loan service went to the fee service as `127.0.0.1:8751`; the
aggregation layers before that went to the catalog service as `127.0.0.1:8741`. In a single-copy
setup, this is correct. The library loan system's catalog service, though, is not a single copy:
copies are added as load grows, a copy crashes, a copy goes into maintenance. An address with no
live counterpart behind a fixed string is as certain a failure as the call itself.

This lesson's job is to build address resolution as an **implementation**. What service discovery
and a registry are was defined in The Application Layer and Service Interaction course; here the
registry actually runs as a process, records are given a lease, health information is tied to
routing, and the result is compared against fixed configuration through two numbers: the file
touched in deployment when a copy is added, the requests sent to a dead address when a copy dies.

Assumptions: **SB16** the catalog service has three copies, and all three serve the same data.
**SB17** the registry's lease duration is 900 ms and the copies' renewal interval is 300 ms;
these ratios are smaller than in a real system, to shorten the measurement time. **SB18** there
are three clients calling the catalog service (loan, notification, fee), and in fixed
configuration each carries its own address file.

#### The Registry Process

The registry is a single process holding addresses and record times. A record's validity depends
on time: a record whose last renewal is older than the lease duration drops off the list. The
resolution response returns two numbers — the live record count and the routable addresses. The
reason the two are separate is health information.

```js
// discovery/registry.mjs — registry 8761: leased records and health information
import { createServer } from "node:http";

const LEASE = 900;                     // ms: a record not renewed by this deadline drops off the list
const records = new Map();              // address -> { name, healthy, last renewal }

createServer((q, y) => {
  const [root, value] = q.url.split("/").filter(Boolean);
  if (root === "register") {                                   // /register/<name>|<address>|<healthy>
    const [name, address, healthy] = decodeURIComponent(value).split("|");
    records.set(address, { name, healthy: healthy === "1", last: Date.now() });
  }
  const name = root === "resolve" ? value : "catalog";
  const live = [...records.entries()]
    .filter(([, k]) => k.name === name && Date.now() - k.last < LEASE);
  y.writeHead(200, { "Content-Type": "application/json" });
  y.end(JSON.stringify({ lease: LEASE, records: live.length,
    addresses: live.filter(([, k]) => k.healthy).map(([a]) => a) }));
}).listen(8761, "127.0.0.1", () => console.log("registry 127.0.0.1:8761"));
```

The registry has no delete endpoint. A copy shutting down cleanly could have deleted its own
record, but a copy that crashes cannot; so a record's validity is tied to time, not to the copy's
good intentions. The registry never asks any copy a question itself.

#### The Registering Copy

Every copy registers itself as it comes up, and renews its record at an interval one third of the
lease duration. The renewal call also carries health information. The `/break` endpoint exists to
produce the state where the process is alive but cannot do the work: the process keeps renewing,
but the health information it reports changes.

```js
// discovery/catalog.mjs — catalog copy: node discovery/catalog.mjs 8762
import { createServer } from "node:http";

const port = Number(process.argv[2] ?? 8762);
const RENEWAL = 300;                        // ms: one third of the lease duration
let healthy = true;

const register = () => fetch("http://127.0.0.1:8761/register/" +
  encodeURIComponent(`catalog|127.0.0.1:${port}|${healthy ? "1" : "0"}`)).catch(() => {});

createServer((q, y) => {
  if (q.url === "/break") healthy = false;    // the process is alive and renewing its record, but cannot do the work
  const code = healthy ? 200 : 503;
  y.writeHead(code, { "Content-Type": "application/json" });
  y.end(JSON.stringify({ port, healthy }));
}).listen(port, "127.0.0.1", () => {
  register();
  setInterval(register, RENEWAL).unref();
});
```

The renewal interval must be smaller than the lease duration; the ratio determines how many
renewals a copy can miss before it drops off the list. Here the ratio is three: a copy that
misses two renewals is still on the list, one that also misses the third drops off.

#### Two Resolution Paths, the Same Job

The measurement script starts the registry and the three copies itself, writes the fixed
configuration file itself, and at each of five stages distributes the same six requests through
both paths. On the fixed path, addresses are read from the file; on the discovery path, the
registry is asked at the start of every stage.

```js
// discovery/measure.mjs — fixed configuration is run against the same job as the registry
import { spawn } from "node:child_process";
import { writeFileSync, readFileSync } from "node:fs";

const wait = (ms) => new Promise((c) => setTimeout(c, ms));
const proc = {};
const start = (p) => (proc[p] = spawn("node", ["discovery/catalog.mjs", String(p)], { stdio: "ignore" }));
const REQUESTS = 6;                        // requests per stage; distributed round-robin across addresses
const CLIENTS = 3;                      // services calling the catalog: loan, notification, fee

const registry = spawn("node", ["discovery/registry.mjs"], { stdio: "ignore" });
await wait(400);
[8762, 8763, 8764].forEach(start);
writeFileSync("discovery/addresses.json",     // configuration file hand-written at deployment time
  JSON.stringify([8762, 8763, 8764].map((p) => `127.0.0.1:${p}`)));
await wait(500);

const call = async (address) => {
  try { return (await fetch(`http://${address}/book/978-0`)).status === 200; } catch { return false; }
};
const distribute = async (addresses) => {
  let ok = 0;
  for (let i = 0; i < REQUESTS && addresses.length; i += 1) if (await call(addresses[i % addresses.length])) ok += 1;
  return ok;
};
const fixed = async () => {
  const a = JSON.parse(readFileSync("discovery/addresses.json", "utf8"));
  return { ok: await distribute(a), records: a.length, addresses: a.length };
};
const discovery = async () => {
  const d = await (await fetch("http://127.0.0.1:8761/resolve/catalog")).json();
  return { ok: await distribute(d.addresses), records: d.records, addresses: d.addresses.length };
};

const stage = [];
const measure = async (name) => stage.push({ name, f: await fixed(), d: await discovery() });

await measure("all alive");
proc[8764].kill();
await measure("copy killed (immediately)");
await wait(1200);
await measure("lease expired");
await fetch("http://127.0.0.1:8763/break").catch(() => {});
await wait(400);
await measure("copy reported broken");
start(8765);
await wait(600);
await measure("new copy added");

const B = ["fixed ok/req", "fixed addrs", "registry ok/req", "registry records", "registry addrs"];
console.log(`${"stage".padEnd(26)}${B.map((b) => b.padStart(18)).join("")}`);
for (const a of stage) {
  console.log(`${a.name.padEnd(26)}${[`${a.f.ok}/${REQUESTS}`, a.f.addresses, `${a.d.ok}/${REQUESTS}`,
    a.d.records, a.d.addresses].map((v) => String(v).padStart(18)).join("")}`);
}
console.log(`\nlease duration 900 ms, renewal interval 300 ms; requests sent to a dead address ` +
  `in the discovery window = ${REQUESTS - stage[1].d.ok}/${REQUESTS}`);
console.log(`configuration files touched when a new copy is added: fixed ${CLIENTS}, registry 0; ` +
  `clients restarted: fixed ${CLIENTS}, registry 0`);
console.log(`registry = 1 extra process, 1 extra resolution request before every call`);

Object.values(proc).forEach((s) => s.kill());
registry.kill();
```

```
stage                           fixed ok/req       fixed addrs   registry ok/req  registry records    registry addrs
all alive                                6/6                 3               6/6                 3                 3
copy killed (immediately)                4/6                 3               4/6                 3                 3
lease expired                            4/6                 3               6/6                 2                 2
copy reported broken                     2/6                 3               6/6                 2                 1
new copy added                           2/6                 3               6/6                 3                 2

lease duration 900 ms, renewal interval 300 ms; requests sent to a dead address in the discovery window = 2/6
configuration files touched when a new copy is added: fixed 3, registry 0; clients restarted: fixed 3, registry 0
registry = 1 extra process, 1 extra resolution request before every call
```

#### Reading the Numbers

At the first stage, both paths give the same result: six requests, six successes, three
addresses. Discovery shows no contribution at all, and that is correct — fixed configuration
works fine while everything is fine. Discovery's measure only shows up once the layout changes.

The second stage is measured immediately after a copy is killed. Both paths give 4/6. The
registry is still returning three addresses, because the lease duration has not yet passed since
the dead copy's last renewal. This interval is called the **discovery window**, and its measured
counterpart is 2/6: of every six requests sent in the window, two went to a dead address. The
registry does not reduce the time it takes to learn of a death to zero; it only bounds it.

The third stage shows the window closing. Once the lease duration has passed, the registry
returns two addresses, and the six requests split three each across the two: 6/6. Fixed
configuration, though, is still carrying three addresses and keeps giving 4/6. This difference is
the whole return on discovery: fixed configuration's error persists until a person fixes the
file.

The fourth stage shows why health information is a separate piece of information. Copy `8763`
keeps living and renewing its record after the `/break` call; the registry's live record count
stays at 2, but the routable address count drops to 1. A process being up does not mean it can do
the work. Fixed configuration cannot make this distinction at all and drops to 2/6: two requests
go to a dead address, and two go to a copy returning 503.

The fifth stage measures the deployment side. When a fourth copy is started, no file is touched
on the registry side at all; the copy registers itself, and at the next resolution the routable
address count rises to 2. On the fixed-configuration side, the result does not change: the new
copy serves no requests at all, because the three clients' address file still carries the old
three. The cost of the change is in the last lines of the output: configuration files touched, 3
against 0; clients restarted, 3 against 0.

#### The Registry's Cost and the Failure Modes It Creates

The cost paid has three items. The first is one extra process, and this process is not an
ordinary service: when the registry goes down, no client can resolve an address. Fixed
configuration's one advantage is exactly here — the file is read without depending on anything.
This is why clients that lean on a registry usually hold on to the result of their last
resolution; and then the held list can go stale, and the discovery window grows again.

The second is the resolution request. In the measurement, one resolution was done at the start of
every stage; if resolution had been done before every call, every request to the catalog service
would have turned into two requests. This is discovery's counterpart to the previous lesson's
local-hop measure, and it makes caching the resolution mandatory.

The third is the lease duration itself, and both of its directions create a failure. Lengthen it,
and the discovery window grows — that is, requests to dead addresses increase. Shorten it, and
copies renew more often; the write load on the registry is multiplied by the copy count, and a
short network interruption causes live copies to drop off the list. When a dropped copy comes
back, requests redistribute again; this oscillation is registry implementations' most familiar
failure.

The common source of the failure modes born here is this: the registry does not hold the truth —
it holds **a delayed copy of the truth.** In fixed configuration, the delay is unbounded and
depends on a person fixing the file; in the registry, the delay is bounded by the lease duration.
What discovery sells is not correctness — it is **an upper bound placed on how long the wrongness
lasts.**

#### Summary

- The registry keeps leased records; there is no delete endpoint, because a copy that crashes
  cannot delete its own record. Copies renew their record at an interval one third of the lease
  duration.
- While everything is fine, both paths give 6/6; the difference only shows up once the layout
  changes.
- Immediately after a copy is killed, both paths give 4/6: two of six requests in the discovery
  window go to a dead address. Once the lease duration has passed, the registry returns to 6/6,
  fixed configuration stays at 4/6.
- Health information is separate from a live record: a copy that reports broken stays in the
  record count (records 2) but drops from routing (addresses 1); fixed configuration cannot make
  this distinction and drops to 2/6.
- When a new copy is added, configuration files touched are 3 against 0, clients restarted are 3
  against 0.
- Cost paid: 1 extra process and a single point of failure, 1 extra resolution request per call,
  and a two-directional failure in the lease duration (a long duration grows the discovery
  window, a short one grows the renewal load and the oscillation).

#### Next Step

This topic began with where the boundary should pass, continued with how the two sides of the
boundary talk, and ended with how the address gets resolved. The boundary was drawn, synchronous
and asynchronous communication were built, edge responsibilities were collected, a client-specific
layer was added, network concerns were pulled out of the application, and the address was
resolved from outside the code. But one assumption sitting underneath everything the seven
lessons measured was never tested: the data on the two sides of the boundary, in the middle of a
job, was treated as if it were under the guarantee of a single transaction. The loan service wrote
the record, and the fee service collected the penalty; the two happened in separate processes, on
separate data, at separate times. The previous lesson's double-collection measurement was the
first crack in that assumption. The next topic takes on exactly this: when a workflow touches more
than one service's data, how is it ensured that all of them happen together or none of them
happen, what happens when that cannot be ensured, and what units its cost is measured in.

## Distributed Consistency

Preserving correctness across the two sides of a boundary: the cost of two-phase commit and its alternatives, workflows carried out with compensating transactions, the outbox pattern that gives atomicity to a database write and a message publish, meeting event reordering and duplication with idempotency on the consumer side, and contract evolution that protects the consumer through schema compatibility.

### Distributed Transaction Alternatives

Source: https://academia.sh/en/courses/service-architectures/distributed-transaction-alternatives

The implementation cost of two-phase commit: how long the lock held between the vote and the decision, how many connections and requests stay blocked during that window, the transaction left pending at a participant when the coordinator crashes — and a comparison against a single-round implementation of the same work.

The previous topic drew the boundary, established communication across the two sides of it,
and showed how the address gets resolved. None of those three tasks touched data. The catalog
service keeps the book, the membership service keeps the loan count, each in its own store;
yet the "issue loan" request is still written on the assumption that both writes happen
together or neither happens at all. That assumption rested on a single transaction boundary,
and the boundary disappeared the moment the two sides split.

This particular loss was already established and measured in the Distributed Transaction
Problem lesson of M16/K04: when writes go to two separate stores and the second call fails,
the first stays committed. That measurement is not repeated here. What gets measured here is
the implementation-side cost of the classic fix that lesson mentioned in a single sentence —
**two-phase commit**: how long the lock stays held, how many requests get blocked during that
window, and what remains behind when the coordinator crashes.

#### Mechanism

The two services are **genuinely two separate processes**; each opens its own `node:sqlite`
file, and the only connection between them is a call over the local network.

**DC1.** The store holds its write lock at the file level, not the row level. So the "blocked
request" figure counted below is an **upper bound**; in a store that locks rows, the blocked
set would stay limited to the rows the prepared transaction touches. The shape of the
measurement — a resource held while a decision is awaited — is the same in both cases; only
the size differs.

**DC2.** The membership service spends a fixed 400 ms before voting. This models a slow
participant; it is not a delay measured from a real network. So the size of the lock window is
a chosen quantity — what gets measured is what happens inside a window of that size.

```sh
# setup.sh — closes the previous run, rebuilds each service's own store from scratch
pkill -f participant.mjs 2>/dev/null; sleep 0.3
rm -f catalog.db* membership.db* service.log
sqlite3 catalog.db >/dev/null <<'SQL'
PRAGMA journal_mode=WAL;
CREATE TABLE book (book_id INTEGER PRIMARY KEY, title TEXT NOT NULL, status TEXT NOT NULL);
INSERT INTO book VALUES (1,'Blindness','on_shelf'),(2,'The Disconnected','on_shelf'),(3,'The Book of Sand','on_shelf');
SQL
sqlite3 membership.db >/dev/null <<'SQL'
PRAGMA journal_mode=WAL;
CREATE TABLE member (member_id INTEGER PRIMARY KEY, name TEXT NOT NULL, open_loans INTEGER NOT NULL, loan_limit INTEGER NOT NULL);
INSERT INTO member VALUES (4,'Ethan',2,3),(5,'Sara',3,3),(6,'Blake',0,9);
SQL
```

The participant side is the same module for both services; the business rule and the store
are supplied from outside. The distinguishing point is the `/prepare` endpoint: the write
happens, the transaction is **not committed**, and the connection stays held until the
decision arrives.

```js
// participant.mjs — one service's participant side; each service is a separate process, a separate store
import { createServer } from "node:http";
import { DatabaseSync } from "node:sqlite";

const [name, file, port, ownSql, rivalSql] = process.argv.slice(2);
const slowMs = Number(process.env.SLOW_MS ?? 0);
const pool = Array.from({ length: 2 }, () => {
  const db = new DatabaseSync(file);
  db.exec("PRAGMA busy_timeout = 50");
  return { db, busy: false };
});
const pending = new Map();
const wait = (ms) => Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
const available = () => pool.find((b) => !b.busy);

createServer((request, response) => {
  const u = new URL(request.url, "http://y");
  const send = (o) => response.end(JSON.stringify(o) + "\n");
  const tx = u.searchParams.get("tx");
  const key = Number(u.searchParams.get("key"));

  if (u.pathname === "/prepare" || u.pathname === "/single") {
    const b = available();
    if (!b) return send({ vote: "no", reason: "no available connection" });
    b.busy = true;
    if (slowMs) wait(slowMs);
    b.db.exec("BEGIN IMMEDIATE");
    if (b.db.prepare(ownSql).run(key).changes !== 1) {
      b.db.exec("ROLLBACK"); b.busy = false; return send({ vote: "no", reason: "condition not met" });
    }
    if (u.pathname === "/single") { b.db.exec("COMMIT"); b.busy = false; return send({ vote: "yes" }); }
    pending.set(tx, b);                    // the lock stays held after this response too
    return send({ vote: "yes" });
  }
  if (u.pathname === "/commit" || u.pathname === "/abort") {
    const b = pending.get(tx);
    if (!b) return send({ result: "unknown transaction" });
    b.db.exec(u.pathname === "/commit" ? "COMMIT" : "ROLLBACK");
    b.busy = false; pending.delete(tx);
    return send({ result: u.pathname.slice(1) });
  }
  if (u.pathname === "/rival") {           // an independent local write that is not part of the transaction
    const b = available();
    if (!b) return send({ result: "blocked", reason: "no available connection" });
    b.busy = true;
    try { b.db.exec("BEGIN IMMEDIATE"); b.db.prepare(rivalSql).run(key); b.db.exec("COMMIT");
          return send({ result: "written" }); }
    catch (h) { return send({ result: "blocked", reason: h.code ?? h.message }); }
    finally { b.busy = false; }
  }
  if (u.pathname === "/pending")
    return send({ service: name, pending: [...pending.keys()], held: pool.filter((b) => b.busy).length });
  send({ error: "unknown endpoint" });
}).listen(Number(port), () => console.log(`${name} up: ${port}`));
```

```sh
# start.sh — two services, two separate processes, two separate stores; membership set up as the slow participant
node participant.mjs catalog catalog.db 8801 \
  "UPDATE book SET status='on_loan' WHERE book_id=? AND status='on_shelf'" \
  "UPDATE book SET status = CASE status WHEN 'on_shelf' THEN 'on_loan' ELSE 'on_shelf' END WHERE book_id=?" \
  >>service.log 2>&1 &
SLOW_MS=400 node participant.mjs membership membership.db 8802 \
  "UPDATE member SET open_loans=open_loans+1 WHERE member_id=? AND open_loans<loan_limit" \
  "UPDATE member SET open_loans=open_loans WHERE member_id=?" \
  >>service.log 2>&1 &
sleep 1
```

#### Same Work, Two Implementations

The coordinator is a single file and runs the same workflow in two forms. `--sequential`
mode has each service commit its own transaction immediately — the arrangement whose
inconsistency was measured in M16/K04. The default mode takes two rounds: votes are collected
from every participant first, then the decision is distributed.

```js
// coordinator.mjs — same work, two implementations: --two-phase and --sequential
const mode = process.argv.includes("--sequential") ? "sequential" : "two-phase";
const crash = process.argv.includes("--crash");       // coordinator drops before the decision is distributed
const [bookId, memberId] = process.argv.slice(2).filter((a) => !a.startsWith("--")).map(Number);
const tx = `tx-${bookId}-${memberId}`;
const participants = [
  { name: "catalog",    endpoint: "http://localhost:8801", key: bookId },
  { name: "membership", endpoint: "http://localhost:8802", key: memberId },
];
let messages = 0;
const call = async (p, path) => {
  messages += 1;
  const c = await fetch(`${p.endpoint}/${path}?tx=${tx}&key=${p.key}`);
  return c.json();
};
const ms = (a, b) => (b - a).toFixed(1);

if (mode === "sequential") {                          // one round: each service commits its own work
  const t0 = performance.now();
  for (const p of participants) {
    const c = await call(p, "single");
    console.log(`  ${p.name}: ${c.vote}${c.reason ? " (" + c.reason + ")" : ""}`);
  }
  console.log(`sequential | round=1 messages=${messages} | coordinator lock window=0 ms | duration=${ms(t0, performance.now())} ms`);
} else {                                              // two rounds: vote first, then decision
  const votes = [];
  const t0 = performance.now();
  for (const p of participants) {
    const c = await call(p, "prepare");
    votes.push([p, c, performance.now()]);
    console.log(`  ${p.name}: vote=${c.vote}${c.reason ? " (" + c.reason + ")" : ""}`);
  }
  const decision = votes.every(([, c]) => c.vote === "yes") ? "commit" : "abort";
  if (crash) {
    console.log(`coordinator dropped before distributing the decision (decision=${decision} reached no participant)`);
    process.exit(1);
  }
  for (const [p, c] of votes) if (c.vote === "yes") await call(p, decision);
  const t1 = performance.now();
  const [, , tVote] = votes[0];
  console.log(`two-phase | round=2 messages=${messages} | decision=${decision}`);
  console.log(`  catalog lock window=${ms(tVote, t1)} ms | total=${ms(t0, t1)} ms (in this run)`);
}
```

Being able to count the blocked resource needs the system given ordinary, unrelated load.

```js
// rival.mjs — requests that are not part of the transaction; tries to write at regular intervals through the window
const [endpoint, key, count, interval] = process.argv.slice(2);
let written = 0, blocked = 0, totalWait = 0;
for (let n = 0; n < Number(count); n++) {
  const t = performance.now();
  const c = await (await fetch(`${endpoint}/rival?key=${key}`)).json();
  totalWait += performance.now() - t;
  c.result === "written" ? (written += 1) : (blocked += 1);
  await new Promise((r) => setTimeout(r, Number(interval)));
}
console.log(`rival: requests=${count} written=${written} blocked=${blocked}` +
            ` | average wait=${(totalWait / Number(count)).toFixed(1)} ms (in this run)`);
```

#### Resources Blocked in the Lock Window

The same load, the same work, two implementations. The load goes to the catalog service; it
has nothing to do with the loan request, it only wants to write to the same store.

```sh
sh setup.sh; sh start.sh
node rival.mjs http://localhost:8801 2 40 10 &
node coordinator.mjs 1 4
wait
```

```
  catalog: vote=yes
  membership: vote=yes
two-phase | round=2 messages=4 | decision=commit
  catalog lock window=459.5 ms | total=489.6 ms (in this run)
rival: requests=40 written=34 blocked=6 | average wait=12.4 ms (in this run)
```

```sh
sh setup.sh; sh start.sh
node rival.mjs http://localhost:8801 2 40 10 &
node coordinator.mjs 1 4 --sequential
wait
```

```
  catalog: yes
  membership: yes
sequential | round=1 messages=2 | coordinator lock window=0 ms | duration=443.0 ms
rival: requests=40 written=40 blocked=0 | average wait=2.6 ms (in this run)
```

Two quantities that do not depend on the run are decisive here: round count, 2 versus 1, and
message count, 4 versus 2. As the number of participants grows, the message count grows as
`2n`, and the window becomes tied to the speed of the **slowest** participant.

The run-dependent figures are consistent too: the catalog lock was held for 459.5 ms, and 6 of
the 40 rival requests in that window were blocked, and the average wait rose from 2.6 ms to
12.4 ms. The catalog had finished its own work in 30.1 ms; the reason it held the lock for
459.5 ms was not its own work, it was **waiting for another service to vote**.

#### When the Coordinator Crashes

This is where the real cost of two-phase commit lies. Once a participant has said "yes," it
cannot decide on its own: if it commits, the other side may have aborted; if it aborts, the
other side may have committed. Only the coordinator knows the decision.

```sh
sh setup.sh; sh start.sh
node coordinator.mjs 1 4 --crash
echo "-- participants' view --"
curl -s "http://localhost:8801/pending"; curl -s "http://localhost:8802/pending"
echo "-- reading from outside, and a rival write --"
sqlite3 catalog.db "SELECT book_id, status FROM book WHERE book_id=1;"
node rival.mjs http://localhost:8801 2 10 10
echo "-- if the decision is supplied from outside --"
TX=$(curl -s "http://localhost:8801/pending" | sed 's/.*\["//;s/"\].*//')
curl -s "http://localhost:8801/commit?tx=$TX"
curl -s "http://localhost:8802/commit?tx=$TX"
node rival.mjs http://localhost:8801 2 10 10
```

```
  catalog: vote=yes
  membership: vote=yes
coordinator dropped before distributing the decision (decision=commit reached no participant)
-- participants' view --
{"service":"catalog","pending":["tx-1-4"],"held":1}
{"service":"membership","pending":["tx-1-4"],"held":1}
-- reading from outside, and a rival write --
1|on_shelf
rival: requests=10 written=0 blocked=10 | average wait=67.4 ms (in this run)
-- if the decision is supplied from outside --
{"result":"commit"}
{"result":"commit"}
rival: requests=10 written=10 blocked=0 | average wait=3.8 ms (in this run)
```

Both services show **1 pending transaction** each, and **1 connection** is held out of each
two-connection pool. From the outside, the book's status still shows as `on_shelf` — the
write is invisible, but the lock is real: **all 10** of the 10 rival requests were blocked,
and the average wait rose to 67.4 ms. This figure does not drop until the coordinator comes
back up. The lock duration is no longer a delay; it is the length of an outage.

The last two calls supply the decision from outside. In real systems this step is the job of
a person or a recovery procedure, and it **must know the correct decision**; whether a
pending transaction should be committed or aborted cannot be derived from the information the
participants hold.

#### Comparison

| Measure | Sequential local commit | Two-phase commit |
|---|---|---|
| Round count | 1 | 2 |
| Message count (2 participants) | 2 | 4 |
| Lock window | 0 ms | 459.5 ms (in this run) |
| Rival requests blocked in the window | 0 / 40 | 6 / 40 (in this run) |
| Pending transactions if the coordinator crashes | 0 | 2 (1 per service) |
| Rival requests blocked if the coordinator crashes | 0 / 10 | 10 / 10 |

**What it made cheaper:** two-phase commit eliminated the partial outcome. In `--sequential`
mode, when the catalog commits and membership rejects, a loaned book with no owner is left
behind; in two-phase mode this cannot happen, because no participant commits without seeing
the other's vote. No compensating logic needs to be written in the application code.

**What it made more expensive:** the round count rose to two, the message count to `2n`; a
resource stayed held for as long as the system's slowest participant took, and unrelated
requests were rejected during that time.

**Which new failure mode was born:** *indeterminate state*. A participant that has cast its
vote but not received the decision holds the lock indefinitely; the only way to free the
resource is a decision supplied from outside, and whoever supplies it cannot find the correct
answer from local information. This is a mode that sequential commit does not have.

The alternatives choose where this cost gets moved. A compensating arrangement never holds a
lock; it makes the inconsistency visible for a while and constrains the ordering. An outbox
arrangement, on the other hand, confines atomicity inside a single store and hands the
publishing off to a separate job. Both give up "all or nothing" and choose "eventually
correct" instead.

#### Summary

- Two-phase commit was built across two service processes; between the vote round and the
  decision round, the catalog service held its lock for 459.5 ms, even though its own work
  took 30.1 ms.
- The window's length is set by the slowest participant; the run-independent cost is 2 rounds
  instead of 1, and `2n` messages instead of `n`.
- In that window, 6 of 40 unrelated requests were blocked (in this run), and the average wait
  rose almost fivefold; in the single-round implementation the blocked request count was zero.
- When the coordinator dropped before distributing the decision, each participant was left
  with 1 pending transaction and 1 held connection; every rival request was blocked, and this
  lasted until a decision was supplied from outside.
- Whether a pending transaction should be committed or aborted cannot be derived from the
  participants' local information; this is a new failure mode that the single-round
  implementation does not have.

#### Next Step

The way to zero out the lock window is to **commit each step individually** instead of
preparing them, and to roll back whatever completed when a step fails. M16/K04 built a
compensating step for this arrangement, and M19/K05 measured the cost of a compensating
transaction. What remains is a constraint neither of them touched: steps with unequal
compensability have to be placed in an order, and that order is not arbitrary. The next lesson
runs the same loan flow through two different step orders and counts the unrecoverable side
effect at each one, then implements the same flow twice, as choreography and as an executor,
and measures the difference on the code side — the services touched and the number of steps
each one knows about.

### Saga Pattern

Source: https://academia.sh/en/courses/service-architectures/saga-pattern

The two implementation constraints of a workflow that runs on compensating transactions: choosing the step order by compensability, and counting the unrecoverable side effect left behind at each failure point; the code-side difference between choreography and an executor — the file carrying the flow information, the named step, and the file touched when a step is inserted.

The previous lesson showed the cost of two-phase commit: a resource stayed held for as long
as the slowest participant took, and it was left pending when the coordinator crashed. The
saga pattern removes this window entirely. Each step commits its own transaction immediately;
if a step fails, whatever completed is **compensated** in reverse.

The compensating step itself was built in M16/K04's Distributed Transaction Problem lesson,
and the cost of a compensating transaction was measured in M19/K05's Distributed Correctness
topic. Neither is repeated here. What this lesson measures is a constraint neither of them
touched: **the order of the steps**. The order is not arbitrary, because not every step is
reversible; and the choice of order determines how many effects have left the system and
cannot be recalled at the moment of failure.

#### Mechanism

**DC3.** In this lesson the services are not separate processes; they are modules that run in
the same process and write to **separate stores**. The property the model carries is that
each step commits within its own transaction boundary and there is no shared rollback call.
Network latency and partial failure are not part of the model.

**DC4.** The notification step's external effect is modeled by writing a line to a file.
Deleting the line does not undo the delivery, so this step has **no compensation**; the only
thing that can be done is a fix through a second notification.

```sh
# setup.sh — four services' separate stores and the log going out to the outside world
rm -f catalog.db membership.db fee.db sent.log
sqlite3 catalog.db "CREATE TABLE book (book_id INTEGER PRIMARY KEY, status TEXT NOT NULL);
                    INSERT INTO book VALUES (1,'on_shelf'),(2,'on_shelf');"
sqlite3 membership.db  "CREATE TABLE member (member_id INTEGER PRIMARY KEY, open_loans INTEGER NOT NULL);
                    INSERT INTO member VALUES (4,0);"
sqlite3 fee.db   "CREATE TABLE entry (id INTEGER PRIMARY KEY, member_id INTEGER, amount INTEGER);"
```

```js
// steps.mjs — the loan flow's four steps; each step either has a compensation or does not
import { DatabaseSync } from "node:sqlite";
import { appendFileSync, readFileSync, writeFileSync } from "node:fs";

const open = (d) => new DatabaseSync(d);
const catalogDb = open("catalog.db"), membershipDb = open("membership.db"), feeDb = open("fee.db");
const send = (line) => appendFileSync("sent.log", line + "\n");

export const steps = {
  "catalog.checkout": {
    run: (i) => catalogDb.prepare("UPDATE book SET status='on_loan' WHERE book_id=?").run(i.book),
    compensate: (i) => catalogDb.prepare("UPDATE book SET status='on_shelf' WHERE book_id=?").run(i.book) },
  "fee.charge": {
    run: (i) => feeDb.prepare("INSERT INTO entry (member_id,amount) VALUES (?,5)").run(i.member),
    compensate: (i) => feeDb.prepare("INSERT INTO entry (member_id,amount) VALUES (?,-5)").run(i.member) },
  "membership.addLoan": {
    run: (i) => membershipDb.prepare("UPDATE member SET open_loans=open_loans+1 WHERE member_id=?").run(i.member),
    compensate: (i) => membershipDb.prepare("UPDATE member SET open_loans=open_loans-1 WHERE member_id=?").run(i.member) },
  "notification.send": {                       // leaves the system: cannot be undone
    run: (i) => send(`member ${i.member}: book ${i.book} ready`),
    compensate: null, fix: (i) => send(`member ${i.member}: previous notification for book ${i.book} void`) },
};

export function reset() {
  catalogDb.exec("UPDATE book SET status='on_shelf'");
  membershipDb.exec("UPDATE member SET open_loans=0");
  feeDb.exec("DELETE FROM entry");
  writeFileSync("sent.log", "");
}
export const sent = () =>
  readFileSync("sent.log", "utf8").split("\n").filter((s) => s.length > 0);
```

Three of the four steps have a compensation, one does not. This is the only distinguishing
difference, and the entire ordering constraint arises from it.

#### The Ordering Constraint

The same four steps are placed into two different orders, and each order is run separately
for every failure point. The measured quantity is the number of **unrecoverable side
effects** left behind once compensation finishes.

```js
// sweep.mjs — same four steps, two different orders; every failure point is swept separately
import { steps, reset, sent } from "./steps.mjs";

const orders = {
  "A (notification early)": ["catalog.checkout", "notification.send", "fee.charge", "membership.addLoan"],
  "B (notification last)":  ["catalog.checkout", "fee.charge", "membership.addLoan", "notification.send"],
};

function runOrder(order, failStep, request) {
  reset();
  const done = [];
  let compensated = 0, unrecoverable = 0, fixed = 0;
  try {
    for (const name of order) {
      if (done.length + 1 === failStep) throw new Error(`${name} failed`);
      steps[name].run(request); done.push(name);
    }
  } catch {
    for (const name of [...done].reverse()) {
      const s = steps[name];
      if (s.compensate) { s.compensate(request); compensated += 1; }
      else { unrecoverable += 1; if (s.fix) { s.fix(request); fixed += 1; } }
    }
  }
  return { done: done.length, compensated, unrecoverable, fixed, sent: sent().length };
}

for (const [name, order] of Object.entries(orders)) {
  console.log(`\n${name}: ${order.join(" > ")}`);
  console.log("fail_step done compensated unrecoverable fixed total_sent");
  let total = 0;
  for (let k = 1; k <= order.length; k++) {
    const s = runOrder(order, k, { book: 1, member: 4 });
    total += s.unrecoverable;
    console.log(`${String(k).padStart(9)} ${String(s.done).padStart(4)} ${String(s.compensated).padStart(11)}` +
      ` ${String(s.unrecoverable).padStart(13)} ${String(s.fixed).padStart(5)} ${String(s.sent).padStart(10)}`);
  }
  console.log(`total unrecoverable side effects: ${total}`);
}
```

```sh
sh setup.sh
node sweep.mjs
```

```
A (notification early): catalog.checkout > notification.send > fee.charge > membership.addLoan
fail_step done compensated unrecoverable fixed total_sent
        1    0           0             0     0          0
        2    1           1             0     0          0
        3    2           1             1     1          2
        4    3           2             1     1          2
total unrecoverable side effects: 2

B (notification last): catalog.checkout > fee.charge > membership.addLoan > notification.send
fail_step done compensated unrecoverable fixed total_sent
        1    0           0             0     0          0
        2    1           1             0     0          0
        3    2           2             0     0          0
        4    3           3             0     0          0
total unrecoverable side effects: 0
```

Same steps, same work, same compensation code. The only difference is the order, and the
result across four failure points is **2 versus 0**. In order A, because the notification is
the second step, every failure after it means a wrong message has already gone out to the
member; since it cannot be recalled, the only thing that can be done is send the member a
second message. Total sent is `2`, not `0`: two deliveries were made even though the work
never happened.

Order B brings this down to zero, but it is not free. In B's last row the compensation count
is **3**; in A's same row it is **2**. When the uncompensable step is moved to the end, the
work that has to be rolled back when that step fails is at its largest. So the ordering
decision is made between two quantities: **unrecoverable side effect** and **completed step
rolled back**. The first is visible to the user and cannot be fixed; the second is merely
expensive. The rule follows from this asymmetry: uncompensable steps go at the end of the
order.

#### Same Flow, Two Implementations

Once the order is fixed, what remains is who carries it out. There are two implementations,
and both produce the same result. In **executor** mode the flow lives in a single module; the
service modules do not know their place in the order.

```js
// executor.mjs — the whole flow in a single module; service modules never see the flow
import { steps } from "./steps.mjs";
export const order = ["catalog.checkout", "fee.charge", "membership.addLoan", "notification.send"];
export function execute(request) {
  const done = [];
  for (const name of order) { steps[name].run(request); done.push(name); }
  return done;                          // the compensation order is also read from this list
}
```

In **choreography** mode there is no central executor; each service triggers the one after
it. Because compensation also walks backward, every node has to know its previous neighbor
too.

```js
// d-catalog.mjs — the catalog node: its own step and its two neighbors in the flow
export const node = { name: "catalog", step: "catalog.checkout", previous: null, next: "fee" };
```

```js
// d-fee.mjs — the fee node: its own step and its two neighbors in the flow
export const node = { name: "fee", step: "fee.charge", previous: "catalog", next: "membership" };
```

```js
// d-membership.mjs — the membership node: its own step and its two neighbors in the flow
export const node = { name: "membership", step: "membership.addLoan", previous: "fee", next: "notification" };
```

```js
// d-notification.mjs — the notification node: its own step and its two neighbors in the flow
export const node = { name: "notification", step: "notification.send", previous: "membership", next: null };
```

```js
// choreography.mjs — no central executor; the data path does not know the order, each node knows its neighbor
import { steps } from "./steps.mjs";
import { node as a } from "./d-catalog.mjs";
import { node as b } from "./d-fee.mjs";
import { node as c } from "./d-membership.mjs";
import { node as d } from "./d-notification.mjs";
const path = Object.fromEntries([a, b, c, d].map((x) => [x.name, x]));
export function start(firstNode, request) {
  const done = [];
  for (let x = path[firstNode]; x; x = x.next ? path[x.next] : null) {
    steps[x.step].run(request); done.push(x.step);   // each node triggers the next one
  }
  return done;
}
```

`choreography.mjs` imports all four nodes, but it does not know the order: which node
connects to which lives in the nodes' own files. The module is a registry, not a flow
definition.

#### The Difference on the Code Side

The measurement is read from the source itself: in each file, how many service names other
than its own appear?

```js
// measure-code.mjs — counts how many files the flow information is scattered across in the source text
import { readFileSync } from "node:fs";
import { reset, sent } from "./steps.mjs";
import { execute, order } from "./executor.mjs";
import { start } from "./choreography.mjs";
const services = ["catalog", "fee", "membership", "notification"];
const edge = ["fee", "membership"];                // a new step would land between these two

function measure(label, files) {
  let named = 0, carrying = 0, touched = 0;
  for (const f of files) {
    const text = readFileSync(f, "utf8");
    const own = f.replace(/^d-/, "").replace(/\.mjs$/, "");
    const n = services.filter((s) => s !== own && text.includes(s)).length;
    named += n; if (n > 0) carrying += 1;
    if (edge.every((s) => text.includes(s))) touched += 1;
  }
  console.log(`${label}: files carrying flow name=${carrying}/${files.length}` +
    ` named steps=${named} | files to open to read the flow=${carrying}` +
    ` | existing files touched when a step is inserted=${touched}`);
}

reset(); const e = execute({ book: 1, member: 4 });
console.log(`executor    result: steps=${e.length} sent=${sent().length} [${e.join(",")}]`);
reset(); const k = start("catalog", { book: 1, member: 4 });
console.log(`choreography result: steps=${k.length} sent=${sent().length} [${k.join(",")}]`);
console.log(`same result: ${JSON.stringify(e) === JSON.stringify(k)}`);
measure("executor    ", ["executor.mjs"]);
measure("choreography", ["d-catalog.mjs", "d-fee.mjs", "d-membership.mjs", "d-notification.mjs"]);
console.log(`flow length (in both implementations): ${order.length} steps`);
```

```sh
sh setup.sh
node measure-code.mjs
```

```
executor    result: steps=4 sent=1 [catalog.checkout,fee.charge,membership.addLoan,notification.send]
choreography result: steps=4 sent=1 [catalog.checkout,fee.charge,membership.addLoan,notification.send]
same result: true
executor    : files carrying flow name=1/1 named steps=4 | files to open to read the flow=1 | existing files touched when a step is inserted=1
choreography: files carrying flow name=4/4 named steps=6 | files to open to read the flow=4 | existing files touched when a step is inserted=2
flow length (in both implementations): 4 steps
```

The run side is identical: four steps, one delivery, the same step list. The difference is
entirely on the code side. In the executor, the flow information sits in 1 file as 4 named
steps. In choreography it spreads across 4 files and rises to 6 named steps; the extra comes
from the `previous` edges needed so compensation can walk backward. The executor's `done`
list accumulates in a single place, so it needs none of those edges.

Inserting a new step touches 1 existing file in the executor and 2 in choreography: the
`next` field of the node before the new step and the `previous` field of the node after it
have to change together. Choreography, in exchange, has no module that knows the whole flow;
a service changing does not change the executor, and adding a new listener to the flow can be
done without touching any existing file.

**What it made cheaper:** the saga zeroed out two-phase commit's lock window; because each
step commits its own transaction immediately, no resource stays held while another service's
response is awaited. Executor mode also made the entire flow readable in a single file.

**What it made more expensive:** a second implementation — the compensation — has to be
written for every step. The step order is no longer free; it is constrained by
compensability. In choreography mode, the flow information spread from 1 file to 4, and from
4 named steps to 6.

**Which new failure mode was born:** *a half-finished flow*. A failure that occurs after the
uncompensable step has completed leaves the system in neither its starting nor its ending
state; in order A this was measured as 2 deliveries to the member even though the work never
happened. In executor mode there is an additional single point of failure: the executor
itself. If the `done` list disappears with it, no one is left holding the compensation order.

#### Summary

- The saga removes two-phase commit's lock window; in exchange it requires writing a
  compensation for every step and ordering the steps by compensability.
- When the same four steps were placed into two orders, the unrecoverable side effect across
  four failure points came out to 2 in order A and 0 in order B; in A, 2 deliveries were made
  even though the work never happened.
- The cost is reciprocal: in order B, the last step's failure required 3 compensation calls,
  versus 2 in A. The uncompensable step goes last, because rolled-back work is expensive but
  fixable; an unrecoverable effect is not.
- The executor and choreography produced the same result — 4 steps, 1 delivery, the same step
  list — the difference is entirely in the code: flow information in 1 file and 4 named steps
  versus 4 files and 6 named steps.
- Inserting a step touches 1 existing file in the executor and 2 in choreography; in exchange,
  no module in choreography knows the whole flow, and adding a new listener does not touch
  existing files.

#### Next Step

In this lesson every step made its `run` call directly; it was treated as if there were no
gap between a step committing and the next one being notified. In reality these are two
separate jobs: the service writes to its own store, then publishes the message — and it can
drop in between. The next lesson measures this gap: how many messages get lost when the
publisher crashes, how many get delivered a second time under the same load with an outbox,
what the relay's delay is, and how many rows accumulate in the box.

### Outbox Pattern

Source: https://academia.sh/en/courses/service-architectures/outbox-pattern

The measured cost of taking the database write and the message publish into the same transaction: the message lost without an outbox when the publisher crashes, the message delivered a second time with the outbox, the relay delay set by the polling interval, and the rows that accumulate in the table when the relay stops.

In the previous lesson, each step called the next one directly: the catalog wrote to its own
store and immediately notified membership of the loan. The code was one line, but there were
two jobs — one inside the local transaction, the other outside it. A process that drops
between the two has done the work but told no one.

M16/K04's Distributed Transaction Problem lesson built the outbox arrangement and showed a
single message being delivered twice. That measurement is not repeated here. The measurement
here is **over a load**: how many messages get lost when the publisher crashes, how many get
delivered a second time under the same crash sequence with the outbox, what the relay's delay
becomes, and how many rows accumulate in the table.

#### Mechanism

**DC5.** The publisher crashing is modeled not by actually killing the process, but by a
decision drawn from a seeded generator at the point between committing and publishing. Seed
`20260731`, crash rate `0.10`. Because the decision sequence is read by **delivery attempt
number**, the `k`-th delivery attempt shares the same fate in both implementations.

**DC6.** The relay is not triggered, it polls the box. The measured delay is a direct
consequence of this choice; in a triggered relay, the delay's lower bound would not depend on
the polling interval.

```sh
# setup.sh — catalog keeps its own state and outbox, membership records deliveries
rm -f catalog.db* membership.db*
sqlite3 catalog.db "PRAGMA journal_mode=WAL;
  CREATE TABLE entry (id INTEGER PRIMARY KEY, book_id INTEGER, member_id INTEGER);
  CREATE TABLE outbox (message_id INTEGER PRIMARY KEY, book_id INTEGER, member_id INTEGER,
                             written_at INTEGER NOT NULL, status TEXT NOT NULL DEFAULT 'pending');" >/dev/null
sqlite3 membership.db "PRAGMA journal_mode=WAL;
  CREATE TABLE delivery (seq INTEGER PRIMARY KEY, message_id INTEGER, arrived_at INTEGER);" >/dev/null
```

```js
// generator.mjs — our own generator; the seed is visible, the sequence is the same on every run
export function generator(seed) {
  let d = seed >>> 0;
  return () => { d = (Math.imul(d, 1103515245) + 12345) >>> 0; return (d >>> 8) / 16777216; };
}
```

The membership side's `delivery` table accumulates every delivery as a separate row; a
duplicated delivery shows up a second time with the same `message_id`. Lost messages and
duplicated messages are both counted from this one table.

#### Same Load, Two Implementations

Two hundred loan requests. The first implementation commits the state, then publishes. The
second writes the state and the message in a single transaction, leaving the publish to a
separate pass.

```js
// publisher.mjs — same load, two implementations: --direct and --outbox
import { DatabaseSync } from "node:sqlite";
import { generator } from "./generator.mjs";

const mode = process.argv.includes("--direct") ? "direct" : "outbox";
const N = 200, SEED = 20260731, RATE = 0.10;
const roll = generator(SEED);
// fixed crash sequence keyed by attempt number: the k-th delivery attempt shares the same fate in both modes
const willCrash = Array.from({ length: N * 4 }, () => roll() < RATE);
let attempt = 0, crashed = 0;

const catalog = new DatabaseSync("catalog.db");
const membership = new DatabaseSync("membership.db");
const deliver = (messageId) =>
  membership.prepare("INSERT INTO delivery (message_id, arrived_at) VALUES (?,?)").run(messageId, Date.now());

if (mode === "direct") {
  for (let n = 1; n <= N; n++) {
    catalog.exec("BEGIN IMMEDIATE");
    const r = catalog.prepare("INSERT INTO entry (book_id, member_id) VALUES (?,4)").run(n);
    catalog.exec("COMMIT");
    if (willCrash[attempt++]) { crashed += 1; continue; }   // the process drops here: the message was never produced
    deliver(Number(r.lastInsertRowid));
  }
} else {
  for (let n = 1; n <= N; n++) {
    catalog.exec("BEGIN IMMEDIATE");                 // state and message in one transaction
    catalog.prepare("INSERT INTO entry (book_id, member_id) VALUES (?,4)").run(n);
    catalog.prepare("INSERT INTO outbox (book_id, member_id, written_at) VALUES (?,4,?)")
      .run(n, Date.now());
    catalog.exec("COMMIT");
  }
  for (let pass = 0; pass < 500; pass++) {           // relay passes
    const pending = catalog.prepare(
      "SELECT message_id FROM outbox WHERE status='pending' ORDER BY message_id").all();
    if (pending.length === 0) break;
    for (const i of pending) {
      deliver(i.message_id);
      if (willCrash[attempt++]) { crashed += 1; break; }    // drops before marking: will be redelivered
      catalog.prepare("UPDATE outbox SET status='sent' WHERE message_id=?").run(i.message_id);
    }
  }
}

const rows = membership.prepare("SELECT COUNT(*) c FROM delivery").get().c;
const unique = membership.prepare("SELECT COUNT(DISTINCT message_id) c FROM delivery").get().c;
const boxed = catalog.prepare("SELECT COUNT(*) c FROM outbox").get().c;
console.log(`${mode.padEnd(13)}| produced=${N} crashed=${crashed} | delivery rows=${rows}` +
  ` unique=${unique} lost=${N - unique} duplicated=${rows - unique}` +
  ` | rows in box=${boxed}`);
```

```sh
sh setup.sh
node publisher.mjs --direct
sh setup.sh
node publisher.mjs --outbox
```

```
direct       | produced=200 crashed=22 | delivery rows=178 unique=178 lost=22 duplicated=0 | rows in box=0
outbox       | produced=200 crashed=24 | delivery rows=224 unique=200 lost=0 duplicated=24 | rows in box=200
```

In direct publishing, no message was produced for 22 of the 200 state changes. These 22 books
showed as on loan in the catalog, never appeared on the membership side, and **no record of
this state remains**: the box has 0 rows, so there is no place in the system to look for
"should have been sent but was not." The loss is silent.

In the outbox, the lost count is zero. The crash count is 24, not 22, because this mode made
more delivery attempts (224 versus 200), and although the first 200 attempts share the same
fate as the direct mode, 2 more crashes turned up in the following 24 attempts. Each crash
produced exactly one second delivery: `224 − 200 = 24` duplicates. The loss turned into
duplication.

A third number should not be missed: the box has 200 rows. These are delivered rows whose job
is done, and the table does not shrink on its own.

#### Relay Delay

The passes above ran back-to-back. In reality, the relay runs separately and polls the box at
set intervals; the interval directly determines the time between a message being written and
being delivered.

```js
// writer.mjs — writes the state change and the message in one transaction, at regular intervals
import { DatabaseSync } from "node:sqlite";
const [count, interval] = process.argv.slice(2).map(Number);
const db = new DatabaseSync("catalog.db");
db.exec("PRAGMA busy_timeout = 500");
for (let n = 1; n <= count; n++) {
  db.exec("BEGIN IMMEDIATE");
  db.prepare("INSERT INTO entry (book_id, member_id) VALUES (?,4)").run(n);
  db.prepare("INSERT INTO outbox (book_id, member_id, written_at) VALUES (?,4,?)").run(n, Date.now());
  db.exec("COMMIT");
  await new Promise((r) => setTimeout(r, interval));
}
console.log(`writer: ${count} messages, written ${interval} ms apart`);
```

```js
// relay.mjs — polls the box at set intervals; measures the delay and the empty polls
import { DatabaseSync } from "node:sqlite";
const [interval, duration] = process.argv.slice(2).map(Number);
const catalog = new DatabaseSync("catalog.db"); catalog.exec("PRAGMA busy_timeout = 500");
const membership = new DatabaseSync("membership.db");
const delays = [];
let polls = 0, emptyPolls = 0, mostPending = 0;
const end = Date.now() + duration;
while (Date.now() < end) {
  const pending = catalog.prepare(
    "SELECT message_id, written_at FROM outbox WHERE status='pending' ORDER BY message_id").all();
  polls += 1;
  if (pending.length === 0) emptyPolls += 1;
  mostPending = Math.max(mostPending, pending.length);
  for (const i of pending) {
    const now = Date.now();
    membership.prepare("INSERT INTO delivery (message_id, arrived_at) VALUES (?,?)").run(i.message_id, now);
    catalog.exec("BEGIN IMMEDIATE");
    catalog.prepare("UPDATE outbox SET status='sent' WHERE message_id=?").run(i.message_id);
    catalog.exec("COMMIT");
    delays.push(now - i.written_at);
  }
  await new Promise((r) => setTimeout(r, interval));
}
delays.sort((a, b) => a - b);
const median = delays[Math.floor(delays.length / 2)] ?? 0;
console.log(`relay(interval=${interval} ms): polls=${polls} empty polls=${emptyPolls}` +
  ` delivered=${delays.length} most pending rows=${mostPending}` +
  ` | delay median=${median} ms max=${delays.at(-1) ?? 0} ms (in this run)`);
```

```sh
# same write rate (60 messages, 50 ms apart), two different polling intervals
for INTERVAL in 20 250; do
  sh setup.sh
  node writer.mjs 60 50 &
  node relay.mjs $INTERVAL 4500
  wait
  sqlite3 catalog.db "SELECT status, COUNT(*) FROM outbox GROUP BY status;"
done
```

```
writer: 60 messages, written 50 ms apart
relay(interval=20 ms): polls=204 empty polls=144 delivered=60 most pending rows=1 | delay median=13 ms max=23 ms (in this run)
sent|60
writer: 60 messages, written 50 ms apart
relay(interval=250 ms): polls=18 empty polls=4 delivered=60 most pending rows=5 | delay median=131 ms max=252 ms (in this run)
sent|60
```

The part that does not depend on the run is clear: in both runs 60 messages were written, and
60 were delivered. What changes is how many queries and how much waiting the same work takes.
At a twenty-millisecond interval, 144 of 204 polls are empty (in this run) — about 3.4 queries
per delivery, seven-tenths of them unrewarded. At a two-hundred-fifty-millisecond interval,
polls drop to 18 and empty polls nearly vanish, but the median delay rises from 13 ms to
131 ms, and a single pass accumulates as many as 5 rows in the box instead of 1. The interval
is a direct trade-off between empty queries and delay.

#### When the Relay Stops

Because the relay is a process independent of the publisher, it can go down on its own. This
is the situation where the outbox's gain is most visible and its cost is most concrete.

```sh
sh setup.sh
node writer.mjs 60 20
sqlite3 catalog.db "SELECT status, COUNT(*) FROM outbox GROUP BY status;"
node relay.mjs 20 1500
sqlite3 catalog.db "SELECT status, COUNT(*) FROM outbox GROUP BY status;"
sqlite3 catalog.db "DELETE FROM outbox WHERE status='sent';
                    SELECT 'rows after pruning: ' || COUNT(*) FROM outbox;"
```

```
writer: 60 messages, written 20 ms apart
pending|60
relay(interval=20 ms): polls=68 empty polls=67 delivered=60 most pending rows=60 | delay median=743 ms max=1382 ms (in this run)
sent|60
rows after pruning: 0
```

While the relay was down, 60 rows accumulated in the box, and none were lost. When the relay
came back, it delivered all of them in a single pass; the cost is that the median delay rose
from 13 ms to 743 ms, and the maximum delay to 1382 ms (in this run). The longer the relay
stays down, the larger the accumulated rows and the catch-up delay grow — this is a quantity
that has to be monitored, it is not bounded on its own.

The last line shows the table growing: the 60 delivered rows stay in the table and are only
removed by an explicit pruning. The outbox adds to the write path a table that has nothing to
do with the business rule; cleaning that table is a job in itself.

#### Three Columns

**What it made cheaper:** writing the state change and the message together eliminated the
"done but not announced" state. Under the same crash sequence, direct publishing lost 22
messages while the outbox lost 0; and the loss is not silent, it shows up as a row waiting in
the box.

**What it made more expensive:** every loan request now writes to two tables; there is also a
new process (the relay) and a new setting (the polling interval). Empty polling, rising to as
much as 3.4 queries per delivery, is the cost of shortening the delay. The box does not shrink
on its own; pruning is a separate job.

**Which new failure modes were born:** two of them. The first is **a second delivery**: the
relay dropping between delivering and marking caused 24 messages to arrive twice. The second
is **relay lag**: when the relay stops, the system does not raise an error, it just falls
behind; without measurement, the accumulating rows and growing delay go unnoticed.

#### Summary

- Under the same crash sequence, direct publishing produced no message for 22 of 200 state
  changes; the outbox lost no message at all.
- Duplication took the place of loss: the outbox made 224 deliveries, 24 of which are second
  deliveries; each crash produced exactly one duplicate.
- The polling interval is a trade-off between delay and empty queries: in this run, at 20 ms
  the median delay is 13 ms and 144 of 204 polls are empty; at 250 ms the delay rises to
  131 ms and polls drop to 18.
- When the relay stopped, 60 rows accumulated in the box, and none were lost; during the
  catch-up, the median delay was 743 ms and the maximum was 1382 ms (in this run).
- Delivered rows stay in the table (200 and 60 rows); the outbox requires an explicit pruning
  job.

#### Next Step

The 24 second deliveries the outbox leaves behind cannot be solved on the publisher side: as
long as the relay and the destination sit in separate stores, "I sent it" and "I processed it"
cannot enter the same transaction. Only one path remains — moving the cost to the consumer.
The next lesson delivers messages to the consumer both **duplicated** and **out of order**,
counts the side effect produced by two consumer implementations, and measures how a version
stamp discards a stale message.

### Event Ordering and Duplication

Source: https://academia.sh/en/courses/service-architectures/event-ordering-and-duplication

Idempotency on the consumer side: the extra side effect a plain consumer produces and the view it leaves in a wrong final state when the same event stream is delivered duplicated and out of order, how a version stamp discards a stale message, and the idempotent consumer's cost measured in queries.

The previous lesson closed out the publisher side: the outbox lost no message, but delivered
24 twice. This is no longer a problem the publisher can solve. As long as the relay's "I sent
it" record and the consumer's "I processed it" record sit in separate stores, the two cannot
enter the same transaction. The cost passes to the consumer.

On the consumer side there are two separate kinds of corruption, and their effects differ.
The first is **duplication**: the same message arrives twice. The second is **reordering**: a
message that is running late arrives after a message that follows it. M19/K05's Distributed
Correctness topic built and measured the idempotency key and the key ledger; that measurement
is not repeated here. The key ledger cuts off duplication but does nothing against
reordering — this lesson separates the two and measures the cost of an implementation that
handles both at once.

#### Mechanism

**DC7.** Network corruption is modeled with a seeded generator: seed `20260731`, delay rate
`0.20`, duplication rate `0.15`. At the same rates, the delivery stream is identical across
every run; so the difference between runs comes only from the **consumer's** implementation.

**DC8.** The consumer is an in-process module and writes to its own store. The query count is
read from the code's own counter, not from the store; what gets measured is the **amount of
work the consumer does**.

```sh
# setup.sh — the consumer side's own store: view table and side-effect log
rm -f view.db*
sqlite3 view.db "PRAGMA journal_mode=WAL;
  CREATE TABLE book_view (book_id INTEGER PRIMARY KEY, status TEXT, last_version INTEGER DEFAULT 0);
  CREATE TABLE notification (id INTEGER PRIMARY KEY, book_id INTEGER, status TEXT);" >/dev/null
```

The catalog produces events with an increasing version for each book: odd versions are loans,
even versions are returns. So the correct order can be read from a single number, and so can
reordering.

```js
// stream.mjs — produces the event log, then breaks delivery order and duplicates messages
export function generator(seed) {                  // seed is visible, sequence is the same on every run
  let d = seed >>> 0;
  return () => { d = (Math.imul(d, 1103515245) + 12345) >>> 0; return (d >>> 8) / 16777216; };
}

export function journal(books, versionCount) {      // the catalog's correctly ordered journal
  const g = [];
  for (let s = 1; s <= versionCount; s++)
    for (let k = 1; k <= books; k++)
      g.push({ message_id: g.length + 1, book: k, version: s, status: s % 2 ? "on_loan" : "on_shelf" });
  return g;
}

export function deliveryOrder(log, seed, delay, duplication) {
  const roll = generator(seed), output = [], held = [];
  for (const o of log) {
    if (roll() < delay) { held.push(o); continue; }             // this message will be late
    output.push(o);
    if (held.length && roll() < 0.10) output.push(held.shift());
    if (roll() < duplication) output.push(o);                   // at-least-once delivery: duplicate
  }
  output.push(...held);                                         // the latest stragglers go last
  return output;
}
```

The consumer is a single file and contains two implementations. `--plain` applies whatever
arrives, as is. `--versioned` stores, for each book, the last version it processed, and
discards any message that is not greater than that version.

```js
// consumer.mjs — same stream, two consumer implementations: --plain and --versioned
import { DatabaseSync } from "node:sqlite";
import { journal, deliveryOrder } from "./stream.mjs";

const versioned = process.argv.includes("--versioned");
const BOOKS = 10, VERSIONS = 300, SEED = 20260731;
const delay = Number(process.env.DELAY ?? 0.20);        // fraction of messages that break the order
const duplication = Number(process.env.DUPLICATION ?? 0.15);  // fraction delivered a second time
const correct = journal(BOOKS, VERSIONS);
const stream = deliveryOrder(correct, SEED, delay, duplication);

const db = new DatabaseSync("view.db");
const readVersion = db.prepare("SELECT last_version FROM book_view WHERE book_id = ?");
const write = db.prepare(
  "INSERT INTO book_view (book_id, status, last_version) VALUES (?,?,?)" +
  " ON CONFLICT(book_id) DO UPDATE SET status = excluded.status, last_version = excluded.last_version");
const notify = db.prepare("INSERT INTO notification (book_id, status) VALUES (?,?)");

let processed = 0, discarded = 0, queries = 0, staleArrival = 0, discardedDuplicate = 0, discardedStale = 0;
const seen = new Map(), seenMessages = new Set();
const t0 = performance.now();
db.exec("BEGIN IMMEDIATE");
for (const o of stream) {
  if ((seen.get(o.book) ?? 0) > o.version) staleArrival += 1;   // an out-of-order arrival
  seen.set(o.book, Math.max(seen.get(o.book) ?? 0, o.version));
  if (versioned) {
    const s = readVersion.get(o.book); queries += 1;            // extra read: version stamp
    if (s && s.last_version >= o.version) {
      discarded += 1; seenMessages.has(o.message_id) ? (discardedDuplicate += 1) : (discardedStale += 1);
      continue;
    }
  }
  seenMessages.add(o.message_id);
  write.run(o.book, o.status, o.version); queries += 1;
  notify.run(o.book, o.status); queries += 1;
  processed += 1;
}
db.exec("COMMIT");
const duration = performance.now() - t0;

const finalState = new Map(correct.map((o) => [o.book, o.status]));
let wrong = 0;
for (const r of db.prepare("SELECT book_id, status FROM book_view").all())
  if (finalState.get(r.book_id) !== r.status) wrong += 1;
const sideEffects = db.prepare("SELECT COUNT(*) c FROM notification").get().c;

console.log(`${versioned ? "versioned" : "plain    "} (delay=${delay} duplication=${duplication})` +
  ` | log=${correct.length} delivered=${stream.length}` +
  ` out-of-order arrivals=${staleArrival} | processed=${processed} discarded=${discarded} (duplicate=${discardedDuplicate} stale=${discardedStale})` +
  ` side effects=${sideEffects} wrong-final books=${wrong}/${BOOKS}` +
  ` | queries=${queries} duration=${duration.toFixed(1)} ms (in this run)`);
```

#### Two Kinds of Corruption, Two Different Harms

The first two runs apply the corruptions one at a time; the third gives both together; the
fourth feeds the same stream to the version-stamped consumer.

```sh
sh setup.sh; DELAY=0 node consumer.mjs --plain
sh setup.sh; DUPLICATION=0 node consumer.mjs --plain
sh setup.sh; node consumer.mjs --plain
sh setup.sh; node consumer.mjs --versioned
```

```
plain     (delay=0 duplication=0.15) | log=3000 delivered=3452 out-of-order arrivals=0 | processed=3452 discarded=0 (duplicate=0 stale=0) side effects=3452 wrong-final books=0/10 | queries=6904 duration=2.8 ms (in this run)
plain     (delay=0.2 duplication=0) | log=3000 delivered=3000 out-of-order arrivals=585 | processed=3000 discarded=0 (duplicate=0 stale=0) side effects=3000 wrong-final books=4/10 | queries=6000 duration=2.3 ms (in this run)
plain     (delay=0.2 duplication=0.15) | log=3000 delivered=3359 out-of-order arrivals=585 | processed=3359 discarded=0 (duplicate=0 stale=0) side effects=3359 wrong-final books=4/10 | queries=6718 duration=2.5 ms (in this run)
versioned (delay=0.2 duplication=0.15) | log=3000 delivered=3359 out-of-order arrivals=585 | processed=2415 discarded=944 (duplicate=359 stale=585) side effects=2415 wrong-final books=0/10 | queries=8189 duration=3.4 ms (in this run)
```

The first row is duplication only. Three thousand events were delivered 3452 times, and the
consumer produced 3452 side effects: **452 extra notifications**. The final state, by
contrast, is correct — because applying the same message a second time does not change the
view, it only repeats the notification. The harm of duplication is **in the side effect**,
not in the state.

The second row is reordering only. The delivery count equals the log, the side effect count
is correct (3000), but 585 messages arrived after the one that followed them, and **4 of the
10 books ended up in the wrong state**. The harm of reordering is **in the state**, not in the
side effect count.

When both corruptions are given together, both show up: 3359 side effects and, again, 4 wrong
books. Even if the key ledger cut off duplication, these 4 books would stay wrong; the ledger
recognizes the same message arriving a second time, it does not recognize a **different**,
late-arriving message.

#### Version Stamp

The fourth row feeds the same stream to the version-stamped consumer. The result: wrong-final
books **0**, side effects drop from 3359 to 2415, and 944 messages are discarded.

The breakdown of the discards matters: `944 = 359 duplicates + 585 stale`. The duplicates are
the same message arriving a second time; discarding them is pure gain. **The stale ones,
though, are real events** — a newer version was written over them purely because they arrived
late. The version stamp discards these too, because rolling a version backward has no
meaning.

This is why the side effect count is 2415, not 3000. The view ended up correct, but for 585
real state changes, no notification was **ever sent**. This is the cost of handling both
corruptions with a single mechanism: the rule that fixes reordering also erases late-arriving
information.

Which behavior is right depends on what the consumer does. For a consumer that carries only
the **final state**, like a view table, this is the correct behavior; the discarded message
costs nothing. For a consumer that has to react to every transition — a service computing
late fees, or one that logs every movement — the 585 losses are a real loss, and that consumer
has to work with a ledger that recognizes each message individually, not a version stamp.

#### The Idempotent Consumer's Cost

The run-independent measure of this cost is the query count. The plain consumer makes two
writes per delivery: `3359 × 2 = 6718`. The versioned consumer makes one read first for every
delivery (`3359`), then two writes only for what it processes (`2415 × 2 = 4830`); total
`8189`. The difference is `+1471` queries, about `0.44` extra queries per delivery. The
duration difference points the same way (2.5 ms versus 3.4 ms), but the real fixed cost is
the read: **every message is queried first, even the ones that end up discarded.**

The cost on the store side is visible too: a `last_version` column was added to the
`book_view` table and updated on every write. The version stamp is a permanent field that
enters the consumer's schema; the service producing the message also has to make publishing
an increasing version on every event **part of its contract**.

**What it made cheaper:** the version stamp derived the correct final state from a
duplicated, out-of-order stream — 4 wrong books dropped to 0 — and zeroed out the extra
notifications. It did this without raising any guarantee on the publisher side; the outbox's
"at least once" delivery stayed exactly as it was.

**What it made more expensive:** a column was added to the consumer schema, and a read was
added for every message (`+1471` queries); on top of that, the publisher is now obligated to
publish an increasing version on every event.

**Which new failure mode was born:** *silent discarding*. The consumer swallowed 585 real
events without raising a single error. Because the view is correct, this loss cannot be seen
from the state; it only shows up if the side effect count is compared against the log length.
If the version field is produced incorrectly — for instance if two services write to the
same record with separate counters — discarded messages increase and no one notices.

#### Summary

- Duplication and reordering cause different harms: with duplication alone, side effects came
  out to 3452 instead of 3000 but the final state was correct; with reordering alone, the
  side effect count was correct but 4 of 10 books ended up in the wrong state.
- So a ledger that only cuts off duplication is not enough on its own; a late message is a
  different message and cannot be recognized.
- The version-stamped consumer derived the correct final state from the same stream (0/10
  wrong) and discarded 944 messages: 359 duplicates, 585 stale.
- The discarded stale messages are real events; side effects stayed at 2415 instead of 3000.
  For a consumer that carries only the final state this is correct; for a consumer that
  reacts to every transition it is a loss.
- The cost is about 0.44 extra queries per delivery (`8189` instead of `6718`), a column in
  the consumer schema, and an increasing-version obligation in the publisher's contract.

#### Next Step

This lesson's solution silently made a promise: the publisher will publish an increasing
`version` field on every event, and the consumer will read that field. The same kind of
promise was made in earlier lessons — `message_id` will stay permanent, the `book` field will
always carry the same meaning. These fields are now a contract two separate teams hold
together, and the system's correctness depends on it. Because the services are separate
deployment units, the contract cannot change at a single instant: for a while, two versions
run together. The next lesson takes up how the contract can change — the order of adding and
removing a field, two versions running together, and protecting the consumer.

### Contract Evolution

Source: https://academia.sh/en/courses/service-architectures/contract-evolution

Adding a contract field, removing it, renaming it, and making it required: how many consumers each step breaks in each direction, whether the break is silent or noisy, which direction a guarded read rescues, and the rollout order measured in deployment steps, broken calls, and extra fields carried.

The previous lesson made a promise: the publisher would publish an increasing `version` field
on every event, and the consumer would read it. The same kind of promise built up across the
course: `message_id` stays permanent, `days` always means the same thing. These fields are a
contract that separate deployment units hold in common.

The problem is not the contract's existence, it is its changing. Because the services are
separate units, the contract cannot change at one instant: producer and consumers ship at
different times, opening a window where two versions run together. M21/K03 established the
**testing** side of this contract, and M19/K03 established the **expand–contract** pattern;
neither is repeated here. This lesson covers the evolution **steps themselves**: which change
breaks whom, whether it breaks silently, and what order lets the window pass harmlessly.

#### Mechanism

**DC9.** Producer and consumers are in-process functions; what gets measured is the contract's
shape, not the delivery mechanism.

**DC10.** The break criterion: the consumer's computed result differs from the value correct
before the change. A value mismatch counts as **silent wrong**, a thrown exception as
**error**.

**DC11.** The window is fixed at 50 messages flowing between two deployment steps. Real length
depends on deployment speed; fixing it is what makes the plans comparable.

#### Four Steps, Two Directions

The loan service publishes the `loan_issued` message, and the fee service computes the late
charge. Each of the four change types is tested both ways: old consumer reading a new
message, new consumer reading an old one.

```js
// compatibility.mjs — four evolution steps, two directions: old consumer reads the new message, new consumer reads the old one
const base = { message_id: 1, book: 101, member: 4 };

const steps = [
  { name: "add field     ", correct: 28,
    oldMessage: { ...base, days: 14 },
    newMessage: { ...base, days: 14, dailyFee: 2 },
    oldConsumer: (i) => 2 * i.days,
    newConsumer: (i) => i.dailyFee * i.days,
    guardedConsumer: (i) => (i.dailyFee ?? 2) * i.days },
  { name: "remove field  ", correct: 336,
    oldMessage: { ...base, days: 14, hours: 336 },
    newMessage: { ...base, days: 14 },
    oldConsumer: (i) => i.hours,
    newConsumer: (i) => i.days * 24,
    guardedConsumer: (i) => i.hours ?? i.days * 24 },
  { name: "rename field  ", correct: 28,
    oldMessage: { ...base, days: 14 },
    newMessage: { ...base, durationDays: 14 },
    oldConsumer: (i) => 2 * i.days,
    newConsumer: (i) => 2 * i.durationDays,
    guardedConsumer: (i) => 2 * (i.durationDays ?? i.days) },
  { name: "make required ", correct: 28,
    oldMessage: { ...base, days: 14 },
    newMessage: { ...base, days: 14, dailyFee: 2 },
    oldConsumer: (i) => 2 * i.days,
    newConsumer: (i) => { if (i.dailyFee === undefined) throw new Error("dailyFee required");
                          return i.dailyFee * i.days; },
    guardedConsumer: (i) => { if (i.dailyFee === undefined) throw new Error("dailyFee required");
                              return i.dailyFee * i.days; } },
];

const attempt = (f, message, correct) => {
  try { return f(message) === correct ? "ok" : "silent wrong"; }
  catch { return "error"; }
};
const pad = (s) => s.padEnd(13);

const counts = { ok: 0, silent: 0, error: 0 }, guardedCounts = { ok: 0, silent: 0, error: 0 };
const tally = (k, s) => { k[s === "ok" ? "ok" : s === "error" ? "error" : "silent"] += 1; };

console.log("step            | old consumer <- new message | new consumer <- old message | guarded consumer <- old message");
for (const s of steps) {
  const r1 = attempt(s.oldConsumer, s.newMessage, s.correct);
  const r2 = attempt(s.newConsumer, s.oldMessage, s.correct);
  const r3 = attempt(s.guardedConsumer, s.oldMessage, s.correct);
  tally(counts, r1); tally(counts, r2); tally(guardedCounts, r1); tally(guardedCounts, r3);
  console.log(`${s.name} | ${pad(r1)}        | ${pad(r2)}        | ${pad(r3)}`);
}
console.log(`plain consumer: ok=${counts.ok} silent wrong=${counts.silent} error=${counts.error} (8 combinations)`);
console.log(`guarded       : ok=${guardedCounts.ok} silent wrong=${guardedCounts.silent} error=${guardedCounts.error} (8 combinations)`);
```

```sh
node compatibility.mjs
```

```
step            | old consumer <- new message | new consumer <- old message | guarded consumer <- old message
add field      | ok                   | silent wrong         | ok           
remove field   | silent wrong         | ok                   | ok           
rename field   | silent wrong         | silent wrong         | ok           
make required  | ok                   | error                | error        
plain consumer: ok=3 silent wrong=4 error=1 (8 combinations)
guarded       : ok=5 silent wrong=2 error=1 (8 combinations)
```

Only 3 of eight combinations are clean. The real finding is how the remaining 5 break down:
**4 silent wrong, 1 error.** A missing field becomes `undefined`, enters the computation, and
the result breaks silently; no error is logged anywhere. Most contract-breaking changes make
no noise.

The one noisy row is "make required," and the noise comes not from the change type but from
the consumer **explicitly checking** the field. The silence is a consequence of missing
validation, not of the contract itself.

The ordering rule reads directly off this table. In field addition the broken direction is
"new consumer ← old message," so the **producer ships first**. In field removal the broken
direction is "old consumer ← new message," so the **consumer ships first**. Renaming is the
sum of both, and both directions break: no single order can make it safe.

#### Consumer Protection

The last column is the new consumer rewritten to also read the old field name. Silent wrong
drops from 4 to 2, ok rises from 3 to 5. Both rescued combinations share one direction:
**new consumer ← old message.** The remaining 2 silent-wrong cases sit in the other
direction, which consumer code cannot rescue — the consumer reading that message is
already-shipped old code and cannot be touched.

From this comes the tightest rule in the course: **consumer protection closes only one
direction; closing the other is the producer's job**, done by continuing to send the old
field name for a while longer.

#### Rollout Order

The rename is run through three plans. Two of the four consumers read the field, two do not.

```js
// rollout.mjs — same field rename (days -> durationDays), three rollout plans; windows are counted
const WINDOW = 50;                       // messages flowing between two deployment steps

const message = {
  v1:         (n) => ({ message_id: n, book: 101, member: 4, days: 14 }),
  transition: (n) => ({ message_id: n, book: 101, member: 4, days: 14, durationDays: 14 }),
  v2:         (n) => ({ message_id: n, book: 101, member: 4, durationDays: 14 }),
};
const consumers = {                       // the first two never read the field, the last two do
  membership:   { correct: 4,   v1: (i) => i.member, v2: (i) => i.member },
  notification: { correct: 101, v1: (i) => i.book,   v2: (i) => i.book },
  fee:          { correct: 28,  v1: (i) => 2 * i.days, v2: (i) => 2 * i.durationDays },
  view:         { correct: 14,  v1: (i) => i.days,     v2: (i) => i.durationDays },
};
const plans = {
  "single step, producer first ": [["producer", "v2"], ["fee", "v2"], ["view", "v2"]],
  "single step, consumer first ": [["fee", "v2"], ["view", "v2"], ["producer", "v2"]],
  "expand-contract             ": [["producer", "transition"], ["fee", "v2"], ["view", "v2"],
                                    ["producer", "v2"]],
};

for (const [name, plan] of Object.entries(plans)) {
  const state = { producer: "v1", membership: "v1", notification: "v1", fee: "v1", view: "v1" };
  let broken = 0, calls = 0, extraFields = 0, extraBytes = 0;
  for (const [unit, version] of plan) {
    state[unit] = version;                // one deployment step
    for (let n = 1; n <= WINDOW; n++) {
      const i = message[state.producer](n);
      if (state.producer === "transition") {          // both field names carried together
        extraFields += Object.keys(i).length - Object.keys(message.v2(n)).length;
        extraBytes += JSON.stringify(i).length - JSON.stringify(message.v2(n)).length;
      }
      for (const [t, k] of Object.entries(consumers)) {
        calls += 1;
        let s; try { s = k[state[t]](i); } catch { s = null; }
        if (s !== k.correct) broken += 1;
      }
    }
  }
  console.log(`${name} | deployment steps=${plan.length} consumer calls=${calls}` +
    ` | broken calls=${broken} | extra fields carried=${extraFields} extra bytes=${extraBytes}`);
}
const readers = Object.values(consumers).filter((k) => String(k.v1).includes("days")).length;
console.log(`messages per window=${WINDOW} | consumers=${Object.keys(consumers).length}` +
  ` | consumers reading the field=${readers}` +
  ` | extra field in the transition version=${Object.keys(message.transition(1)).length - Object.keys(message.v2(1)).length}`);
```

```sh
node rollout.mjs
```

```
single step, producer first  | deployment steps=3 consumer calls=600 | broken calls=150 | extra fields carried=0 extra bytes=0
single step, consumer first  | deployment steps=3 consumer calls=600 | broken calls=150 | extra fields carried=0 extra bytes=0
expand-contract              | deployment steps=4 consumer calls=800 | broken calls=0 | extra fields carried=150 extra bytes=1500
messages per window=50 | consumers=4 | consumers reading the field=2 | extra field in the transition version=1
```

The two single-step plans gave the **same number**: 150 of 600 consumer calls broke. Order
moves the break's location, not its amount — a rename breaks both directions, so whichever
side ships first, the other reads wrong for that entire span.

The expand–contract plan brings broken calls to **0**. The cost is three items: deployment
steps rose from 3 to 4, 150 messages carried an extra field across the window (1500 bytes, in
this run), and the producer had to produce the same information under two names for that
whole span. Where the duplicate code sits is a choice: kept in the producer, 1 unit carries
both names; kept in the consumers, the 2 reading units each carry a guarded-read line. The
measurement says the second option costs more — the gap grows with the number of reading
consumers.

The fourth step — the contract — looks skippable. Skip it and the system keeps working; a
dead field stays behind, and no one knows who reads it.

#### Summary

- Of the eight change–direction combinations, 3 were clean, 4 came out silent wrong, and 1
  errored; most changes that break a contract make no noise.
- The one noisy row is the one where the consumer explicitly checks the field; the silence is
  a consequence of missing validation, not of the contract.
- The ordering rule reads off the measurement: for adding a field the producer ships first,
  for removing one the consumer ships first. Renaming is the sum of both, and no single order
  makes it safe.
- A guarded read brought silent wrong from 4 to 2; both rescued combinations are in the
  "new consumer ← old message" direction. The other direction closes only by the producer
  continuing to send the old name for a while.
- For the rename, both single-step plans broke 150 of 600 calls; expand–contract broke 0. Its
  cost is 1 extra deployment step, 150 extra fields, and 1500 extra bytes (in this run).
- Skipping the contract step breaks nothing; a dead field stays in the contract permanently.

#### Course Wrap-Up

Across the course, the same library loan system was built eighteen times, in eighteen forms.
Every lesson wrote the same three columns.

| Lesson | What it made cheaper | What it made more expensive | The failure mode born |
|---|---|---|---|
| Monolithic Application | Local calls, a shared transaction boundary, one deployment step | Scaling unit is the whole codebase; a one-line change republishes 7 modules | A leak or infinite loop in one module stops every domain together |
| Modular Monolith | A domain's internals changing touched files dropped from 3 to 1 | Public surface rose from 7 to 11; every cross-domain need adds indirection | The rule holds only while the linter runs; a one-line import silently rolls back the boundary |
| Service-Oriented Architecture | Units stopped importing each other; adding a unit leaves existing ones unchanged | Units bound to the shared model rose from 1 to 3; a field-name change forces 3 files to ship together | No error when the owner updates and the consumer does not; the boundary check silently disappears |
| Microservices | A rate change restarted 1 of 5 deployment units | 9 network hops per request, 7 sequential rounds; startup went from 1 step to 6 | A mid-chain break leaves a half-finished state; the failure is partial |
| Serverless Approach | Processes kept running dropped from 5 to 1; the deployment unit is a single file | 4 new processes per request; store calls rose from 2 to 9 | The time limit cuts work off mid-way: no error, an incomplete result, a partial effect stays in the store |
| Architectural Migration Strategies | Rollback is 1 step, redeployment 0; the new form is tested on a live path | The same rule lives in 2 codebases; every change is written to 2 files | Updating only one codebase makes the same request give two different results |
| Boundary-Drawing Criteria | A well-drawn boundary keeps a change inside one deployment unit: one deploy, one rollback | A poorly-drawn one spreads the same change across 2 units; 1 import edge becomes a network call, 2 writes become boundary violations | A window opens between two deploys: the new rate is live in one unit while the other still sends the old field |
| Synchronous Service Calls | Four work steps finish in three rounds; when the service at the top of the chain goes down, three of the four entry points stay up | 3 network hops per request, 4 processes, requests processed 4 times over | Partial failure; an unchecked downstream status code returns success for incomplete work |
| Asynchronous Integration | Response rounds dropped from 2 to 1, units on the response path from 3 to 1 | The contract spread across 3 deployment units; 2.17 units bound to each field on average | A consistency window, and a consumer failure invisible on the response path |
| API Gateway | Duplicated edge-concern lines dropped from 15 to 6; touched files and client calls from 3 to 1 | Hops per request rose from 3 to 4; a new deployment unit; an endpoint path change still touches 2 files | The gateway is the dependency closure of all outside traffic; bare services no longer defend themselves |
| Backend for Frontend | Requests dropped to 1, extra bytes to 0; downstream field names the client reads went from 6 to 0 | 2 new deployment units, 13 of 38 lines identical; the request reaching downstream does not change | A downstream field-name change with only one layer updated leaves clients silently drifting |
| Service Mesh | Application code dropped from 38 lines to 13; 18 lines of network concerns left the application | Processes rose from 2 to 3, local hops from 1 to 2, requests reaching the fee service from 4 to 5 | A status-code-only retry repeated a non-idempotent charge: 2 charges |
| Service Discovery | A new replica touched 3 configs versus 0; once the lease expired, recovery returned to 6/6 | 1 extra process and a single point of failure, 1 extra resolution request per call | The registry holds a delayed copy of the truth; in the discovery window, 2 of 6 requests hit a dead address |
| Distributed Transaction Alternatives | The partial outcome disappeared; no compensating logic in the application code | Rounds went from 1 to 2, messages from `n` to `2n`; the lock held 459.5 ms, 6 of 40 requests blocked | Indeterminate state: a participant that voted but never got the decision holds the lock indefinitely |
| Saga Pattern | The lock window was zeroed out; no resource stays held awaiting another service | Every step needs a compensation; order bound to compensability; choreography spread flow information from 1 file to 4 | A half-finished flow: a failure after the uncompensable step left 2 unrecoverable deliveries |
| Outbox Pattern | "Done but not announced" disappeared: 22 losses dropped to 0, and the loss stopped being silent | Every request writes two tables; a new process and polling interval; the box grows if never pruned | A second delivery (24 messages) and relay lag: no error, just falling behind |
| Event Ordering and Duplication | Correct final state from a corrupted stream: 4 wrong books to 0, extra notifications to 0 | A column in the consumer schema, one read per message (+1471 queries), a version obligation on the publisher | Silent discarding: 585 real events swallowed without a single error |
| Contract Evolution | Expand–contract brought broken calls from 150 to 0; a guarded read brought silent wrong from 4 to 2 | 4 deployment steps instead of 3; 150 extra fields and 1500 extra bytes through the window | Skipping the contract breaks nothing; a dead field stays in the contract permanently |

The table carries the course's rule: **an architecture is defended only with these three
columns.** "More scalable" is not a decision; a sentence that omits which unit scales, how
many hops and deployment steps get added in exchange, and which new failure mode is born does
not count as an architectural justification. In none of the eighteen rows above is the third
column empty: every boundary traded one problem for a new one.

One thing stayed missing. Most of these failure modes are **silent**: a silently rolled-back
boundary, silently drifting clients, a silently swallowed event, a silently misread field. The
system now runs across multiple processes, and every boundary gave birth to a new failure
mode; but no tool was ever built for **seeing where a request slows down or gets lost**. Every
lesson measured with a counter it wrote itself; the running system carries none of these
counters on its own. The next course, **M16/K07 Observability and Reliability**, takes up
exactly this gap.