Stateless Application Design

The previous lesson treated copy count as a setting: a number written to the process manager, four processes coming up, requests split between the copies. In all three layouts, every one of sixty requests could go to any copy, and the result did not change.

This is scaling’s invisible precondition, and it is not automatic. Copies can only be multiplied if it does not matter which one a request reaches. This lesson looks for where that precondition breaks in the code, and answers two questions with numbers: how many call sites are pinned to a node, and how many requests get the wrong result once a second process is added.

Four Ties

Four places in the loan application are tied to the process itself. In-memory session: login produces a token and writes it to a table. Local file: the loan receipt is written to the process’s own directory. In-process cache: the catalog value is cached in memory on first read. Timer: the overdue penalty is processed by a counter set up inside the process.

• SD1. The only shared resources are catalog.json and the penalty ledger; the receipt directory is copy-specific and stands in for the copy’s local disk.
• SD2. Requests are split between copies in turn. The dispatch rule was measured in the previous lesson; what this one measures is not dispatch, it is how the copies diverge.
• SD3. The timer processes exactly three rounds per process, then stops. The line count landing in the penalty ledger is therefore run-independent.

// topology/app.mjs — a copy of the loan application. Four kinds of state live inside this
// process: the session table, the local receipt directory, the catalog cache, and the overdue-
// penalty timer. The only shared spot is catalog.json and ledger.txt; the receipt directory is copy-specific.
import { createServer } from "node:http";
import { appendFileSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";

const port = Number(process.argv[2]);
const session = new Map();                       // in-memory session
const cache = new Map();                          // in-process catalog cache
let tokenCounter = 0;

const readCatalog = () => JSON.parse(readFileSync("catalog.json", "utf8"));
const body = (req) => new Promise((resolve) => { let b = ""; req.on("data", (p) => { b += p; }); req.on("end", () => resolve(b)); });

if (Number.isInteger(port) === false) console.log("usage: node topology/app.mjs <port>");
else {
  mkdirSync(`receipt-${port}`, { recursive: true });          // the copy's own local disk
  let round = 0;
  const timer = setInterval(() => {                          // each copy sets up its own timer
    if (++round > 3) { clearInterval(timer); return; }
    appendFileSync("ledger.txt", `member-7 overdue penalty 2 (copy ${port}, round ${round})\n`);
  }, 150);

  createServer(async (req, res) => {
    res.sendDate = false;
    const [, root, sub] = req.url.split("/");
    if (root === "login") {
      const token = `B${port}-${++tokenCounter}`;
      session.set(token, "member-7");
      res.writeHead(200, { "x-copy": String(port) }).end(token);
    } else if (root === "my-loans") {
      const member = session.get(req.headers["x-token"]);
      if (member === undefined) res.writeHead(401, { "x-copy": String(port) }).end("no session");
      else res.writeHead(200, { "x-copy": String(port) }).end(member);
    } else if (root === "receipt" && req.method === "POST") {
      const id = `M${port}-${Date.now() % 100000}`;
      writeFileSync(`receipt-${port}/${id}.txt`, "loan received");
      res.writeHead(200, { "x-copy": String(port) }).end(id);
    } else if (root === "receipt") {
      let content = null;
      try { content = readFileSync(`receipt-${port}/${sub}.txt`, "utf8"); } catch { content = null; }
      if (content === null) res.writeHead(404, { "x-copy": String(port) }).end("no receipt");
      else res.writeHead(200, { "x-copy": String(port) }).end(content);
    } else if (root === "book" && req.method === "POST") {
      const k = readCatalog();
      k[sub] = Number(await body(req));
      writeFileSync("catalog.json", JSON.stringify(k));        // the shared resource is updated
      cache.delete(sub);                                       // only this process's cache is refreshed
      res.writeHead(200, { "x-copy": String(port) }).end(String(k[sub]));
    } else if (root === "book") {
      if (cache.has(sub) === false) cache.set(sub, readCatalog()[sub]);
      res.writeHead(200, { "x-copy": String(port) }).end(String(cache.get(sub)));
    } else res.writeHead(404, { "x-copy": String(port) }).end("not found");
  }).listen(port, "127.0.0.1");
}

The first measurement looks at the code. The scanner searches the source for four patterns and lists the node-pinned lines.

// topology/scanner.mjs — counts node-pinned call sites in the source. Each pattern is a state
// type; the lines found are tied to that process and diverge once a second copy is added.
// Import lines are not counted; what is counted is where the state is declared and used.
import { readFileSync } from "node:fs";

const PATTERN = [
  ["in-memory session", /session\.(set|get|has|delete)|new Map\(\).*session|const session/],
  ["local file", /receipt-\$\{port\}|mkdirSync/],
  ["in-process cache", /cache\.(set|get|has|delete)|const cache/],
  ["timer", /setInterval|appendFileSync/],
];

const file = process.argv[2];
if (file === undefined) console.log("usage: node topology/scanner.mjs <source-file>");
else {
  const lines = readFileSync(file, "utf8").split("\n");
  let total = 0;
  console.log("state type           call sites  lines");
  for (const [name, pattern] of PATTERN) {
    const hits = lines.map((s, i) => (pattern.test(s) && s.startsWith("import") === false ? i + 1 : 0)).filter(Boolean);
    total += hits.length;
    console.log(`${name.padEnd(21)}${String(hits.length).padStart(10)}  ${hits.join(", ")}`);
  }
  console.log(`${"total".padEnd(21)}${String(total).padStart(10)}`);
}

The second measurement looks at runtime. The client runs four scenarios in turn and counts how many requests get the wrong result in each.

// topology/client.mjs — dispatches requests among copies in turn (the previous lesson's
// dispatcher did this too; what is measured here is not dispatch, it is how the copies diverge)
// and counts, for each of four state types, how many requests got the wrong result.
import { request } from "node:http";
import { readFileSync } from "node:fs";

const [copiesText, label] = [process.argv[2] ?? "", process.argv[3] ?? "-"];
const COPIES = copiesText.split(",").filter(Boolean).map(Number);
const COUNT = 6;
let turn = 0;

const one = (path, { method = "GET", headers = {}, body } = {}) => new Promise((resolve) => {
  const target = COPIES[turn++ % COPIES.length];
  const r = request({ port: target, path, method, headers }, (y) => {
    let b = "";
    y.on("data", (p) => { b += p; });
    y.on("end", () => resolve({ code: y.statusCode, body: b, copy: y.headers["x-copy"] }));
  });
  r.on("error", () => resolve({ code: 0, body: "", copy: "-" }));
  r.end(body);
});

const report = (name, wrong, note) => {
  console.log(`${label} ${name.padEnd(20)} ${COUNT - wrong}/${COUNT} correct  ${String(wrong).padStart(2)} wrong  ${note}`);
  return wrong;
};

if (COPIES.length === 0) console.log("usage: node topology/client.mjs <copy,copy> <label>");
else {
  const login = await one("/login", { method: "POST" });
  const o = await Promise.all(Array.from({ length: COUNT }, () => one("/my-loans", { headers: { "x-token": login.body } })));
  const a = report("in-memory session", o.filter((y) => y.code !== 200).length, `session on copy ${login.copy}`);

  const receipt = await one("/receipt", { method: "POST" });
  const m = await Promise.all(Array.from({ length: COUNT }, () => one(`/receipt/${receipt.body}`)));
  const b = report("local file", m.filter((y) => y.code !== 200).length, `receipt on copy ${receipt.copy}`);

  await Promise.all([one("/book/K1"), one("/book/K1")]);              // two copies' caches warm up
  await one("/book/K1", { method: "POST", body: "2" });                // shared catalog goes from 1 to 2
  const k = await Promise.all(Array.from({ length: COUNT }, () => one("/book/K1")));
  const c = report("in-process cache", k.filter((y) => y.body !== "2").length, "current value 2");

  const lines = readFileSync("ledger.txt", "utf8").trim().split("\n").filter((s) => s.includes("member-7"));
  const d = Math.max(0, lines.length - 3);
  console.log(`${label} ${"timer".padEnd(20)} ${3}/${lines.length} correct  ${String(d).padStart(2)} wrong  ` +
    `member-7 penalty processed ${lines.length} times`);
  console.log(`${label} ${"total".padEnd(20)} requests with wrong result ${a + b + c + d}/${3 * COUNT + 3}`);
}

The same application runs under two layouts. Not one line of code changes; the only thing that changes is how many processes are brought up.

# measure.sh — same application, first one copy, then two. The shared catalog resets before
# every run; the ledger and the copies' receipt directories are removed too.
setup() {                                 # setup <copy-count>
  rm -rf receipt-897* ledger.txt
  echo '{"K1":1}' > catalog.json
  : > ledger.txt
  P=""; N=""
  for i in $(seq 1 "$1"); do node topology/app.mjs $((8970 + i)) & P="$P $!"; N="$N,$((8970 + i))"; done
  sleep 0.9
}
teardown() { kill $P 2>/dev/null; wait $P 2>/dev/null; sleep 0.2; }

setup 1; node topology/client.mjs "${N#,}" "one copy  "; teardown
setup 2; node topology/client.mjs "${N#,}" "two copies"; teardown
echo
node topology/scanner.mjs topology/app.mjs

one copy   in-memory session    6/6 correct   0 wrong  session on copy 8971
one copy   local file           6/6 correct   0 wrong  receipt on copy 8971
one copy   in-process cache     6/6 correct   0 wrong  current value 2
one copy   timer                3/3 correct   0 wrong  member-7 penalty processed 3 times
one copy   total                requests with wrong result 0/21
two copies in-memory session    3/6 correct   3 wrong  session on copy 8971
two copies local file           3/6 correct   3 wrong  receipt on copy 8972
two copies in-process cache     3/6 correct   3 wrong  current value 2
two copies timer                3/6 correct   3 wrong  member-7 penalty processed 6 times
two copies total                requests with wrong result 12/21

state type           call sites  lines
in-memory session             3  8, 29, 32
local file                    3  17, 37, 41
in-process cache              4  9, 48, 51, 52
timer                         2  19, 21
total                        12

All the numbers are deterministic: request count is fixed, dispatch is round robin, timer round count is three.

What the Code Says

The scan finds twelve call sites in a fifty-five-line source. What matters is less the number itself than its spread. Session appears in three places: where it is declared, where it is written, where it is read. Cache in four. The timer in only two, and those two are not even inside an endpoint — they sit in the module body, before the server is even set up.

This spread is the cost of a fix. Moving the session to a shared store touches three lines, the cache four. The timer is two lines, but touching it changes not the code but the layout: the work needs to run in one copy and not the others; that is not a line of code, it is a deployment decision.

One number needs a careful reading. The scan gives twelve call sites, the run gives twelve wrong results — the two are not the same thing, and their matching here is specific to this setup. One is a static measure of the source, independent of copy count; the other is a runtime measure of the requests, and it grows with copy count.

Twelve of Twenty-One Requests

In the single-copy layout, all twenty-one requests got the correct result. The application works correctly, passes its tests, and shows no leak at all. Every one of these leaks is correct behavior with a single copy: keeping the session in memory is fast, writing the receipt to local disk is cheap, the catalog cache saves a read, the timer does its job.

Once a second process is added, the same code gets the wrong result on twelve of twenty-one requests. Exactly three requests in each of the four types, because dispatch is round robin and half the requests land on the copy holding the state.

• Session opened on copy 8971; the three requests landing on 8972 got a 401. The member knows they logged in, and the system fails to recognize them on every other request.
• Receipt was written to copy 8972’s local directory; the three requests landing on 8971 got a 404. The receipt is not lost, it sits on the wrong node.
• Cache warmed on both copies, but the update invalidated only one of them. The shared catalog holds the value 2, and three requests get 1 — with a 200.
• Timer was set up on both copies, and the overdue penalty was processed six times instead of three. Nothing corresponds to these three extra runs; no one asked anything, no status code was produced.

Four Leaks, Four Visibilities

The four do not carry equal weight, and what sets the order is neither frequency nor count — it is what they look like from the outside.

Leak	What the client sees	Visible in the log	Touches data
In-memory session	401	yes, as an error	no
Local file	404	yes, as an error	no
In-process cache	200, stale value	no	no
Timer	nothing	no	yes

The first two rows are loud: the wrong result comes back as an error code, lands in the log, shows up in a counter. It is annoying, and exactly for that reason it gets found early.

The third row is silent. The request succeeds, the log is clean, latency is low. In the loan system, this means a book that is not available shows up as on the shelf. The only reason this row can be measured is that the client already knows the correct answer; a user does not.

The fourth row is both silent and, alone among the four, corrupts shared data. Member seven’s overdue penalty was processed twice. No request, no error, no deviation in the log — only twice the lines in the ledger. Raise copy count to three and the penalty would run three times; the leak’s severity is tied directly to copy count, and copy count is the scaling decision itself.

Where the Setting Lives

Setting	Location	How many places	Silent result of a wrong value
Copy count	process manager	1 line	12 call sites in the application diverge, 12 of 21 requests are wrong
Where the session is held	application code	3	session lost on every other request
Receipt directory	application code	3	no receipt outside the copy that wrote it
Cache invalidation	application code	4	stale value returns with a 200
Where the timer is set up	application code, module body	2	the work repeats once per copy

The table’s asymmetry is this lesson’s rule. The setting that changes is one line, in the process manager; the lines that decide its outcome are twelve, in application code. A change raising copy count touches no line of application code, so application code never gets opened in the review.

Summary

• The same application got the correct result on all twenty-one requests with one copy, and the wrong result on twelve with two; not one line of code changed.
• Each of the four leak types wronged exactly three requests: session with a 401, receipt with a 404, cache with a stale value at 200, and the timer by processing member seven’s overdue penalty six times instead of three.
• The source scan found twelve node-pinned call sites in fifty-five lines: session 3, local file 3, cache 4, timer 2; this spread is the line count a fix will touch.
• The static twelve and the runtime twelve are different measures; the first is independent of copy count, the second grows with it.
• The quietest two leaks are the cache and the timer: one returns the wrong value with a 200, the other corrupts shared data without producing a single request, and its severity scales with copy count.

Next Step

For three of the four leaks the direction is clear: state moves out of the process, to somewhere the copies share. For the session, this is not the only solution that comes to mind, and it is often not the first one tried either. A cheaper-looking path exists: instead of moving the state, move the request — send all of a member’s requests to the copy where their session opened. This drops the three wrong requests in the measurement to zero and touches no line of application code. It makes something else pay the cost. The next lesson names this path and works out its bill: how much it skews distribution, how many sessions are lost when a copy goes down, and what moving the session to a shared store changes in the same measurement.