Process Managers

The previous lesson configured the layer in front of the application, and every one of its eight runs assumed one thing: the process was up, started by hand, stopped by hand when the job finished, and never crashed in between.

In production, another component starts the process, restarts it when it crashes, decides how much memory it may use, and warns it before stopping it: the process manager. The proxy’s settings changed what a request carried; this layer’s settings change whether it gets answered at all.

Two Settings, One Worker

The process manager has two settings to measure. The first is the restart policy: how long to wait before restarting an exited process, and after how many attempts to give up. The second is the resource limit: how much memory a process may hold, and what happens when it is crossed. A third setting stands apart from both, just as silent: the wait time granted after a stop signal.

• SD1. The worker allocates 2 MB per open loan record and reports its running total to the manager; the limit is enforced through that report. In a real deployment the value is read from the operating system.
• SD2. A request’s work takes a fixed duration. In the memory measurement, responses are staggered 100 milliseconds apart by acceptance order, so both the instant the limit is crossed and how many requests are in flight then are determined.
• SD3. This lesson runs a single worker process; copy count is the next lesson’s subject.

The worker is the running form of the loan application, and takes four behaviors from a mode string.

// topology/worker.mjs — loan worker process. A mode string sets its behavior:
//   count=<n>  exits after answering n requests   |  start-fail=1  exits before it starts listening
//   work=<ms>  one request's duration   step=<ms>  work grows by acceptance order (responses stagger)
//   memory=soft  at the limit, discards the oldest half of records and stays up
//   shutdown=graceful|hard  on the stop signal, drains pending requests or exits immediately
import { createServer } from "node:http";

const [port, modeText] = [Number(process.argv[2]), process.argv[3] ?? ""];
const MODE = Object.fromEntries(modeText.split(",").filter(Boolean).map((p) => p.split("=")));
const RECORD_MB = 2;                      // memory held per open loan record
const LIMIT_MB = 40;                      // soft-limit threshold
const held = [];
let responded = 0, discarded = 0, accepted = 0, closing = false;

if (Number.isInteger(port) === false) { console.log("usage: node topology/worker.mjs <port> <mode>"); process.exit(0); }
if (MODE["start-fail"] === "1") process.exit(9);

const server = createServer((req, res) => {
  res.sendDate = false;
  if (closing) { res.writeHead(503).end("closing"); return; }
  const order = accepted++;
  held.push(new Array((RECORD_MB * 1024 * 1024) / 8).fill(1));    // an open loan record sits in memory
  if (MODE.memory === "soft" && held.length * RECORD_MB > LIMIT_MB) {
    discarded += held.splice(0, held.length >> 1).length;         // oldest half discarded, process lives
  }
  setTimeout(() => {
    responded += 1;
    res.writeHead(200, { "x-held-mb": String(held.length * RECORD_MB) }).end("loan");
    if (process.send !== undefined) process.send({ heldMB: held.length * RECORD_MB, responded, discarded });
    if (MODE.count !== undefined && responded >= Number(MODE.count)) process.exit(9);
  }, Number(MODE.work ?? 40) + order * Number(MODE.step ?? 0));
});

process.on("SIGTERM", () => {
  if (MODE.shutdown === "graceful") { closing = true; server.close(() => process.exit(0)); }
  else process.exit(0);                    // pending requests drop without ever seeing a response
});
server.listen(port, "127.0.0.1");

The manager starts the worker, listens for its reports, catches its exit, and applies policy.

// topology/manager.mjs — process manager. Setting string:
//   policy=immediate|backoff|limited:<n>   memory=none|hard:<mb>
//   duration=<ms>  0 waits for the stop signal and prints a summary; >0 starts a graceful stop then
//   wait=<ms>  time granted after the stop signal before a hard kill
import { fork } from "node:child_process";

const [port, configText, workerMode, label] = [process.argv[2], process.argv[3] ?? "", process.argv[4] ?? "", process.argv[5] ?? "-"];
const OPTS = Object.fromEntries(configText.split(",").filter(Boolean).map((p) => p.split("=")));
const BACKOFF = [100, 200, 400, 800, 1600];       // backoff ladder (ms)
let attempts = 0, delay = 0, kills = 0, peak = 0, discarded = 0, gaveUp = false, stopping = false, child = null;

function start() {
  let killed = false;
  child = fork("topology/worker.mjs", [port, workerMode], { stdio: ["ignore", "ignore", "ignore", "ipc"] });
  child.on("message", (m) => {
    peak = Math.max(peak, m.heldMB);
    discarded = m.discarded;
    const hard = (OPTS.memory ?? "none").startsWith("hard:") ? Number(OPTS.memory.split(":")[1]) : Infinity;
    if (m.heldMB > hard && killed === false) {                        // limit crossed: process is killed
      killed = true;
      kills += 1;
      child.kill("SIGKILL");
    }
  });
  child.on("exit", () => { if (stopping === false) retry(); });
}

function retry() {
  const max = (OPTS.policy ?? "").startsWith("limited:") ? Number(OPTS.policy.split(":")[1]) : Infinity;
  if (attempts >= max) { gaveUp = true; return; }
  const g = OPTS.policy === "backoff" ? BACKOFF[Math.min(attempts, BACKOFF.length - 1)] : 0;
  attempts += 1;
  delay += g;
  setTimeout(start, g);
}

if (port === undefined) console.log("usage: node topology/manager.mjs <port> <config> <worker-mode> <label>");
else {
  setInterval(() => {}, 60000);          // stays up for the summary even after giving up
  start();
  process.on("SIGTERM", () => {
    stopping = true;
    child?.kill("SIGKILL");
    console.log(`${label} manager: ${attempts} restarts, ${delay} ms delay, ${kills} kills, ` +
      `peak ${peak} MB, ${discarded} records discarded${gaveUp ? ", gave up" : ""}`);
    process.exit(0);
  });
  if (Number(OPTS.duration ?? 0) > 0) setTimeout(() => {
    stopping = true;
    child.kill("SIGTERM");                                            // graceful stop request first
    const hard = setTimeout(() => child.kill("SIGKILL"), Number(OPTS.wait ?? 500));
    child.on("exit", () => { clearTimeout(hard); process.exit(0); }); // clean if it exits before the wait ends
  }, Number(OPTS.duration));
}

The load generator sends requests at a fixed interval or all at once, and separates three outcomes: answered, cleanly rejected, dropped.

// topology/load.mjs — load generator. Mode "spaced": count requests, interval ms apart.
// Mode "concurrent": count requests at once. A request that fails to connect or closes without a
// response counts as "dropped".
import { request } from "node:http";

const [port, mode, count, interval, label] = [Number(process.argv[2]), process.argv[3], Number(process.argv[4]), Number(process.argv[5]), process.argv[6] ?? "-"];
const wait = (ms) => new Promise((c) => setTimeout(c, ms));
let answered = 0, rejected = 0, dropped = 0;

const one = () => new Promise((resolve) => {
  const r = request({ port, path: "/loan", agent: false }, (y) => {
    y.resume();
    y.on("end", () => { if (y.statusCode === 200) answered += 1; else rejected += 1; resolve(); });
    y.on("aborted", () => { dropped += 1; resolve(); });
  });
  r.on("error", () => { dropped += 1; resolve(); });
  r.end();
});

if (Number.isInteger(port) === false) console.log("usage: node topology/load.mjs <port> <mode> <count> <interval> <label>");
else {
  if (mode === "concurrent") await Promise.all(Array.from({ length: count }, one));
  else {
    const open = [];
    for (let i = 0; i < count; i += 1) { open.push(one()); await wait(interval); }
    await Promise.all(open);
  }
  console.log(`${label} load: ${count} requests, ${answered} answered, ${rejected} rejected, ${dropped} dropped`);
}

A Crashing Process and One That Never Starts

The first setup meets the same crash with three policies. The worker exits after every fourth response, and the load sends 24 requests 100 milliseconds apart. The same three policies then run against a worker that never starts, exiting before it begins listening.

# measure1.sh — same crash, three restart policies. Worker exits after every 4th response.
run() {                                   # run <policy> <worker-mode> <count> <label>
  node topology/manager.mjs 8941 "policy=$1,duration=0" "$2" "$4" & M=$!
  sleep 0.5
  node topology/load.mjs 8941 spaced "$3" 100 "$4"
  kill -TERM $M; wait $M; sleep 0.2
}

run immediate       "count=4,work=40" 24 "crash / immediate      "
run backoff         "count=4,work=40" 24 "crash / backoff        "
run limited:2       "count=4,work=40" 24 "crash / limited (2)    "
run immediate       "start-fail=1" 8 "start-fail / immediate "
run backoff         "start-fail=1" 8 "start-fail / backoff   "
run limited:2       "start-fail=1" 8 "start-fail / limited 2 "

crash / immediate       load: 24 requests, 20 answered, 0 rejected, 4 dropped
crash / immediate       manager: 5 restarts, 0 ms delay, 0 kills, peak 8 MB, 0 records discarded
crash / backoff         load: 24 requests, 14 answered, 0 rejected, 10 dropped
crash / backoff         manager: 3 restarts, 700 ms delay, 0 kills, peak 8 MB, 0 records discarded
crash / limited (2)     load: 24 requests, 12 answered, 0 rejected, 12 dropped
crash / limited (2)     manager: 2 restarts, 0 ms delay, 0 kills, peak 8 MB, 0 records discarded, gave up
start-fail / immediate  load: 8 requests, 0 answered, 0 rejected, 8 dropped
start-fail / immediate  manager: 43 restarts, 0 ms delay, 0 kills, peak 0 MB, 0 records discarded
start-fail / backoff    load: 8 requests, 0 answered, 0 rejected, 8 dropped
start-fail / backoff    manager: 4 restarts, 1500 ms delay, 0 kills, peak 0 MB, 0 records discarded
start-fail / limited 2  load: 8 requests, 0 answered, 0 rejected, 8 dropped
start-fail / limited 2  manager: 2 restarts, 0 ms delay, 0 kills, peak 0 MB, 0 records discarded, gave up

The first three pairs meet the same crash, and the ranking flips from what one might expect. Immediate answered 20 of 24 requests, four dropped (this run). Backoff answered 14, ten dropped — its ladder added 100, 200, and 400 milliseconds of waiting, 700 total, and every request arriving during that wait was dropped. The limited policy gave up after two attempts, and all twelve remaining requests dropped; that number comes from the policy’s definition, not the run.

The answered and dropped counts come from a fixed-interval load and depend on the machine’s load at that moment; what stays run-independent is the wait time and attempt count each policy adds.

When the crash is transient, the ranking is clear: backoff is pure loss if the next attempt would have succeeded anyway. The ladder’s reason for existing shows up in the last three rows.

When the worker never starts, all three policies give the same result — eight of eight requests drop. The difference is not on the request side but in the work the machine burns: immediate started process after process in the same 0.8-second window — dozens, 43 in this run — backoff started four, limited two. Two of these numbers are run-independent: the ladder’s 100, 200, 400, and 800 milliseconds fit four attempts into that window, and limited stops at two by definition. Immediate’s count is the ratio of window length to process-start cost, with no upper bound: it grows with the window and with a faster machine.

One more distinction is the quietest of all. The limited policy prints that it gave up; the other two keep trying forever. A system that is not up looks like it is “starting” for as long as something keeps trying to start it.

Crossing the Limit: Killing and Slowing

The second setup applies the memory limit in two places. At the hard limit, the manager reads the report and kills the process that crosses the threshold. At the soft limit, the worker sees its own threshold, discards the oldest half of its records, and stays up. Twenty-four requests are sent at once, so all are in flight when the limit is crossed. A third setup sends the stop signal while six requests are in flight.

# measure2.sh — memory limit two ways, then requests in flight when the stop signal lands.
memory() {                                # memory <manager-config> <worker-mode> <label>
  node topology/manager.mjs 8942 "$1" "$2" "$3" & M=$!
  sleep 0.5
  node topology/load.mjs 8942 concurrent 24 0 "$3"
  kill -TERM $M; wait $M; sleep 0.2
}
memory "policy=immediate,memory=hard:40,duration=0" "work=200,step=100"             "hard limit 40 MB"
memory "policy=immediate,memory=none,duration=0"    "work=200,step=100,memory=soft" "soft limit 40 MB"

shutdown() {                              # shutdown <manager-config> <worker-mode> <label>
  node topology/manager.mjs 8943 "$1" "$2" "$3" & M=$!
  sleep 0.5
  node topology/load.mjs 8943 concurrent 6 0 "$3"
  wait $M; sleep 0.2
}
shutdown "policy=immediate,duration=700,wait=500" "work=300,shutdown=graceful" "graceful, wait 500 ms"
shutdown "policy=immediate,duration=700,wait=50"  "work=300,shutdown=graceful" "graceful, wait  50 ms"
shutdown "policy=immediate,duration=700,wait=500" "work=300,shutdown=hard"     "hard shutdown        "

hard limit 40 MB load: 24 requests, 1 answered, 0 rejected, 23 dropped
hard limit 40 MB manager: 1 restarts, 0 ms delay, 1 kills, peak 48 MB, 0 records discarded
soft limit 40 MB load: 24 requests, 24 answered, 0 rejected, 0 dropped
soft limit 40 MB manager: 0 restarts, 0 ms delay, 0 kills, peak 28 MB, 10 records discarded
graceful, wait 500 ms load: 6 requests, 6 answered, 0 rejected, 0 dropped
graceful, wait  50 ms load: 6 requests, 0 answered, 0 rejected, 6 dropped
hard shutdown         load: 6 requests, 0 answered, 0 rejected, 6 dropped

The hard limit is crossed at 48 MB and the process is killed: one of twenty-four requests is answered, twenty-three drop, one restart happens. At the soft limit, all twenty-four are answered, none drop, no restart happens; the cost is ten discarded records and a peak of 28 MB instead of 48.

Both rows meet the same threshold under the same load, and what gets lost changes. Under killing, the requests in flight are lost: twenty-three loan operations are cut off mid-way with no response. Under discarding, memory history is lost: the process lives, the request is answered, and a discarded record has to be read again if needed. The first is a visible failure, the second a slowdown.

Twenty-three is the number of requests in flight at the instant the limit was crossed: killing’s bill is measured by the work being carried at that instant.

The soft limit’s silent side is here too: no counter ever turns red. In a setup that does not report a discarded-record count, this row looks flawless, and the fact that the memory threshold is being crossed constantly is written down nowhere.

Wait Time

The last three rows send the stop signal while six requests are in flight. Each request’s work takes 300 milliseconds.

Under graceful shutdown, the worker stops accepting new requests and drains the ones waiting; at a 500-millisecond wait, all six were drained. The same code, wait cut to 50 milliseconds, lost all six: the worker began draining, the manager did not wait for it to finish, and killed the process. Under hard shutdown, the worker never drained at all, and the result was the same.

Two of the three rows give the same result for different reasons: one has no draining, the other has draining but not enough time granted. On the code side the second is written correctly; on the setting side both are equally a loss. Graceful-shutdown code does not run to completion unless the wait time exceeds the application’s longest request — and nowhere is it recorded that the code never ran.

Where the Setting Lives

Setting	Location	How many places	Silent result of a wrong value
Restart policy	process manager	1 (each node)	extra dropped requests on a transient crash, an endless start loop on a permanent one
Attempt limit	process manager	1	without a limit, the failure looks like “starting”
Memory limit and form	manager (hard) or application code (soft)	2	every request in flight lost under killing, a silent slowdown under discarding
Wait time	process manager	1	graceful-shutdown code is written and never runs

Three of the four rows sit in a single place and are a single number. Their effects show up only during a crash, or during a release.

Summary

• Under the same transient crash, immediate answered 20 of 24 requests, backoff 14, and the policy limited to two attempts 12 (the first two figures are from this run); the 700 milliseconds backoff added turned directly into dropped requests.
• When the process never started, all three policies lost eight of eight requests; the difference was in work spent — immediate started dozens of processes in the same window (43 in this run), backoff four, limited two, and only the last one reported it gave up.
• The numbers from the backoff ladder and the attempt limit are run-independent; immediate’s start count has no upper bound.
• Enforced as a hard limit, twenty-three of twenty-four requests dropped and one restart happened; enforced as a soft limit, none dropped and the cost was ten discarded records. The dropped-request count equals the work in flight at the instant the limit was crossed.
• Under graceful shutdown, a 500-millisecond wait drained all six requests in flight; the same code at 50 milliseconds lost all six, matching hard shutdown.

Next Step

These two lessons worked with a single process: the proxy forwarded to a single target, the manager started a single worker, and the memory limit was that one process’s limit. There are two ways to meet more load, and both touch this number: give the process more resources, or open more processes. Both meet the same request and return the same response, and produce very different bills behind the scenes. The next lesson meets the same load with one large process and with four small ones; it measures total memory, startup time, and the multiplier from each process opening its own connection pool — a setting correct for one process sends the store four times the connections once four processes share it.