The Need for Orchestration

All of the previous course’s measurements ran on a single machine: the output became portable, its identity was derived from its content, and its isolation and authority were counted item by item. One question remained — a human being still decided how many copies of that output would run, on which machine, and when. This lesson measures that decision itself: how it grows with scale when given manually, what a system needs to know when the decision is delegated to it, and where delegating fails to pay off.

Orchestration is the work of delegating placement, replica count, and post-failure restart decisions for workloads running across multiple machines to a system. This course’s unit of measure is the cost of delegation: every delegated decision requires the information it depends on to be produced continuously and on time.

The measurement runs, once again, on the regional measurement network: fictional software that collects, verifies, and bills water-meter readings and opens field work orders; the nightly batch job is part of the same network. What runs here is not a real cluster but a model built with node; the numbers are the model’s numbers, and the model’s assumptions are written below.

CA1. The network consists of four services and each service has three copies; machine count and service count are scanned separately. CA2. The manual step cost is fixed: placing one copy takes 4 steps, preparing one machine takes 6 steps, visually inspecting one copy takes 20 seconds. CA3. Failures come from a 32-bit generator we wrote ourselves; the seed 20260401 appears in the output, and the daily failure probability per copy is 0.15. CA4. A human inspects the copies in sequence, making rounds; two scenarios are run — a continuous watch and a single eight-hour shift. CA5. Under delegated management, preparation per machine is 9 steps instead of 6 (the node agent is also installed), the reporting period is 10 seconds, and the reconciliation period is 15 seconds. CA6. The placement constraint is real: collector copies can run only on machines with a physical connection to the meter network; four of the sixteen machines qualify.

Manual Management Grows at Two Different Speeds

// scale.mjs — scans manual management across machine and service counts. This is a model; a
// real cluster does not run. Time is in seconds, a "step" is one human operation.
export const generator = (seed) => { let s = seed >>> 0;  // our own generator: 32-bit, seed visible
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };

const A_COPY = 4, A_MACHINE = 6, D_TIME = 20, DAY = 86400, SHIFT = 8 * 3600;

export function manual({ M, S, r, failureRate, seed, continuous }) {
  const N = S * r, rnd = generator(seed), T = N * D_TIME;   // T: the duration of one inspection round
  const failures = [];
  for (let i = 0; i < N; i++) if (rnd() < failureRate) failures.push({ i, t: rnd() * DAY });
  let delay = 0;
  for (const a of failures) {
    let t = a.t;
    if (!continuous && t % DAY >= SHIFT) t = Math.floor(t / DAY) * DAY + DAY; // outside the shift
    const position = a.i * D_TIME;                          // the copy's position within the round
    let d = Math.ceil((t - position) / T) * T + position;
    if (d < t) d += T;
    if (!continuous && d % DAY >= SHIFT) d = Math.floor(d / DAY) * DAY + DAY + position;
    delay += d - a.t;
  }
  const decision = N + S + failures.length;                 // placement + how many copies + re-place
  return { N, decision, scan: (N + failures.length) * M, roundDuration: T,
    steps: A_MACHINE * M + A_COPY * N + A_COPY * failures.length, failureCount: failures.length,
    avgDelay: failures.length ? delay / failures.length : 0 };
}

if (import.meta.url === `file://${process.argv[1]}`) {        // the second block imports this file
  console.log("M  S  copies  decisions  scan  steps  round(min)  failures  delay(min)  delay-single-shift");
  for (const [M, S] of [[1,4],[2,4],[4,6],[8,8],[16,12],[32,16]]) {
    const g = { M, S, r: 3, failureRate: 0.15, seed: 20260401 };
    const k = manual({ ...g, continuous: true }), v = manual({ ...g, continuous: false });
    console.log(`${String(M).padStart(2)} ${String(S).padStart(2)} ${String(k.N).padStart(6)}` +
      `${String(k.decision).padStart(7)}${String(k.scan).padStart(7)}${String(k.steps).padStart(6)}` +
      `${(k.roundDuration/60).toFixed(1).padStart(9)}${String(k.failureCount).padStart(7)}` +
      `${(k.avgDelay/60).toFixed(1).padStart(13)}${(v.avgDelay/60).toFixed(1).padStart(21)}`);
  }
}

M  S  copies  decisions  scan  steps  round(min)  failures  delay(min)  delay-single-shift
 1  4     12     21     17    74      4.0      5          1.2                394.8
 2  4     12     21     34    80      4.0      5          1.2                394.8
 4  6     18     29     92   116      6.0      5          2.4                394.8
 8  8     24     38    240   168      8.0      6          3.6                330.2
16 12     36     56    704   272     12.0      8          6.5                363.5
32 16     48     74   1856   424     16.0     10          7.7                324.5

Two columns grow at two different speeds. Decision count is linear: from 21 to 74, three and a half times. Manual steps are linear too: from 74 to 424. Looking at these two numbers alone, someone could conclude that manual management scales without trouble, because copy count also grew by exactly a factor of four.

The scan column, over the same scale, rises from 17 to 1,856 — one hundred and nine times. Scan is how many machines’ state gets examined while making one placement decision. Decision count grows with copy count; the information examined per decision grows with machine count; the product of the two does not grow at the same speed as decision count itself. What makes manual management expensive is not the decision count but the information-gathering work underneath every decision.

The third measure is detection delay. Under a continuous watch, delay rises from 1.2 minutes to 7.7 minutes and looks linear: one inspection round lengthens with copy count, and average wait runs about half a round. The single-shift column says something entirely different — from 394.8 to 324.5 minutes, meaning it does not grow with scale; it even falls. This is not a computation error but a change in the dominant term: under a single shift, the source of the delay is not round duration but the night itself. The measurement network’s nightly batch job starts at 02:30; a collector copy that goes down around then stays down until the shift opens, and adding more copies does not lengthen this wait.

The Threshold for Delegation

// threshold.mjs — compares manual management against delegated management on the same items.
import { manual, generator } from "./scale.mjs";
const SETUP = 120, REPORT = 9, A_NODE = 9, A_COPY = 4, DAY = 86400;
const P = 10, U = 15;                                    // reporting and reconciliation period (seconds)

function delegated({ M, S, r, failureRate, seed }) {
  const N = S * r, rnd = generator(seed), failures = [];
  for (let i = 0; i < N; i++) if (rnd() < failureRate) failures.push({ i, t: rnd() * DAY });
  let g = 0;
  for (const a of failures) g += Math.ceil(Math.ceil(a.t / P) * P / U) * U - a.t;
  return { setup: SETUP + REPORT * S + A_NODE * M, failureCount: failures.length,
    avgDelay: failures.length ? g / failures.length : 0 };
}

console.log("copies  manual-setup  manual/day  deleg-setup  threshold(day)  manual-delay(min)  deleg-delay(sec)");
for (const [M, S] of [[1,2],[2,4],[4,6],[8,8],[16,12],[32,16]]) {
  const g = { M, S, r: 3, failureRate: 0.15, seed: 20260401 };
  const e = manual({ ...g, continuous: true }), d = delegated(g);
  const eSetup = e.steps - A_COPY * e.failureCount, eDay = A_COPY * e.failureCount;
  console.log(`${String(e.N).padStart(6)}${String(eSetup).padStart(14)}${String(eDay).padStart(12)}` +
    `${String(d.setup).padStart(13)}${Math.ceil((d.setup - eSetup) / eDay).toString().padStart(16)}` +
    `${(e.avgDelay/60).toFixed(1).padStart(19)}${d.avgDelay.toFixed(1).padStart(18)}`);
}

console.log("\nfailure rate  failures/day  manual/day  deleg-setup  threshold(day)   [M=8, S=8, 24 copies]");
for (const o of [0, 0.02, 0.05, 0.10, 0.15, 0.30]) {
  const g = { M: 8, S: 8, r: 3, failureRate: o, seed: 20260401 };
  const e = manual({ ...g, continuous: true }), d = delegated(g);
  const eSetup = e.steps - A_COPY * e.failureCount, eDay = A_COPY * e.failureCount;
  console.log(`${o.toFixed(2).padStart(11)}${String(e.failureCount).padStart(11)}` +
    `${String(eDay).padStart(10)}${String(d.setup).padStart(11)}` +
    `${(eDay ? String(Math.ceil((d.setup - eSetup) / eDay)) : "infinite").padStart(11)}`);
}

// Unwritten constraint: collector copies can run only on machines connected to the meter network.
const machines = Array.from({ length: 16 }, (_, i) => ({ name: i, meterNetwork: i % 5 === 0 }));
const copies = [...Array(6).fill("collector"), ...Array(4).fill("verifier"),
  ...Array(3).fill("billing"), ...Array(3).fill("work-order")];
for (const written of [false, true]) {
  const load = machines.map(() => 0), placement = [];
  for (const k of copies) {
    const fit = machines.filter((m) => !(written && k === "collector") || m.meterNetwork);
    const s = fit.reduce((a, b) => (load[a.name] <= load[b.name] ? a : b));
    load[s.name]++; placement.push({ k, m: s });
  }
  const wrong = placement.filter((y) => y.k === "collector" && !y.m.meterNetwork).length;
  console.log(`constraint reported ${written ? "written  " : "unwritten"}: copies ${placement.length}` +
    `   wrong placement ${wrong}   fix steps ${wrong * A_COPY}` +
    `   report items ${written ? 5 : 4}`);
}

copies  manual-setup  manual/day  deleg-setup  threshold(day)  manual-delay(min)  deleg-delay(sec)
     6            30           4          147              30                1.0               2.0
    12            60          20          174               6                1.2               8.1
    18            96          20          210               6                2.4               8.1
    24           144          24          264               5                3.6               8.7
    36           240          32          372               5                6.5               9.1
    48           384          40          552               5                7.7               8.0

failure rate  failures/day  manual/day  deleg-setup  threshold(day)   [M=8, S=8, 24 copies]
       0.00          0         0        264   infinite
       0.02          2         8        264         15
       0.05          3        12        264         10
       0.10          6        24        264          5
       0.15          6        24        264          5
       0.30         10        40        264          3
constraint reported unwritten: copies 16   wrong placement 4   fix steps 16   report items 4
constraint reported written  : copies 16   wrong placement 0   fix steps 0   report items 5

The setup cost of delegating is higher than manual setup at every scale: 147 against 30 steps at 6 copies, 552 against 384 at 48 copies. Delegating is always a loss on day one. The difference shows up in the daily item: under manual management, failures are handled by hand every day; under delegated management, the daily manual step count is zero. The threshold column gives the day on which the two costs cross.

This threshold drops fast with scale. In a six-copy setup, manual management stays cheaper for thirty days; at twelve copies that falls to six days, and above twenty-four it falls to five. Thirty days is a real answer for a single-service trial environment or a one-month pilot — at that scale, delegating is unnecessary work. Five days is not an answer; the measurement network’s four services and twelve copies sit in the region where delegating the decision pays for itself within a week.

The delay columns show a sharper split than the threshold does: 1.0–7.7 minutes under manual management, 2.0–9.1 seconds under delegated management. Under delegated management, this number does not grow with scale, because the source of the delay is not scanning but two fixed periods — the reporting period and the reconciliation period. As scale grows, manual management’s real loss accumulates not in step count but here.

The Six Items the System Needs to Know

The delegated decision can be written in one sentence: which copy runs on which machine, and what happens when it goes down. For the system to make this decision, it must continuously know the following items.

Information item	Source	If it goes stale
machine list	node registration	placement onto a machine that no longer exists
machine capacity	node report	placement onto a machine whose capacity is full
machine health	periodic report	assuming a copy is running on a dead machine
desired copy count	human-written entry	too few or too many copies
copies’ actual state	report from the node	detecting a dropped copy late
placement constraints	human-written entry	placement onto a machine that cannot run it

Four of the six items are measurements the system collects itself; two are items a human must write. This split is also a split in cost. The four collected items produce a continuous reporting load: every machine must report its capacity, its health, and the state of the copies on it in every period, and delegated management’s nine-second detection delay is exactly the cost of that period. Widening the period reduces the load and widens the delay. The two written items, by contrast, are one-time, but if left unmaintained they go quietly wrong — once the desired copy count is written and forgotten, the system keeps working toward that number and no one suspects it.

The cost of the last item sits in the output’s final two lines. While the constraint was unwritten, the placer put four of the six collector copies on machines without a meter-network connection; those four copies start but cannot pull readings. Once the constraint was added to the report, wrong placement dropped to zero and the report’s item count rose from four to five. One written item prevented four wrong placements — this is exactly the cost of delegation, and it is paid up front.

Three Cases Where Delegating Does Not Pay Off

The first is below the threshold: in a six-copy setup, delegating is a loss for thirty days, and if the setup will not live thirty days, it never pays off at all.

The second is a low failure rate, and the second scan gives it a number. In the same twenty-four-copy setup, the threshold is three days at a failure rate of 0.30; it rises to ten days at 0.05, to fifteen days at 0.02, and becomes infinite once the rate hits zero — a setup with no failures at all never earns delegating back. Delegating the decision makes failure cheaper, not the absence of failure; the entire gain comes from the failure item, and the gain shrinks as that item shrinks.

The third, and the most expensive, is that the information the decision depends on is not writable. The constraint experiment was the easy version of this: the constraint could be written in one line. There are constraints that cannot be written — a machine being physically reachable by the field crew, a service having to sit in a specific building under a contract with a specific customer. As long as these items never enter a report, the system stays unaware of them, and every item it is unaware of produces a class of wrong decisions. Under manual management, this information sits in a person’s head and looks like it costs nothing; once delegated, it shows up as a cost for the first time. The decision to delegate is therefore as much a writability decision as a scale decision: how much of the information feeding the decision can actually go into a file.

Summary

• Under manual management, decision count grew linearly (21 → 74) while information examined per decision grew as a product (17 → 1,856); what was expensive was not the decision but the information-gathering work underneath it.
• Failure-detection delay grew linearly with round duration under a continuous watch (1.2 → 7.7 minutes); under a single shift it stayed in the 325–395-minute range and was independent of scale, because the dominant term was the night itself.
• Delegating is a loss at every scale on day one; the threshold is 30 days at 6 copies, 6 days at 12 copies, and 5 days above 24 copies.
• Under delegated management, detection delay stayed in the 2–9 second range and did not grow with scale; its source was not scanning but two fixed periods.
• The placement decision depends on six information items; when the constraint item went unwritten, four of six collector copies were placed on machines that could not run them, and one written item brought that to zero.

Next Step

Delegating the decision requires a system in exchange, and that system has to collect and store the six information items, produce the decision, and apply the result. None of this work sits in a single part. The next lesson splits the party that takes on the decision — the control plane — into its parts: the API server, the scheduler, the controller manager, and the state store. What gets measured is how many components a request passes through end to end, which information each component reads and writes, and, when one component goes down, which decisions stop and which keep going.