---
title: 'Control Plane Components'
source: 'https://academia.sh/en/courses/orchestration/control-plane-components'
course: 'Container Orchestration'
language: en
updated: '2026-09-13T05:08:26+00:00'
license: 'CC BY-SA 4.0'
objectives:
    - 'Traces a request end to end: how many components, how many store reads, how many writes, how many rounds'
    - 'Measures what it means for components to depend on the state store rather than on each other'
    - "Counts each component's failure separately: which decision stops, what the cluster does not know"
---

# Control Plane Components

The party that takes on the decision is split into four parts, and a single request is traced end to end: how many components it passes through, which information each component reads and writes, and how many reads and writes land on the state store are counted. Then each component is brought down one at a time, and which decisions stop and which keep going is measured.

The previous lesson counted that the placement decision depends on six information items, and that four of these items must be collected continuously. The party that collects, stores, produces the decision, and applies the result is not a single part. This lesson splits that party — the **control plane** — into four components and measures two things: a request's end-to-end trace, and which decisions stop when one component goes down.

The components are named by their duties: the **API server** (the single write gate, plus validation), the **state store** (the durable home for every item), the **scheduler** (the placement decision), the **controller manager** (the work of closing the gap between desired and actual). The node side stands here as a single collective part; the next lesson splits it into its own parts.

The measurement runs, once again, on the **regional measurement network**: **fictional** software that collects and verifies water-meter readings. What runs is not a real cluster but a **model** built with `node`.

**CA7.** The schema is our own: three key prefixes — `service/<name>`, `copy/<name>`, `node/<name>`.
**CA8.** The control plane is four components; the node side is modeled as a single collective part.
**CA9.** The API server is the single write gate and has no state of its own; every write lands on the state store.
**CA10.** A read is a single operation that scans one prefix; the number of items scanned counts as read load.
**CA11.** The starting state is two services (3 collector copies, 2 verifier copies) and three nodes; node capacity is four.
**CA12.** When one component goes down, only that component stops; the network and the nodes stay up. A copy's death is also recorded in the ledger by passing through the API server.

## Four Components, One Write Gate

```js
// plane.mjs — the control plane's four components and the state store. Model; not a real cluster.
// The schema is our own: service/<name>, copy/<name>, node/<name>.
export function create(initial) {
  const data = new Map(Object.entries(initial)); const trace = [];
  let rev = 0, active = "-";
  const d = {
    data, trace, get rev() { return rev; },
    read(prefix) { const c = [...data].filter(([a]) => a.startsWith(prefix));
      trace.push({ b: active, i: "read", a: prefix + "*", n: c.length }); return c; },
    write(a, v) { data.set(a, v); rev++; trace.push({ b: active, i: "write", a, n: 1 }); },
    remove(a) { data.delete(a); rev++; trace.push({ b: active, i: "remove", a, n: 1 }); },
    component(name) { active = name; },
  };
  return d;
}

// 1) API server: the single write gate. Validates the schema, writes to the store. Has no state of its own.
export const apiServer = (d, request) => {
  d.component("API server");
  if (!request.key.match(/^(service|copy|node)\//)) return "out of schema";
  request.value === null ? d.remove(request.key) : d.write(request.key, request.value);
  return "written";
};

// 2) Controller manager: compares desired copy count against actual copy count.
export const controllerManager = (d) => {
  d.component("controller manager"); let work = 0;
  const services = d.read("service/"), all = d.read("copy/");
  for (const [a, v] of all) if (v.status === "dead") { d.remove(a); work++; }   // clean up dead records
  for (const [a, s] of services) {
    const name = a.slice(8), prefix = `copy/${name}-`;
    const k = all.filter(([, v]) => v.service === name && v.status !== "dead");
    let n = Math.max(0, ...all.filter(([b]) => b.startsWith(prefix)).map(([b]) => +b.slice(prefix.length)));
    for (let i = k.length; i < s.desired; i++, work++)
      d.write(prefix + ++n, { service: name, node: null, status: "pending" });
    for (let i = k.length; i > s.desired; i--, work++) d.remove(k[i - 1][0]);
  }
  return work;
};

// 3) Scheduler: picks the emptiest node with enough capacity for nodeless copies.
export const scheduler = (d) => {
  d.component("scheduler"); let work = 0;
  const nodes = d.read("node/").filter(([, v]) => v.health === "alive");
  const copies = d.read("copy/");
  const load = new Map(nodes.map(([a]) => [a, copies.filter(([, v]) => v.node === a).length]));
  for (const [a, k] of copies.filter(([, v]) => v.node === null && v.status === "pending")) {
    const eligible = nodes.filter(([n, v]) => load.get(n) < v.capacity);
    if (!eligible.length) continue;                                  // unsolved: no room
    const s = eligible.reduce((x, y) => (load.get(x[0]) <= load.get(y[0]) ? x : y))[0];
    load.set(s, load.get(s) + 1); d.write(a, { ...k, node: s }); work++;
  }
  return work;
};

// 4) The node agents' collective stand-in: starts an assigned copy and reports its status.
export const nodes = (d) => {
  d.component("node agents"); let work = 0;
  for (const [a, k] of d.read("copy/"))
    if (k.node && k.status === "pending") { d.write(a, { ...k, status: "running" }); work++; }
  return work;
};
```

The point worth noticing is that the components **never call one another**. The controller manager does not notify the scheduler; it writes the copy object into the ledger without a node and leaves it there. The scheduler notifies no one either; it reads nodeless copies, picks a node, writes to the ledger. The four components' shared language is a single place — the state store — and this choice also determines how they behave when something goes down. The **revision** counter the store increments on every write is that language's timestamp: only that number tells a component which moment's ledger it is looking at.

## The End-to-End Trace of a Request

```js
// trace.mjs — the end-to-end trace of a single request: which component reads what, writes what.
import { create, apiServer, controllerManager, scheduler, nodes } from "./plane.mjs";

export const initial = () => {
  const b = { "service/collector": { desired: 3 }, "service/verifier": { desired: 2 } };
  for (let i = 1; i <= 3; i++)
    b[`copy/collector-${i}`] = { service: "collector", node: `m${i}`, status: "running" };
  for (let i = 1; i <= 2; i++)
    b[`copy/verifier-${i}`] = { service: "verifier", node: `m${i}`, status: "running" };
  for (let i = 1; i <= 3; i++) b[`node/m${i}`] = { capacity: 4, health: "alive" };
  return b;
};

if (import.meta.url === `file://${process.argv[1]}`) {   // the third block imports this file
  const d = create(initial());
  apiServer(d, { key: "service/collector", value: { desired: 5 } });
  let round = 0, work;
  do { work = controllerManager(d) + scheduler(d) + nodes(d); round++; } while (work > 0 && round < 8);

  let reads = 0, writes = 0;
  console.log("request: collector service's desired copy count 3 -> 5");
  console.log("#  component               op      key                        items  rd  wr");
  d.trace.forEach((e, i) => {
    e.i === "read" ? (reads += e.n) : writes++;
    console.log(`${String(i + 1).padStart(2)} ${e.b.padEnd(23)} ${e.i.padEnd(7)} ` +
      `${e.a.padEnd(26)} ${String(e.n).padStart(5)} ${String(reads).padStart(4)} ${String(writes).padStart(4)}`);
  });
  const components = new Set(d.trace.map((e) => e.b));
  console.log(`\ncomponents visited ${components.size}   trace steps ${d.trace.length}   store reads ${reads}` +
    `   store writes ${writes}   revision 0 -> ${d.rev}   round ${round}`);
}
```

```
request: collector service's desired copy count 3 -> 5
#  component               op      key                        items  rd  wr
 1 API server              write   service/collector              1    0    1
 2 controller manager      read    service/*                      2    2    1
 3 controller manager      read    copy/*                         5    7    1
 4 controller manager      write   copy/collector-4               1    7    2
 5 controller manager      write   copy/collector-5               1    7    3
 6 scheduler               read    node/*                         3   10    3
 7 scheduler               read    copy/*                         7   17    3
 8 scheduler               write   copy/collector-4               1   17    4
 9 scheduler               write   copy/collector-5               1   17    5
10 node agents             read    copy/*                         7   24    5
11 node agents             write   copy/collector-4               1   24    6
12 node agents             write   copy/collector-5               1   24    7
13 controller manager      read    service/*                      2   26    7
14 controller manager      read    copy/*                         7   33    7
15 scheduler               read    node/*                         3   36    7
16 scheduler               read    copy/*                         7   43    7
17 node agents             read    copy/*                         7   50    7

components visited 4   trace steps 17   store reads 50   store writes 7   revision 0 -> 7   round 2
```

A human wrote a single item: the collector service's desired copy count, 3 instead of 5. The counterpart of that was **four components, seventeen trace steps, 50 item reads, and 7 writes**. The write count is the most instructive number: against the human's 1 write, the plane produced 6 more writes — creating two copy objects, assigning both to a node, reporting both as running. **One item gets written three times**, because each component fills in a separate field of the same object.

The copy object's journey is visible in the rows: at step 4 it is born nodeless and `pending`, at step 9 it gets a node, at step 12 it becomes `running`. Three components touch the same key in sequence, and none of them waits for the other; each one only looks at the ledger and does its own work.

The second round — steps 13 through 17 — carries this lesson's most expensive number: **0 writes, 26 item reads**. After the gap closed, every component still read the ledger from the top and found no work to do. This is the price the cluster pays for "doing nothing," and it never stops; even on a quiet night for the measurement network, the controller manager and the scheduler keep scanning the ledger. Read load grows with item count, not with work done.

## When One Component Goes Down

```js
// matrix.mjs — when one component goes down, which decisions stop, which keep going.
import { create, apiServer, controllerManager, scheduler, nodes } from "./plane.mjs";
import { initial } from "./trace.mjs";

const EVENTS = [["service/collector", { desired: 5 }],                       // scale request
  ["copy/collector-2", { service: "collector", node: "m2", status: "dead" }],  // node report
  ["node/m4", { capacity: 4, health: "alive" }]];                        // new node

function run(down) {
  const d = create(initial());
  if (down === "state store") { d.read = () => []; d.write = () => {}; d.remove = () => {}; }
  for (const [a, v] of EVENTS)
    if (down !== "API server") apiServer(d, { key: a, value: v });
  const reported = d.data.get("copy/collector-2")?.status === "dead";   // was the death recorded
  let round = 0, work;
  do { work = (down === "controller manager" ? 0 : controllerManager(d)) +
    (down === "scheduler" ? 0 : scheduler(d)) + nodes(d); round++; } while (work > 0 && round < 8);

  const k = [...d.data].filter(([a]) => a.startsWith("copy/collector"));
  const count = (t) => k.filter(([, v]) => v.status === t).length;
  const outcome = (done, half) => (done ? "done" : half ? "half" : "halted");
  const believed = count("running"), actual = believed - (reported ? 0 : 1);  // collector-2 had died
  return [down, outcome(believed === 5, count("pending") > 0),
    reported ? outcome(count("dead") === 0 && believed === 5, count("dead") === 0) : "unknown",
    d.data.has("node/m4") ? "done" : "halted",
    down === "API server" || down === "state store" ? "halted" : "done",
    "continues", `${believed}/${actual}`, d.trace.filter((e) => e.i === "read").reduce((t, e) => t + e.n, 0),
    d.trace.filter((e) => e.i !== "read").length];
}

console.log("down component          scale     failure   node      reads     traffic      believed/actual rd wr");
for (const b of ["-", "API server", "state store", "scheduler", "controller manager"]) {
  const r = run(b);
  console.log(`${r[0].padEnd(23)} ${r[1].padEnd(8)} ${r[2].padEnd(9)} ${r[3].padEnd(8)}` +
    ` ${r[4].padEnd(8)} ${r[5].padEnd(11)} ${r[6].padStart(11)} ${String(r[7]).padStart(5)}` +
    ` ${String(r[8]).padStart(3)}`);
}
```

```
down component          scale     failure   node      reads     traffic      believed/actual rd wr
-                       done     done      done     done     continues           5/5    52  13
API server              halted   unknown   halted   halted   continues           3/2    20   0
state store             halted   unknown   halted   halted   continues           3/2     0   0
scheduler               half     half      done     done     continues           2/2    30   7
controller manager      halted   halted    done     done     continues           2/2    14   3
```

**The traffic column reads `continues` on every row.** Even if the entire control plane goes down, running copies keep serving; meter readings keep getting collected, verification keeps running, the nightly batch job still starts. The cluster does not become "unable to work" — it becomes **unable to decide**. This distinction holds for the whole course: the control plane does not do the work, it tells the parties that do the work what to do.

The four failure scenarios produce three distinct failure types.

**When the API server or the state store goes down,** no write lands on the ledger at all. The scale request is lost, the new node's registration is lost, a read request gets no answer. The heaviest row is the last one: `unknown`. Because a copy's death also has to pass through the API server, the report never gets recorded, and the ledger still counts that copy as running — believed 3, actual 2. Even after repair, this gap does not close on its own; what was lost was not a decision but an **event**, and the event itself does not come back. The only way to relearn it is for the node to report the same information again.

**When the scheduler goes down,** the decision gets made but no place gets found. The controller manager sees the missing copies and creates their objects; the objects pile up without a node. The scale request stays `half`: the ledger holds five copies, two of which run nowhere. Repairing this case is cheap — once the scheduler comes back, it reads the accumulated copies and distributes them, because **the work is waiting in the ledger**.

**When the controller manager goes down,** the information is correct, but nothing acts on it. The dead copy's report lands on the ledger, the ledger knows it is dead, but no one creates a replacement. The cluster knows what is wrong and does not fix it. This case, too, closes on repair.

The distinction comes down to one sentence: **not knowing and knowing but not acting are separate failures.** The second closes on its own once the component comes back, because the gap is still on the ledger. The first does not close, because the information needed to close it was never written.

The last two columns give the cost side of this. With everything up, the ledger took 52 item reads and 13 writes; once the scheduler went down, reads fell to 30, and once the controller manager went down, to 14. **Most of the store's load comes from the decision-making components**, not from requests. A single human request produced 1 write; in the same span, the decision components read 52 items.

## Summary

- A single human write (desired copies 3 → 5) passed through four components and produced 17 trace steps, 50 item reads, and 7 store writes; the copy object was written three times by three separate components.
- Components do not call one another, only the state store; the verification round after the gap closed took 0 writes and 26 item reads — read load grows with item count, not with work done.
- Even with the whole control plane down, running copies kept serving; the ability the cluster lost was not working but deciding.
- When the API server or the state store went down, the death report was never recorded, and the ledger kept counting the copy as running: believed 3, actual 2.
- When the scheduler went down, the scale request stayed half-done (5 copies on the ledger, 2 nodeless); when the controller manager went down, the information stayed correct but nothing acted on it — both close on their own with repair, an unrecorded report does not.
- Store reads were 52 with everything up, 30 with the scheduler down, and 14 with the controller manager down; the source of the load was the decision-making components, not requests.

## Next Step

All of this lesson's measurements sat on one side of the ledger: how the decision gets produced and where it gets written. On the ledger's other side stood an assumption — that nodes correctly report what they are running. The last scenario showed a moment where that assumption broke: once a report could not be recorded, the cluster kept believing an old fact. The next lesson splits the node side into its parts — the **node agent, the runtime, the network proxy** — and measures the report itself: what gets reported, how often, with how much delay, and, when a report is lost, for how long the cluster keeps believing the old fact.