Capacity Planning

The previous lesson used three tests to measure what a single process does at today’s load, and never asked one question: how many are needed. The number the load test gives is not a limit but a confirmation — 514 requests/s is met. The limit itself is not remeasured in this lesson; the Introduction to System Design course measured a process’s saturation throughput and built the queuing model showing that latency does not grow linearly as utilization rises. Both are inputs here.

This lesson turns a measurement into a forecast. The input is a rate and an upper bound; the output is one number: how many nodes. The one decision in between is a ratio.

Headroom and Two Constraints

Headroom is the portion of a resource deliberately left unused, and it is written as a utilization ceiling: 30% headroom means utilization must not pass 70%. The rationale for headroom is in K01’s queuing model — as utilization rises, latency grows faster than linearly, so headroom is not waste but a latency decision.

Two separate constraints determine the node count, and the larger one wins. The load constraint: the number of nodes needed for the peak rate to be met below the utilization ceiling. The failure constraint: the number needed for the remaining nodes to stay below that same ceiling once one node drops — which means the remaining nodes must meet the load constraint, so there is always one extra node.

The calculation below has these inputs: peak rate 513.89 requests/s (K01, computed); saturation throughput per node 1,176.47 requests/s — since the previous lesson’s server spends 0.85 ms of processor time per request, this follows from that arithmetic on a single thread, it is not a measurement; K01’s measured 4,268 requests/s at the bare edge is used in the sensitivity line. Monthly demand growth (IZ8) and accumulation’s effect on effective capacity (IZ9) are assumptions.

// capacity/nodes.mjs — from measured saturation throughput to node count. Headroom is a
// utilization ceiling; node count is the greater of the load constraint and the failure
// constraint. All of it is arithmetic.
const PEAK = 513.89;        // K01 (computed): peak edge requests/s
const BATCH = 833.33;       // K01 (computed): end-of-day job's scan rate (records/s)
const RECORD_MS = 1.20;     // K01 (computed): batch job time per record, single worker
const MU = 1 / 0.00085;     // computed: previous lesson's server spends 0.85 ms processor per request
const MU_BARE = 4268;       // K01 (measurement): bare tracking endpoint's saturation throughput, on that machine
const DEGRADED = 0.7;       // IZ9 (assumption): accumulation drops a node's effective capacity by 30%
const GROWTH = 0.06;        // IZ8 (assumption): monthly demand growth
const b = (x, n = 2) => x.toFixed(n);

const loadConstraint = (peak, mu, ceiling) => Math.ceil(peak / (mu * ceiling));
const failureConstraint = (peak, mu, ceiling) => loadConstraint(peak, mu, ceiling) + 1;
const latencyMs = (mu, lambda) => 1000 / (mu - lambda);   // K01's queuing MODEL, as an input

console.log(`saturation throughput per node ${b(MU)} requests/s (computed)`);
console.log(`${"peak requests/s".padStart(16)}${"utilization ceiling".padStart(21)}${"load constraint".padStart(17)}` +
  `${"failure constraint".padStart(20)}${"chosen n".padStart(10)}${"utilization".padStart(13)}` +
  `${"utilization at loss".padStart(21)}${"latency at loss".padStart(17)}`);
for (const peak of [PEAK, PEAK * 3])
  for (const ceiling of [0.5, 0.6, 0.7, 0.8, 0.9]) {
    const n = Math.max(loadConstraint(peak, MU, ceiling), failureConstraint(peak, MU, ceiling));
    const atLoss = peak / (n - 1);
    console.log(`${b(peak).padStart(16)}${(b(ceiling * 100, 0) + "%").padStart(21)}` +
      `${String(loadConstraint(peak, MU, ceiling)).padStart(17)}${String(failureConstraint(peak, MU, ceiling)).padStart(20)}` +
      `${String(n).padStart(10)}${(b((peak / (n * MU)) * 100, 1) + "%").padStart(13)}` +
      `${(b((atLoss / MU) * 100, 1) + "%").padStart(21)}${(b(latencyMs(MU, atLoss)) + " ms").padStart(17)}`);
  }

console.log(`\nredundancy premium shrinks with scale (utilization ceiling 70%):`);
console.log(`${"scale".padStart(6)}${"peak requests/s".padStart(17)}${"load constraint".padStart(17)}` +
  `${"chosen n".padStart(10)}${"redundancy premium".padStart(20)}${"monthly node-hours".padStart(20)}`);
for (const k of [1, 2, 3, 4, 6, 10]) {
  const peak = PEAK * k;
  const y = loadConstraint(peak, MU, 0.7);
  const n = Math.max(y, failureConstraint(peak, MU, 0.7));
  console.log(`${(k + "x").padStart(6)}${b(peak).padStart(17)}${String(y).padStart(17)}` +
    `${String(n).padStart(10)}${(b((100 * (n - y)) / y, 1) + "%").padStart(20)}` +
    `${(n * 720).toLocaleString("en-US").padStart(20)}`);
}

console.log(`\ndemand growth (monthly ${b(GROWTH * 100, 0)}%, utilization ceiling 70%):`);
console.log(`${"month".padStart(6)}${"peak requests/s".padStart(17)}${"chosen n".padStart(10)}` +
  `${"utilization".padStart(13)}${"n with degraded node".padStart(23)}`);
for (const month of [0, 6, 12, 18, 24]) {
  const peak = PEAK * (1 + GROWTH) ** month;
  const n = Math.max(loadConstraint(peak, MU, 0.7), failureConstraint(peak, MU, 0.7));
  const nDeg = Math.max(loadConstraint(peak, MU * DEGRADED, 0.7), failureConstraint(peak, MU * DEGRADED, 0.7));
  console.log(`${String(month).padStart(6)}${b(peak).padStart(17)}${String(n).padStart(10)}` +
    `${(b((peak / (n * MU)) * 100, 1) + "%").padStart(13)}${String(nDeg).padStart(23)}`);
}

console.log(`\nsensitivity: if saturation throughput per node were ${MU_BARE} requests/s (K01's bare edge)`);
console.log(`  at today's peak n = ${Math.max(loadConstraint(PEAK, MU_BARE, 0.7), failureConstraint(PEAK, MU_BARE, 0.7))}` +
  `, at ${b(MU)} requests/s n = ${Math.max(loadConstraint(PEAK, MU, 0.7), failureConstraint(PEAK, MU, 0.7))}` +
  ` (${b(MU_BARE / MU)}x the throughput, same node count)`);
const worker = Math.max(loadConstraint(BATCH, 1000 / RECORD_MS, 0.7), 1);
console.log(`the batch job is a separate capacity question: ${BATCH} records/s needed, one worker does ` +
  `${b(1000 / RECORD_MS)} records/s -> at a 70% ceiling ${worker} workers, the window narrows to ` +
  `${b((BATCH * 4) / (worker * (1000 / RECORD_MS)), 2)} hours`);

saturation throughput per node 1176.47 requests/s (computed)
 peak requests/s  utilization ceiling  load constraint  failure constraint  chosen n  utilization  utilization at loss  latency at loss
          513.89                  50%                1                   2         2        21.8%                43.7%          1.51 ms
          513.89                  60%                1                   2         2        21.8%                43.7%          1.51 ms
          513.89                  70%                1                   2         2        21.8%                43.7%          1.51 ms
          513.89                  80%                1                   2         2        21.8%                43.7%          1.51 ms
          513.89                  90%                1                   2         2        21.8%                43.7%          1.51 ms
         1541.67                  50%                3                   4         4        32.8%                43.7%          1.51 ms
         1541.67                  60%                3                   4         4        32.8%                43.7%          1.51 ms
         1541.67                  70%                2                   3         3        43.7%                65.5%          2.47 ms
         1541.67                  80%                2                   3         3        43.7%                65.5%          2.47 ms
         1541.67                  90%                2                   3         3        43.7%                65.5%          2.47 ms

redundancy premium shrinks with scale (utilization ceiling 70%):
 scale  peak requests/s  load constraint  chosen n  redundancy premium  monthly node-hours
    1x           513.89                1         2              100.0%               1,440
    2x          1027.78                2         3               50.0%               2,160
    3x          1541.67                2         3               50.0%               2,160
    4x          2055.56                3         4               33.3%               2,880
    6x          3083.34                4         5               25.0%               3,600
   10x          5138.90                7         8               14.3%               5,760

demand growth (monthly 6%, utilization ceiling 70%):
 month  peak requests/s  chosen n  utilization   n with degraded node
     0           513.89         2        21.8%                      2
     6           728.96         2        31.0%                      3
    12          1034.05         3        29.3%                      3
    18          1466.82         3        41.6%                      4
    24          2080.71         4        44.2%                      5

sensitivity: if saturation throughput per node were 4268 requests/s (K01's bare edge)
  at today's peak n = 2, at 1176.47 requests/s n = 2 (3.63x the throughput, same node count)
the batch job is a separate capacity question: 833.33 records/s needed, one worker does 833.33 records/s -> at a 70% ceiling 2 workers, the window narrows to 2.00 hours

Headroom Has No Effect at Today’s Load

The first table’s first five rows give an unexpected result. Even as the utilization ceiling is raised from 50% to 90%, the node count stays at 2, utilization stays at 21.8%, and utilization at the loss of one node stays at 43.7%. At today’s peak rate, the choice of headroom changes nothing, because the load constraint gives 1 node at every ceiling, and the constraint that decides is the failure constraint.

This is the most often overlooked side of capacity planning. The question “how many nodes are needed” has two independent answers; one comes from load, the other from failure behavior. A single node meets today’s load with room to spare, but when that single node drops, capacity is zero, and K01’s outage budget cannot exceed that one node’s availability. What makes the node count two is not the load — it is this constraint.

The next five rows show when headroom becomes binding. When the peak rate triples — K01’s peak factor was also three — a ceiling of 70% and above needs 3 nodes, while 50% and 60% need 4. What the fourth node buys is in the last two columns: at the loss of one node, utilization is 43.7% instead of 65.5%, and the latency the queuing model gives is 1.51 ms instead of 2.47 ms. Headroom looks like a node-count decision, and it is a failure-day latency decision.

Redundancy Premium, Growth, and Accumulation

The second table gives the failure constraint’s bill alongside scale. At today’s scale, the extra node doubles the fleet: a redundancy premium of 100%. At ten times the scale, the same rule adds only 14.3%. Redundancy is disproportionately expensive in small systems, and this is one of scale’s few defensible advantages. The right column writes the resource on a monthly basis: 1,440 node-hours today, 5,760 node-hours at ten times the scale. This course uses a resource unit in place of a currency.

The third table shows that capacity is not a number but a time series. At 6% monthly growth, the node count climbs to 3 by the twelfth month and to 4 by the twenty-fourth; utilization drops with each addition and climbs back up. The last column accounts for the previous lesson’s soak-test finding: if a node’s effective capacity drops 30% because of accumulation (IZ9), the third node is needed right away instead of at month six, and the count reaches five nodes by month twenty-four. A soak test’s finding, if it had not been run, gets written into the capacity plan as an extra node.

Performance Does Not Lower the Node Count

The sensitivity line is the lesson’s harshest result. If saturation throughput per node were 4,268 requests/s — K01’s measurement at the bare edge — instead of 1,176.47, that is, an edge 3.63x as fast, the required node count would still be 2. At today’s scale, a performance improvement’s resource payoff is zero, because the deciding constraint is failure, not performance. Translating a performance effort into capacity only makes sense once the load constraint is binding; in this table, that only happens after the fourth row.

The last line is a reminder that capacity is asked per flow. End-of-day billing needs 833.33 records per second, and a single worker’s rate is exactly 833.33 records/s — that is, zero headroom. At a 70% ceiling, two workers are needed, and in exchange, the four-hour window narrows to two hours. The tracking query’s node count never affects this calculation: the two flows are part of the same system, and each asks its own capacity question.

Summary

• Headroom is a utilization ceiling; its rationale is K01’s queuing model, and it is not waste but a latency decision.
• Node count is the greater of the load constraint and the failure constraint; the failure constraint always demands one extra node, because the remaining nodes must also meet the same ceiling.
• At today’s peak of 513.89 requests/s, the choice of headroom has no effect at all: every ceiling gives 2 nodes, utilization 21.8%. When the peak triples, a 50% ceiling gives 4 nodes and a 70% ceiling gives 3, and the difference is a failure-day latency (1.51 ms instead of 2.47 ms).
• The redundancy premium shrinks with scale: at today’s scale, the extra node grows the fleet by 100%; at ten times the scale, by 14.3%; the resource goes from 1,440 node-hours a month to 5,760.
• Capacity is a time series: at 6% monthly growth, the node count reaches 3 by month 12 and 4 by month 24; if accumulation drops effective capacity by 30%, that same calendar becomes 3 and 5.
• Improving performance 3.63x does not change the node count at today’s scale; translating performance into capacity only makes sense once the load constraint is binding. The batch job is a separate question: at a 70% ceiling, two workers narrow the window from four hours to two.

Next Step

This lesson produced one number: how many nodes. Alongside that number is a resource unit — 1,440 node-hours a month today, 5,760 at ten times the scale. But what those nodes cost was never asked. A node-hour is not a single line item; a request carries data over the network, a record gets stored, a metric gets produced and stored, a cache sits in memory. More importantly, every design decision made across all the courses up to this point moved one of these line items, and none of them was counted as operating cost: what does raising the cache hit ratio reduce, what does partitioning the data multiply, what line item does adding a replica multiply and by how much, in which line item does monitoring’s own cost show up. The next lesson takes up this question and answers it in resource units.