Capacity Planning

Everything counted across this topic — dropped requests, downtime seconds, rollback steps — was measured in a layout where the resource was a given; nodes, the doubled processes during a release, and the store the migration ran against were always there. This lesson produces the number that builds that layout: how much load one process carries, how many processes are needed, what release and failure headroom adds, and how many nodes that comes to.

Load-testing method was established in Performance Anti-Patterns and Monitoring, and finding a capacity limit by testing in Non-Functional Testing; here that measurement is an input. The course’s rule holds: a wrong estimate raises no error, the measurement shows the difference.

• SD19. Target load is 150 requests per second; a spike brings four times that.
• SD20. The store accepts 16 concurrent connections; a socket past that is closed unread.
• SD21. A node allocates 256 MiB to the application.

The Measured Setup

The store genuinely enforces its connection limit.

// store.mjs — the store that returns loan records; concurrent open connections are capped by LIMIT
import { createServer } from 'node:http';

const { LIMIT = 16, DELAY = 12, PORT = 8830 } = process.env;   // plan.mjs passes these
let open = 0, peak = 0;

const store = createServer((req, res) => {
  if (req.url === '/measure') return res.end(JSON.stringify({ peak }));
  setTimeout(() => res.end('{"record":1}'), Number(DELAY));
});
store.on('connection', (socket) => {          // a socket past the limit is closed unread
  if (open >= Number(LIMIT)) return socket.destroy();
  open += 1;
  peak = Math.max(peak, open);
  socket.on('close', () => { open -= 1; });
});
store.listen(Number(PORT), () => console.log('store ready'));

At the loan endpoint, the pool holds both the store socket and the working buffer; the work per request is fixed.

// service.mjs — a copy of the loan endpoint; POOL sockets and POOL buffers are allocated up front
import { createServer, Agent, get } from 'node:http';
import { createHash } from 'node:crypto';

const { POOL = 2, QUEUE = 32, ROUNDS = 40000, BUFFER = 4194304, STORE = 8830,
  PORT = 8831 } = process.env;                    // plan.mjs passes these
const pool = Number(POOL), rounds = Number(ROUNDS);
const connections = new Agent({ keepAlive: true, maxSockets: pool });
const buffers = Array.from({ length: pool }, () => Buffer.allocUnsafe(Number(BUFFER)).fill(0x2d));
const free = buffers.map((_, i) => i);
const waiting = [];
let longestQueue = 0;

function acquire() {                // no free slot: join the queue; queue full too: -1
  if (free.length) return Promise.resolve(free.pop());
  if (waiting.length >= Number(QUEUE)) return Promise.resolve(-1);
  longestQueue = Math.max(longestQueue, waiting.length + 1);
  return new Promise((c) => waiting.push(c));
}
const release = (y) => { const s = waiting.shift(); if (s) s(y); else free.push(y); };
const storeQuery = () => new Promise((c, r) => get({ port: Number(STORE), path: '/record',
  agent: connections }, (y) => { y.resume(); y.on('end', c); }).on('error', r));

function work(no, slot) {           // fixed work per request: as many digest rounds as `rounds`
  let digest = createHash('sha256').update(String(no)).digest();
  for (let i = 0; i < rounds; i += 1) digest = createHash('sha256').update(digest).digest();
  buffers[slot].write(digest.toString('hex'));
  return buffers[slot].subarray(0, 8).toString('hex');
}

createServer(async (req, res) => {
  if (req.url === '/measure') {
    return res.end(JSON.stringify({ longestQueue, rss: process.memoryUsage().rss }));
  }
  const slot = await acquire();
  if (slot < 0) return res.writeHead(503).end('{"status":"queue full"}');
  try {
    await storeQuery();
    res.end(JSON.stringify({ stamp: work(req.url, slot) }));
  } catch (e) {
    res.writeHead(502).end(JSON.stringify({ error: e.code ?? e.message }));
  } finally { release(slot); }
}).listen(Number(PORT), () => console.log('service ready'));

From Measurement to Estimate

The driver saturates a single process by varying pool size, calculates process and node count from capacity, and runs three estimates under the same load and the same spike. 40,000 digest rounds, a 4 MiB buffer, and the store limit are run-independent; rate and memory are from this run.

// plan.mjs — a single process's capacity is measured per pool size, process and node count are
// calculated, and three estimates are run under the same target load and the same spike
import { spawn } from 'node:child_process';

const TARGET = 150, SPIKE = 4;              // SD: requests per second and spike multiplier
const DURATION = 1800, SPIKE_DURATION = 1000; // measurement and spike durations (ms)
const ROUNDS = 40000, BUFFER = 4 * 1024 * 1024, QUEUE = 32;
const STORE_LIMIT = 16, STORE_DELAY = 12;   // SD: concurrent connections the store accepts
const NODE_MEMORY = 256, HEADROOM = 2;      // SD: node memory (MiB); release + failure headroom

let base = 8830;
const children = [];
const start = (file, env) => new Promise((k, r) => {
  const c = spawn(process.execPath, [file], { stdio: ['ignore', 'pipe', 'inherit'], env: { ...process.env, ...env } });
  children.push(c);
  c.stdout.once('data', k);
  c.once('exit', (code) => r(new Error(`${file} exited with ${code}`)));
});
const teardown = () => { children.forEach((c) => c.kill('SIGKILL')); children.length = 0; };

async function setup(processes, pool) {
  const store = base;
  base += 20;
  await start('store.mjs', { PORT: store, LIMIT: STORE_LIMIT, DELAY: STORE_DELAY });
  const ports = [];
  for (let i = 1; i <= processes; i += 1) {
    await start('service.mjs', { PORT: store + i, STORE: store, POOL: pool, QUEUE, ROUNDS, BUFFER });
    ports.push(store + i);
  }
  return { store, ports };
}
const measure = (p) => fetch(`http://127.0.0.1:${p}/measure`).then((y) => y.json());
const request = (p, i) => fetch(`http://127.0.0.1:${p}/loan/${i}`)
  .then((y) => { y.body.cancel().catch(() => {}); return y.status; }).catch(() => 0);
function table(header, rows) {
  const widths = header.map((b, i) => Math.max(b.length, ...rows.map((s) => String(s[i]).length)));
  for (const r of [header, ...rows]) console.log(r.map((v, i) => String(v).padEnd(widths[i])).join('  ').trimEnd());
}

// open loop: sent at a fixed rate, response codes are tallied
async function trial(processes, pool, rate, duration) {
  const { store, ports } = await setup(processes, pool);
  const tally = { ok: 0, queued: 0, broken: 0 }, inflight = [];
  let debt = 0, n = 0;
  for (let t = 0; t < duration; t += 20) {
    for (debt += (rate * 20) / 1000; debt >= 1; debt -= 1) {
      n += 1;
      inflight.push(request(ports[n % ports.length], n)
        .then((k) => { tally[k === 200 ? 'ok' : k === 503 ? 'queued' : 'broken'] += 1; }));
    }
    await new Promise((k) => setTimeout(k, 20));
  }
  await Promise.all(inflight);
  const perProcess = await Promise.all(ports.map((x) => measure(x)));
  children.slice(1).forEach((c) => c.kill('SIGKILL'));   // measuring the store needs one connection
  await new Promise((k) => setTimeout(k, 150));
  const storeStats = await measure(store);
  teardown();
  return [tally.ok, tally.queued, tally.broken, Math.max(...perProcess.map((x) => x.longestQueue)),
    storeStats.peak, (perProcess.reduce((t, x) => t + x.rss, 0) / 2 ** 20).toFixed(0)];
}

// stage 1: closed loop; the client keeps 16 requests in flight, the pool is the limiting factor
const plan = [];
for (const h of [1, 2, 4, 8]) {
  const { store, ports } = await setup(1, h);
  const deadline = Date.now() + DURATION;
  let n = 0;
  await Promise.all(Array.from({ length: 16 }, async () => {
    while (Date.now() < deadline) if (await request(ports[0], n) === 200) n += 1;
  }));
  const { rss } = await measure(ports[0]);
  const { peak } = await measure(store);
  teardown();
  const rate = n / (DURATION / 1000), forLoad = Math.ceil(TARGET / rate);
  plan.push({ h, rate, mib: rss / 2 ** 20, socket: peak, forLoad, processes: forLoad + HEADROOM, connections: (forLoad + HEADROOM) * h });
}
table(['pool', 'completed/sec', 'store socket', 'rss (MiB)', 'processes for load', '+headroom',
  'total connections', 'store limit'],
  plan.map((p) => [p.h, p.rate.toFixed(1), p.socket, p.mib.toFixed(0), p.forLoad, p.processes, p.connections,
    p.connections <= STORE_LIMIT ? 'passes' : 'exceeds']));
const chosen = plan.filter((p) => p.connections <= STORE_LIMIT).sort((a, b) => a.processes - b.processes)[0];
const perNode = Math.floor(NODE_MEMORY / chosen.mib);
console.log(`\nchosen plan: pool ${chosen.h}, ${chosen.processes} processes (${chosen.forLoad} for load + ${HEADROOM} headroom), `
  + `${chosen.connections} connections, ${Math.ceil(chosen.processes / perNode)} nodes (${perNode} processes per node)`);

// stage 2: three estimates, first at target load then at the spike
const settings = [['low estimate', chosen.forLoad - 1, chosen.h], ['chosen plan', chosen.processes, chosen.h],
  ['high estimate, pool untouched', chosen.processes + 2, 4]];
for (const [name, rate, duration] of [['target load', TARGET, DURATION], ['spike', TARGET * SPIKE, SPIKE_DURATION]]) {
  const rows = [];
  for (const [label, processes, pool] of settings) {
    rows.push([label, processes, pool, processes * pool, ...await trial(processes, pool, rate, duration)]);
  }
  console.log(`\n${name} — ${rate} requests/sec`);
  table(['setting', 'processes', 'pool', 'pool product', '200', '503', '502', 'longest queue',
    'store socket', 'rss'], rows);
}

pool  completed/sec  store socket  rss (MiB)  processes for load  +headroom  total connections  store limit
1     41.7           2             83         4                   6          6                  passes
2     69.4           3             90         3                   5          10                 passes
4     68.9           5             98         3                   5          20                 exceeds
8     70.6           9             112        3                   5          40                 exceeds

chosen plan: pool 2, 5 processes (3 for load + 2 headroom), 10 connections, 3 nodes (2 processes per node)

target load — 150 requests/sec
setting                        processes  pool  pool product  200  503  502  longest queue  store socket  rss
low estimate                   2          2     4             270  0    0    23             4             181
chosen plan                    5          2     10            270  0    0    0              6             449
high estimate, pool untouched  7          4     28            270  0    0    0              7             642

spike — 600 requests/sec
setting                        processes  pool  pool product  200  503  502  longest queue  store socket  rss
low estimate                   2          2     4             279  321  0    32             4             183
chosen plan                    5          2     10            460  140  0    32             10            520
high estimate, pool untouched  7          4     28            351  0    249  17             16            683

The Pool’s Three Bills

At pool one, a process completed 41.7 requests per second; at two it jumped to 69.4, staying in the same band at four and eight. At pool one, the store’s wait cannot overlap with any work; at two, one request waits while the other is processed. Above two, no wait is left to overlap.

The next two columns are the setting’s price: the socket opened at the store rose to 2, 3, 5, and 9, process memory to between 83 and 112 MiB. Pool is not a ceiling, it is the connection count reached at saturation.

From Process to Node

At pool two, ceil(150 / 69.4) = 3 processes suffice for 150 requests; adding release and failure headroom makes it five. Five processes at pool two drop ten connections on the store. Had pool four been chosen, process count would still be five, but connections would rise to twenty — forty at pool eight.

This reverses the calculation’s direction: store limit divided by process count caps the pool, the cap determines capacity, and capacity determines process count.

Node count is a division: a 90 MiB process fits two to a 256 MiB node, five processes make three nodes. There is a mismatch here — headroom was calculated in process terms, but failure arrives in node terms: lose one node and two processes go at once.

A Two-Sided Wrong Estimate

At target load, all three settings look the same: 270 of 270 requests completed, error zero in all three. This is where the wrong estimate stays silent.

The low estimate’s trace is in a single column: with two processes, longest queue climbed to 23, staying zero in the correct plan. Two processes carry roughly 139 requests per second against a target of 150; the accumulating queue fills within a few seconds, shorter than the measurement window. Under the spike, the same setting lost 321 requests. Queue depth is the indicator that arrives before a dropped request and, alone, produces no error.

The high estimate’s only trace at target load is in memory: seven processes held 642 MiB, five held 449. Under the spike, seven processes asked the store for twenty-eight connections; the socket saturated at sixteen, and 249 requests got a 502. The 351 completed requests are fewer than the correct plan’s 460. The correct plan rejected the 140 requests it could not carry with a 503 in its own queue; the high estimate spent a shared resource.

The three inputs sit in three separate layers: process count in the process manager, pool in the application’s environment, connection limit at the store. What gets measured is the product of the first two, and it is written in no file.

Summary

• Pool size determines capacity (41.7 to 69.4 requests), process memory (83 to 112 MiB), and the socket count landing on the store (2 to 9) at once; above two, only the price kept rising.
• The calculation from measurement to plan is three steps — process count for load, release and failure headroom, node count divided by memory: pool 2, 5 processes, 10 connections, 3 nodes.
• The store’s connection limit caps the pool: at the same process count, pool 4 would mean twenty connections, pool 8 forty.
• Headroom is calculated in process terms while failure arrives in node terms; since two processes fit per node, losing one node costs twice the single-process headroom.
• At target load, all three settings met 270 of 270 requests; the low estimate’s trace was a queue climbing to 23, the high estimate’s a 193 MiB sitting idle. Under the spike, the low estimate rejected 321 requests, the high estimate broke 249 when the store socket filled.

Course Wrap-Up

Lesson	Measure of the correct setting	Silent result of the wrong setting	Where the setting lives
Input Validation	Allowlist 0 and 1, blocklist 11 and 4 wrong	0 errors across three wrong settings	0 at the endpoint, 22 wasted steps at the store
Injection Class	Corruption 0 on the separately handled path	Escaped concatenation drifts on 4 of 10 inputs	Decision separate across 12 call sites
Cross-Origin Access	Origin list open to 2 outside origins	Wildcard setting returned 200, the client discarded it	Separate across 5 endpoints, 1 in a single layer
Security Headers	1 write site in a single layer, 0 lines for a new endpoint	One wrong value dropped 10/10 to 0/10	10 files per endpoint, 1 in a single layer
Secrets Management	Window 390 seconds; rotation with no window gets 36 rejections	The leaked key got a 200 at second 900	Key sits in 14 places, 56 steps a year
Dependency Risk	Daily upgrades: 113 exposure days, 237 intervention days	Under a loose range, two deployments diverged on 14 packages	Version range sits in the manifest
Audit Logging	Every decision point answered 8 of 8 questions	Deleting the last 25 records went uncaught	Anchor period: 3,012 with no anchor, 499 at one in 500
Abuse Defense	The layered key stopped 2,446 requests, 0 false rejects	Without normalization, stopped count fell to 2,266	Key selection sits in the limiter
Reverse Proxy	Forwarding on: 11 of 16 requests passed, 4 identities logged	Off: 6 good requests stopped	Forwarding at the proxy, body limit at two layers
Process Managers	Immediate restart answered 20 of 24 requests; 500 ms wait drained 6 requests	Dozens of restarts on a process that never starts (43 in that run)	Policy and wait time in the process manager
Horizontal/Vertical Scaling	Pool cut to two: 60 of 60 requests answered	Pool untouched: connections rose to 32, 30 requests unanswered	Process count and pool sit apart
Stateless Application	21 of 21 requests correct with one copy	12 requests wronged with two copies, no code changed	12 call sites in 55 lines
Session and Stickiness	0 sessions lost in the shared store	Process memory: 8 additions never accumulated, all returned 200	Signing key one per node
Zero-Downtime Deployment	Blue-green and canary both dropped 0 requests	8 in-flight requests dropped under hard shutdown, invisible in the counter	Release layout at the frontend
Migration and Release Order	Expand–contract met 120 of 120 requests in 5 steps	Skipping dual writing left a field empty, request returned 200	Order sits in a written list
Capacity Planning	Pool 2 gives 5 processes, 10 connections, 3 nodes	Under the low estimate the queue rose to 23, under the high, 193 MiB sat idle	Process manager, application, and store

The middle column writes the course’s rule sixteen times over: a wrong configuration raises no error; the measurement shows the difference. A missing header, a key never rotated, a skipped dual write, an over-provisioned process — none threw an exception. What separated right from wrong in every lesson was a counted difference: a passed request, an open origin, an unrecoverable record, idle memory. Every setting gets its measurement written down beside it.

This lesson closes the Server Security and Going to Production course, and the curriculum with it. Server-Side Fundamentals built the request’s path through the server, Web API Design the contract, Authentication and Authorization access, the Data Access Layer persistence, Caching, Queues and Asynchronous Processing the work leaving the request, Service Architectures the boundary, Observability and Reliability visibility; this course built production decisions. What comes next is the reader’s own system: which endpoint’s validator is missing, how many processes run, and which measurement that number came from.