--- title: 'Capacity Planning' source: 'https://academia.sh/en/courses/backend-production/capacity-planning' course: 'Server Security and Going to Production' language: en updated: '2026-08-23T07:00:26+00:00' license: 'CC BY-SA 4.0' --- # Capacity Planning The calculation that goes from measured load to process and node count: a single process's capacity measured against pool size, the process, connection, and node counts that follow from it, how the store's connection limit determines pool size working backward, and the difference low and high estimates produce — invisible at target load, visible under a spike. Everything counted across this topic — dropped requests, downtime seconds, rollback steps — was measured in a layout where the resource was a given; nodes, the doubled processes during a release, and the store the migration ran against were always there. This lesson produces the number that builds that layout: how much load one process carries, how many processes are needed, what release and failure headroom adds, and how many nodes that comes to. Load-testing method was established in **Performance Anti-Patterns and Monitoring**, and finding a capacity limit by testing in **Non-Functional Testing**; here that measurement is an input. The course's rule holds: a wrong estimate raises no error, the measurement shows the difference. - **SD19.** Target load is 150 requests per second; a spike brings four times that. - **SD20.** The store accepts 16 concurrent connections; a socket past that is closed unread. - **SD21.** A node allocates 256 MiB to the application. ## The Measured Setup The store genuinely enforces its connection limit. ```js // store.mjs — the store that returns loan records; concurrent open connections are capped by LIMIT import { createServer } from 'node:http'; const { LIMIT = 16, DELAY = 12, PORT = 8830 } = process.env; // plan.mjs passes these let open = 0, peak = 0; const store = createServer((req, res) => { if (req.url === '/measure') return res.end(JSON.stringify({ peak })); setTimeout(() => res.end('{"record":1}'), Number(DELAY)); }); store.on('connection', (socket) => { // a socket past the limit is closed unread if (open >= Number(LIMIT)) return socket.destroy(); open += 1; peak = Math.max(peak, open); socket.on('close', () => { open -= 1; }); }); store.listen(Number(PORT), () => console.log('store ready')); ``` At the loan endpoint, the pool holds both the store socket and the working buffer; the work per request is fixed. ```js // service.mjs — a copy of the loan endpoint; POOL sockets and POOL buffers are allocated up front import { createServer, Agent, get } from 'node:http'; import { createHash } from 'node:crypto'; const { POOL = 2, QUEUE = 32, ROUNDS = 40000, BUFFER = 4194304, STORE = 8830, PORT = 8831 } = process.env; // plan.mjs passes these const pool = Number(POOL), rounds = Number(ROUNDS); const connections = new Agent({ keepAlive: true, maxSockets: pool }); const buffers = Array.from({ length: pool }, () => Buffer.allocUnsafe(Number(BUFFER)).fill(0x2d)); const free = buffers.map((_, i) => i); const waiting = []; let longestQueue = 0; function acquire() { // no free slot: join the queue; queue full too: -1 if (free.length) return Promise.resolve(free.pop()); if (waiting.length >= Number(QUEUE)) return Promise.resolve(-1); longestQueue = Math.max(longestQueue, waiting.length + 1); return new Promise((c) => waiting.push(c)); } const release = (y) => { const s = waiting.shift(); if (s) s(y); else free.push(y); }; const storeQuery = () => new Promise((c, r) => get({ port: Number(STORE), path: '/record', agent: connections }, (y) => { y.resume(); y.on('end', c); }).on('error', r)); function work(no, slot) { // fixed work per request: as many digest rounds as `rounds` let digest = createHash('sha256').update(String(no)).digest(); for (let i = 0; i < rounds; i += 1) digest = createHash('sha256').update(digest).digest(); buffers[slot].write(digest.toString('hex')); return buffers[slot].subarray(0, 8).toString('hex'); } createServer(async (req, res) => { if (req.url === '/measure') { return res.end(JSON.stringify({ longestQueue, rss: process.memoryUsage().rss })); } const slot = await acquire(); if (slot < 0) return res.writeHead(503).end('{"status":"queue full"}'); try { await storeQuery(); res.end(JSON.stringify({ stamp: work(req.url, slot) })); } catch (e) { res.writeHead(502).end(JSON.stringify({ error: e.code ?? e.message })); } finally { release(slot); } }).listen(Number(PORT), () => console.log('service ready')); ``` ## From Measurement to Estimate The driver saturates a single process by varying pool size, calculates process and node count from capacity, and runs three estimates under the same load and the same spike. 40,000 digest rounds, a 4 MiB buffer, and the store limit are run-independent; rate and memory are from this run. ```js // plan.mjs — a single process's capacity is measured per pool size, process and node count are // calculated, and three estimates are run under the same target load and the same spike import { spawn } from 'node:child_process'; const TARGET = 150, SPIKE = 4; // SD: requests per second and spike multiplier const DURATION = 1800, SPIKE_DURATION = 1000; // measurement and spike durations (ms) const ROUNDS = 40000, BUFFER = 4 * 1024 * 1024, QUEUE = 32; const STORE_LIMIT = 16, STORE_DELAY = 12; // SD: concurrent connections the store accepts const NODE_MEMORY = 256, HEADROOM = 2; // SD: node memory (MiB); release + failure headroom let base = 8830; const children = []; const start = (file, env) => new Promise((k, r) => { const c = spawn(process.execPath, [file], { stdio: ['ignore', 'pipe', 'inherit'], env: { ...process.env, ...env } }); children.push(c); c.stdout.once('data', k); c.once('exit', (code) => r(new Error(`${file} exited with ${code}`))); }); const teardown = () => { children.forEach((c) => c.kill('SIGKILL')); children.length = 0; }; async function setup(processes, pool) { const store = base; base += 20; await start('store.mjs', { PORT: store, LIMIT: STORE_LIMIT, DELAY: STORE_DELAY }); const ports = []; for (let i = 1; i <= processes; i += 1) { await start('service.mjs', { PORT: store + i, STORE: store, POOL: pool, QUEUE, ROUNDS, BUFFER }); ports.push(store + i); } return { store, ports }; } const measure = (p) => fetch(`http://127.0.0.1:${p}/measure`).then((y) => y.json()); const request = (p, i) => fetch(`http://127.0.0.1:${p}/loan/${i}`) .then((y) => { y.body.cancel().catch(() => {}); return y.status; }).catch(() => 0); function table(header, rows) { const widths = header.map((b, i) => Math.max(b.length, ...rows.map((s) => String(s[i]).length))); for (const r of [header, ...rows]) console.log(r.map((v, i) => String(v).padEnd(widths[i])).join(' ').trimEnd()); } // open loop: sent at a fixed rate, response codes are tallied async function trial(processes, pool, rate, duration) { const { store, ports } = await setup(processes, pool); const tally = { ok: 0, queued: 0, broken: 0 }, inflight = []; let debt = 0, n = 0; for (let t = 0; t < duration; t += 20) { for (debt += (rate * 20) / 1000; debt >= 1; debt -= 1) { n += 1; inflight.push(request(ports[n % ports.length], n) .then((k) => { tally[k === 200 ? 'ok' : k === 503 ? 'queued' : 'broken'] += 1; })); } await new Promise((k) => setTimeout(k, 20)); } await Promise.all(inflight); const perProcess = await Promise.all(ports.map((x) => measure(x))); children.slice(1).forEach((c) => c.kill('SIGKILL')); // measuring the store needs one connection await new Promise((k) => setTimeout(k, 150)); const storeStats = await measure(store); teardown(); return [tally.ok, tally.queued, tally.broken, Math.max(...perProcess.map((x) => x.longestQueue)), storeStats.peak, (perProcess.reduce((t, x) => t + x.rss, 0) / 2 ** 20).toFixed(0)]; } // stage 1: closed loop; the client keeps 16 requests in flight, the pool is the limiting factor const plan = []; for (const h of [1, 2, 4, 8]) { const { store, ports } = await setup(1, h); const deadline = Date.now() + DURATION; let n = 0; await Promise.all(Array.from({ length: 16 }, async () => { while (Date.now() < deadline) if (await request(ports[0], n) === 200) n += 1; })); const { rss } = await measure(ports[0]); const { peak } = await measure(store); teardown(); const rate = n / (DURATION / 1000), forLoad = Math.ceil(TARGET / rate); plan.push({ h, rate, mib: rss / 2 ** 20, socket: peak, forLoad, processes: forLoad + HEADROOM, connections: (forLoad + HEADROOM) * h }); } table(['pool', 'completed/sec', 'store socket', 'rss (MiB)', 'processes for load', '+headroom', 'total connections', 'store limit'], plan.map((p) => [p.h, p.rate.toFixed(1), p.socket, p.mib.toFixed(0), p.forLoad, p.processes, p.connections, p.connections <= STORE_LIMIT ? 'passes' : 'exceeds'])); const chosen = plan.filter((p) => p.connections <= STORE_LIMIT).sort((a, b) => a.processes - b.processes)[0]; const perNode = Math.floor(NODE_MEMORY / chosen.mib); console.log(`\nchosen plan: pool ${chosen.h}, ${chosen.processes} processes (${chosen.forLoad} for load + ${HEADROOM} headroom), ` + `${chosen.connections} connections, ${Math.ceil(chosen.processes / perNode)} nodes (${perNode} processes per node)`); // stage 2: three estimates, first at target load then at the spike const settings = [['low estimate', chosen.forLoad - 1, chosen.h], ['chosen plan', chosen.processes, chosen.h], ['high estimate, pool untouched', chosen.processes + 2, 4]]; for (const [name, rate, duration] of [['target load', TARGET, DURATION], ['spike', TARGET * SPIKE, SPIKE_DURATION]]) { const rows = []; for (const [label, processes, pool] of settings) { rows.push([label, processes, pool, processes * pool, ...await trial(processes, pool, rate, duration)]); } console.log(`\n${name} — ${rate} requests/sec`); table(['setting', 'processes', 'pool', 'pool product', '200', '503', '502', 'longest queue', 'store socket', 'rss'], rows); } ``` ``` pool completed/sec store socket rss (MiB) processes for load +headroom total connections store limit 1 41.7 2 83 4 6 6 passes 2 69.4 3 90 3 5 10 passes 4 68.9 5 98 3 5 20 exceeds 8 70.6 9 112 3 5 40 exceeds chosen plan: pool 2, 5 processes (3 for load + 2 headroom), 10 connections, 3 nodes (2 processes per node) target load — 150 requests/sec setting processes pool pool product 200 503 502 longest queue store socket rss low estimate 2 2 4 270 0 0 23 4 181 chosen plan 5 2 10 270 0 0 0 6 449 high estimate, pool untouched 7 4 28 270 0 0 0 7 642 spike — 600 requests/sec setting processes pool pool product 200 503 502 longest queue store socket rss low estimate 2 2 4 279 321 0 32 4 183 chosen plan 5 2 10 460 140 0 32 10 520 high estimate, pool untouched 7 4 28 351 0 249 17 16 683 ``` ## The Pool's Three Bills At pool one, a process completed 41.7 requests per second; at two it jumped to 69.4, staying in the same band at four and eight. At pool one, the store's wait cannot overlap with any work; at two, one request waits while the other is processed. Above two, no wait is left to overlap. The next two columns are the setting's price: the socket opened at the store rose to 2, 3, 5, and 9, process memory to between 83 and 112 MiB. Pool is not a ceiling, it is the connection count reached at saturation. ## From Process to Node At pool two, `ceil(150 / 69.4) = 3` processes suffice for 150 requests; adding release and failure headroom makes it five. Five processes at pool two drop ten connections on the store. Had pool four been chosen, process count would still be five, but connections would rise to twenty — forty at pool eight. This reverses the calculation's direction: store limit divided by process count caps the pool, the cap determines capacity, and capacity determines process count. Node count is a division: a 90 MiB process fits two to a 256 MiB node, five processes make three nodes. There is a mismatch here — headroom was calculated in process terms, but failure arrives in node terms: lose one node and two processes go at once. ## A Two-Sided Wrong Estimate At target load, all three settings look the same: 270 of 270 requests completed, error zero in all three. This is where the wrong estimate stays silent. The low estimate's trace is in a single column: with two processes, longest queue climbed to 23, staying zero in the correct plan. Two processes carry roughly 139 requests per second against a target of 150; the accumulating queue fills within a few seconds, shorter than the measurement window. Under the spike, the same setting lost 321 requests. Queue depth is the indicator that arrives before a dropped request and, alone, produces no error. The high estimate's only trace at target load is in memory: seven processes held 642 MiB, five held 449. Under the spike, seven processes asked the store for twenty-eight connections; the socket saturated at sixteen, and 249 requests got a 502. The 351 completed requests are fewer than the correct plan's 460. The correct plan rejected the 140 requests it could not carry with a 503 in its own queue; the high estimate spent a shared resource. The three inputs sit in three separate layers: process count in the process manager, pool in the application's environment, connection limit at the store. What gets measured is the product of the first two, and it is written in no file. ## Summary - Pool size determines capacity (41.7 to 69.4 requests), process memory (83 to 112 MiB), and the socket count landing on the store (2 to 9) at once; above two, only the price kept rising. - The calculation from measurement to plan is three steps — process count for load, release and failure headroom, node count divided by memory: pool 2, 5 processes, 10 connections, 3 nodes. - The store's connection limit caps the pool: at the same process count, pool 4 would mean twenty connections, pool 8 forty. - Headroom is calculated in process terms while failure arrives in node terms; since two processes fit per node, losing one node costs twice the single-process headroom. - At target load, all three settings met 270 of 270 requests; the low estimate's trace was a queue climbing to 23, the high estimate's a 193 MiB sitting idle. Under the spike, the low estimate rejected 321 requests, the high estimate broke 249 when the store socket filled. ## Course Wrap-Up | Lesson | Measure of the correct setting | Silent result of the wrong setting | Where the setting lives | |---|---|---|---| | Input Validation | Allowlist 0 and 1, blocklist 11 and 4 wrong | 0 errors across three wrong settings | 0 at the endpoint, 22 wasted steps at the store | | Injection Class | Corruption 0 on the separately handled path | Escaped concatenation drifts on 4 of 10 inputs | Decision separate across 12 call sites | | Cross-Origin Access | Origin list open to 2 outside origins | Wildcard setting returned 200, the client discarded it | Separate across 5 endpoints, 1 in a single layer | | Security Headers | 1 write site in a single layer, 0 lines for a new endpoint | One wrong value dropped 10/10 to 0/10 | 10 files per endpoint, 1 in a single layer | | Secrets Management | Window 390 seconds; rotation with no window gets 36 rejections | The leaked key got a 200 at second 900 | Key sits in 14 places, 56 steps a year | | Dependency Risk | Daily upgrades: 113 exposure days, 237 intervention days | Under a loose range, two deployments diverged on 14 packages | Version range sits in the manifest | | Audit Logging | Every decision point answered 8 of 8 questions | Deleting the last 25 records went uncaught | Anchor period: 3,012 with no anchor, 499 at one in 500 | | Abuse Defense | The layered key stopped 2,446 requests, 0 false rejects | Without normalization, stopped count fell to 2,266 | Key selection sits in the limiter | | Reverse Proxy | Forwarding on: 11 of 16 requests passed, 4 identities logged | Off: 6 good requests stopped | Forwarding at the proxy, body limit at two layers | | Process Managers | Immediate restart answered 20 of 24 requests; 500 ms wait drained 6 requests | Dozens of restarts on a process that never starts (43 in that run) | Policy and wait time in the process manager | | Horizontal/Vertical Scaling | Pool cut to two: 60 of 60 requests answered | Pool untouched: connections rose to 32, 30 requests unanswered | Process count and pool sit apart | | Stateless Application | 21 of 21 requests correct with one copy | 12 requests wronged with two copies, no code changed | 12 call sites in 55 lines | | Session and Stickiness | 0 sessions lost in the shared store | Process memory: 8 additions never accumulated, all returned 200 | Signing key one per node | | Zero-Downtime Deployment | Blue-green and canary both dropped 0 requests | 8 in-flight requests dropped under hard shutdown, invisible in the counter | Release layout at the frontend | | Migration and Release Order | Expand–contract met 120 of 120 requests in 5 steps | Skipping dual writing left a field empty, request returned 200 | Order sits in a written list | | Capacity Planning | Pool 2 gives 5 processes, 10 connections, 3 nodes | Under the low estimate the queue rose to 23, under the high, 193 MiB sat idle | Process manager, application, and store | The middle column writes the course's rule sixteen times over: a wrong configuration raises no error; the measurement shows the difference. A missing header, a key never rotated, a skipped dual write, an over-provisioned process — none threw an exception. What separated right from wrong in every lesson was a counted difference: a passed request, an open origin, an unrecoverable record, idle memory. Every setting gets its measurement written down beside it. This lesson closes the **Server Security and Going to Production** course, and the curriculum with it. Server-Side Fundamentals built the request's path through the server, Web API Design the contract, Authentication and Authorization access, the Data Access Layer persistence, Caching, Queues and Asynchronous Processing the work leaving the request, Service Architectures the boundary, Observability and Reliability visibility; this course built production decisions. What comes next is the reader's own system: which endpoint's validator is missing, how many processes run, and which measurement that number came from.