--- title: 'Finding the Capacity Limit' source: 'https://academia.sh/en/courses/non-functional-testing/finding-the-capacity-limit' course: 'Non-Functional Testing' language: en updated: '2026-08-23T14:25:17+00:00' license: 'CC BY-SA 4.0' --- # Finding the Capacity Limit Searching for the breaking point: raising load step by step and reading throughput, wait round, and successful request count separately at each step, the saturation step where throughput stops rising and the knee step where delay crosses the threshold not landing at the same place, the rule that declares the limit by throughput letting three steps pass falsely, and lengthening the queue growing throughput while leaving the successful request count unchanged. The three experiments in this topic were all built the same way: a part of the system was chosen and deliberately broken — the catalog slowed down, a batch was tagged, a table dropped out of the backup. None of them pushed the system to its own limit. A **breaking point test** raises load in stages and searches for the step where the system starts to break down. The difficulty is not in measuring but in **recognizing**: the step where throughput stops rising and the step where delay steepens may not be the same place. The **saturation** and **utilization** terms from the Introduction to System Design course name these two readings; the capacity planning from the Performance Anti-Patterns and Monitoring course is this test's **input**. The question is: **which one gets declared the limit, and what do the two error types of that decision come to.** ## The Load Ladder and the Wait Round The loan endpoint is built as a set of service slots with a fixed service duration and a bounded queue. Instead of duration, **rounds** are counted: one round is one service duration, and a request's **wait round** comes from its queue position at the moment it is accepted. The numbers below do not change from run to run. **NF22 — one service round is 40 ms, 4 service slots.** **NF23 — queue length 16, 5 waves per step.** Both are assumptions; the requests in a wave arrive before the first service finishes, which makes the queue position deterministic. ```js // capacity.mjs — the loan endpoint runs with a fixed service duration, S service slots, and a // queue of length Q. Every response carries the wait round computed from the queue position at // the moment the request was accepted; if the queue is full the request is rejected (load shedding). import { createServer } from 'node:http'; export const ROUND = 40; // NF22: one service round is 40 ms export function setupEndpoint({ slots, queue }) { const measure = { accepted: 0, rejected: 0 }; const waiting = []; let active = 0; const start = (finish) => { active += 1; setTimeout(() => { active -= 1; finish(); if (waiting.length > 0) start(waiting.shift()); }, ROUND); }; const server = createServer((request, response) => { const occupied = active + waiting.length; if (occupied >= slots + queue) { measure.rejected += 1; response.writeHead(503, { 'content-type': 'application/json' }); return response.end('{"status":"rejected"}'); } measure.accepted += 1; const round = Math.floor(occupied / slots); // which round it will enter service in const finish = () => { response.writeHead(200, { 'content-type': 'application/json' }); response.end(`{"round":${round}}`); }; if (active < slots) start(finish); else waiting.push(finish); }); server.listen(0, '127.0.0.1'); return { server, measure }; } ``` ## The Steps The measure of acceptability comes from a number already measured in this topic: the timeout the loan service places on its catalog call in the first lesson is 100 ms; with a 40 ms round, that means a request can wait **at most one round**. One that waits longer is processed but never reaches the caller. **NF24 — the acceptable wait is the number of rounds the timeout allows**; it is not an assumption but a calculation derived from the first lesson's measurement. ```js // ladder.mjs — load is raised step by step; at every step, throughput, wait round, and // successful requests are counted. All numbers are integers and run-independent; duration is not printed. import { writeFileSync } from 'node:fs'; import { setupEndpoint, ROUND } from './capacity.mjs'; const SLOTS = 4, QUEUE = 16, WAVE = 5; // NF22, NF23 const STEP = [1, 2, 4, 8, 12, 16, 20, 24, 32]; const TIMEOUT = 100; // lesson 01's NF16: 100 ms const MAX_ROUND = Math.ceil(TIMEOUT / ROUND) - 2; // NF24: wait round cap async function runStep(base, c) { const rounds = []; let rejected = 0; for (let d = 0; d < WAVE; d += 1) { const s = await Promise.all(Array.from({ length: c }, async () => { const y = await fetch(base); const g = await y.json(); return y.status === 503 ? null : g.round; })); for (const x of s) { if (x === null) rejected += 1; else rounds.push(x); } } const s = [...rounds].sort((a, b) => a - b); return { c, sent: c * WAVE, processed: rounds.length, rejected, p95: s.length > 0 ? s[Math.ceil(0.95 * s.length) - 1] : 0, successful: rounds.filter((x) => x <= MAX_ROUND).length }; } async function ladder(queue, steps) { const { server } = setupEndpoint({ slots: SLOTS, queue }); await new Promise((c) => server.once('listening', c)); const base = `http://127.0.0.1:${server.address().port}/loan`; const r = []; for (const c of steps) r.push(await runStep(base, c)); server.closeAllConnections(); await new Promise((c) => server.close(c)); return r; } const p = (x, n) => String(x).padStart(n); const result = await ladder(QUEUE, STEP); writeFileSync('ladder.json', `${JSON.stringify({ MAX_ROUND, result }, null, 0)}\n`); console.log(`${SLOTS} service slots, ${QUEUE} queue, round ${ROUND} ms, ${WAVE} waves per step; ` + `timeout ${TIMEOUT} ms -> acceptable wait at most ${MAX_ROUND} rounds`); console.log(`\n${'concurrent'.padStart(11)}${'sent'.padStart(10)}${'processed'.padStart(11)}` + `${'rejected'.padStart(10)}${'p95 wait rounds'.padStart(18)}${'successful'.padStart(12)}`); for (const r of result) console.log(`${p(r.c, 11)}${p(r.sent, 10)}${p(r.processed, 11)}${p(r.rejected, 10)}` + `${p(r.p95, 18)}${p(r.successful, 12)}`); const saturation = result.find((r, i) => i > 0 && r.processed === result[i - 1].processed); const knee = result.find((r) => r.p95 > MAX_ROUND); console.log(`\nsaturation step (where throughput stops rising): c=${saturation.c}, throughput ${saturation.processed} requests`); console.log(`knee step (where p95 wait exceeds ${MAX_ROUND} rounds): c=${knee.c}`); console.log(`\nsame step (c=32), two queue lengths:`); for (const q of [QUEUE, 64]) { const [r] = await ladder(q, [32]); console.log(`queue ${p(q, 2)}: processed ${p(r.processed, 3)}, rejected ${p(r.rejected, 2)}, ` + `p95 ${r.p95} rounds, successful ${r.successful}`); } ``` ``` 4 service slots, 16 queue, round 40 ms, 5 waves per step; timeout 100 ms -> acceptable wait at most 1 rounds concurrent sent processed rejected p95 wait rounds successful 1 5 5 0 0 5 2 10 10 0 0 10 4 20 20 0 0 20 8 40 40 0 1 40 12 60 60 0 2 40 16 80 80 0 3 40 20 100 100 0 4 40 24 120 100 20 4 40 32 160 100 60 4 40 saturation step (where throughput stops rising): c=24, throughput 100 requests knee step (where p95 wait exceeds 1 rounds): c=12 same step (c=32), two queue lengths: queue 16: processed 100, rejected 60, p95 4 rounds, successful 40 queue 64: processed 160, rejected 0, p95 7 rounds, successful 40 ``` ## Saturation or Knee The three columns tell three different stories. **Processed requests** — throughput — rise up to c = 24 and stop there: this is the **saturation step**. **p95 wait rounds** moves much earlier and crosses the threshold at c = 12: this is the **knee step**. Three steps sit between them, and across those three steps the system looks like it is "processing more requests" while most of its responses are timing out. The third column settles it. **Successful requests never rise after c = 8: fixed at 40.** Every added request only joins the queue, lengthens the wait of the ones already there, and times out. The real capacity is 40 successful requests, and it was found far earlier than where throughput flattens out. The last two lines say why this cannot be seen from throughput alone. The same step (c = 32) was run at two queue lengths: at queue 16, throughput is 100 with 60 rejected; at queue 64, throughput is 160 with none rejected — 160/100, that is, **1.60x**. Successful requests are 40 in both, that is, **1.00x**. Lengthening the queue shifts the saturation step to the right and changes nothing about what reaches the user; **a throughput criterion makes the breaking point something a queue setting can hide.** ## The Rule for Declaring the Limit The limit is written down as a number, and the rule by which it was chosen **is** the threshold. The scan compares two families of rules: one that looks at delay (p95 wait ≤ t rounds) and one that looks at throughput (the last step before saturation). **NF25 — a step is acceptable as long as the successful-request share does not drop below 99%.** This is an assumption and is kept outside the rules; the rules decide without knowing this share. ```js // limit.mjs — a scan of limit-declaration rules over ladder.json. The actual acceptance // criterion is the successful-request share; the rules decide without knowing it. import { readFileSync } from 'node:fs'; const { result } = JSON.parse(readFileSync('ladder.json', 'utf8')); const SHARE = 0.99; // NF25 const actual = (r) => r.successful / r.sent >= SHARE; const saturation = result.findIndex((r, i) => i > 0 && r.processed === result[i - 1].processed); const count = (accept) => { let falsePass = 0, falseFail = 0; result.forEach((r, i) => { if (accept(r, i) && actual(r) === false) falsePass += 1; if (accept(r, i) === false && actual(r)) falseFail += 1; }); return { falsePass, falseFail, limit: result.filter(accept).at(-1)?.c }; }; const p = (x, n) => String(x).padStart(n); console.log(`actual acceptance criterion: successful-request share >= ${100 * SHARE}%; ` + `the highest step meeting this criterion is c=${result.filter(actual).at(-1).c}`); console.log(`\n${'rule'.padEnd(30)}${'declared limit'.padStart(18)}` + `${'false pass'.padStart(14)}${'false fail'.padStart(14)}`); for (const t of [0, 1, 2, 3, 4]) { const r = count((x) => x.p95 <= t); console.log(`${`delay: p95 <= ${t} rounds`.padEnd(30)}${p(`c=${r.limit}`, 18)}${p(r.falsePass, 14)}${p(r.falseFail, 14)}`); } const v = count((_, i) => i < saturation); console.log(`${'throughput: before saturation'.padEnd(30)}${p(`c=${v.limit}`, 18)}${p(v.falsePass, 14)}${p(v.falseFail, 14)}`); ``` ``` actual acceptance criterion: successful-request share >= 99%; the highest step meeting this criterion is c=8 rule declared limit false pass false fail delay: p95 <= 0 rounds c=4 0 1 delay: p95 <= 1 rounds c=8 0 0 delay: p95 <= 2 rounds c=12 1 0 delay: p95 <= 3 rounds c=16 2 0 delay: p95 <= 4 rounds c=32 5 0 throughput: before saturation c=20 3 0 ``` The last line is this lesson's outcome. **The throughput rule declares the limit at c = 20 and lets three steps through falsely — 12, 16, and 20.** In all three, the system processes more requests; in none of them do more users get an answer. When the delay rule's threshold is set to one round, it declares the limit at c = 8 and zeroes out both error types. The other end of the scan shows the cost of tightening the threshold: the p95 ≤ 0 rounds rule accepts no wait at all and eliminates c = 8 too — a **false fail**; at that step the wait is below the timeout, and half the capacity is written off for nothing. **The chosen threshold is p95 wait ≤ 1 round**, and its source is the timeout the first lesson measured. The decision belongs to the capacity plan: this number does not stop a release, it feeds into a scaling decision. ## Cost and the Limit Not Seen The run-independent cost: nine steps, 595 requests sent, 515 accepted, and 80 rejected. That is not the real cost — **the only way to find the limit is to go past it.** The last five steps deliberately cause harm: of the 400 requests processed there, 240 timed out. In a production-like environment, this cannot be done without bounding the blast radius, and it needs the previous lesson's abort condition. There are three classes the ladder cannot see. The first is a **limit tied to the workload mix**: the ladder sent the same request to a single endpoint. The second is **cumulative degradation**: a five-wave step is not the same window as load sustained for hours. The third is **collapse behavior**: because this endpoint does load shedding, throughput stayed flat past saturation; in a system without shedding, throughput would drop. ## Summary - A breaking point test raises load step by step; across nine steps, throughput, p95 wait round, and successful requests were read separately, and every number is an integer. - The saturation step is c = 24, the knee step is c = 12; three steps sit between them. - Successful requests stayed fixed at 40 after c = 8: every added request only lengthened the queue and timed out. - When the queue was raised from 16 to 64, throughput grew 1.60x while successful requests stayed at 1.00x. - The chosen threshold is p95 wait ≤ 1 round, its source is the first lesson's 100 ms timeout, and it gives 0 / 0; the throughput rule lets three steps through falsely. - The cost is 595 requests sent, and it requires going past the limit: of the 400 requests processed in the last five steps, 240 timed out. ## Course Wrap-Up This course's fifteen lessons tested the same system with three questions — what happens under load, under attack, and under failure — and every lesson tied its outcome to a threshold. | Lesson | Threshold tested | Source of threshold | False pass / false fail | Cost | |---|---|---|---|---| | Performance Testing Types | p95 ≤ 20 ms, across four decision windows | NF1 assumption+calculation: the 60 ms budget split three ways | 1 / 0; at 3 ms 0 / 0, at 1 ms 0 / 3 | 9 processes, 55,000 requests, 1.00 billion row steps | | Test Scenario Modeling | same 20 ms, as the scenario changes | inherited from NF1 | 1 / 0; at 5 ms 0 / 1; at 100 ms 3 / 0 | 3,000 requests per run; the scan step spreads 10.7x | | Metric Selection | at least $1/(1-p)$ samples per percentile | NF5 calculation | at 20 samples 22/200 / 0; at 100 samples 0 / 0 | 8,496 and 8,494 latency values, 1,068,000 samples drawn | | Bottleneck Analysis | layer share ≥ 0.50 | NF6 assumption | 1 misattribution / 0 unattributable; at 0.80 0 / 1 | 5 processes, 4 clock readings per request | | Benchmarking Discipline | candidate/baseline ±5%, 8 rounds | NF9 assumption; lower bound is the measured spread | 0 / 0; at 2% and one round 0 / 179; at 20% 500 / 0 | 30 processes, 2.4 million rows of fill, 20,940 requests | | Static Application Security Testing | finding score ≥ 40 | NF8 assumption: a missed defect = 5 false positives | 1 / 2; at 50 2 / 0 | 75 comparisons, 0 processes, 7 manual reviews | | Dynamic Application Security Testing | finding score ≥ 50 | measurement: lowest cost at 50 | 1 / 0 | 1 process, 8 requests, a hand-built route table | | Dependency and Component Scanning | severity ≥ 80 | measurement; weight from NF8 | 1 / 0 | 5 of 10 packages upgraded, each with a regression test | | The Role of Penetration Testing | chain length ≤ 4 steps | budget (NF12), not a requirement | at k=3 2/0; at k=4 0/2; at k=5 0/8 | 40 steps at k=3, 84 at k=4, 169 at k=5 | | Accessibility Testing | zero automated findings and 6 manual criteria | measurement: the automated check comes back green with three flaws | 3 / 1 | 1 tree traversal, 14 elements; 6 criteria per screen | | Compatibility Testing | covered-user share ≥ 90% | fallacy: the share does not represent the defect distribution | 3 / 3; with the profile-and-layout criterion 0 / 5 | 8 environment runs, a hand-built support table, multiplicative growth | | Fault Injection | drop rate ≤ 5% and one degraded response per injected request | measurement: 10/200; the second threshold is the harness's own counter | 0 / 0; with the drop rate alone 1 / 0 | 13 of 38 lines, 5 decision points, 1 opening path | | Chaos Experiments | window full-response share ≥ baseline − 10% | measurement: the control run's average, 96.00% | 0 / 0; at 5% 0 / 4; at 20% detection shifts to window 8 | blast radius multiplier 4.00; 44 damaged requests at k = 3 | | Recovery Verification | the whole of D1+D2+D3+D4 must pass | measurement: the set scan | 0 / 0; with D1 alone 2 / 0; with D5 0 / 1 | restore grows 10.00x, verification stays at 7 queries | | Finding the Capacity Limit | p95 wait ≤ 1 round | measurement: the first lesson's 100 ms timeout | 0 / 0; with the throughput criterion 3 / 0 | 595 requests, 9 steps, 240 timeouts | The third column is this course's rule: **a threshold without a source carries no decision.** The fourth column is the other half of that rule. In functional testing, the outcome is a pass/fail, because the expected value is known. In non-functional testing, the outcome is a **distribution**, and calling it "successful" is choosing a threshold; that threshold can only be defended with two error types — how many real defects it lets through, and how many healthy runs it turns red. None of the rows in the table that zeroed out both did so by tightening the threshold; every one of them changed **what it measured**. Across four courses, testing itself was measured: which test catches which class of defect, which it misses, and what it costs in return. But one question was never asked — **when will each test run.** Some tests here are cheap enough to run on every change; some are expensive enough to run once per release; one needs a production-like environment. How limited time gets divided among these tests, and how a red result turns into a team decision, was never addressed in any lesson. The Testing Process and Automation Infrastructure course opens with that question.