--- title: 'Horizontal and Vertical Scaling' source: 'https://academia.sh/en/courses/introduction-to-system-design/horizontal-and-vertical-scaling' course: 'Introduction to System Design' language: en updated: '2026-08-23T14:25:23+00:00' license: 'CC BY-SA 4.0' --- # Horizontal and Vertical Scaling The two ways to change a resource and each one's hard limit: measuring the speedup and efficiency of splitting the same job across one to sixteen workers, counting how the partitioning decision changes the serial fraction and the upper bound, measuring the limit that core count places on vertical scaling, and comparing the two paths' cost curves under the same target. The previous lesson varied the load and held the resource fixed; every measurement was taken with a single worker. The capacity computation was valid under that same assumption. The resource, however, is a quantity that can be changed, and there are two ways to change it: grow the same unit, or increase the number of units. This lesson separates the two paths, measures where each one stops, and compares their costs under the same target. The names of the distinction are as they appear in the catalog. **Vertical scaling** is growing the scaling unit itself: more CPU, more memory to the same unit. **Horizontal scaling** is increasing the number of units. The Server-Side Fundamentals course established that the scaling unit is the process, and that processes do not share memory; this lesson turns that rule's consequence into a measurement. ## Same Job, Growing Worker Count The job measured is end-of-day billing: 12 million records spanning thirty days are priced, and an audit trail is produced for every invoice line. The grouping unit set up in the previous lesson still holds — a **partition** is one seller's one day, so the thirty-day scan has 120,000 partitions (4000 sellers times 30 days). The job is split across one to sixteen workers, and the total duration is measured at each level. Records are not read, they are generated deterministically; what is measured is the parallelization of the compute share, not I/O. ```js // scale/worker.mjs — a worker: prices the items in its share, signs them, and returns partition totals import { parentPort, workerData } from "node:worker_threads"; const TARIFF = { tier: [[1, 3000], [5, 4800], [20, 9600]], zone: { "34": 100, "06": 115, "35": 125 }, minimum: 2500 }; const ZONES = ["34", "06", "35"]; function net(weight, zone, rate) { const tier = TARIFF.tier.find(([s]) => weight <= s) ?? [0, 9600]; const base = Math.max(TARIFF.minimum, Math.round((tier[1] * TARIFF.zone[zone]) / 100)); return base - Math.round(base * Math.min(rate, 0.4)); } function signature(text) { // audit trail of an invoice line let h = 2166136261; for (let i = 0; i < text.length; i += 1) h = Math.imul(h ^ text.charCodeAt(i), 16777619) >>> 0; return h; } function share({ start, end, step, partitions }) { const totals = new Map(); for (let i = start; i < end; i += step) { const b = i % partitions; const value = net((i % 20) + 1, ZONES[i % 3], (i % 5) / 20); const sig = signature(`G${i};${ZONES[i % 3]};${value}`); const previous = totals.get(b) ?? { net: 0, sig: 0 }; totals.set(b, { net: previous.net + value, sig: (previous.sig ^ sig) >>> 0 }); } return [...totals]; } if (workerData === null) console.log("this file is run as a worker by scale/run.mjs"); else parentPort.postMessage(share(workerData)); ``` ``` this file is run as a worker by scale/run.mjs ``` The item count is the product of V3 and V11 and comes from the Back-of-the-Envelope Estimation lesson's table; this lesson does not choose its own volume figure. The partition count is derived from V13: since a hundred items are grouped into an invoice line, twelve million items come down to 120,000 partitions. The partition count is not the seller count — 4000 sellers' thirty days make 120,000 seller-days. The driver splits the job in two different ways. **Index** partitioning divides items into contiguous blocks; a single partition's items are spread across every worker. **Key** partitioning advances with a stride; every partition falls to a single worker. Both process the same number of items and must produce the same result. ```js // scale/run.mjs — the same job split across 1..16 workers; speedup, efficiency, and merge share are measured import { Worker } from "node:worker_threads"; import { availableParallelism } from "node:os"; const ITEMS = 12_000_000; // V3 x V11: record count spanning thirty days const PARTITIONS = ITEMS / 100; // assumption: 100 items per invoice line -> 4000 sellers x 30 days const WORKERS = [1, 2, 4, 8, 12, 16]; function worker(data) { return new Promise((resolve, reject) => { const w = new Worker(new URL("./worker.mjs", import.meta.url), { workerData: data }); w.on("message", resolve); w.on("error", reject); }); } function ranges(n, mode) { // "index": contiguous block, "key": strided return mode === "index" ? Array.from({ length: n }, (_, s) => ({ start: Math.floor((s * ITEMS) / n), end: Math.floor(((s + 1) * ITEMS) / n), step: 1, partitions: PARTITIONS })) : Array.from({ length: n }, (_, s) => ({ start: s, end: ITEMS, step: n, partitions: PARTITIONS })); } async function run(n, mode) { const start = performance.now(); const shares = await Promise.all(ranges(n, mode).map(worker)); const parallel = performance.now() - start; const mergeStart = performance.now(); const invoice = new Map(); let collision = 0; for (const share of shares) { for (const [b, v] of share) { const previous = invoice.get(b); if (previous === undefined) invoice.set(b, { ...v }); else { previous.net += v.net; previous.sig = (previous.sig ^ v.sig) >>> 0; collision += 1; } } } const merge = performance.now() - mergeStart; const signature = [...invoice].reduce((a, [, v]) => (a ^ v.sig) >>> 0, 0); return { parallel, merge, total: parallel + merge, partitions: invoice.size, collision, signature }; } console.log(`cores = ${availableParallelism()}, items = ${ITEMS}, partitions = ${PARTITIONS}`); for (const mode of ["index", "key"]) { console.log(`\npartitioning = ${mode}`); console.log("worker parallel(ms) merge(ms) total(ms) speedup efficiency colliding records signature"); let base = 0; for (const n of WORKERS) { const r = await run(n, mode); if (n === 1) base = r.total; console.log(`${String(n).padStart(4)} ${r.parallel.toFixed(0).padStart(11)} ${r.merge.toFixed(1).padStart(15)} ` + `${r.total.toFixed(0).padStart(10)} ${(base / r.total).toFixed(2).padStart(8)} ` + `${(base / r.total / n).toFixed(2).padStart(10)} ${String(r.collision).padStart(13)} ${r.signature}`); } } ``` ``` cores = 12, items = 12000000, partitions = 120000 partitioning = index worker parallel(ms) merge(ms) total(ms) speedup efficiency colliding records signature 1 1674 8.4 1683 1.00 1.00 0 799471432 2 941 13.3 954 1.76 0.88 120000 799471432 4 602 24.6 626 2.69 0.67 360000 799471432 8 580 42.9 622 2.70 0.34 840000 799471432 12 883 70.6 953 1.76 0.15 1320000 799471432 16 882 68.3 950 1.77 0.11 1800000 799471432 partitioning = key worker parallel(ms) merge(ms) total(ms) speedup efficiency colliding records signature 1 1662 9.9 1672 1.00 1.00 0 799471432 2 879 5.6 885 1.89 0.94 0 799471432 4 549 5.5 554 3.02 0.75 0 799471432 8 321 6.8 328 5.11 0.64 0 799471432 12 284 5.5 289 5.79 0.48 0 799471432 16 364 6.9 370 4.51 0.28 0 799471432 ``` These numbers are measurements, taken on this machine; the core count is 12, and on another machine the table's break point falls elsewhere. The signature column gives the same value in all twelve runs (799471432), meaning every partitioning produced the same invoice; the durations of runs that give different results cannot be compared. The two single-worker base durations are 1683 and 1672 milliseconds; the difference between them is measurement noise, which is why speedup is read against each mode's own base. ## A Hard Limit With key partitioning, speedup grows from 1 to 12 workers: 1.89, 3.02, 5.11, and 5.79. At sixteen workers it **drops**: 4.51. This drop is not a measurement error; it is the vertical-scaling limit itself. The machine has 12 cores; sixteen workers are forced to share twelve cores, and the sharing itself is a cost. That growing a unit stops at some point is not an intuitive claim but a number read off the table. The efficiency column says the same thing from another angle. **Parallel efficiency** is the speedup obtained per worker, that is, speedup divided by worker count; it should not be confused with the Quality Attributes lesson's performance-efficiency family, which is the name of a quality family. In the measurement it is 0.94 at two workers, 0.75 at four, 0.64 at eight, 0.48 at twelve. **Every added worker returns less than the one before it.** The value dropping below 1 is not a flaw but the nature of parallel work; what matters for the decision is how fast it drops. ## The Condition for Horizontal Scaling The difference between the two partitionings is the lesson's second result. Same job, same worker count, same machine — at eight workers, speedup is 5.11 with key partitioning, 2.70 with index partitioning. A difference of nearly two times comes solely from **how the job is split**. The source of the difference is in the colliding-records column. With index partitioning, sixteen workers produced 1,800,000 colliding records: because a partition's total sits in pieces across multiple workers, every piece has to be summed at the merge step. With key partitioning this number is zero. Merge duration splits accordingly: 68.3 milliseconds versus 6.9 milliseconds at sixteen workers. The Server-Side Fundamentals course laid down the stateless-process rule: state that must survive between requests is moved outside the process. The measurement here is that rule's counterpart on the scaling side. Horizontal scaling does not happen just by increasing the number of units; it happens because **the job can be split in a way that requires no state shared between units.** The choice of partition key is therefore not a detail but the scalability decision itself. The terms partition and partitioning were set up in the Advanced SQL course in the context of table partitioning; the context here is splitting work across workers, not data across disk. ## Serial Fraction, Upper Bound, and Cost One more number can be derived from the measured speedup. Part of the job cannot be split — starting the workers, summing the shares, writing the result. If this **serial fraction** is $s$, the best possible speedup with $n$ workers is: $$ S(n) = \frac{1}{s + \dfrac{1 - s}{n}} $$ Solving the relation backward turns a measured speedup into a serial fraction, and as $n$ goes to infinity, speedup is bounded by $1/s$. The computation below does this for both partitionings, then compares the cost of meeting a target under the two paths. The vertical cost needs an **assumption**: how many times the unit's cost multiplies when its power doubles. Three values are given together because the result is sensitive to this assumption. ```js // calc/cost.mjs — serial fraction, upper bound, and the cost of the two paths from measured speedup const MEASURED = { key: { workers: 8, speedup: 5.11 }, index: { workers: 8, speedup: 2.70 } }; const MACHINE_LIMIT = 5.79; // measured: this machine's highest speedup for a single unit (12 cores) const serialFraction = (n, S) => (1 / S - 1 / n) / (1 - 1 / n); const amdahl = (s, n) => 1 / (s + (1 - s) / n); const requiredUnits = (s, K) => (1 - s) / (1 / K - s); console.log("partitioning measured(workers/speedup) serial fraction upper bound(1/s)"); for (const [name, o] of Object.entries(MEASURED)) { const s = serialFraction(o.workers, o.speedup); console.log(`${name.padEnd(12)}${`${o.workers} / ${o.speedup.toFixed(2)}`.padStart(21)}${(s * 100).toFixed(1).padStart(11)}%` + `${(1 / s).toFixed(1).padStart(17)}`); } const s = serialFraction(MEASURED.key.workers, MEASURED.key.speedup); console.log("\ntarget horizontal units horizontal cost vertical x2.0 vertical x2.5 vertical x3.0 vertical reachable"); for (const K of [2, 4, 8, 16]) { const units = requiredUnits(s, K); const vertical = (c) => c ** Math.log2(K); console.log(`${`x${K}`.padStart(5)}${(units > 0 ? units.toFixed(1) : "unreachable").padStart(13)}` + `${(units > 0 ? units.toFixed(1) : "-").padStart(16)}${vertical(2.0).toFixed(1).padStart(13)}` + `${vertical(2.5).toFixed(1).padStart(13)}${vertical(3.0).toFixed(1).padStart(13)}` + `${String(K <= MACHINE_LIMIT).padStart(15)}`); } console.log(`\namdahl forecast up to 16 workers (key partitioning, s = ${(s * 100).toFixed(1)}%):`); console.log([1, 2, 4, 8, 12, 16].map((n) => `${n}:${amdahl(s, n).toFixed(2)}`).join(" ")); ``` ``` partitioning measured(workers/speedup) serial fraction upper bound(1/s) key 8 / 5.11 8.1% 12.4 index 8 / 2.70 28.0% 3.6 target horizontal units horizontal cost vertical x2.0 vertical x2.5 vertical x3.0 vertical reachable x2 2.2 2.2 2.0 2.5 3.0 true x4 5.4 5.4 4.0 6.3 9.0 true x8 20.8 20.8 8.0 15.6 27.0 false x16 unreachable - 16.0 39.1 81.0 false amdahl forecast up to 16 workers (key partitioning, s = 8.1%): 1:1.00 2:1.85 4:3.22 8:5.11 12:6.35 16:7.23 ``` The serial-fraction lines reduce the partitioning decision's cost to a single number: 8.1% and an upper bound of 12.4 with key partitioning; 28.0% and an upper bound of 3.6 with index partitioning. Same job, same hardware — the upper bound diverges by more than three times. In a system where index partitioning was chosen, adding a thousand units instead of sixteen still would not push speedup past 3.6. The cost table shows the shape of the two curves. Horizontal cost is paid linearly in unit count, but the required unit count is not linear: 2.2 units for double, 5.4 for four times, 20.8 for eight times. The sixteen-times row shows "unreachable" instead of a number, because the target sits above the 12.4 upper bound the serial fraction imposes — no number of added units reaches it. Vertical cost, in contrast, depends entirely on the assumption: at a multiplier of 2.0, eight times costs 8.0 units — cheaper than horizontal; at 3.0 it costs 27.0 units — more expensive than horizontal. The last column settles the argument. Reaching eight times through the vertical path is **not possible** on this machine, because the highest measured speedup is 5.79. Vertical scaling's real problem is not the cost curve but the curve ending somewhere. The horizontal path is expensive, but its limit comes from the serial fraction, not the hardware — and the serial fraction is a design decision, as the measurement table's first row shows. The last line shows the model's own limit. The relation forecasts a speedup of 6.35 for twelve workers, 7.23 for sixteen; the measured values are 5.79 and 4.51. The model does not know the core count and never drops. **When a model's forecast diverges from a measurement, what needs correcting is not the measurement but the model's assumption** — here the missing assumption is that there is one core per worker. ## Summary - Vertical scaling is growing the scaling unit, horizontal scaling is increasing the number of units; the two hit different limits. - With key partitioning, speedup rose to 5.79 at 12 workers and dropped to 4.51 at 16: core count is the measured hard limit of the vertical path. - Parallel efficiency drops with every added worker: 0.94 at 2 workers, 0.64 at 8, 0.48 at 12. - How the job is split changed speedup by nearly two times: 5.11 versus 2.70 at eight workers. Index partitioning produced 1,800,000 colliding records and a 68.3 ms merge at 16 workers, key partitioning produced 0 and 6.9 ms. - The serial fraction derived from the measured speedup is 8.1% with key partitioning (upper bound 12.4), 28.0% with index partitioning (upper bound 3.6); the partition key is a scalability decision. - Reaching an eightfold target requires 20.8 units on the horizontal path, and is unreachable on the vertical path on this machine; sixteenfold is unreachable on the horizontal path too, because the serial fraction holds the upper bound at 12.4. ## Next Step The three lessons so far measured under the assumption that the system **works**. Load, capacity, speedup, and cost — all of these are numbers for a system that is up. The rationale for scaling is not load alone either: when a setup that scans twelve million records in 289 milliseconds is already sufficient, there must be another reason to increase the unit count. That reason is units not staying up. When a unit goes down, how much of the system goes down, for how long, and is that duration acceptable? The third question is answered with a percentage, and a percentage alone says nothing: the difference between ninety-nine percent and ninety-nine point nine percent stays invisible until it is written in minutes. The next lesson turns service availability into an outage budget and computes what combining components does to that budget.