--- title: 'The Traffic Layer' source: 'https://academia.sh/en/courses/traffic-layer' language: en updated: '2026-08-23T07:01:34+00:00' license: 'CC BY-SA 4.0' lessons: 16 outcomes: - 'Designs the layers a request passes through from the user to the application' - 'Justifies load balancing and content distribution decisions' - 'Places gateway responsibilities in the right layer' --- # The Traffic Layer Designing the stops a request passes through on its way from the user to the application: domain name routing policies and failover, content distribution with an edge cache, push- and pull-based propagation, the load balancer's layer 4 and layer 7 decisions and its distribution algorithms, the scaling cost of session stickiness, and the responsibilities the gateway takes to the edge; each stop is treated together with which of the numbers calculated in the introductory course it changes. ## Entry Points The first stops where a request enters the system: using domain name resolution as a routing policy and failover instrument, the edge cache's effect on the number of requests reaching the application, propagating content with push and pull models, and separating the content path from the application path. ### Domain Name System Design Source: https://academia.sh/en/courses/traffic-layer/domain-name-system-design Designing name resolution as a routing decision: measuring the geolocation, latency-based, and weighted policies on the same request flow, showing through resolver shares why the weighted policy misses its target share, and converting the TTL's determination of the failover window into dropped requests against the introductory course's downtime budget. The Introduction to System Design course defined every property, set the thresholds, and chose the failover patterns. One thing was missing: the request had not yet entered the system. A tracking query begins with a domain name resolving, may pass through an edge cache, lands on a load balancer, and reaches the application through a gateway. There, the cache was only a multiplier in the arithmetic, and routing was a one-line choice in the model. This course takes up that path, starting with name resolution. The Domain Name System itself was built in the How the Internet Works course in the Computer Networks curriculum — hierarchy, resolution chain, record types, and TTL are covered there and not repeated here. The question here is different: when a name has more than one zone behind it, which address gets returned is a design decision with measurable consequences. #### The Name Layer Is a Decision Point The authoritative server need not give the same answer to every query: it can return a different address depending on where the query came from and the state of the zones. This choice is a routing policy, and it takes three forms. The geolocation policy answers from a coarse, hand-written table keyed on the querier's location — this client cluster goes to that zone — and the table can be wrong. The latency-based policy picks whichever zone measures closer; because geographic and network proximity are not the same thing, the two policies produce different answers. The weighted policy assigns each zone a target share and distributes answers accordingly; its purpose is capacity control, not proximity. A fourth use is not a policy but a failure response: when a zone becomes unreachable, the authoritative server removes its address from the answers. This is the name layer's form of failover, and the second half of the lesson measures its cost. #### Setup The measurement uses an in-process model with one authoritative server, forty resolvers, and three zones — `b06` added to the `b34` and `b35` zones from the Availability Patterns lesson. The request rate comes from the introductory course: peak edge requests/s is 513.89, rounded up to 514. The model adds four new assumptions of its own; they are not folded into the introductory course's V1–V13 table. | Code | Assumption | Value | Rationale | |---|---|---|---| | T1 | TTL (baseline) | 300 s | the duration the name owner chooses, varied here | | T2 | zone count | 3 | a backup zone is added to the two-zone failover | | T3 | target share | 0.50 / 0.30 / 0.20 | the old zone carries more load, the new zone less | | T4 | resolver count and share | 40, share ~ 1/(j+1) | most queries come from a small number of large resolvers | | T5 | dead-zone detection time | 10 s | the health check counts a zone dead after this delay | The distance matrix is in abstract units, not measured latency — the same kind of abstraction as the introductory course's round. The resolvers' cache phases are spread evenly, since all of them expiring at the same instant is a separate problem, measured in the Caching, Queues and Asynchronous Processing course as the stampede effect. ```js // name/model.mjs -- name layer model. Distance is an abstract unit, not measured latency; // TTL, detection time, and outage start are model parameters. export const ZONE = ["b34", "b35", "b06"]; export const GROUP = ["g1", "g2", "g3", "g4"]; export const DISTANCE = { // unit; not measured latency g1: { b34: 1, b35: 3, b06: 2 }, g2: { b34: 3, b35: 1, b06: 2 }, g3: { b34: 2, b35: 4, b06: 1 }, g4: { b34: 2, b35: 3, b06: 4 }, }; export const GEOLOCATION = { g1: "b34", g2: "b35", g3: "b34", g4: "b06" }; // coarse zone table export const FALLBACK = { b34: "b06", b35: "b34", b06: "b35" }; // the table's fallback export const PATTERN = ["b34", "b35", "b06", "b34", "b35", "b34", "b35", "b06", "b34", "b34"]; export const WEIGHT = { b34: 0.5, b35: 0.3, b06: 0.2 }; // T3: target share, PATTERN yields this export const RESOLVERS = 40; // T4 export const DETECTION = 10; // T5: health check marks a dead zone (s) export const REQUESTS_PER_SECOND = 514; // K01 "peak at the edge 513.89 requests/s", rounded up // T4: resolver shares are not equal, share ~ 1/(j+1); the per-second requests are divided, remainder goes to the largest fractional parts. const raw = Array.from({ length: RESOLVERS }, (_, j) => 1 / (j + 1)); const t0 = raw.reduce((a, b) => a + b, 0); export const REQUESTS = raw.map((p) => Math.floor((p / t0) * REQUESTS_PER_SECOND)); for (const [, j] of raw.map((p, j) => [(p / t0) * REQUESTS_PER_SECOND % 1, j]).sort((a, b) => b[0] - a[0]) .slice(0, REQUESTS_PER_SECOND - REQUESTS.reduce((a, b) => a + b, 0))) REQUESTS[j] += 1; export function run({ policy, ttl, outageStart = null, duration = 3600, requests = REQUESTS }) { const s = { accepted: 0, dropped: 0, upstreamQueries: 0, distance: 0, window: 0 }; const zoneRequests = Object.fromEntries(ZONE.map((b) => [b, 0])); const bound = new Array(RESOLVERS).fill(null); // warm-up: cache phases are spread evenly, counting starts at t >= 1 const expires = Array.from({ length: RESOLVERS }, (_, j) => 1 - ttl + ((j * 13) % ttl)); let patternIndex = 0; const isDown = (b, t) => outageStart !== null && b === "b34" && t >= outageStart; const isDetected = (b, t) => isDown(b, t) && t >= outageStart + DETECTION; const authoritative = (group, t) => { if (policy === "geolocation") { const b = GEOLOCATION[group]; return isDetected(b, t) ? FALLBACK[b] : b; } if (policy === "latency-based") { return ZONE.filter((b) => !isDetected(b, t)) .reduce((a, b) => (DISTANCE[group][b] < DISTANCE[group][a] ? b : a)); } for (let k = 0; k < PATTERN.length; k++) { // weighted: next live zone in the pattern const b = PATTERN[(patternIndex + k) % PATTERN.length]; if (!isDetected(b, t)) { patternIndex = (patternIndex + k + 1) % PATTERN.length; return b; } } }; for (let t = 1 - ttl; t <= duration; t++) { for (let j = 0; j < RESOLVERS; j++) { if (t >= expires[j]) { // TTL expired: asks the authoritative server bound[j] = authoritative(GROUP[j % GROUP.length], t); expires[j] = t + ttl; if (t >= 1) s.upstreamQueries += 1; } if (t < 1) continue; const n = requests[j], target = bound[j]; if (isDown(target, t)) { s.dropped += n; s.window = t - outageStart + 1; continue; } s.accepted += n; s.distance += n * DISTANCE[GROUP[j % GROUP.length]][target]; zoneRequests[target] += n; } } const share = Object.fromEntries(ZONE.map((b) => [b, zoneRequests[b] / s.accepted])); const deviation = ZONE.reduce((a, b) => a + Math.abs(share[b] - WEIGHT[b]), 0) / 2; return { ...s, share, deviation, unitDistance: s.distance / s.accepted }; } ``` #### Three Policies on the Same Flow ```js // name/policy.mjs -- three routing policies on the same request flow: zone share, deviation from target share, // distance unit per request, and how many names resolvers ask the authoritative server for import { run, REQUESTS, REQUESTS_PER_SECOND, WEIGHT, ZONE } from "./model.mjs"; const TTL = 300, DURATION = 3600; console.log(`${DURATION} s, ${REQUESTS_PER_SECOND} requests/s, TTL ${TTL} s, ` + `largest resolver's share %${((REQUESTS[0] / REQUESTS_PER_SECOND) * 100).toFixed(2)}`); console.log(`target share: ${ZONE.map((b) => `${b}=%${WEIGHT[b] * 100}`).join(" ")}`); console.log(); console.log("policy | b34 b35 b06 | deviation | unit distance | upstream query"); console.log("----------------|-------------------|-----------|---------------|---------------"); for (const policy of ["geolocation", "latency-based", "weighted"]) { const r = run({ policy, ttl: TTL, duration: DURATION }); console.log(`${policy.padEnd(15)} |` + `${ZONE.map((b) => `%${(r.share[b] * 100).toFixed(1)}`.padStart(6)).join("")} | ` + `${r.deviation.toFixed(3).padStart(9)} | ${r.unitDistance.toFixed(3).padStart(13)} | ` + `${String(r.upstreamQueries).padStart(14)}`); } console.log(); for (const ttl of [30, 60, 300, 900]) { const r = run({ policy: "weighted", ttl, duration: DURATION }); console.log(`weighted, ttl ${String(ttl).padStart(3)} s -> deviation ${r.deviation.toFixed(3)}, ` + `upstream query ${String(r.upstreamQueries).padStart(4)}`); } // T4 sensitivity: how much the same policy would deviate if resolver shares were equal const EQUAL = Array.from({ length: 40 }, (_, j) => (j < 34 ? 13 : 12)); // totals 514 const e = run({ policy: "weighted", ttl: TTL, duration: DURATION, requests: EQUAL }); console.log(`\nT4 sensitivity: with equal shares (ttl ${TTL}) deviation ${e.deviation.toFixed(3)}`); ``` ``` 3600 s, 514 requests/s, TTL 300 s, largest resolver's share %23.35 target share: b34=%50 b35=%30 b06=%20 policy | b34 b35 b06 | deviation | unit distance | upstream query ----------------|-------------------|-----------|---------------|--------------- geolocation | %57.6 %25.3 %17.1 | 0.076 | 1.710 | 480 latency-based | %55.1 %25.3 %19.6 | 0.051 | 1.171 | 480 weighted | %50.6 %27.8 %21.6 | 0.022 | 2.144 | 480 weighted, ttl 30 s -> deviation 0.082, upstream query 4800 weighted, ttl 60 s -> deviation 0.195, upstream query 2400 weighted, ttl 300 s -> deviation 0.022, upstream query 480 weighted, ttl 900 s -> deviation 0.037, upstream query 160 T4 sensitivity: with equal shares (ttl 300) deviation 0.000 ``` These numbers belong to the measurement class: they come out of a model run on this machine. The distance unit depends on the model's parameters; the request counts do not. The first table shows two metrics moving in opposite directions. The latency-based policy finds the closest placement, 1.171 distance units per request, against 1.710 for geolocation and 2.144 for weighted — 1.83 times the distance of the closest-zone policy, a ratio independent of unit choice since both runs share the same matrix. But weighted gives the distribution closest to the target share: deviation 0.022, against 0.076 for geolocation. Proximity and share control are not achieved together in the same policy. Geolocation's 1.710 comes from the hand-written table: in two of the four client clusters it misses the nearest zone. Cluster `g3`'s entry says `b34`, but the nearest zone in the matrix is `b06`; `g4`'s says `b06`, but the nearest is `b34`. When geographic and network proximity diverge, the policy silently sends traffic to the wrong zone — visible only once it is measured. The second table exposes the weighted policy's real problem: deviation does not improve as the TTL shortens — 0.082, 0.195, 0.022, 0.037, oscillating without a pattern — while upstream queries climb from 160 to 4,800, thirtyfold. A short TTL multiplies the price paid by thirty without bringing the result any closer to the target share. The last line gives the reason: when resolver shares are equalized, the same policy's deviation drops to 0.000. The problem is not the TTL but the control unit — the policy distributes share per resolution, while load arrives per request. The single largest resolver alone carries 23.35% of all requests, more than zone `b06`'s entire 20% target share; whichever zone it is bound to overshoots. The name layer is not a request-level control instrument. #### The Failover Window When a zone becomes unreachable, the authoritative server removes it from the answers, but the answers already sitting in caches at that moment stay valid. The failover window therefore has two parts: the zone being marked dead (T5) and the cached answer's TTL running out (T1). ```js // name/failover.mjs -- b34 becomes unreachable at second 100. TTL determines the failover // window and the requests dropped in it; the budget figures are taken from the K01 computations. import { run, DETECTION, REQUESTS_PER_SECOND } from "./model.mjs"; const OUTAGE = 100, DURATION = 1200; const MONTHLY_BUDGET_S = (1 - 99.9 / 100) * 30 * 24 * 3600; // K01: 99.9% -> 43.2 min/month const K01_PASSIVE_DROPPED = 1541.67; // K01: active-passive, 3 s window console.log(`b34 goes down at second ${OUTAGE}, the health check marks it ${DETECTION} s later.`); console.log(`monthly downtime budget = ${(MONTHLY_BUDGET_S / 60).toFixed(1)} min (99.9%)`); console.log(); console.log("ttl | window | dropped req | equiv. downtime | budget share | upstream query"); console.log("----|--------|-------------|------------------|---------------|---------------"); for (const ttl of [30, 60, 300, 900]) { const r = run({ policy: "geolocation", ttl, outageStart: OUTAGE, duration: DURATION }); const equivalent = r.dropped / REQUESTS_PER_SECOND; console.log(`${String(ttl).padStart(3)} | ${String(r.window).padStart(4)} s | ` + `${String(r.dropped).padStart(11)} | ${equivalent.toFixed(2).padStart(14)} s | ` + `${`%${((equivalent / MONTHLY_BUDGET_S) * 100).toFixed(2)}`.padStart(13)} | ` + `${String(r.upstreamQueries).padStart(15)}`); } const r = run({ policy: "geolocation", ttl: 300, outageStart: OUTAGE, duration: DURATION }); console.log(); console.log(`ttl 300: window ${r.window} s, equivalent downtime ` + `${(r.dropped / REQUESTS_PER_SECOND).toFixed(2)} s -> ` + `%${((r.dropped / (r.window * REQUESTS_PER_SECOND)) * 100).toFixed(1)} of the window's traffic was dropped`); console.log(`K01's active-passive failover dropped ${K01_PASSIVE_DROPPED} requests in 3 s; ` + `this failover drops ${(r.dropped / K01_PASSIVE_DROPPED).toFixed(1)} times that`); ``` ``` b34 goes down at second 100, the health check marks it 10 s later. monthly downtime budget = 43.2 min (99.9%) ttl | window | dropped req | equiv. downtime | budget share | upstream query ----|--------|-------------|------------------|---------------|--------------- 30 | 39 s | 6666 | 12.97 s | %0.50 | 1600 60 | 69 s | 9486 | 18.46 s | %0.71 | 800 300 | 305 s | 57906 | 112.66 s | %4.35 | 160 900 | 905 s | 190206 | 370.05 s | %14.28 | 64 ttl 300: window 305 s, equivalent downtime 112.66 s -> %36.9 of the window's traffic was dropped K01's active-passive failover dropped 1541.67 requests in 3 s; this failover drops 37.6 times that ``` The window tracks the TTL almost one to one — 39, 69, 305, 905 seconds — so the TTL stops being merely a cache setting and becomes the failover duration itself. The number the name owner picks to reduce query load is the same number that decides how long traffic keeps hitting a dead address once a zone fails. The gap between dropped requests and equivalent downtime confirms a distinction from the introductory course: at a 300-second TTL the window is 305 seconds, but the dropped requests' equivalent is only 112.66 — 36.9% of the window's traffic, not all of it. The Availability in Numbers lesson said time-based and request-based measurement diverge; a zone failover is exactly where that lands. Someone counting by time sees 305 seconds of downtime, someone counting by requests sees 112.66 — both describing the same event. The budget share column settles the decision. At a 99.9% target, the monthly budget is 43.2 minutes; a single zone failover spends 14.28% of it at a 900-second TTL, 4.35% at 300 seconds, 0.50% at 30 seconds. The cost of the same event changes twenty-eightfold, and the only thing that changed is a number in the name record. The comparison closes on the last line: the active–passive failover in the Availability Patterns lesson dropped 1,541.67 requests in a three-second window; the name layer's failover, at the baseline TTL, drops 37.6 times that many. The name layer is the slowest failover instrument available — seconds at the replica level become minutes here. Design consequence: use the name layer for zone selection, not as the sole failure response — the stop that closes the window is the subject of the following lessons. #### Summary - Name resolution is a routing decision: the geolocation policy answers from a coarse table, the latency-based policy from measured proximity, and the weighted policy from a target share. - On the same flow, distance units per request were 1.171 for latency-based, 1.710 for geolocation, and 2.144 for weighted; weighted gave the distribution closest to the target share (deviation 0.022). - Writing the geolocation table by hand is a measurable flaw: in two of the four client clusters, the table did not point to the nearest zone. - The weighted policy's deviation did not improve as the TTL shortened (0.082–0.195, oscillating), but upstream queries rose thirtyfold; equalizing shares dropped the deviation to 0.000 — the control unit is the resolution, but load arrives per request. - The failover window tracks the TTL (39, 69, 305, 905 s), and a single zone failover takes up anywhere from 0.50% to 14.28% of the monthly downtime budget. - At a 300-second TTL the window is 305 seconds but the equivalent downtime is 112.66 seconds; the name layer's failover drops 37.6 times as many requests as the replica-level one. #### Next Step The name layer chose which zone the request would go to, paying for that choice in the distance unit and the failover window. But what it chose was an address — the request still has not reached the application once it arrives there. In that gap sits a stop that can answer the request without letting it reach the application. Caching was built in the Caching, Queues and Asynchronous Processing course, but there the cache sat next to the application, as the introductory course's arithmetic assumed too: a 90% hit rate brought `reads behind cache/s` down to 41.67. The next lesson relocates the same cache — next to the user instead of the application — and measures what happens to the requests reaching the application, the bytes crossing the boundary, and the introductory course's computations. ### Content Delivery Networks Source: https://academia.sh/en/courses/traffic-layer/content-delivery-networks Treating the edge cache as a location decision: measuring wire bytes with local origin and edge processes, counting the edge hit rate and the requests reaching the application with an in-process trace, showing that chaining two caches in series leaves the second one useless, and showing which of the introductory course's computations changes. The name layer chose which zone the request would go to, but what it chose was an address — the request still has not reached the application once it arrives there. In that gap sits a stop that can answer the request without letting it reach the application. Caching was built in the Caching, Queues and Asynchronous Processing course, where the cache sat next to the application, as the Introduction to System Design course's arithmetic assumed too: a 90% hit rate brought `reads behind cache/s` down to 41.67. This lesson relocates the same cache; strategy, invalidation, and key design are not repeated, since the only thing that changes is where the cache sits. A **content delivery network** keeps the same content at many points close to the user, and each such point is an **edge cache**. The question: at the same hit rate, next to the user instead of next to the application, which number changes and which does not. #### What the Location Decision Changes A cache next to the application versus next to the user separates three things at the same hit rate. First, a hit never reaches the application — it drops out of the load application processes see. Second, a hit's response does not use the link between origin and edge, so bytes crossing the boundary fall. Third, a miss now passes through three stops instead of two. None of this grows the hit rate on its own. What this lesson measures is that location does not change the rate — it changes what the rate is worth. #### Bytes on the Wire The first setup is two local processes: origin and edge, measuring how many bytes the same body takes up on a real connection. The introductory course's calculation left this out, saying header and protocol overhead would be a separate assumption; here a measurement replaces it. ```js // edge/server.mjs — two local processes: the origin listens on 8471, the edge cache on 8472. // Cache capacities are three keys to keep the example small; both expose their counters at /counter. import http from "node:http"; const BODY = 480; // K01 V5: tracking response body const body = (num) => { const head = JSON.stringify({ tracking: num, state: "in-transit", zone: "b34" }); return head + " ".repeat(BODY - head.length); }; const newCache = (capacity) => ({ capacity, m: new Map(), get(num) { const b = this.m.get(num); if (b !== undefined) this.m.delete(num); return b; }, put(num, b) { if (this.m.size >= this.capacity) this.m.delete(this.m.keys().next().value); this.m.set(num, b); }, }); const counterRoute = (s) => (request, response) => { response.writeHead(200, { "content-type": "application/json" }); response.end(JSON.stringify(s)); }; const originStats = { requests: 0, hits: 0, stored: 0 }; const originCache = newCache(3); http.createServer((request, response) => { if (request.url === "/counter") return counterRoute(originStats)(request, response); const num = request.url.slice("/tracking/".length); originStats.requests += 1; let b = originCache.get(num); if (b === undefined) { originStats.stored += 1; b = body(num); } else originStats.hits += 1; originCache.put(num, b); response.writeHead(200, { "content-type": "application/json", "content-length": BODY }); response.end(b); }).listen(8471); const edgeStats = { requests: 0, hits: 0, misses: 0, originRequests: 0, originBytes: 0 }; const edgeCache = newCache(3); http.createServer(async (request, response) => { if (request.url === "/counter") return counterRoute(edgeStats)(request, response); const num = request.url.slice("/tracking/".length); edgeStats.requests += 1; let b = edgeCache.get(num), status = "hit"; if (b === undefined) { status = "miss"; edgeStats.misses += 1; edgeStats.originRequests += 1; b = await (await fetch(`http://127.0.0.1:8471/tracking/${num}`)).text(); edgeStats.originBytes += Buffer.byteLength(b); } else edgeStats.hits += 1; edgeCache.put(num, b); response.writeHead(200, { "content-type": "application/json", "content-length": Buffer.byteLength(b), "x-edge": status }); response.end(b); }).listen(8472); ``` ```bash # edge-measure.sh — first one direct request to the origin, then the same two tracking numbers # queried three times each through the edge. Header bytes depend on the header set the server # produces; the ratio between body and wire is not tied to that. node edge/server.mjs & SERVER=$! until curl -sf http://127.0.0.1:8472/counter >/dev/null; do sleep 0.2; done measure() { curl -s -o /dev/null -D - "$1" -w '%{size_download} %{size_header}\n' 2>/dev/null; } printf '%-18s %-7s %6s %7s %5s\n' request edge body header wire measure http://127.0.0.1:8471/tracking/TR900010 | awk 'END{printf "%-18s %-7s %6d %7d %5d\n", "origin direct", "-", $1, $2, $1+$2}' for round in 1 2 3; do for num in TR900011 TR900012; do measure "http://127.0.0.1:8472/tracking/$num" | awk -v n="$num" -v t="$round" '/^x-edge/{k=$2; sub(/\r/, "", k)} END{printf "%-18s %-7s %6d %7d %5d\n", "edge " n " r" t, k, $1, $2, $1+$2}' done done printf '\nedge counter = %s\n' "$(curl -s http://127.0.0.1:8472/counter)" printf 'origin counter = %s\n' "$(curl -s http://127.0.0.1:8471/counter)" kill $SERVER ``` ``` request edge body header wire origin direct - 480 156 636 edge TR900011 r1 miss 480 170 650 edge TR900012 r1 miss 480 170 650 edge TR900011 r2 hit 480 169 649 edge TR900012 r2 hit 480 169 649 edge TR900011 r3 hit 480 169 649 edge TR900012 r3 hit 480 169 649 edge counter = {"requests":6,"hits":4,"misses":2,"originRequests":2,"originBytes":960} origin counter = {"requests":3,"hits":0,"stored":3} ``` These numbers belong to the measurement class. Header bytes depend on the server's header set and differ in another implementation; the ratio between body and wire, 649 / 480 = 1.352, carries forward as a lower bound, since the header set can only grow. The counters give the location decision's first result: the edge saw six requests, two misses, four hits. Only two requests reached the origin — the second and third query for the same two tracking numbers never got there. The origin's own counter shows three, because the direct measurement counted too, and its hit counter is zero: requests reaching it are so sparse that its own cache never helped. A small preview of the result the second half of the lesson measures. #### Where the Hit Rate Comes From The second setup is an in-process model, labeled as one: the same request trace runs first with a single cache, then with an edge cache placed in front of it. The trace is this course's assumption; the rates and counts are measurements. The model's input derives from the introductory course: 12,000,000 daily tracking queries, 400,000 new shipments a day. One assumption is added — **T6: a shipment stays in transit for 3 days** — giving 1,200,000 active shipments; daily queries per shipment is 12,000,000 / 1,200,000 = 10, a computed value. Ten queries are spread across a shipment's three-day life. A cache entry cannot live three days, so what sets the rate is how many of the ten land in the same short window: a user opens the tracking page and refreshes it a few times. This is the second assumption — **T7: burst length**, how many of the ten queries share a burst. The third is **T8: 100 bursts** open at once; the fourth is **T9: each cache holds 1,000 keys**. ```js // edge/trace.mjs — in-process model: the same request trace runs first with a single cache, // then with an edge cache placed in front of it. The trace is an assumption (T7); ratios and counts are measurements. const REQUESTS = 200_000; // K01 peak read 416.67 requests/s x 480 s const QUERIES = 10; // computed value: V1*V2 / (V3*T6) = 12,000,000 / 1,200,000 const OPEN = 100; // T8: tracking pages open at the same time (burst) const CAPACITY = 1_000; // T9: each cache's key capacity function* trace(burst) { // burst: how many of a shipment's 10 queries land in the same burst const open = []; let fresh = 1, single = 1, index = 0; for (let i = 0; i < REQUESTS; i++) { if (i % QUERIES >= burst) { yield `D${single++}`; continue; } // key seen once in the window while (open.length < OPEN) open.push({ id: `G${fresh++}`, remaining: burst }); index = (index + 1) % open.length; const o = open[index]; yield o.id; if (--o.remaining === 0) open.splice(index, 1); } } class Cache { // most-recently-used sits last; the Map's insertion order is enough constructor(k) { this.k = k; this.m = new Map(); this.hits = 0; this.misses = 0; } probe(a) { if (this.m.has(a)) { this.hits += 1; this.m.delete(a); this.m.set(a, 1); return true; } this.misses += 1; if (this.m.size >= this.k) this.m.delete(this.m.keys().next().value); this.m.set(a, 1); return false; } get rate() { return this.hits / (this.hits + this.misses); } } export function run(burst) { const single = new Cache(CAPACITY); const edge = new Cache(CAPACITY), app = new Cache(CAPACITY); for (const a of trace(burst)) { single.probe(a); if (!edge.probe(a)) app.probe(a); } return { requests: REQUESTS, singleRate: single.rate, singleStored: single.misses, edgeRate: edge.rate, appRequests: edge.misses, appRate: app.rate, edgeStored: app.misses }; } if (process.argv[1].endsWith("trace.mjs")) { // does not print when imported, only exposes run console.log(`${REQUESTS} requests, ${QUERIES} queries per shipment, ${OPEN} open bursts, ` + `cache capacity ${CAPACITY} keys`); console.log(); console.log("burst | single cache | edge hit | reaching app | app hit | reaching store"); console.log("------|--------------|----------|---------------|---------|----------------"); for (const burst of [10, 5, 2]) { const r = run(burst); console.log(`${String(burst).padStart(5)} | ${r.singleRate.toFixed(4).padStart(12)} | ` + `${r.edgeRate.toFixed(4).padStart(8)} | ${String(r.appRequests).padStart(13)} | ` + `${r.appRate.toFixed(4).padStart(7)} | ${String(r.edgeStored).padStart(14)}`); } const r = run(10); console.log(); console.log(`burst 10: requests reaching store without edge ${r.singleStored}, with edge ${r.edgeStored} ` + `(x${(r.edgeStored / r.singleStored).toFixed(3)})`); console.log(`requests reaching app ${r.requests} -> ${r.appRequests} ` + `(x${(r.appRequests / r.requests).toFixed(3)})`); console.log(`measured edge hit rate ${r.edgeRate.toFixed(4)}, V9 assumption 0.90 -> difference ` + `${(Math.abs(r.edgeRate - 0.9) * 100).toFixed(2)} points`); } ``` ``` 200000 requests, 10 queries per shipment, 100 open bursts, cache capacity 1000 keys burst | single cache | edge hit | reaching app | app hit | reaching store ------|--------------|----------|---------------|---------|---------------- 10 | 0.8998 | 0.8998 | 20046 | 0.0000 | 20046 5 | 0.3998 | 0.3998 | 120044 | 0.0000 | 120044 2 | 0.0759 | 0.0759 | 184815 | 0.0021 | 184418 burst 10: requests reaching store without edge 20046, with edge 20046 (x1.000) requests reaching app 200000 -> 20046 (x0.100) measured edge hit rate 0.8998, V9 assumption 0.90 -> difference 0.02 points ``` The first line exposes an assumption from the introductory course: the cache hit ratio there, V9 = 0.90, was justified only as "the same tracking number gets asked again within a short interval." At burst length 10, the measured rate is 0.8998 — a 0.02-point difference — so V9 = 0.90 was quietly assuming all ten of a shipment's queries land in the same burst. Sensitivity shows in the next two lines: at burst 5 the rate is 0.3998, at burst 2 it is 0.0759. Half the queries falling outside the burst drops the rate from 0.90 to 0.40. A hit rate is not a cache property — it is a trace property. The second result is the location decision's real finding: adding the edge cache dropped requests reaching the application from 200,000 to 20,046 — a tenth. Requests reaching the store did not change at all — 20,046 with or without the edge, a ratio of 1.000 — because the application cache's hit rate is 0.0000. The edge cache absorbs every repeat, so only first-seen keys reach the cache behind it, and a first-seen key cannot be found in any cache. This is a general property of chaining two caches in series: if the last cache holds every distinct key in the window, requests reaching the store are independent of the caches in front of it. The third row shows the limit: at burst 2 the edge cache is already failing (0.0759), and the cache behind it adds a small contribution (0.0021), bringing requests reaching the store from 184,815 to 184,418. The second cache only earns its keep once the first fails. Capacity and hit rate's relationship was measured in the Cache Metrics lesson; added here is that caching the same key twice is not free the second time. #### Back to the Numbers ```js // edge/compute.mjs — folding the measured hit rate and wire bytes back into the K01 computations import { run } from "./trace.mjs"; const PEAK_READ = 416.67, PEAK_WRITE = 97.22; // K01 calculation const V5 = 480, V9 = 0.90; // K01 assumption table const K01_BEHIND_CACHE = 41.67, K01_STORE = 138.89, K01_EGRESS = 1.60; const WIRE_EDGE = 649, WIRE_ORIGIN = 636; // measurement: body + header (local processes) const r = run(10); const app = PEAK_READ * (1 - r.edgeRate); const storeReads = PEAK_READ * (r.edgeStored / r.requests); const mbit = (requests, bytes) => (requests * bytes * 8) / 1e6; console.log(`measured edge hit rate = ${r.edgeRate.toFixed(4)} (V9 assumption ${V9})`); console.log(); console.log(`${"calculation".padEnd(28)}${"K01".padStart(9)}${"with edge".padStart(11)} note`); const row = (name, before, after, note) => console.log(`${name.padEnd(28)}${before.toFixed(2).padStart(9)}${after.toFixed(2).padStart(11)} ${note}`); row("reads reaching app/s", PEAK_READ, app, "edge cache removed these"); row("reads behind cache/s", K01_BEHIND_CACHE, storeReads, "unchanged"); row("requests reaching store/s", K01_STORE, storeReads + PEAK_WRITE, "unchanged"); row("read egress Mbit/s", K01_EGRESS, mbit(PEAK_READ, WIRE_EDGE), "K01 counted body only"); row("origin egress Mbit/s", K01_EGRESS, mbit(app, WIRE_ORIGIN), "edge sits in front of the origin"); console.log(); console.log(`wire/body ratio = ${(WIRE_EDGE / V5).toFixed(3)}; K01's ${K01_EGRESS.toFixed(2)} Mbit/s is a lower bound`); console.log(`origin egress / user egress = ` + `${(mbit(app, WIRE_ORIGIN) / mbit(PEAK_READ, WIRE_EDGE)).toFixed(4)}`); console.log(`app cache's contribution = ${(r.appRate * 100).toFixed(2)}% hit`); ``` ``` measured edge hit rate = 0.8998 (V9 assumption 0.9) calculation K01 with edge note reads reaching app/s 416.67 41.76 edge cache removed these reads behind cache/s 41.67 41.76 unchanged requests reaching store/s 138.89 138.98 unchanged read egress Mbit/s 1.60 2.16 K01 counted body only origin egress Mbit/s 1.60 0.21 edge sits in front of the origin wire/body ratio = 1.352; K01's 1.60 Mbit/s is a lower bound origin egress / user egress = 0.0982 app cache's contribution = 0.00% hit ``` The table gives four results at once. Reads reaching the application drop from 416.67 to 41.76 — a number the introductory course never computed, since there the cache sat inside the application and reaching it was unavoidable. `reads behind cache/s` and `requests reaching store/s` do not change (41.67 against 41.76, 138.89 against 138.98; the gap is the trace measurement's 0.02-point deviation). The edge cache takes the same 0.90 and moves it from one place to another. Bandwidth gets corrected in both directions: egress to the user, with measured wire bytes, rises from 1.60 to 2.16 Mbit/s — the introductory course's number was a lower bound, since it counted only the body. Origin egress is 0.21 Mbit/s, 0.0982 of egress to the user. One link in the same design demands 2.16 Mbit/s, the other 0.21, and the introductory course's single number cannot stand for both. #### Summary - The edge cache is a location decision: a hit never reaches the application and does not use the link between origin and edge, but the hit rate does not grow because of this decision. - The same body's footprint on a real connection was measured: a 480-byte body takes up 649 bytes on the wire, a ratio of 1.352; the introductory course's 1.60 Mbit/s is therefore a lower bound. - At a burst length of 10, the measured hit rate came out to 0.8998: the V9 = 0.90 assumption assumed all ten of a shipment's queries land in the same burst. At burst 5 the rate falls to 0.3998. - The edge cache brought requests reaching the application from 200,000 down to 20,046, but requests reaching the store stayed at 20,046 (ratio 1.000), and the application cache's hit rate was 0.0000. - If the last cache can hold every distinct key in the window, requests reaching the store are independent of the caches in front of it; the second cache only contributes once the first one fails. - Back to the numbers: reads reaching the application fell from 416.67 to 41.76, `requests reaching store/s` stayed at 138.89, and origin egress was 0.21 Mbit/s, 0.0982 of egress to the user. #### Next Step The edge cache in this lesson waited for the request: the first time a key was asked for, it missed, went to the origin, and stored the response. The measurement also counted those first requests — 20,046 of the 200,000 missed at the edge, each a trip to the origin. This waiting is not mandatory: content can be sent to the edge before it is ever asked for. The choice between the two models changes the miss count, the content volume held at the edge, and how long an update takes to become visible at every edge. The next lesson measures both on the same setup and shows which fits which kind of content. ### Push and Pull Based Distribution Source: https://academia.sh/en/courses/traffic-layer/push-and-pull-based-distribution Propagating content to edges with two distinct models: pull-based distribution producing the first miss and a stale-version window, push-based distribution producing wasted copies and a message multiplier, choosing between the two by content class, and the choice's effect on the introductory course's write bandwidth and storage calculations. The edge cache in the previous lesson waited for the request: the first time a key was asked for, it missed, went to the origin, and stored the response. The measurement also counted those first requests — 20,046 of the 200,000 requests missed at the edge, each a trip to the origin. This waiting is not mandatory. Content can be sent to the edge before it is ever asked for. There are two ways, and the choice between them is a propagation model decision. **Pull-based distribution** pulls content on the first request: the edge misses, goes to the origin, stores the response. **Push-based distribution** sends content to edges before any request arrives: the edge never misses, but whether the pushed content will ever be requested is unknown. This lesson measures both models on the same content set. #### What Marks the Difference The difference between the two models shows up in three places. First, **origin requests**: in pull, every edge goes to the origin at least once per object; in push, never. Second, **wasted copies**: push sends an object even to edges that will never request it. Third, **stale versions**: when an object changes, pull's edges keep serving the old version until their TTL expires, while push sends the new version to every edge in one step. All three tie back to a single quantity: how many edges request an object. If every edge requests it, push wastes nothing; if only one edge does, all but one of push's copies are wasted. This lesson measures that. #### Setup The model is in-process, and a round is an abstract step, the same kind of abstraction as the introductory course's round. Content splits into two classes. **Common** objects are the tracking page's shell, style, and script files: requested from every edge. **Unique** objects are per-shipment tracking responses, and their count comes from the previous lesson's measurement — 20,046 distinct responses in the window. | Code | Assumption | Value | Rationale | |---|---|---|---| | T10 | edge count | 12 | four entry points in each of the three zones | | T11 | common object count | 20 | shell, style, script, and icon files | | T12 | edges requesting a unique object | 1 | a shipment's queries come from the same user, the same edge | ```js // propagation/model.mjs — in-process model: the same content set is distributed first pull // based, then push based. A round is an abstract step; TTL and the update round are parameters. export const EDGES = 12; // T10: edge count export const TTL = 300; // T1: TTL (rounds) export const ROUNDS = 900, UPDATE = 100; // T11: common objects are requested from every edge. T12: a shipment's queries come from a // single edge, so a unique object is requested from only one edge. UNIQUE is the previous lesson's measurement. export const CLASS = [ { name: "common", objects: 20, requesting: EDGES }, { name: "unique", objects: 20_046, requesting: 1 }, ]; export function cost(cls) { const pull = cls.objects * cls.requesting; // one trip to the origin on the first miss const push = cls.objects * EDGES; // the set is pushed to every edge return { pull, push, wasted: push - pull, wastedShare: 1 - cls.requesting / EDGES }; } // mode: "pull", "push", or "hybrid" (common objects pushed, unique objects pulled) const modeOf = (mode, name) => (mode === "hybrid" ? (name === "common" ? "push" : "pull") : mode); export function propagate(mode) { const s = { origin: 0, pushed: 0, wasted: 0, stale: 0, messages: 0 }; for (const cls of CLASS) { const m = cost(cls); if (modeOf(mode, cls.name) === "push") { s.pushed += m.push; s.wasted += m.wasted; s.messages += m.push; } else { s.origin += m.pull; s.messages += m.pull; } } const commonMode = modeOf(mode, "common"); const version = new Array(EDGES).fill(1); // the common object's version each edge sees const expires = Array.from({ length: EDGES }, (_, k) => 1 + ((k * 27) % TTL)); for (let round = 1; round <= ROUNDS; round++) { if (commonMode === "push" && round === UPDATE) { version.fill(2); s.messages += EDGES; } for (let k = 0; k < EDGES; k++) { if (commonMode === "pull" && round >= expires[k]) { // TTL expired, refetched version[k] = round >= UPDATE ? 2 : 1; expires[k] = round + TTL; s.origin += 1; s.messages += 1; } if (round >= UPDATE && version[k] === 1) s.stale += 1; // round in which the stale version was served } } return s; } ``` ```js // propagation/measure.mjs — comparing two propagation modes on the same content set import { propagate, cost, CLASS, EDGES, TTL, ROUNDS, UPDATE } from "./model.mjs"; console.log(`${EDGES} edges, TTL ${TTL} rounds, ${ROUNDS} rounds, ` + `common object's version changes at round ${UPDATE}`); console.log(); console.log("class | objects | requesting edges | pull req | push req | wasted | wasted share"); console.log("-------|---------|-------------------|----------|----------|--------|-------------"); for (const cls of CLASS) { const m = cost(cls); console.log(`${cls.name.padEnd(6)} | ${String(cls.objects).padStart(7)} | ` + `${String(cls.requesting).padStart(17)} | ${String(m.pull).padStart(8)} | ` + `${String(m.push).padStart(8)} | ${String(m.wasted).padStart(6)} | ` + `${`%${(m.wastedShare * 100).toFixed(1)}`.padStart(11)}`); } console.log(); console.log("mode | origin req | pushed obj | wasted | stale version rounds | messages"); console.log("-------|------------|------------|--------|-----------------------|---------"); const record = {}; for (const mode of ["pull", "push", "hybrid"]) { const r = record[mode] = propagate(mode); console.log(`${mode.padEnd(6)} | ${String(r.origin).padStart(10)} | ` + `${String(r.pushed).padStart(10)} | ${String(r.wasted).padStart(6)} | ` + `${String(r.stale).padStart(21)} | ${String(r.messages).padStart(8)}`); } const pull = record.pull, push = record.push; console.log(); console.log(`message ratio push/pull = ${(push.messages / pull.messages).toFixed(2)}`); console.log(`stale version rounds, average per edge = ${(pull.stale / EDGES).toFixed(1)} rounds ` + `(${((pull.stale / EDGES) / TTL).toFixed(2)} times the TTL)`); const two = cost({ ...CLASS[1], requesting: 2 }); console.log(`T12 sensitivity: if the unique object were requested from two edges, wasted share ` + `%${(two.wastedShare * 100).toFixed(1)} (%${(cost(CLASS[1]).wastedShare * 100).toFixed(1)} at one edge)`); ``` ``` 12 edges, TTL 300 rounds, 900 rounds, common object's version changes at round 100 class | objects | requesting edges | pull req | push req | wasted | wasted share -------|---------|-------------------|----------|----------|--------|------------- common | 20 | 12 | 240 | 240 | 0 | %0.0 unique | 20046 | 1 | 20046 | 240552 | 220506 | %91.7 mode | origin req | pushed obj | wasted | stale version rounds | messages -------|------------|------------|--------|-----------------------|--------- pull | 20322 | 0 | 0 | 1794 | 20322 push | 0 | 240792 | 220506 | 0 | 240804 hybrid | 20046 | 240 | 0 | 0 | 20298 message ratio push/pull = 11.85 stale version rounds, average per edge = 149.5 rounds (0.50 times the TTL) T12 sensitivity: if the unique object were requested from two edges, wasted share %83.3 (%91.7 at one edge) ``` These numbers belong to the measurement class; their inputs are the T10–T12 assumptions and the previous lesson's 20,046 measurement. The first table confirms the selection criterion. In the common class, pull and push request the same amount (240 to 240) with zero wasted copies, since the object is already requested from all twelve edges. In the unique class, push sends 240,552 copies, 220,506 of which will never be requested: a wasted share of 91.7%. The same criterion shows up as a formula — the wasted share is the complement of the ratio of requesting edges to total edges. Playing T12 drops the share from 91.7% to 83.3%; even doubled, the assumption still leaves push heavily wasted in the unique class. The second table shows where the cost lands. Pull generates 20,322 requests to the origin and, in return, serves stale versions for 1,794 rounds — an average of 149.5 rounds per edge, half the TTL. Push produces no stale versions but sends 240,804 messages, 11.85 times pull's count. The two models pay for the same work in different currencies: one in stale responses, the other in wasted messages. This mirrors the tradeoff between the two failover patterns in the introductory course, where the cost split between downtime and lost data. The third row shows the decision is made at the content-class level, not the system level. The hybrid mode pushes common objects and pulls unique ones: 20,046 origin requests, 0 wasted copies, 0 stale-version rounds, 20,298 messages — better than both other modes on all three counts. A design forced to pick a single model pays unnecessary cost on one of the two classes. #### Back to the Numbers ```js // propagation/compute.mjs — which of K01's computations change when a propagation mode is chosen import { propagate, cost, CLASS, EDGES } from "./model.mjs"; const PEAK_READ = 416.67, PEAK_WRITE = 97.22, DAILY_EVENTS = 2_800_000; // K01 calculation const V5 = 480, ACTIVE = 1_200_000; // K01 V5; active shipment count computed with T6 const K01_BEHIND_CACHE = 41.67, K01_WRITE = 0.17, K01_READ = 1.60; const WINDOW = 480; // s: the previous lesson's measurement window const pull = propagate("pull"), hybrid = propagate("hybrid"); const mbit = (rate, bytes) => (rate * bytes * 8) / 1e6; const pushRate = PEAK_WRITE * EDGES; const staleResponses = pull.stale * (PEAK_READ / EDGES); const pushGB = (ACTIVE * V5 * EDGES) / 1e9, pullMB = (CLASS[1].objects * V5) / 1e6; console.log(`pull: ${pull.origin} requests to the origin / ${WINDOW} s = ${(pull.origin / WINDOW).toFixed(2)} ` + `requests/s -- K01 reads behind cache/s = ${K01_BEHIND_CACHE}`); console.log(`hybrid: ${(hybrid.origin / WINDOW).toFixed(2)} requests/s to the origin, no cold start ` + `miss because the common object is pushed`); console.log(); console.log(`push: every state event goes to ${EDGES} edges`); console.log(` peak push rate = ${pushRate.toFixed(2)} pushes/s, ${DAILY_EVENTS * EDGES} pushes/day`); console.log(` push bandwidth = ${mbit(pushRate, V5).toFixed(2)} Mbit/s = ` + `${(mbit(pushRate, V5) / K01_WRITE).toFixed(1)} times K01's write ingress (${K01_WRITE}), ` + `${(mbit(pushRate, V5) / K01_READ).toFixed(2)} times its read egress (${K01_READ.toFixed(2)})`); console.log(` unique content held at the edges = ${pushGB.toFixed(2)} GB ` + `(${(ACTIVE * V5 / 1e6).toFixed(0)} MB x ${EDGES} edges)`); console.log(` same content in pull mode = ${pullMB.toFixed(2)} MB, ratio ` + `${((pushGB * 1e3) / pullMB).toFixed(0)}`); console.log(); console.log(`stale responses from one version change in pull mode = ` + `${staleResponses.toFixed(0)} (${pull.stale} rounds x ${(PEAK_READ / EDGES).toFixed(2)} requests/s)`); console.log(`unique object's wasted push share = ` + `%${(cost(CLASS[1]).wastedShare * 100).toFixed(1)}; a push is only not wasted when the object is ` + `requested from all ${EDGES} edges`); ``` ``` pull: 20322 requests to the origin / 480 s = 42.34 requests/s -- K01 reads behind cache/s = 41.67 hybrid: 41.76 requests/s to the origin, no cold start miss because the common object is pushed push: every state event goes to 12 edges peak push rate = 1166.64 pushes/s, 33600000 pushes/day push bandwidth = 4.48 Mbit/s = 26.4 times K01's write ingress (0.17), 2.80 times its read egress (1.60) unique content held at the edges = 6.91 GB (576 MB x 12 edges) same content in pull mode = 9.62 MB, ratio 718 stale responses from one version change in pull mode = 62292 (1794 rounds x 34.72 requests/s) unique object's wasted push share = %91.7; a push is only not wasted when the object is requested from all 12 edges ``` The first line shows pull staying consistent with the introductory course: the request rate to the origin is 42.34, against the course's `reads behind cache/s` of 41.67. The two numbers come from separate paths — one from the assumption table, the other from this lesson's model run — and land within two percent of each other. When pull is chosen, the introductory course's arithmetic stays valid as it is. Push breaks that same arithmetic. Since every state event changes the corresponding tracking response, the new version must go to all twelve edges: peak push rate is 1,166.64 pushes/s, 33,600,000 pushes a day. That bandwidth is 4.48 Mbit/s — 26.4 times the course's `write ingress Mbit/s` (0.17) and 2.80 times its `read egress Mbit/s` (1.60). Push turns read load into write load and multiplies it by the edge count. A design's largest bandwidth line item shifts, once push is chosen, to the write side flowing at 97 events per second. Storage moves the same way. Holding 1,200,000 active shipments' tracking responses at every edge costs 576 MB per edge, 6.91 GB across twelve edges. In pull mode, the same content's total footprint at the edges is 9.62 MB — a 718-fold difference. This figure is not added to the course's `stored data GB` calculation, because the edge copies do not replace the store — they sit on top of it. The last line gives pull's bill. When a common object's version changes, the stale version is served for 1,794 rounds, and at 34.72 requests per second per edge, that means 62,292 stale responses. The number ties directly to the TTL and repeats the first lesson's finding: the TTL is not a cache setting but a single decision that determines both the failover duration and the version propagation time. #### Summary - The propagation model is a decision: pull-based distribution pulls content on the first request, push-based distribution sends it before any request arrives. - The selection criterion is how many edges request an object; the wasted push share is the complement of the requesting-edge ratio. The measured share was 0% in the common class, 91.7% in the unique class. - Pull generated 20,322 requests to the origin and served stale versions for 1,794 rounds; push produced no stale versions but sent 240,804 messages — 11.85 times pull's count. - Hybrid mode beats both other modes on all three counts (20,046 origin requests, 0 wasted, 0 stale rounds): the decision is made at the content-class level, not the system level. - Pull preserves the introductory course's arithmetic: 42.34 requests/s reach the origin, against the course's `reads behind cache/s` of 41.67. - Push breaks it: a peak of 1,166.64 pushes/s and 4.48 Mbit/s — 26.4 times write ingress; content held at the edges is 6.91 GB, 718 times pull mode's footprint. #### Next Step These two lessons kept content at the edge, but the content always came from the same place: the application itself. The twenty common-class objects — shell, style, script, and icon files — touch none of the application's logic; they sit as fixed byte sequences and are still requested from the application processes' address. The next lesson separates that path: it measures what happens to the requests reaching the application, the number of stops a request touches, and the introductory course's peak edge request rate once content is served from a place separate from the application. ### Static Content Hosting Source: https://academia.sh/en/courses/traffic-layer/static-content-hosting Separating the content path from the application path: measuring the combined and split arrangements with local processes, counting the requests and bytes a page load drops on the application process, and folding asset traffic into the introductory course's peak edge request rate and read egress calculations. The previous two lessons kept content at the edge, but the content always came from the same place: the application itself. The twenty common-class objects — shell, style, script, and icon files — touch none of the application's logic; they sit as fixed byte sequences and are still requested from the application processes' address. This lesson separates that path. The **content path** is what an unchanging file follows up to the request; the **application path** is where a request gets answered by running code. Static file serving was built in the Server-Side Fundamentals course, asset naming and versioning in Rendering Strategies and Infrastructure, and neither is repeated here. The only question: once the paths split, how many requests and bytes reach the application process, and what that changes in the introductory course's arithmetic. #### Two Arrangements **Combined arrangement** means the same process carries both the `/tracking` path and the `/asset` path. There is a single deployment unit and little configuration; in return, the application process does twenty file reads on every page load. **Split arrangement** hands the asset path to a separate process. The application process sees only requests that run code. The difference between the two is not a capability difference — both return the same bytes — it is a placement difference, and its result can be counted. ```js // static/server.mjs — brings up both arrangements at once: the app process on 8473 can carry // both the /tracking and /asset paths, the content process on 8474 carries only /asset. // Asset files are generated at startup. /counter returns the counters and resets them when called. import http from "node:http"; import fs from "node:fs"; const V5 = 480; // K01 V5: tracking response body const ASSET = [["shell.html", 6_000], ["style.css", 40_000], ["app.js", 120_000], ["font.woff2", 28_000], ["logo.svg", 4_000]]; for (let i = 1; i <= 15; i += 1) ASSET.push([`icon-${i}.svg`, 1_200]); // T13 fs.mkdirSync("asset", { recursive: true }); for (const [name, bytes] of ASSET) fs.writeFileSync(`asset/${name}`, "x".repeat(bytes)); const fresh = () => ({ tracking: 0, asset: 0, bytes: 0 }); const writeCounter = (s, response) => { // plain text so it can be read from the shell response.writeHead(200, { "content-type": "text/plain" }); response.end(`${s.tracking} ${s.asset} ${s.bytes}`); Object.assign(s, fresh()); // read, reset }; function startProcess(port, carriesAssets) { const s = fresh(); http.createServer((request, response) => { if (request.url === "/counter") return writeCounter(s, response); if (request.url.startsWith("/asset/")) { if (!carriesAssets) { response.writeHead(404); return response.end(); } const b = fs.readFileSync(`asset/${request.url.slice("/asset/".length)}`); s.asset += 1; s.bytes += b.length; response.writeHead(200, { "content-length": b.length }); return response.end(b); } const b = JSON.stringify({ tracking: request.url.slice("/tracking/".length), state: "in-transit" }); const body = b + " ".repeat(V5 - b.length); s.tracking += 1; s.bytes += V5; response.writeHead(200, { "content-type": "application/json", "content-length": V5 }); response.end(body); }).listen(port); return s; } startProcess(8473, true); // combined arrangement: the app process also carries assets startProcess(8474, true); // content process in the split arrangement ``` ```bash # static-measure.sh — one page load in two arrangements: first the app process also carries the # assets, then the asset path is handed to a separate process. The counter resets when read. node static/server.mjs & SERVER=$! until curl -sf http://127.0.0.1:8473/counter >/dev/null; do sleep 0.2; done ASSET=$(ls asset) load() { # $1 = the asset process's port curl -s -o /dev/null "http://127.0.0.1:8473/tracking/TR900011" for a in $ASSET; do curl -s -o /dev/null "http://127.0.0.1:$1/asset/$a"; done } curl -s -o /dev/null http://127.0.0.1:8473/counter; curl -s -o /dev/null http://127.0.0.1:8474/counter printf '%s assets, %s bytes total\n\n' "$(ls asset | wc -l | tr -d ' ')" "$(cat asset/* | wc -c | tr -d ' ')" printf '%-11s %-9s %8s %6s %9s\n' arrangement process tracking asset bytes load 8473 printf '%-11s %-9s %8s %6s %9s\n' combined app $(curl -s http://127.0.0.1:8473/counter) printf '%-11s %-9s %8s %6s %9s\n' combined content $(curl -s http://127.0.0.1:8474/counter) load 8474 printf '%-11s %-9s %8s %6s %9s\n' split app $(curl -s http://127.0.0.1:8473/counter) printf '%-11s %-9s %8s %6s %9s\n' split content $(curl -s http://127.0.0.1:8474/counter) kill $SERVER ``` ``` 20 assets, 216000 bytes total arrangement process tracking asset bytes combined app 1 20 216480 combined content 0 0 0 split app 1 0 480 split content 0 20 216000 ``` These numbers belong to the measurement class; the asset set is this course's assumption — **T13: 20 assets per page, 216,000 bytes total** — because a tracking page consists of shell, style, script, font, and icon files. The measurement gives what a page load drops on the application process in each arrangement. In the combined arrangement, it handled 21 requests and sent 216,480 bytes. In the split arrangement, the same page load left it with 1 request and 480 bytes; the remaining 20 requests and 216,000 bytes went to the content process. Requests on the application process fell from 21 to 1, bytes from 216,480 to 480, and the only thing that produced this was splitting the path — no cache, no compression was added. The split's second result is the scaling unit. In the combined arrangement, the number of application processes is set by the sum of two separate needs: requests running code and requests reading files. The two needs do not grow in the same direction — more assets on a page does not change the code load. In the split arrangement, the two counts are set independently. #### Two Names, Two TTLs A direct consequence of splitting the path is that the content path gets its own name, which returns to the first lesson's decision. There, the TTL was a single number doing two jobs at once: cutting query load and setting the failover window. The measurement showed a tradeoff — a 900-second TTL brings upstream queries down to 64 but a single zone failover spends 14.28% of the monthly downtime budget; a 30-second TTL cuts that budget share to 0.50% but raises upstream queries to 1,600. With a single name, only one point on that tradeoff can be picked; with two names, two points can be picked, because their failover needs differ. The application name must fail over fast when a zone goes down, paying for it in a short TTL and its query load. The content name points to unchanging files held at twelve edges — when one goes down, the others carry the same bytes, so the failover window never becomes an outage, and a long TTL costs it nothing. The result is that two separate rows of the first lesson's table can be chosen at once: a short TTL for the name that resolves the application, a long one for the content name. Taking both ends of a tradeoff at once happens by splitting the responsibility that created the tradeoff. #### Back to the Numbers The measured twenty-one requests have no counterpart in the introductory course. The course's peak edge requests/s, 513.89, counted only tracking queries and state events; the page itself and its assets never entered the arithmetic. Filling that gap needs one more assumption — **T14: 2 page loads per user per day** — because two of the six tracking queries are a fresh page open and four are refreshes of the same page. ```js // static/compute.mjs — the effect of splitting the content path on K01's computations. The // asset count and bytes per page are taken from the measurement in the previous block. const V1 = 2_000_000, PEAK_FACTOR = 3, DAY = 86_400; // K01 V1, V8 const PEAK_EDGE = 513.89, K01_EGRESS = 1.60; // K01 calculation const T14 = 2; // page loads per user per day const ASSETS = 20, ASSET_BYTES = 216_000; // measurement: previous block const EDGES = 12; // T10: previous lesson's edge count const pagePeak = ((V1 * T14) / DAY) * PEAK_FACTOR; const assetPeak = pagePeak * ASSETS; const mbit = (rate, bytes) => (rate * bytes * 8) / 1e6; const row = (name, requests, bytes) => console.log(`${name.padEnd(34)}${requests.toFixed(2).padStart(10)}${bytes.toFixed(2).padStart(12)}`); console.log(`daily page loads = ${V1 * T14}, peak = ${pagePeak.toFixed(2)} pages/s`); console.log(`${ASSETS} assets per page, ${ASSET_BYTES} bytes -> ` + `peak asset requests = ${assetPeak.toFixed(2)} requests/s`); console.log(); console.log(`${"arrangement".padEnd(34)}${"requests/s".padStart(10)}${"Mbit/s".padStart(12)}`); row("K01 calc. (assets not counted)", PEAK_EDGE, K01_EGRESS); row("combined path, app process", PEAK_EDGE + assetPeak, K01_EGRESS + mbit(pagePeak, ASSET_BYTES)); row("split path, app process", PEAK_EDGE, K01_EGRESS); row("split path, content process", assetPeak, mbit(pagePeak, ASSET_BYTES)); console.log(); console.log(`split path + edge: with assets held at ${EDGES} edges, the origin sees ` + `${ASSETS * EDGES} asset requests/day, ${((ASSET_BYTES * EDGES) / 1e6).toFixed(2)} MB`); console.log(`in the combined path, the app process sees ${((PEAK_EDGE + assetPeak) / PEAK_EDGE).toFixed(2)} ` + `times K01's requests, ${((K01_EGRESS + mbit(pagePeak, ASSET_BYTES)) / K01_EGRESS).toFixed(1)} times its bytes`); console.log(`requests the app sees in one page load: combined ${1 + ASSETS}, split 1`); console.log(`T14 sensitivity: at 1 page load per user, asset requests would be ` + `${(assetPeak / 2).toFixed(2)} requests/s, edge total ` + `${(PEAK_EDGE + assetPeak / 2).toFixed(2)} requests/s`); ``` ``` daily page loads = 4000000, peak = 138.89 pages/s 20 assets per page, 216000 bytes -> peak asset requests = 2777.78 requests/s arrangement requests/s Mbit/s K01 calc. (assets not counted) 513.89 1.60 combined path, app process 3291.67 241.60 split path, app process 513.89 1.60 split path, content process 2777.78 240.00 split path + edge: with assets held at 12 edges, the origin sees 240 asset requests/day, 2.59 MB in the combined path, the app process sees 6.41 times K01's requests, 151.0 times its bytes requests the app sees in one page load: combined 21, split 1 T14 sensitivity: at 1 page load per user, asset requests would be 1388.89 requests/s, edge total 1902.78 requests/s ``` The table's first row names the introductory course's gap: 513.89 requests/s and 1.60 Mbit/s were calculated without ever counting the asset path. The second row shows what asset traffic adds: in the combined arrangement, the application process sees 3,291.67 requests/s — 6.41 times the course's number — and sends 241.60 Mbit/s, 151.0 times its read egress. A design's largest resource line item can be the one never calculated. The introductory course reached the same conclusion with its end-of-day job, which demanded 3.75 times the bandwidth of a continuously running flow; here the ratio is 151.0, and the line item never entered the arithmetic at all. The third and fourth rows say what the split does: the total work does not shrink, it divides. The application process returns to the course's 513.89 requests/s and 1.60 Mbit/s; 2,777.78 requests/s and 240.00 Mbit/s move to the content process. Splitting the content path does not fix the introductory course's arithmetic — it names the arrangement under which that arithmetic holds. The course's 513.89 is the number the application process sees only once the asset path is separated from it. The fifth row connects to the previous two lessons. The twenty assets were the common class, requested from every edge in the previous lesson's measurement, with a zero wasted share under push-based distribution. Pushed to twelve edges, the origin sees only 240 asset requests a day and sends 2.59 MB. That is how much of the 240.00 Mbit/s flowing to users actually reaches the origin. Three decisions combined — a split content path, an edge cache, and pushing the common class — bring a 240.00 Mbit/s line item down to 2.59 MB a day at the origin. The last line is the assumption's sensitivity. Had T14 been halved, asset requests would be 1,388.89/s, edge total 1,902.78/s — still more than three times the course's number. Whether the assumption doubles or halves, the conclusion does not change: the calculation is wrong whenever the asset path is not counted. #### Summary - Separating the content path from the application path is a placement decision; both return the same bytes, and what changes is which process a request lands on. - Measurement: on one page load, the application process saw 21 requests and 216,480 bytes in the combined arrangement, 1 request and 480 bytes in the split arrangement. - In the split arrangement, scaling units become independent and the content path gets its own name; both ends of the first lesson's TTL tradeoff — short for the application name, long for the content name — can be chosen at once. - The introductory course's 513.89 requests/s at the edge never counted the asset path; once asset traffic is added, the application process in the combined arrangement sees 3,291.67 requests/s and 241.60 Mbit/s — 6.41 times the requests, 151.0 times the bytes. - The split does not reduce the total, it divides it: the application process returns to 513.89 requests/s and 1.60 Mbit/s, while 2,777.78 requests/s and 240.00 Mbit/s move to the content process. The course's number is valid only under this arrangement. - With the split path, edge cache, and pushing the common class combined, the origin sees 240 asset requests and 2.59 MB a day; even if T14 were halved, the calculation stays wrong whenever the asset path is not counted. #### Next Step This topic kept a share of requests from ever reaching the application: the edge cache satisfied nine-tenths of tracking responses at the boundary, the split content path kept twenty of a page load's twenty-one requests away from the application, and the name layer chose which zone a request went to. What remains are the requests that genuinely must reach the application — 513.89 a second in the measured arrangement. One question stays unanswered: once a zone is chosen, with multiple replicas doing the same work, which one does a request land on, and who notices when that replica is down. The next lesson ties both questions to a single component and counts its responsibilities. ## Load Balancing The layer that decides which replica a request lands on: the load balancer's distribution, health check, and TLS termination responsibilities, the separation of connection-level distribution from content-aware routing, the comparison of round robin, least connections, and consistent hashing algorithms, the functional overlap with the reverse proxy, and the cost of session stickiness to scaling. ### Role of the Load Balancer Source: https://academia.sh/en/courses/traffic-layer/role-of-the-load-balancer The three responsibilities of the layer that distributes requests among replicas doing the same work: measuring distribution with local processes, comparing the answered-request rate between setups with and without health checking when a replica goes down, treating TLS termination as a placement decision, and dividing the peak request rate computed in the introductory course by the replica count. The previous topic managed to keep some requests from reaching the application **at all**: the edge cache met nine-tenths of tracking responses at the boundary, the separated content path pulled away almost all of a page load's requests, and the DNS layer chose which zone the request would go to. What remains are the requests that genuinely need to reach the application — 513.89 of them per second at the edge. One question is still unanswered: once the zone is chosen, which of the several replicas doing the same work does a request land on, and if that replica is down, who notices. This lesson ties those two questions to a single component. The **load balancer** sits in front of replicas doing the same work and routes an incoming request to one of them. The lesson enumerates this device's responsibilities, gets each one's code-level measurable result, and shows what dividing the peak request rate computed in the Introduction to System Design course by the replica count yields. #### Three Responsibilities The balancer's jobs can be named separately, each requiring its own decision. **Distribution.** Choosing which replica an incoming request goes to. The selection rule is the subject of later lessons; here, in its plainest form, it is **round robin** — replicas chosen in turn. **Health checking.** Independently testing whether a replica can still accept requests, and removing the ones that cannot from the pool. The check request is separate from the user's request: the balancer polls the replicas at fixed intervals and drops one from distribution when it gets no answer. **TLS termination.** Choosing where the encrypted connection ends. If it ends at the balancer, the balancer sees the request's content; if it stays encrypted all the way to the replica, it does not. Some of these jobs are also done by the **reverse proxy** from the Server-Side Fundamentals course; their overlap and distinction is this topic's fifth lesson. Here the balancer keeps a single role: sharing requests among replicas doing the same work. #### Setup The measurement is set up with local processes. Three replicas are each a `node:http` server that writes its own name into its response; the balancer does round robin distribution and can run with health checking on or off. No real load balancer, cloud environment, or container is set up. ```js // balance/replica.mjs — an application replica: returns the tracking response, writes its own name to the header import { createServer } from "node:http"; const [port, name] = [Number(process.argv[2]), process.argv[3]]; const BODY = (no) => JSON.stringify({ no, state: "at transfer hub", zone: "35", step: 4, route: ["accepted", "departed", "transfer-34"], }); if (Number.isInteger(port) === false) console.log("usage: node balance/replica.mjs "); else createServer((req, res) => { const url = new URL(req.url, "http://local"); if (url.pathname === "/health") { res.writeHead(200, { "x-replica": name }).end("up"); return; } const body = BODY(url.searchParams.get("no") ?? "-"); res.writeHead(200, { "content-type": "application/json", "x-replica": name }); res.end(body); }).listen(port, "127.0.0.1"); ``` The balancer does two of the three jobs: it chooses a replica in turn and checks the replicas' health on a separate timer. The check interval is 200 milliseconds, and a single failed round counts a replica as down; both are this topic's assumptions, referred to below as Y3. ```js // balance/balancer.mjs — round robin distribution + health check. Usage: // node balance/balancer.mjs import { createServer, request, get } from "node:http"; const INTERVAL = 200; // assumption Y3: health check interval (ms) const THRESHOLD = 1; // assumption Y3: consecutive failures counted as down const port = Number(process.argv[2]); const healthOn = process.argv[3] === "on"; const replicas = process.argv.slice(4).map((p) => ({ port: Number(p), up: true, failures: 0 })); let next = 0, unreachable = 0; function check(r) { const req = get({ port: r.port, path: "/health", timeout: 100 }, (res) => { res.resume(); if (res.statusCode === 200) { r.failures = 0; r.up = true; } else fail(r); }); req.on("error", () => fail(r)); req.on("timeout", () => req.destroy()); } function fail(r) { r.failures += 1; if (r.failures >= THRESHOLD) r.up = false; } if (Number.isInteger(port) === false || replicas.length === 0) { console.log("usage: node balance/balancer.mjs "); } else { if (healthOn) setInterval(() => replicas.forEach(check), INTERVAL).unref(); createServer((outerReq, outerRes) => { if (outerReq.url === "/manage/status") { outerRes.writeHead(200, { "content-type": "application/json" }); outerRes.end(JSON.stringify({ unreachable, up: replicas.filter((r) => r.up).map((r) => r.port) })); return; } const pool = replicas.filter((r) => r.up); if (pool.length === 0) { outerRes.writeHead(503).end("pool empty"); return; } const chosen = pool[next++ % pool.length]; const innerReq = request({ port: chosen.port, path: outerReq.url, method: outerReq.method, headers: outerReq.headers }, (innerRes) => { outerRes.writeHead(innerRes.statusCode, innerRes.headers); innerRes.pipe(outerRes); }); innerReq.on("error", () => { unreachable += 1; outerRes.writeHead(502).end("replica unreachable"); }); outerReq.pipe(innerReq); }).listen(port, "127.0.0.1"); } ``` The client sends requests **in sequence**; keeping concurrency at 1 makes the distribution counts deterministic — otherwise the same setup would give a different split on every run. ```js // balance/client.mjs — sends requests in sequence, counts which replica answered each one import { Agent, request } from "node:http"; const [port, count, label] = [Number(process.argv[2]), Number(process.argv[3]), process.argv[4]]; const agent = new Agent({ keepAlive: true, maxSockets: 1 }); function one(no) { return new Promise((resolve, reject) => { const r = request({ port, path: `/tracking?no=${no}`, agent }, (res) => { res.resume(); res.on("end", () => resolve({ code: res.statusCode, replica: res.headers["x-replica"] ?? "-" })); }); r.on("error", reject); r.end(); }); } if (Number.isInteger(port) === false) console.log("usage: node balance/client.mjs