Content Distribution Design

In all three cases, the response was a record, the bytes carried were small, and the expensive line item was compute or memory. In this case the same system carries two different kinds of traffic: personal responses of a few hundred bytes, and large, unchanging static assets resent on every request. When the two travel the same path, one’s property breaks the other — a personal response cannot be cached, and the entire path is treated as uncacheable because of it. This lesson measures what the split does to the request count and bytes reaching the origin.

Constraints

Functional requirements. F1: the application shell and its assets (scripts, styles, images) reach the client. F2: a request for personal data is answered. F3: when a new version is published, the client does not use the old one. F4: a published version can be rolled back.

Scope reduction. Resizing and encoding images, offline operation, and whether the shell is built on the server or the client are outside this design; all three change the number of bytes carried, not where traffic gets split.

Code	Threshold	Threshold’s source
G1	the median personal response does not exceed 150 ms	how long data is expected on screen
G2	no more than 2000 requests per second reach the origin	the origin’s provisioned capacity
G3	a new version is active on every client within 300 seconds	for a bad version to be rolled back
G4	the cache per edge node does not exceed 8 GB	the node’s local storage share

Assumptions

Code	Assumption	Value	Rationale
VI1	daily sessions	4,000,000	number of users who open the application
VI2	static asset requests per session	25	scripts, styles, and images combined
VI3	personal requests per session	8	data calls that fill the screen
VI4	peak factor	3	the ratio of the peak-hour rate to the daily average
VI5	static asset size	base 20 KB, exponent 2, cap 4 MB	a few large bundles, many small assets; average is 40 KB
VI6	personal response	6 KB	fields that fill the screen
VI7	distinct static assets	500,000	versioned names and images included
VI8	static request distribution	Zipf, exponent 0.9	a small number of assets in every session, most requested rarely

Back-of-the-Envelope Estimate

// distribution/estimate.mjs — the estimate from VI1-VI8 and deriving the hit rate G2 requires
const VI = { sessions: 4e6, staticRequests: 25, dynamicRequests: 8, peakFactor: 3, assetBytes: 40_960, responseBytes: 6144 };
const DAY = 86_400, ORIGIN = 2000;                     // ORIGIN: G2 threshold, requests/s
const b = (x, n = 2) => x.toFixed(n);
const mbit = (x) => (x * 8) / 1e6;

const staticPeak = (VI.sessions * VI.staticRequests / DAY) * VI.peakFactor;
const dynamicPeak = (VI.sessions * VI.dynamicRequests / DAY) * VI.peakFactor;
const staticBytes = staticPeak * VI.assetBytes, dynamicBytes = dynamicPeak * VI.responseBytes;
console.log(`peak static requests/s = ${b(staticPeak)}   peak dynamic requests/s = ${b(dynamicPeak)}   total = ${b(staticPeak + dynamicPeak)}`);
console.log(`request ratio static/dynamic = ${b(staticPeak / dynamicPeak)}   byte ratio = ${b(staticBytes / dynamicBytes)}`);
console.log(`peak egress: static ${b(mbit(staticBytes))} Mbit/s, dynamic ${b(mbit(dynamicBytes))} Mbit/s`);
console.log(`G2 threshold ${ORIGIN} requests/s; the undivided design exceeds it by ${b(staticPeak + dynamicPeak - ORIGIN)} requests/s`);
console.log(`static request hit rate needed to hold G2 = ${b(1 - (ORIGIN - dynamicPeak) / staticPeak, 4)}`);
console.log(`the dynamic floor alone sends ${b(dynamicPeak)} requests/s to the origin = ${b(dynamicPeak / (staticPeak + dynamicPeak), 4)} of the total`);

const doubled = (VI.sessions * VI.staticRequests * 2 / DAY) * VI.peakFactor;
console.log(`VI2 sensitivity: static requests per session 25 -> 50 makes peak static ${b(doubled)} requests/s and required hit rate ${b(1 - (ORIGIN - dynamicPeak) / doubled, 4)}`);

peak static requests/s = 3472.22   peak dynamic requests/s = 1111.11   total = 4583.33
request ratio static/dynamic = 3.13   byte ratio = 20.83
peak egress: static 1137.78 Mbit/s, dynamic 54.61 Mbit/s
G2 threshold 2000 requests/s; the undivided design exceeds it by 2583.33 requests/s
static request hit rate needed to hold G2 = 0.7440
the dynamic floor alone sends 1111.11 requests/s to the origin = 0.2424 of the total
VI2 sensitivity: static requests per session 25 -> 50 makes peak static 6944.44 requests/s and required hit rate 0.8720

Four numbers determine the design. The first is that the two ratios are different from each other: static traffic is 3.13 times dynamic in requests, 20.83 times in bytes. Speaking of a single “traffic” number is therefore misleading — the answer differs depending on which resource is under pressure. The second is that the undivided design exceeds G2 by 2583.33 requests/s. The third is that the threshold turns into a hit rate: keeping the origin under 2000 requests/s requires a static hit rate of at least 0.7440. The fourth is the floor itself: the dynamic stream alone sends 1111.11 requests/s to the origin, and this does not come down no matter the hit rate; 0.2424 of total requests is reduced by no cache.

Measuring the Split

// distribution/separation.mjs — the effect of the static/dynamic split on requests and bytes
// reaching the origin. The edge cache is a Map that keeps the most recently used entry
// (insertion order = use order). Asset sizes and the request stream are generated with a
// hand-written generator, the seed is fixed. IT IS A MODEL.
const SEED = 20260730, ASSETS = 500_000, REQUESTS = 1_800_000, WARMUP = 600_000;
const BASE = 20_480, EXPONENT = 2.0, CAP = 4 << 20, ZIPF = 0.9;   // VI5 and VI8
const STATIC_PEAK = 3472.22, DYNAMIC_PEAK = 1111.11, DYNAMIC_BYTES = 6144;   // from the estimate block
let s = SEED % 2147483647;
const rand = () => (s = (s * 48271) % 2147483647) / 2147483647;

const size = new Int32Array(ASSETS);
for (let i = 0; i < ASSETS; i += 1) size[i] = Math.min(CAP, Math.round(BASE * Math.pow(rand(), -1 / EXPONENT)));
const totalBytes = size.reduce((a, b) => a + b, 0);
// Zipf(0.9) inverse sampling: rank = (1 + u (N^(1-z) - 1))^(1/(1-z))
const scalePow = Math.pow(ASSETS, 1 - ZIPF);
const draw = () => Math.min(ASSETS - 1, Math.floor(Math.pow(1 + rand() * (scalePow - 1), 1 / (1 - ZIPF))) - 1);
const requests = new Int32Array(REQUESTS);
for (let i = 0; i < REQUESTS; i += 1) requests[i] = draw();

function run(capacity) {
  const cache = new Map();
  let cursor = cache.keys();
  let filled = 0, hits = 0, hitBytes = 0, measured = 0, measuredBytes = 0;
  for (let i = 0; i < REQUESTS; i += 1) {
    const v = requests[i], bt = size[v], counted = i >= WARMUP;
    if (counted) { measured += 1; measuredBytes += bt; }
    if (cache.has(v)) {
      cache.delete(v); cache.set(v, bt);
      if (counted) { hits += 1; hitBytes += bt; }
      continue;
    }
    if (bt > capacity) continue;                       // an asset that does not fit is not cached
    while (filled + bt > capacity) {                     // the least recently used entry is evicted
      let n = cursor.next();
      if (n.done) { cursor = cache.keys(); n = cursor.next(); }
      const evictedBytes = cache.get(n.value);
      if (evictedBytes !== undefined) { cache.delete(n.value); filled -= evictedBytes; }
    }
    cache.set(v, bt); filled += bt;
  }
  return { requestHitRate: hits / measured, byteHitRate: hitBytes / measuredBytes, retained: cache.size };
}

const b = (x, n = 2) => x.toFixed(n);
const mbit = (bytesS) => (bytesS * 8) / 1e6;
console.log(`model: ${ASSETS.toLocaleString("en-US")} assets, total ${b(totalBytes / 1e9)} GB, average ` +
  `${b(totalBytes / ASSETS / 1024)} KB, ${REQUESTS.toLocaleString("en-US")} requests (first ${WARMUP.toLocaleString("en-US")} warmup)`);
const undividedRequests = STATIC_PEAK + DYNAMIC_PEAK;
const undividedBytes = STATIC_PEAK * (totalBytes / ASSETS) + DYNAMIC_PEAK * DYNAMIC_BYTES;
console.log(`\nno split (the personal document makes the entire path uncacheable):`);
console.log(`  reaching the origin: ${b(undividedRequests)} requests/s and ${b(mbit(undividedBytes))} Mbit/s`);

console.log(`\n${"edge cache".padStart(12)}${"assets retained".padStart(17)}${"request hit rate".padStart(18)}` +
  `${"byte hit rate".padStart(15)}${"origin requests/s".padStart(19)}${"origin Mbit/s".padStart(15)}${"request factor".padStart(16)}${"byte factor".padStart(13)}`);
for (const gb of [0.5, 2, 8, 20.5]) {
  const r = run(gb * 1e9);
  const originRequests = STATIC_PEAK * (1 - r.requestHitRate) + DYNAMIC_PEAK;
  const originBytes = STATIC_PEAK * (1 - r.byteHitRate) * (totalBytes / ASSETS) + DYNAMIC_PEAK * DYNAMIC_BYTES;
  console.log(`${`${gb} GB`.padStart(12)}${r.retained.toLocaleString("en-US").padStart(17)}${b(r.requestHitRate, 4).padStart(18)}` +
    `${b(r.byteHitRate, 4).padStart(15)}${b(originRequests).padStart(19)}${b(mbit(originBytes)).padStart(15)}${b(undividedRequests / originRequests).padStart(16)}${b(undividedBytes / originBytes).padStart(13)}`);
}

model: 500,000 assets, total 20.44 GB, average 39.93 KB, 1,800,000 requests (first 600,000 warmup)

no split (the personal document makes the entire path uncacheable):
  reaching the origin: 4583.33 requests/s and 1190.40 Mbit/s

  edge cache  assets retained  request hit rate  byte hit rate  origin requests/s  origin Mbit/s  request factor  byte factor
      0.5 GB           12,118            0.4562         0.4390            2999.18         691.80            1.53         1.72
        2 GB           49,111            0.6188         0.6059            2434.84         502.20            1.88         2.37
        8 GB          196,218            0.8272         0.8209            1711.28         258.06            2.68         4.61
     20.5 GB          324,829            0.8773         0.8727            1537.00         199.14            2.98         5.98

The model finds the assets’ average to be 39.93 KB; because the estimate block uses VI5’s theoretical average (40 KB), the two blocks’ byte counts diverge in the second digit. The measured value is the true average of the cap-truncated distribution.

The split holds G2 only with enough cache. At the 8 GB G4 allows, the request hit rate is 0.8272, above the required 0.7440; 1711.28 requests/s reach the origin and the threshold holds. At 2 GB, the hit rate drops to 0.6188, the origin rises to 2434.84 requests/s, and G2 breaks. Cache size is not a convenience in this design; it is the parameter that holds the threshold.

The split does not reduce requests and bytes at the same rate. At 8 GB, requests drop 2.68-fold, bytes 4.61-fold. The reason was already in the estimate: the dynamic stream holds 0.2424 of requests but less than one-twentieth of bytes. As the cache removes the static side, what remains is a floor that is heavy in requests, light in bytes. A design cannot be defended with the sentence “we cut the origin by seventy percent”; which number was cut must be stated.

Request hit rate and byte hit rate are separate numbers. In every row, the byte hit rate is lower than the request hit rate (0.8209 against 0.8272): large assets get evicted from the cache sooner, because dozens of small assets fit in their place. The difference is small but its sign is constant, and it grows as the cache shrinks (0.4390 against 0.4562 at 0.5 GB).

Design

• Content delivery network and edge caching (Traffic Layer, Content Delivery Networks). The parameter is 8 GB per node, and the measured request hit rate of 0.8272 is above G2’s required 0.7440.
• Static content hosting (Traffic Layer, Static Content Hosting). The parameter is the versioned name: an asset whose content changes gets a new name too, so it can be given an unbounded lifetime.
• API gateway (Traffic Layer, API Gateway). The dynamic path goes through here; the parameter is the 1111.11 requests/s floor reaching the origin.
• Backend for frontend (Traffic Layer, Backends for Frontends). The parameter is 8 personal requests per session; if this number drops, the floor drops too, because the floor is reduced not by caching but by merging.
• Push-based distribution (Traffic Layer, Push and Pull Based Distribution). A new version’s assets are pushed to the edge before publishing; the parameter is G3’s 300-second window.
• Graceful degradation (Resilience and Reliability, Graceful Degradation). When the origin drops, the edge keeps serving the assets it holds; the parameter is the share of requests that stay up.

Two deliberately unused patterns. Refresh-ahead (Scaling the Data Layer, Refresh-Ahead) is not used: an asset under a versioned name never goes stale, there is nothing to refresh. Session stickiness (Traffic Layer, Session Stickiness) is not used: the static path holds no state, and stickiness would lower the hit rate by preventing the edge cache from being shared.

Eliminated Alternative: One Path, Short-Lived Cache

The alternative design does not separate the two kinds of traffic; it sends everything through one path and gives static assets a short cache lifetime. What it wins is real: a single distribution line, a single publish step, no version mixing — the document and its assets refresh together. A publish staying atomic is not a small gain.

The number that eliminates it is at the origin. Because the personal document cannot be cached, caching on the single path can be given only to assets, and their lifetime has to stay short since their names never change; this pulls the hit rate close to the table’s 0.5 GB row. In the undivided design, the origin takes 4583.33 requests/s and 1190.40 Mbit/s — it exceeds G2 by 2583.33 requests/s, and no cache size rescues this, because the problem is not size but cacheability. The alternative wins if a different constraint changes: had the origin’s capacity been 5000 requests/s, G2 would hold on its own, and bandwidth would be the split’s only remaining justification.

Failure Behavior and What Is Given Up

If the origin drops, the static path stays up: the edge keeps meeting 0.8272 of requests with the assets it holds, and thanks to the versioned name, what it serves is not stale but the correct version. What drops is the entire dynamic floor — 0.2424 of requests. The application opens but its data does not arrive; this is graceful degradation’s shape in this case.

If an edge node drops, other nodes take the load, but their caches are cold for those assets. The table’s first row gives the lower bound of a cold cache: at a 0.4562 hit rate the origin takes 2999.18 requests/s and G2 breaks. The cost of takeover here is not capacity but warmup time.

What is given up is the atomicity of publishing. Because the two paths advance at two speeds, a client can see a new document with an old asset, or the reverse, within G3’s 300-second window. The versioned name zeroed out the staleness window on the static side, but in exchange it opened a new window: the staleness of each side relative to the other.

Summary

• Static traffic is 3.13 times dynamic in requests, 20.83 times in bytes; defending a design with a single traffic number hides which resource is under pressure.
• G2’s 2000 requests/s threshold turns into a hit rate: static hit rate must be at least 0.7440. At the 8 GB G4 allows, the measured hit rate is 0.8272; at 2 GB it is 0.6188 and the threshold breaks.
• The split cuts requests 2.68-fold and bytes 4.61-fold; the difference comes from the dynamic floor holding 0.2424 of requests but a very small share of bytes.
• The byte hit rate is always lower than the request hit rate (0.8209 against 0.8272 at 8 GB), and the gap grows as the cache shrinks, because large assets are evicted first.
• The versioned name zeroes out the staleness window on the static side; refresh-ahead is therefore unnecessary.
• What is given up is the atomicity of publishing: within the 300-second window, a new document can be paired with an old asset.

Next Step

The four cases belonged to the same family. In URL shortening, reads outnumbered writes a hundred to one; in the news feed, forty to one; in search suggestion, twelve to one; in content distribution, even the dynamic requests stayed small next to cached static traffic. This ratio was the backbone of all four designs: work was shifted from the read side to the write side (fanning out to the box, keeping a ready suggestion list, the prefix tree), because writing was infrequent and the shifted work was cheap. All four shared a second point too: staleness was an affordable price. A redirect 300 seconds old at the edge, a post invisible for 60 seconds, a query not suggested for 600 seconds, one side drifting relative to the other — none of it counted as unacceptable, because a read being a little old is a correctable error.

In a system dominated by writing, neither decision holds. If writing is not infrequent, shifting work to the read side is a multiplier, not a gain; and if a record being wrong is unacceptable, staleness stops being an affordable price and becomes a direct failure. The next topic opens with these reversed constraints: systems where writing outpaces reading, where order and delivery must be guaranteed, and where processing a record twice produces real damage.