Redundancy Zones

Every calculation in the previous lesson carried one implicit assumption: there is a replica to recover, and that replica was untouched by the failure. K01 had already flagged this assumption — replica arithmetic only holds once replicas fail independently, which is why the computed number is an upper bound. If two replicas share the same power line, the same network switch, or the same cooling unit, there is no independence: when one goes down, the other goes down too.

This lesson measures where replicas are placed. Two placement units are separated. An availability zone is a placement unit with its own power and network path, one whose failure is assumed not to cross into its neighbor. A region is a geographically separate set of zones; the distance between two regions lengthens the propagation round.

In this course, zone refers only to this physical placement unit. For the same reason, the set of components a failure touches is called blast radius in this course.

A Placement Plan Selects a Failure Mode

A placement plan determines which replicas fail together. If six replicas sit in a single zone, the loss of that zone takes all six at once and the six replicas mean nothing. If the same six replicas are spread across three zones, the loss of one zone takes two. If they are spread across two regions, the loss of one region takes three.

The setup below is a model: no real region, zone, cloud environment, or container is set up. Replicas are objects carrying a region and zone tag, a round is an abstract step, and the propagation round changes with distance. The mechanics of replication were established in the Scaling the Data Layer course and are not repeated here; the only question here is which write ends up nowhere when a unit is lost.

KU7 — propagation rounds: 1 within the same zone, 2 between different zones in the same region, 4 between regions. Rationale: propagation time grows with distance. Sensitivity is linear; if the remote round doubles, the unpropagated write count at region loss also doubles. This assumption is not added to K01’s table.

// zone/model.mjs — placement is a MODEL: replicas carry a region and zone tag, propagation rounds
// change with distance. A round is an abstract step; no real region, zone, or cloud environment is set up.
export const ROUND = 40, FAILURE_ROUND = 20, WRITES_PER_ROUND = 6;
export const PROPAGATION = { zone: 1, region: 2, remote: 4 };   // KU7 (assumption): propagation rounds

export const PLAN = {
  "single-zone": [["r1", "z1"], ["r1", "z1"], ["r1", "z1"], ["r1", "z1"], ["r1", "z1"], ["r1", "z1"]],
  "three-zone":  [["r1", "z1"], ["r1", "z1"], ["r1", "z2"], ["r1", "z2"], ["r1", "z3"], ["r1", "z3"]],
  "two-region":  [["r1", "z1"], ["r1", "z2"], ["r1", "z3"], ["r2", "z1"], ["r2", "z2"], ["r2", "z3"]],
};
export const SCENARIO = {
  machine: (y, i) => i === 0,
  zone: (y) => y[0] === "r1" && y[1] === "z1",
  region: (y) => y[0] === "r1",
};

const delay = (x, y) => (x[0] !== y[0] ? PROPAGATION.remote : x[1] !== y[1] ? PROPAGATION.region : PROPAGATION.zone);

export function run(planName, scenarioName) {
  const placement = PLAN[planName], n = placement.length;
  const down = placement.map((y, i) => SCENARIO[scenarioName](y, i));
  const alive = (i, t) => !(down[i] && t >= FAILURE_ROUND);
  const writes = [];
  let cursor = 0, s = { accepted: 0, dropped: 0, lost: 0 };

  for (let t = 1; t <= ROUND; t += 1) {
    for (const w of writes) {                          // propagations that arrived become visible at the target
      for (const m of w.inFlight) if (m.arrives === t) w.owner.add(m.target);
      w.inFlight = w.inFlight.filter((m) => m.arrives > t);
    }
    if (t === FAILURE_ROUND) {                          // the failed unit: loss of ownership and in-flight messages
      for (const w of writes) {
        w.inFlight = w.inFlight.filter((m) => alive(m.source, t) && alive(m.target, t));
        const remaining = [...w.owner].filter((i) => alive(i, t));
        if (remaining.length === 0 && w.inFlight.length === 0) { s.lost += 1; w.lost = true; }
        w.owner = new Set(remaining);
      }
    }
    const pool = placement.map((_, i) => i).filter((i) => alive(i, t));
    for (let k = 0; k < WRITES_PER_ROUND; k += 1) {
      if (pool.length === 0) { s.dropped += 1; continue; }
      const i = pool[cursor++ % pool.length];
      s.accepted += 1;
      const w = { owner: new Set([i]), inFlight: [], lost: false };
      for (const j of pool) if (j !== i) w.inFlight.push({ source: i, target: j, arrives: t + delay(placement[i], placement[j]) });
      writes.push(w);
    }
  }
  const survived = placement.filter((_, i) => alive(i, ROUND)).length;
  return { ...s, survived, total: n };
}

// zone/measure.mjs — three placement plans, three failures: surviving replicas, dropped writes, lost writes
import { run, PLAN, SCENARIO, ROUND, FAILURE_ROUND, WRITES_PER_ROUND, PROPAGATION } from "./model.mjs";

const s = (x, n) => String(x).padStart(n);
console.log(`${ROUND} rounds, ${WRITES_PER_ROUND} writes per round, failure at round ${FAILURE_ROUND}.`);
console.log(`propagation rounds: same zone ${PROPAGATION.zone}, same region different zone ${PROPAGATION.region}, ` +
  `different region ${PROPAGATION.remote} (KU7)`);
console.log();
console.log(`${"plan".padEnd(12)} | ${"failure".padEnd(7)} | ${s("surviving replicas", 19)} | ${s("surviving capacity", 19)} | ${s("dropped writes", 15)} | ${s("lost unpropagated", 18)}`);
console.log(`${"-".repeat(12)}-|-${"-".repeat(7)}-|-${"-".repeat(19)}-|-${"-".repeat(19)}-|-${"-".repeat(15)}-|-${"-".repeat(18)}`);
for (const plan of Object.keys(PLAN)) {
  for (const scenario of Object.keys(SCENARIO)) {
    const r = run(plan, scenario);
    console.log(`${plan.padEnd(12)} | ${scenario.padEnd(7)} | ${s(`${r.survived}/${r.total}`, 19)} | ` +
      `${s(`${((100 * r.survived) / r.total).toFixed(1)}%`, 19)} | ${s(r.dropped, 15)} | ${s(r.lost, 18)}`);
  }
}

40 rounds, 6 writes per round, failure at round 20.
propagation rounds: same zone 1, same region different zone 2, different region 4 (KU7)

plan         | failure |  surviving replicas |  surviving capacity |  dropped writes |  lost unpropagated
-------------|---------|---------------------|---------------------|-----------------|-------------------
single-zone  | machine |                 5/6 |               83.3% |               0 |                  0
single-zone  | zone    |                 0/6 |                0.0% |             126 |                114
single-zone  | region  |                 0/6 |                0.0% |             126 |                114
three-zone   | machine |                 5/6 |               83.3% |               0 |                  0
three-zone   | zone    |                 4/6 |               66.7% |               0 |                  2
three-zone   | region  |                 0/6 |                0.0% |             126 |                114
two-region   | machine |                 5/6 |               83.3% |               0 |                  1
two-region   | zone    |                 5/6 |               83.3% |               0 |                  1
two-region   | region  |                 3/6 |               50.0% |               0 |                  9

Reading the Three Plans

The machine rows are the same across all three plans: the loss of one replica does not stop the service in any arrangement. The replica count already covered this failure mode, and the placement plan changes nothing here.

The zone row separates the plans. In the single-zone arrangement, the loss of the zone takes all six replicas: 126 writes drop and 114 writes end up nowhere. The same failure leaves four replicas standing in the three-zone arrangement, no writes drop, and only 2 writes are lost. All six replicas existed in both cases; the only difference was where they were placed.

The region row separates a second level. The three-zone arrangement covers zone loss, but region loss takes it entirely down too — all three zones sit in the same region. Only the two-region arrangement survives region loss: half capacity, zero dropped writes, 9 lost unpropagated writes.

The cost of spreading out shows up in the last column. In the two-region arrangement, even a single machine’s loss costs 1 write, while the single-zone arrangement costs 0. The reason is the propagation round: when replicas sit farther apart, a write takes longer to reach a second replica, and more writes sit in a single replica during that window. Longer propagation is redundancy’s other face; the same count climbs to 9 at region loss.

How Much Capacity Must Sit Idle

The surviving capacity column leaves a question open: is 66.7 percent capacity enough to carry the load? For it to be enough, part of the capacity must sit idle on a normal day. The calculation below gives that share as a function of zone count and counts the load that cross-region replication carries on a failure-free day.

// zone/capacity.mjs — the failure-free-day cost of redundancy: capacity sitting idle and writes
// carried across regions; converting model rounds to K01 scale
const PEAK_EDGE = 513.89;     // K01: peak requests/s at the edge
const PEAK_WRITE = 97.22;     // K01: peak write requests/s
const EVENTS_DAY = 2_800_000; // K01: daily state events
const GROWTH_GB = 0.976;      // K01: daily data growth
const ROUND_SEC = 2;          // KU1 (assumption): one round is 2 seconds
const b = (x, n = 2) => x.toFixed(n);

console.log(`${"zones".padStart(5)}${"load per zone".padStart(17)}${"if one zone drops".padStart(18)}` +
  `${"must be provisioned".padStart(20)}${"sitting idle".padStart(13)}${"extra share".padStart(14)}`);
for (const n of [2, 3, 4, 6]) {
  const normal = PEAK_EDGE / n, dropped = PEAK_EDGE / (n - 1), total = n * dropped;
  console.log(`${String(n).padStart(5)}${b(normal).padStart(17)}${b(dropped).padStart(18)}` +
    `${b(total).padStart(20)}${b(total - PEAK_EDGE).padStart(13)}${(b(100 * (total / PEAK_EDGE - 1), 1) + "%").padStart(14)}`);
}

const bytes = (GROWTH_GB * 1e9) / EVENTS_DAY;
console.log(`\n${b(bytes)} bytes per event (K01: ${GROWTH_GB} GB / ${EVENTS_DAY.toLocaleString("en-US")} events a day)`);
console.log(`replicating to two regions: ${b(PEAK_WRITE * bytes / 1024)} KB/s extra transfer at peak load, ` +
  `${b(GROWTH_GB * 1000)} MB a day`);

console.log(`\nconverting lost unpropagated writes to K01 scale (KU1: round = ${ROUND_SEC} sec):`);
for (const [name, round, share] of [["zone loss (3 zones)", 1, 3], ["region loss (2 regions)", 4, 2]]) {
  console.log(`  ${name.padEnd(24)} propagation ${round} round(s) = ${round * ROUND_SEC} sec, ` +
    `${b(PEAK_WRITE / share)} writes/s per unit -> ${b((round * ROUND_SEC * PEAK_WRITE) / share)} writes`);
}
console.log(`  ratio ${b(((4 * ROUND_SEC * PEAK_WRITE) / 2) / ((1 * ROUND_SEC * PEAK_WRITE) / 3))}: ` +
  `region loss leaves more unpropagated writes in the same failure duration`);

zones    load per zone if one zone drops must be provisioned sitting idle   extra share
    2           256.94            513.89             1027.78       513.89        100.0%
    3           171.30            256.94              770.84       256.95         50.0%
    4           128.47            171.30              685.19       171.30         33.3%
    6            85.65            102.78              616.67       102.78         20.0%

348.57 bytes per event (K01: 0.976 GB / 2,800,000 events a day)
replicating to two regions: 33.09 KB/s extra transfer at peak load, 976.00 MB a day

converting lost unpropagated writes to K01 scale (KU1: round = 2 sec):
  zone loss (3 zones)      propagation 1 round(s) = 2 sec, 32.41 writes/s per unit -> 64.81 writes
  region loss (2 regions)  propagation 4 round(s) = 8 sec, 48.61 writes/s per unit -> 388.88 writes
  ratio 6.00: region loss leaves more unpropagated writes in the same failure duration

Idle capacity shrinks quickly with zone count. Withstanding the loss of one zone with two zones requires double the capacity: 1027.78 is provisioned to carry 513.89 requests/s, and half of it sits idle. The extra share drops to 50 percent with three zones, 33.3 percent with four zones, and 20 percent with six zones. The general form for $n$ zones is $1/(n-1)$. Increasing the zone count makes redundancy cheaper, which means the same service availability can be reached at two different prices.

The transfer bill for cross-region replication, however, is small: 348.57 bytes per event, 33.09 KB/s at peak load, 976 MB a day. For this workload, the bill for region redundancy is not in the transfer but in the capacity and the lost writes. Under a different load — a workload carrying writes with large bodies — the ranking would reverse; the number depends on the workload, not on the principle.

The last block converts model rounds to K01 scale. Unpropagated writes at the loss of one zone in the three-zone arrangement come to 64.81; at the loss of one region in the two-region arrangement, 388.88. The ratio is 6.00 and comes from two causes: cross-region propagation takes four times as long, and the per-region write share is larger. Region redundancy removes the outage but grows the data loss, and this is exactly the number to compare against the previous lesson’s data loss objective: a one-second replication lag lost 97.22 events, region loss loses 388.88.

The Numbers for Two Days

The failure-free day’s cost is three line items. In a three-zone placement, 50 percent of capacity sits idle — 256.95 requests/s never get used. Two-region replication carries every write a second time: 976 MB a day and 33.09 KB/s at peak load. Third, longer propagation produces loss even at small failures: at a single machine’s loss, the two-region arrangement lost 1 write while the single-zone arrangement lost 0.

The failing day’s gain is in the same table’s zone and region rows. In the single-zone arrangement, the loss of one zone drops 126 writes and loses 114, while the same failure in the three-zone arrangement gives 0 dropped writes and 2 lost. At region loss, the three-zone arrangement gives 126/114 while the two-region arrangement gives 0/9. Each level removes one failure mode and leaves the next one open; the placement decision is choosing which mode gets removed.

Summary

• A zone is a placement unit with its own power and network path; a region is a geographically separate set of zones. In this course, zone carries only this meaning.
• A placement plan determines which replicas fail together: when six replicas are pooled in a single zone, zone loss drops 126 writes and loses 114; spread across three zones, the same failure gives 0 dropped writes and 2 lost.
• Region loss is survived only by the two-region arrangement: 50 percent capacity, 0 dropped writes, 9 lost unpropagated writes.
• Spreading out has its own cost in longer propagation rounds: the two-region arrangement loses 1 write even at a single machine’s loss, the single-zone arrangement loses 0.
• The capacity that must sit idle to withstand the loss of one zone is $1/(n-1)$: 100 percent with two zones, 50 percent with three zones, 20 percent with six zones.
• At K01 scale, zone loss leaves 64.81 and region loss leaves 388.88 unpropagated writes (ratio 6.00); region redundancy removes the outage but forces the data loss objective.

Next Step

This lesson spread replicas out, but what got spread was always the same whole: six replicas, one service, one dataset. Surviving capacity at zone or region loss came out to 66.7 percent and 50 percent, because the lost unit carried a fraction of the service. This fraction arithmetic assumes something: the units can do work independently of each other. In most arrangements, though, the units keep something in common — a single routing map, a single number dispenser, a single configuration source — and the loss of that point takes all the units at once. The next lesson takes on splitting the service into self-contained, replicable units: keeping the links inside a unit from crossing it, the load per unit, and the blast radius of the single resource shared across units.