Data Transfer and Egress Cost

The previous lesson showed that putting the cache in the wrong zone writes a 722-unit item onto the bill. That item came from a single flow and carried a single price.

Looking at the whole measurement network, there are eight separate flows: the meter collector writes to the data layer, the application reads from the data layer and from the read replica, writes to the cache, the data layer replicates to the read replica, backups go to a store, logs stream to a collector, and dashboard responses go out to the open network. Except for the last one, none of these is client traffic; seven of them are the system’s own internal traffic. And all of them can carry a fee.

What decides the price is not the byte count, it is where the two ends sit. This lesson runs the same architecture across three placements, works out the egress fee item by item, measures the effect of five design decisions on that fee one at a time, and counts the asymmetry between data coming in and going out.

• MD31. The fictional regional measurement network’s monthly flows, in GB: collector → data layer 1,400, data layer → app 900, app → cache 780, data layer → read replica 1,400, read replica → app 620, data layer → backup 600, app → log 320, app → internet 160. Total 6,180 GB.
• MD32. Prices are fictional, the currency’s name is a fictional unit: within zone 0, cross-zone 1, cross-region 2, to internet 9 units/GB. Bytes coming in from the internet are free. The one point where the model rests on reality is this ordering.
• MD33. Three placements are measured: everything in a single zone; the read replica, backup, and log collector in a second zone; the same three in a second region.
• MD34. Compression applies only to logs, backups, and dashboard responses, at a fictional ratio of 0.38. The replication stream is already compressed, and meter writes are too small to be worth compressing.
• MD35. The edge hit rate is 67%, and edge egress is five units per GB. Both were measured in M23/K02; here they are used as an input and are not remeasured.
• MD36. The data layer and the object store together grow by a net 1,400 GB a month; the bulk export runs over a 1 Gbit/s link. Both are fictional.

// data/egress.mjs — model: the same architecture run across three placements, egress fee
// worked out item by item. Flows, prices, and growth are fictional; the currency's name is "unit".
// The edge's hit rate was measured in M23/K02, here it is an input. No randomness.
const FLOW = [                         // [source, destination, monthly GB]
  ["collector", "data", 1400], ["data", "app", 900], ["app", "cache", 780],
  ["data", "replica", 1400], ["replica", "app", 620], ["data", "backup", 600],
  ["app", "log", 320], ["app", "internet", 160],
];
const SINGLE = { collector: "r1z1", data: "r1z1", app: "r1z1", cache: "r1z1",
  replica: "r1z1", backup: "r1z1", log: "r1z1" };
const ZONE = { ...SINGLE, replica: "r1z2", backup: "r1z2", log: "r1z2" };
const REGION = { ...SINGLE, replica: "r2z1", backup: "r2z1", log: "r2z1" };
const PRICE = { "within zone": 0, "cross-zone": 1, "cross-region": 2, "to internet": 9 };
const classify = (loc, src, dst) => dst === "internet" ? "to internet"
  : loc[src] === loc[dst] ? "within zone" : loc[src][1] === loc[dst][1] ? "cross-zone" : "cross-region";
const compute = (flow, loc) => {
  const gb = { "within zone": 0, "cross-zone": 0, "cross-region": 0, "to internet": 0 };
  for (const [src, dst, v] of flow) gb[classify(loc, src, dst)] += v;
  const total = Object.keys(gb).reduce((a, s) => a + gb[s] * PRICE[s], 0);
  return { gb, total, totalGB: flow.reduce((a, f) => a + f[2], 0) };
};
const columns = (widths, left = 1) => (row) =>
  row.map((d, i) => String(d)[i < left ? "padEnd" : "padStart"](widths[i])).join("");
const fmt1 = columns([16, 14, 16, 17, 14, 11, 15]);
console.log(fmt1(["layout", "within zone", "cross-zone", "cross-region", "to internet",
  "total GB", "fee(unit)"]));
for (const [label, loc] of [["single zone", SINGLE], ["two zones", ZONE], ["two regions", REGION]]) {
  const r = compute(FLOW, loc);
  console.log(fmt1([label, ...Object.values(r.gb), r.totalGB, r.total.toFixed(0)]));
}

// --- What a design decision writes onto the bill ----------------------------------
// Baseline: two zones. Each decision is applied to the baseline on its own.
const BASELINE = compute(FLOW, ZONE);
const scale = (flow, target, factor) =>
  flow.map(([src, dst, gb]) => [src, dst, dst === target ? gb * factor : gb]);
const COMPRESSION = 0.38, HIT = 0.67, EDGE = 5;   // compression ratio; edge hit rate and unit/GB
const edgeCost = 160 * EDGE + 160 * (1 - HIT) * PRICE["to internet"];
const DECISION = [
  ["second read replica", compute([...FLOW, ["data", "replica2", 1400]],
    { ...ZONE, replica2: "r1z2" })],
  ["log sampling 1/5", compute(scale(FLOW, "log", 0.2), ZONE)],
  ["move backup to a remote region", compute(FLOW, { ...ZONE, backup: "r2z1" })],
  ["edge, 67% hit", { totalGB: BASELINE.totalGB, total: BASELINE.total - 160 * 9 + edgeCost }],
  ["compression, ratio 0.38", compute(["log", "backup", "internet"]
    .reduce((a, dst) => scale(a, dst, COMPRESSION), FLOW), ZONE)],
];
const fmt2 = columns([34, 13, 14, 15]);
console.log("\n" + fmt2(["decision", "GB diff", "fee diff", "new total"]));
console.log(fmt2(["baseline: two zones", BASELINE.totalGB.toFixed(0), "-", BASELINE.total.toFixed(0)]));
for (const [label, r] of DECISION)
  console.log(fmt2([label, (r.totalGB - BASELINE.totalGB).toFixed(0),
    (r.total - BASELINE.total).toFixed(0), r.total.toFixed(0)]));

// --- Cheap coming in, expensive going out ------------------------------------------
const fmt3 = columns([22, 14, 18]);
console.log("\n" + fmt3(["1000 GB to", "unit/GB", "fee(unit)"]));
console.log(fmt3(["from the internet", 0, 0]));
for (const s of Object.keys(PRICE)) console.log(fmt3([s, PRICE[s], 1000 * PRICE[s]]));
const GROWTH = 1400, LINK = 1;                 // net growth GB/month; export link Gbit/s
const fmt4 = columns([10, 16, 18, 20, 14]);
console.log("\n" + fmt4(["month", "accumulated GB", "export(unit)", "months of billing", "link(days)"]));
for (const month of [12, 24, 36]) {
  const gb = GROWTH * month, fee = gb * PRICE["to internet"];
  console.log(fmt4([month, gb, fee, (fee / BASELINE.total).toFixed(1), (gb * 8 / LINK / 86400).toFixed(1)]));
}

layout             within zone      cross-zone     cross-region   to internet   total GB      fee(unit)
single zone               6020               0                0           160       6180           1440
two zones                 3080            2940                0           160       6180           4380
two regions               3080               0             2940           160       6180           7320

decision                                GB diff      fee diff      new total
baseline: two zones                        6180             -           4380
second read replica                        1400          1400           5780
log sampling 1/5                           -256          -256           4124
move backup to a remote region                0           600           4980
edge, 67% hit                                 0          -165           4215
compression, ratio 0.38                    -670         -1463           2917

1000 GB to                   unit/GB         fee(unit)
from the internet                  0                 0
within zone                        0                 0
cross-zone                         1              1000
cross-region                       2              2000
to internet                        9              9000

month       accumulated GB      export(unit)   months of billing    link(days)
12                   16800            151200                34.5           1.6
24                   33600            302400                69.0           3.1
36                   50400            453600               103.6           4.7

Same Bytes, Three Bills

The first table’s most important column is total GB, and it is the same in all three rows: 6,180. No placement moves a single byte more or less. Same components, same flows, same volume.

The bill, though, is 1,440, 4,380, and 7,320 units. More than a fivefold difference. The only thing that changes is where each component sits.

In the single-zone row, 6,020 GB stays within the zone, and within-zone traffic is free; only 160 GB of internet egress is left, and its 1,440 units. This row’s entire bill is client-bound traffic; the internal traffic costs nothing.

In the second row, the read replica, backup store, and log collector move to the neighboring zone. Three components moved, four flows moved: replication from the data layer to the replica (1,400), reads from the replica to the app (620), data layer to backup (600), and app to log (320). 2,940 GB in total, and the moment it changes tier it writes one unit per GB. The bill jumps to three times the baseline, and what it buys is surviving the loss of a single zone.

The third row moves the same three components to a second region. The flows are the same, the volume is the same, the only change is the pricing tier: the same 2,940 GB is now priced at two units. The bill is 7,320. The cost of raising durability from a zone to a region is writing 2,940 more units for the exact same bytes.

The 2,940 GB in the second row does not split evenly across the four flows. Replication alone is 1,400 GB, 47% of the traffic crossing the boundary; the rest is shared between reads from the replica (620), backup (600), and log (320). The size of the replication flow is no coincidence either: it is exactly the 1,400 GB the collector writes to the data layer. The read replica mirrors the write flow, which is why half the cross-zone bill depends on write volume, not read volume. There is no way to shrink this item by caching reads.

The decision here has two parts, and only one is yours. The provider decides where each tier starts and ends, and its price; you decide which component sits inside which tier. The model makes this split visible: the price list is a constant, the placement map is a variable.

What a Design Decision Writes onto the Bill

The second table takes the two-zone placement as the baseline and applies five decisions one at a time.

A second read replica adds 1,400 units to the bill. The reason is direct: every replica has the full replication traffic flowing to it from the data layer written out again. Adding a read replica looks like buying read capacity; on the bill it also shows up as a transfer item, and that item grows linearly with replica count.

Log sampling is the cheapest win: cutting logs to a fifth drops 256 GB and 256 units. The GB difference and the fee difference are equal because the cross-zone price is one; the same decision would have gained twice as much in a cross-region placement.

Moving the backup to a remote region adds 600 units while the GB difference stays zero. This is the first row in the table where the two columns diverge, and the lesson’s sentence sits right here: the fee can change without the transferred bytes changing. Those 600 units are the price of the backup surviving a region loss too.

The edge reprices the 160 GB of internet egress: 160 GB of edge egress at five units, the 33% share it misses at nine units from the origin. The net difference is minus 165 units — under 4% of the 4,380-unit bill. The edge’s real gain does not show up in this table; that gain was in origin load, and it was measured earlier.

Compression is by far the biggest item: 1,463 units, a third of the baseline. A 670 GB drop gaining 1,463 units comes down to the dropped bytes sitting in the expensive tiers. Saving 100 GB from going to the internet is worth 900 units; saving it from crossing to the neighboring zone is worth 100. The same compression ratio pays off ninefold differently depending on which flow it is applied to.

The two outer rows should be read together: compression takes off 1,463 units, the second replica adds 1,400. One is a configuration line, the other a durability decision — and on the bill they nearly cancel each other out.

Each of the five rows was applied to the baseline on its own, and adding them up would be wrong: compression applied to a log that is already sampled does not gain the same share a second time. The point where the rows really diverge is reversibility. Log sampling and compression are configuration lines; they can be switched off the next day and the bill returns to what it was. Adding a replica and moving the backup to another region require moving data; reversing those decisions writes a new transfer item. The same table holds decisions that reverse for free and decisions whose reversal costs money.

Cheap Coming In, Expensive Going Out

The third table sends the same 1,000 GB to five destinations. Coming in from the internet is zero, moving within a zone is zero, crossing to the neighboring zone is 1,000, crossing to the neighboring region is 2,000, going out to the internet is 9,000 units.

This table shows a single fact: putting data in is free, taking it out is the top row of the price list. Field photos enter the system at hundreds of GB a month, and none of that entry gets written to any line; the day those same photos need to come back out, nine units a GB will get written. The asymmetry is not a bug, it is the pricing itself.

That is also why entry never shows up in the first table’s flow list: since field photos and meter readings coming in are free, adding them to the model does not change the bill. The flow that grows the system is the flow that never appears on the bill. Watching monthly spend to track data growth therefore does not work; while spend stays flat, the accumulated volume — and with it, the fourth table’s export column — climbs every month.

The fourth table measures what this asymmetry does over time. A data layer growing by a net 1,400 GB a month accumulates 16,800 GB in twelve months, 33,600 in twenty-four, 50,400 in thirty-six. Pushing that accumulation out to the internet all at once costs 151,200, 302,400, and 453,600 units respectively — equal to 34.5, 69.0, and 103.6 months of billing.

The last column shows a second limit. Pushing 33,600 GB down a 1 Gbit/s link takes 3.1 days, 50,400 GB takes 4.7. Through that whole span, the old system keeps running, new data keeps accumulating, and both sides keep getting billed.

Egress Fee and Exit Cost

These two terms sound alike and need to be pulled apart. Egress fee is a line that sits on the bill every month; in this model it is a slice of the 4,380 units, and design decisions can shrink it. Exit cost is the concept built in M23/K01: the total price of leaving a rented place. The link is one sentence: the egress fee is the exit cost’s price tag, and it grows every month along with the accumulated data.

This course’s irreversible item shows up here. The placement decision is reversible on paper; moving a component back from the second zone to the first is a configuration change. But a component does not move without its data moving, and that move itself writes an egress item: bringing 33,600 GB back from a neighboring region is 67,200 units, fifteen months of billing. Getting out of the wrong placement can cost more than staying in it.

The delegated decision also reads off this table. The provider holds four items: the price of the four tiers, where the tier boundaries are drawn, which tier a flow falls into, and whether entry stays free. You hold five items: which component sits where, how many replicas are kept, how much of the log gets shipped, where the backup gets written, and whether compression is on. Five of nine items are yours, but the multipliers that set the size of the numbers in the table belong to the other four.

Summary

• The same 6,180 GB writes 1,440, 4,380, and 7,320 units across three placements. It is not the byte count that sets the price, it is where the two ends sit.
• Spreading across two zones pulls 2,940 GB out of the within-zone tier and triples the bill; moving the same three components to a second region adds 2,940 more units.
• Measured one at a time, compression takes off 1,463 units and a second read replica adds 1,400. Moving the backup to a remote region writes 600 units while the GB difference stays zero: the fee can change without the transferred bytes changing.
• The same 1,000 GB costs zero coming in, 1,000 to the neighboring zone, 2,000 to the neighboring region, 9,000 to the internet. Exporting the 33,600 GB accumulated over 24 months costs 302,400 units, sixty-nine months of billing, and 3.1 days over a 1 Gbit/s link.
• The irreversible item is not the placement itself, it is the data underneath it: moving a component back means moving its data too, and that move writes 67,200 units for 33,600 GB.

Next Step

This lesson worked out the price of moving data from one place to another, and treated the backup flow as an ordinary item: 600 GB a month, 600 or 1,200 units depending on its destination.

A backup exists not because it moves, but because it can be restored. What that 600 GB on the price list buys is which moment you can return to the day a bad write or a corrupted table turns up. In a managed database, that question no longer depends on a file kept by hand, it depends on the provider’s snapshot schedule and the logs it keeps.

The next lesson measures this: where the oldest restorable point sits, how long a restore takes, what window is lost while restoring, and where the balance between retention period and storage fee settles.