Backup and Restore

The previous lesson worked out, item by item, where data sits and what moving it costs. The backup flow was an ordinary line there: 600 GB a month, 600 or 1,200 units depending on its destination.

A backup exists not because it moves, but because it can be restored: which moment you can return to the day a bad bulk update turns up. In a managed data layer, that question no longer depends on a file kept by hand, it depends on the provider’s snapshot schedule and the logs it keeps.

Backup scope and restore drills were measured earlier; this lesson does not repeat that, it takes it as input. Three things get counted here: the trade-off between the lost window and the storage fee, the components of restore time, and whose format the backup sits in.

• MD37. The fictional regional measurement network’s work-order data layer is 220 GB; 4% of the data changes in 24 hours, and the changed share falls off exponentially at shorter intervals, because the same blocks get written over and over.
• MD38. The continuous log produces 46 GB a day and is archived every sixty seconds. Snapshots are kept for thirty days. Both are fictional.
• MD39. The price is fictional, the currency’s name is a fictional unit: backup storage is 0.9 units per GB a month; transfer is 0 units per GB in the same zone, 1 in a neighboring zone, 2 in a neighboring region, 9 to the internet.
• MD40. Restore speeds are fictional: opening a new instance is a fixed 210 seconds, loading runs at 95 MB/s, replaying the log at 12 MB/s, verifying at 260 MB/s. Replay is the slowest step, because records are applied one at a time.
• MD41. A logical export reads at 40 MB/s, the compression ratio is 0.38; restoring from the exported copy runs at 26 MB/s, because indexes are rebuilt. The portable copy is taken weekly, and four are kept a month. All of it is fictional.

// backup/restore.mjs — model: backup and point-in-time restore in the managed data layer.
// Backup scope and restore drills were measured in M22/K07 and are not repeated here.
// Sizes, speeds, and prices are fictional; the currency's name is "unit". No randomness.
const DATA_GB = 220, DAILY_CHANGE = 0.04, EXPONENT = 0.7;   // share changed in 24h; decay exponent
const LOG_GB = 46, PRICE = 0.9;         // continuous log output GB/day; unit/GB-month
const SNAPSHOT_DAYS = 30, ARCHIVE_SEC = 60;   // snapshot retention; log archiving interval
const changedShare = (hours) => DAILY_CHANGE * Math.pow(hours / 24, EXPONENT);
const snapshotGB = (hours) => DATA_GB + (SNAPSHOT_DAYS * 24 / hours) * changedShare(hours) * DATA_GB;
const columns = (widths, left = 1) => (row) =>
  row.map((d, i) => String(d)[i < left ? "padEnd" : "padStart"](widths[i])).join("");
const fmt1 = columns([28, 16, 22, 14, 13, 16]);
console.log(fmt1(["layout", "lost window", "continuous restore", "oldest point", "stored GB",
  "fee(unit/month)"]));
const row = (label, hours, logDays) => {
  const gb = snapshotGB(hours) + logDays * LOG_GB;
  console.log(fmt1([label, logDays ? ARCHIVE_SEC + " s" : hours + " h",
    logDays ? logDays + " d" : "none", SNAPSHOT_DAYS + " d", gb.toFixed(0),
    (gb * PRICE).toFixed(0)]));
};
for (const hours of [24, 12, 6, 3, 1]) row(`snapshot ${hours} h`, hours, 0);
for (const days of [1, 7, 35]) row(`24 h + log ${days} d`, 24, days);

// --- Restore time splits into components -----------------------------------------
const INSTANCE_SEC = 210, LOAD_MBS = 95, REPLAY_MBS = 12, VERIFY_MBS = 260;
const logPerHour = (gb) => LOG_GB / 24 * (gb / DATA_GB);   // log GB produced per hour
const restore = (gb, distance) => {
  const load = gb * 1024 / LOAD_MBS, replay = logPerHour(gb) * distance * 1024 / REPLAY_MBS;
  const verify = gb * 1024 / VERIFY_MBS;
  return { load, replay, verify, total: INSTANCE_SEC + load + replay + verify };
};
const fmt2 = columns([14, 12, 15, 19, 17, 14, 10]);
console.log("\n" + fmt2(["data (GB)", "instance (s)", "load (s)", "replay (s)",
  "verify (s)", "total (s)", "hours"]));
for (const gb of [55, 110, 220, 440, 880]) {
  const r = restore(gb, 18);
  console.log(fmt2([gb, INSTANCE_SEC, r.load.toFixed(0), r.replay.toFixed(0), r.verify.toFixed(0),
    r.total.toFixed(0), (r.total / 3600).toFixed(2)]));
}
const BASELINE = INSTANCE_SEC + DATA_GB * 1024 / LOAD_MBS + DATA_GB * 1024 / VERIFY_MBS;
const fmt3 = columns([22, 17, 19, 15, 17]);
console.log("\n" + fmt3(["snapshot interval", "worst distance", "replay (s)",
  "total (h)", "fee(unit/month)"]));
for (const hours of [24, 12, 6, 3, 1]) {
  const r = restore(DATA_GB, hours);
  console.log(fmt3([hours + " h", hours + " h", r.replay.toFixed(0), (r.total / 3600).toFixed(2),
    (snapshotGB(hours) * PRICE).toFixed(0)]));
}
console.log(`baseline (instance + load + verify, replay zero): ${(BASELINE / 3600).toFixed(2)} hours`);

// --- Portable backup ---------------------------------------------------------------
// The service's own backup is in its own format. The portable copy is a logical export.
const COMPRESSION = 0.38, WEEKLY = 4, EXPORT_MBS = 40, IMPORT_MBS = 26;
const DESTINATION = { "same zone": 0, "neighboring zone": 1, "neighboring region": 2, "to internet": 9 };
const COPY_GB = DATA_GB * COMPRESSION, SERVICE = (snapshotGB(24) + 7 * LOG_GB) * PRICE;
const fmt4 = columns([21, 12, 16, 18, 16, 16]);
console.log("\n" + fmt4(["destination", "unit/GB", "transfer(mo)", "storage(mo)", "extra total",
  "service + extra"]));
for (const [name, rate] of Object.entries(DESTINATION)) {
  const transfer = COPY_GB * WEEKLY * rate, storage = COPY_GB * WEEKLY * PRICE;
  console.log(fmt4([name, rate, transfer.toFixed(0), storage.toFixed(0),
    (transfer + storage).toFixed(0), (SERVICE + transfer + storage).toFixed(0)]));
}
console.log(`the service's own backup is ${SERVICE.toFixed(0)} units/month; the portable copy is ` +
  `${COPY_GB.toFixed(1)} GB x ${WEEKLY}/month (compression ${COMPRESSION}).`);
console.log(`export ${(DATA_GB * 1024 / EXPORT_MBS / 3600).toFixed(2)} h (${EXPORT_MBS} MB/s); ` +
  `restoring from it ${(DATA_GB * 1024 / IMPORT_MBS / 3600).toFixed(2)} h (${IMPORT_MBS} MB/s, ` +
  `indexes are rebuilt) — ` +
  `${(DATA_GB * 1024 / IMPORT_MBS / restore(DATA_GB, 1).total).toFixed(1)}x the service's own restore.`);

layout                           lost window    continuous restore  oldest point    stored GB fee(unit/month)
snapshot 24 h                           24 h                  none          30 d          484             436
snapshot 12 h                           12 h                  none          30 d          545             491
snapshot 6 h                             6 h                  none          30 d          620             558
snapshot 3 h                             3 h                  none          30 d          713             641
snapshot 1 h                             1 h                  none          30 d          905             814
24 h + log 1 d                          60 s                   1 d          30 d          530             477
24 h + log 7 d                          60 s                   7 d          30 d          806             725
24 h + log 35 d                         60 s                  35 d          30 d         2094            1885

data (GB)     instance (s)       load (s)         replay (s)       verify (s)     total (s)     hours
55                     210            593                736              217          1755      0.49
110                    210           1186               1472              433          3301      0.92
220                    210           2371               2944              866          6392      1.78
440                    210           4743               5888             1733         12574      3.49
880                    210           9485              11776             3466         24937      6.93

snapshot interval        worst distance         replay (s)      total (h)  fee(unit/month)
24 h                               24 h               3925           2.05              436
12 h                               12 h               1963           1.50              491
6 h                                 6 h                981           1.23              558
3 h                                 3 h                491           1.09              641
1 h                                 1 h                164           1.00              814
baseline (instance + load + verify, replay zero): 0.96 hours

destination               unit/GB    transfer(mo)       storage(mo)     extra total service + extra
same zone                       0               0               301             301            1026
neighboring zone                1             334               301             635            1361
neighboring region              2             669               301             970            1695
to internet                     9            3010               301            3311            4036
the service's own backup is 725 units/month; the portable copy is 83.6 GB x 4/month (compression 0.38).
export 1.56 h (40 MB/s); restoring from it 2.41 h (26 MB/s, indexes are rebuilt) — 2.4x the service's own restore.

Window or Bill

The first table’s first five rows run on the snapshot alone. Pulling the frequency from twenty-four hours to one hour narrows the lost window twenty-fourfold and pushes storage from 484 GB to 905 GB — the bill rises from 436 to 814 units, an 87% increase.

The growth in stored bytes looks backward: a more frequent snapshot carries fewer changed bytes each time. The number still grows, because as the interval shortens, the same blocks end up sitting in more than one snapshot. The cost of raising frequency is not request count, it is repetition.

The last three rows turn on the continuous log and rewrite the table. Add seven days of log to the 24-hour snapshot and the lost window drops to sixty seconds, with the bill at 725 units; the hourly snapshot, by contrast, gives a one-hour window for 814 units. The cheaper option narrows the window sixty times more. Raising snapshot frequency is no substitute for a continuous log.

The third and fourth columns show the limit. The continuous-restore window is the log’s retention period: under the seven-day layout, you can return to any second of the last seven days. The oldest discrete point, though, is thirty days out — you can return to a snapshot from twenty days ago, not to that day’s afternoon. The bill scales with the first: thirty-five days of log is 2,094 GB and 1,885 units, two and a half times the seven-day setup.

The Floor of a Restore

The second table splits the duration into four components. Opening a new instance is a fixed 210 seconds, and its share falls as the data grows: 12% of the total at 55 GB, 1% at 880 GB. Loading and verifying grow linearly with size. Replay grows twice over — as data grows, log output grows with it — and at 220 GB, at 2,944 seconds, it is the largest component.

A design decision follows from this: what sets restore time is not just data size, it is how far the target point sits from the last snapshot. At a twenty-four-hour interval, the worst case is a 3,925-second replay and a 2.05-hour total; at a one-hour interval, replay drops to 164 seconds and the total to 1.00 hour.

The price of halving that duration is in the first table: from 436 to 814 units. And the drop stops there: opening the instance plus loading plus verifying add up to 0.96 hours, and no matter how close to zero the snapshot interval gets pushed, nothing goes below this floor. The continuous log brings down the lost window, not the restore time. The two numbers are bought from two separate items.

The Delegated Decision

The sentence “backups are automatic in a managed service” hides four items: how the backup is taken, which format it is kept in, where it is stored, and how it is verified all sit with the provider. The 95, 12, and 260 MB/s above are not a configuration line either — they are the number the service hands you.

Three kept items directly set this lesson’s tables: retention period (the first table’s third and fourth columns), who holds the encryption key, and whether it gets copied outside the region (the last table’s destination column). Four of seven items are delegated, three are kept; two of the three kept items directly write a transfer item.

The encryption key is the quietest item: if it sits in the provider’s store, the backup is readable but not portable; if it sits with you, a lost key erases the backup too. This item never gets written onto the bill at all.

Whose Format the Backup Is In

The last table measures this lesson’s irreversible item. The backup the service takes is in its own format: it cannot be loaded anywhere else, cannot be handed to another engine. A portable copy has to be produced separately — a logical export, 1.56 hours of reading for 220 GB.

The extra cost is two items. Four compressed 83.6 GB copies run 301 units a month in storage; the transfer item depends on the destination: zero in the same zone, 334 in a neighboring zone, 669 in a neighboring region, 3,010 to the internet. The service’s own backup is 725 units; adding portability pushes the total to 1,026 units in the same zone (41%), to 1,695 in a neighboring region (134%).

The third cost is time: restoring from the exported copy takes 2.41 hours, 2.4 times the service’s own restore. A portable backup is not just expensive, it is a slow backup.

This item ties to M23/K01’s exit column in one sentence: a portable backup is the exit cost paid in installments. When it is not paid, the saving is real, and the cost comes due all at once on the day you leave.

Summary

• Pulling snapshot frequency from twenty-four hours to one hour narrows the window twenty-fourfold and raises the bill from 436 to 814 units: hot blocks repeat across more than one snapshot.
• A 24-hour snapshot plus seven days of continuous log narrows the window to sixty seconds and costs 725 units: both cheaper than the hourly snapshot and sixty times narrower.
• The continuous-restore window is the log’s retention period, the oldest discrete point is the snapshot’s: you can return to any second up to seven days back, and only to specific moments up to thirty days back.
• A restore has four components and a floor of 0.96 hours; replay is the largest component at 2,944 seconds for 220 GB, and it shortens only with the snapshot interval.
• Four of seven decision items are delegated; two of the three kept items directly write a transfer fee. The backup format cannot travel: a portable copy adds between 301 and 3,311 units a month, and restoring from it is 2.4 times slower.

Course Wrap-Up

Lesson	Delegated decision	Irreversible item	Egress or transfer cost
Object Storage	placement, replication, repair	key layout: 2,481,240 requests	2.77 GiB → 0.055 / 0.236 units
Storage Classes	durability, same across four classes	minimum-retention debt: reverting costs 0.338 units	276.3 GiB retrieved → 23.48 units
Versioning and Lifecycle Rules	enforcing the rule	versioning: 93.6 GiB remains after turning it off	8,931,900 transitions → 239.75 units
Block Storage	disks, replication, repair	size does not shrink: 1,056 units overpaid a year	110 GiB snapshot → 2.20 / 9.35 units
File Storage	metadata server, growth	access protocol: 30 of 54 call sites	640 GiB → 12.80 / 54.40 units
Archive Storage	storage medium, readability	180-day debt: 4.5 months paid for nothing	297.9 GiB → 25.32 units
Access Control and Signed URLs	verification only	issued link: revoking it breaks 3,360,000	leaked bucket: 184.8 GiB → 15.71 units
Managed Relational Database	8 of 28 items; 9 inaccessible	4 of 11 kept items are one-way	220 GB → 3.97 h, 1,980 units
Read Replicas	replication mechanics	promoted replica: full 220 GB copy	1,370 GB/month/replica → 2,741 units
Managed Key-Value Service	partitioning and balancing	partition key: 8 h, 518.4K units	62.2 GB export → 560 units
Secondary Indexes and Streams	index build and update	missed window: 960K events at 30 h	41.5 GB/month of events → 83 / 373 units
Managed Cache Service	7 of 17 items	500 pending records; shard count fixed	other zone → 722 units/month
Data Transfer and Egress Cost	4 of 9 items: the price list	accumulated data: 33,600 GB → 67,200 units	6,180 GB: 1,440 → 7,320 units
Backup and Restore	taking and verifying the backup	backup format: export +301 units	83.6 GB x 4 → 0 / 334 / 669 / 3,010

Across the fourteen rows, the third column splits into three families. Six rows are schema and format: key layout, access protocol, collation choice, partition key, shard count, and backup format — none of them change without rewriting the data. Five rows are prepaid or accumulated bytes: the minimum-retention debt, the versioning backlog, the archive’s one hundred eighty days, the volume that will not shrink, and the accumulated data itself; on these, the decision reverses and no fee is charged. Three rows are a closed window of time: the issued signed link, the broken replication link, and the missed event window.

The fourth column points the same way across all fourteen rows: no row ever charged a fee for data coming in, every row charged for it going out, and the fee was measured by where it went. That is this course’s rule: data is cheap coming in, expensive going out — and in a rented data layer, what really binds you is not the price, it is the irreversible schema. You can choose to pay the bill; you cannot pay your way out of the six-row schema family.

Across three courses, a machine was rented, a network was rented, data was rented. Underneath all of it, the same thing still stands: a server that keeps running. Application instances are up at three in the morning too, the measurement network’s dashboard waits even when no request arrives, and both get billed by the hour. The next course — M23/K04 Serverless and Edge Computing — asks this: can the continuously running server itself be rented away, or only the seconds it actually runs?