Persistence Selection

Both persistence forms have now been measured, and each fell short on its own. A snapshot cannot close the data loss window: whatever the interval, the writes after the last snapshot are unprotected. An append log, meanwhile, sets the data loss window per write, but its file takes up more than three times the space for the same state, and recovery is forced to replay the entire history.

This lesson makes the choice. The choice is made on three measures, and all three are counted in the same run:

Recovery steps. How many records are applied for the process to become ready to serve after restarting. What is counted is steps, not duration; a step is independent of the environment.

Data loss window. How many writes are lost at the moment of the crash.

Steady-state cost. The bytes persistence writes to disk while the store is running normally, the syncs it performs, and the number of full passes that scan every entry.

Four Schemes, One Crash

PM5. The crash happens right after write 55,000. The snapshot is taken every 20,000 writes, so the last completed snapshot belongs to write 40,000. The log is synced once per second and is rewritten once over the course of the run. Every completed snapshot performs one sync.

The run actually builds all four schemes: files are written to disk, recovery reads from those files, and the built state is compared entry by entry against the correct state at write 55,000.

// selection.mjs — recovery, data loss window, and steady-state cost of four persistence schemes
import { writeFileSync, readFileSync, statSync } from 'node:fs';

function generator(seed) {
  let a = seed >>> 0;
  return () => { a = (a + 0x6d2b79f5) >>> 0; let t = a;
    t = Math.imul(t ^ (t >>> 15), t | 1); t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296; };
}
const r = generator(20260731), LOAD = [];
for (let i = 0; i < 60000; i++) {
  const m = Math.floor(r() * 20000);
  LOAD.push(['loan:m' + String(m).padStart(5, '0'),
    `branch=${1 + (m % 8)};day=${[7, 14, 28][i % 3]};renewal=${i % 4}`]);
}
const stateAt = (n) => { const m = new Map();
  for (let i = 0; i < n; i++) m.set(LOAD[i][0], LOAD[i][1]); return m; };
const serialize = (m) => { let s = '';
  for (const [k, v] of m) s += k + '\t' + v + '\n'; return s; };

const SNAPSHOT = 20000, CRASH = 55000;            // snapshot interval and crash instant
writeFileSync('snap40.txt', serialize(stateAt(40000)));    // last completed snapshot
writeFileSync('full.log', LOAD.slice(0, CRASH).map(([k, v]) => `W\t${k}\t${v}\n`).join(''));
writeFileSync('queue.log', LOAD.slice(40000, CRASH).map(([k, v]) => `W\t${k}\t${v}\n`).join(''));
const correct = stateAt(CRASH);

function recover(files) {                      // replays files in order and rebuilds the state
  const m = new Map();
  let steps = 0, bytes = 0;
  for (const f of files) {
    bytes += statSync(f).size;
    for (const s of readFileSync(f, 'utf8').split('\n')) {
      if (s === '') continue;
      const p = s.split('\t');
      m.set(p[p.length - 2], p[p.length - 1]);
      steps++;
    }
  }
  let mismatches = 0;
  for (const [k, v] of correct) if (m.get(k) !== v) mismatches++;
  return { bytes, steps, entries: m.size, mismatches };
}

const schemes = [                                  // name, source, snapshot, log, rewrite, loss
  ['persistence off', [], 0, 0, 0, 60000],
  ['snapshot only', ['snap40.txt'], 1, 0, 0, SNAPSHOT],
  ['log only', ['full.log'], 0, 1, 1, 3000],
  ['snapshot + log', ['snap40.txt', 'queue.log'], 1, 1, 0, 3000],
];
console.log('crash instant: write 55000 | snapshot interval: 20000 writes | sync: once per second');
console.log('scheme             bytes read  recovery steps  entries built  missing entries');
for (const [name, files] of schemes) {
  const k = recover(files);
  console.log(name.padEnd(18), String(k.bytes).padStart(11), String(k.steps).padStart(15),
    String(k.entries).padStart(14), String(k.mismatches).padStart(12));
}

let snapshotBytes = 0;                             // steady-state cost of the 60000-write run
for (let n = SNAPSHOT; n <= 60000; n += SNAPSHOT) snapshotBytes += Buffer.byteLength(serialize(stateAt(n)));
const logBytes = Buffer.byteLength(LOAD.map(([k, v]) => `W\t${k}\t${v}\n`).join(''));
const rewriteBytes = Buffer.byteLength(serialize(stateAt(60000)));
console.log('\nsteady-state cost of the 60000-write run');
console.log('scheme             bytes written  syncs  full scans  worst loss');
for (const [name, , snap, log, rewrite, loss] of schemes) {
  const bytes = snap * snapshotBytes + log * logBytes + rewrite * rewriteBytes;
  console.log(name.padEnd(18), String(bytes).padStart(13),
    String(snap * 3 + log * 20).padStart(12),
    String(snap * 3 + rewrite).padStart(11), String(loss).padStart(14));
}

console.log('\neffect of the snapshot interval on recovery when used together (same crash instant)');
console.log('snapshot interval  snapshot steps  log steps  total steps  snapshot disk');
for (const interval of [5000, 20000, 60000]) {
  const last = Math.floor(CRASH / interval) * interval;
  let disk = 0;
  for (let n = interval; n <= 60000; n += interval) disk += Buffer.byteLength(serialize(stateAt(n)));
  const g = last ? stateAt(last).size : 0;
  console.log(String(interval).padStart(17), String(g).padStart(15),
    String(CRASH - last).padStart(11), String(g + CRASH - last).padStart(12),
    String(disk).padStart(14));
}

crash instant: write 55000 | snapshot interval: 20000 writes | sync: once per second
scheme             bytes read  recovery steps  entries built  missing entries
persistence off              0               0              0        18763
snapshot only           652842           17330          17330         9816
log only               2181666           55000          18763            0
snapshot + log         1247842           32330          18763            0

steady-state cost of the 60000-write run
scheme             bytes written  syncs  full scans  worst loss
persistence off                0            0           0          60000
snapshot only            1849117            3           3          20000
log only                 3097260           20           1           3000
snapshot + log           4229117           23           3           3000

effect of the snapshot interval on recovery when used together (same crash instant)
snapshot interval  snapshot steps  log steps  total steps  snapshot disk
             5000           18763           0        18763        6535929
            20000           17330       15000        32330        1849117
            60000               0       55000        55000         717260

Reading the Recovery

The most telling column in the first table is the last one. The correct state has 18,763 entries; missing entries counts how many of them, once recovery finishes, are either absent altogether or hold the wrong value.

The scheme with persistence off recovers in zero steps and builds all 18,763 entries missing. Zero recovery steps is not a success; it is just another way of writing that there was nothing to recover.

The real warning is in the second row. Recovering with the snapshot alone builds 17,330 entries — 92 percent of the correct count, and on the surface the store looks full. And yet 9,816 of the entries, more than half, are either missing or hold a stale value. From the library’s point of view, this means opening with a loan table where half the rows belong to last week: loan counts do not add up, return dates show the past, and there is no way from outside to tell which record is wrong. Opening with an incomplete state is harder to notice than not opening at all.

The log alone and the combination give the same result: zero missing entries. The difference between them is in cost. The log alone applies 55,000 steps and reads 2,181,666 bytes; the combination builds the same state in 32,330 steps, reading 1,247,842 bytes. Recovery work drops by 41 percent, because instead of replaying all of the first 40,000 writes one by one, only their result is read.

Steady-State Cost

The second table gives the price the same schemes pay while running. The snapshot alone writes 1,849,117 bytes and performs three syncs; it is the cheapest protected scheme and loses up to 20,000 writes in the worst case. The log alone writes 3,097,260 bytes, performs twenty syncs, and brings the loss down to 3,000 writes.

The combination is the most expensive row: 4,229,117 bytes, twenty-three syncs. That number would be expected to be the sum of the log and the snapshot, but it is not the exact sum — in the combination, the log no longer needs its own rewrite, because it can be trimmed after every completed snapshot. The snapshot becomes the log’s pruner.

The full-scan column counts a silent cost. The scan cost measured in the previous lesson — pausing or copying — is paid again on every full pass. The log-only scheme makes this pass once over the course of the run; the schemes with a snapshot make it three times.

The Snapshot’s Job When Used Together

The third table shows the real decision behind using both together. The data loss window is no longer the snapshot’s job; the log holds it, and it stays at 3,000 writes across all three rows. The only thing that changes is the recovery step count.

When the snapshot interval is 60,000 writes — that is, when no snapshot completes before the crash — recovery climbs to 55,000 steps and the snapshot disk usage drops to 717,260 bytes. At an interval of 20,000, recovery drops to 32,330 steps; at 5,000, it drops to 18,763 steps — the lowest possible value, since it cannot go below the entry count. The cost is the snapshot disk usage: it climbs from 717,260 bytes to 6,535,929 bytes, a factor of nine.

This is the lesson’s decision. When a snapshot and an append log are used together, the snapshot’s job is recovery time, not durability. The interval is set not by the question “how much data loss am I willing to accept” but by “how long am I willing to let a restart take.”

The Decision

The library’s data does not call for a single scheme; each kind of data carries its own price.

Data	Scheme	Rationale
Catalog cache	persistence off	Regenerable from its source; zero recovery steps, loss is inconsequential
Session records	snapshot only	Reopening a session is cheap; a 20,000-write window is acceptable, lowest steady-state cost
Loan counters, waiting list	snapshot + log	Opening with an incomplete state is not acceptable; recovery is 32,330 steps, window is 3,000 writes
Late-fee records	snapshot + log, sync on every write	A lost record maps to money; the window drops to zero, the cost is a sync per write

The one scheme missing from the table is “log only,” and the reason is visible in the measurement: while giving the same durability, its recovery takes 1.7 times as many steps as the combination, and stopping the log’s growth still requires the same full scan. Using the log alone does not escape the snapshot’s cost; it only postpones it.

Summary

• Recovery is evaluated on three measures: the number of records applied, the data loss window, and the steady-state cost while running.
• Recovering with the snapshot alone built 92 percent of the correct entries but left 9,816 missing or stale; opening with an incomplete state is harder to notice than not opening at all.
• The log alone built the same correct state in 55,000 steps, the snapshot and log together in 32,330; bytes read dropped from 2,181,666 to 1,247,842.
• The combination carries the most expensive steady-state cost (4,229,117 bytes, 23 syncs), but because the snapshot trims the log, it needs no separate rewrite.
• In the combination, the snapshot interval sets the recovery step count, not the data loss window: the 5,000 interval brought recovery down to 18,763 steps and raised the snapshot disk usage ninefold.

Next Step

Once the persistence decision is made, the store has learned how to recover after a crash. A limit that arrives before a crash has not been addressed yet: memory runs out. Every measurement in this topic let the store keep growing; 19,040 entries held 1,593,100 bytes and nothing stopped it. The next lesson takes up the question of which key gets evicted once the memory limit is reached. Four policies — least recently used, least frequently used, random, and time-based — are compared on the same loan access trace along the dimensions of hit rate and bytes held; the overhead a policy adds to each entry is counted as well.