Snapshot Persistence

The previous topic built the loan system entirely in memory: catalog entries, session records, loan counters, waiting lists, leaderboards. For each structure, the bytes held were counted and what they bought in return was measured. All of those measurements shared one quiet assumption: the process is running. When the process stops, none of it survives. Durability was called a setting, but what that setting does, what it costs, and exactly what is lost while it is off were never asked.

This lesson takes up the first form of that setting: writing the entire store to disk at fixed intervals. It raises two questions. How many writes are lost when the process stops, and how much does taking the snapshot itself cost the store while it keeps running? Both answers depend on the same dial — the interval — and pull in opposite directions.

Setting Up the Store and the Measurement

For the measurement, the loan store is written directly. The store is a key-value mapping; on every write it also updates the byte count it holds, so the memory cost is not estimated — it is counted.

PM1. An entry’s memory cost is the key’s bytes, the value’s bytes, and a fixed 48 bytes of overhead per entry. Those 48 bytes represent the hash table slot, the two length fields, and the entry pointer; a real runtime would move the number slightly, but not the direction of the measurement.

// store.mjs — in-memory loan store and snapshot taker
export const OVERHEAD = 48;          // PM1: fixed overhead per entry (slot + pointer + length)

export class Store {
  constructor() { this.entries = new Map(); this.bytes = 0; }
  #cost(k, v) { return Buffer.byteLength(k) + Buffer.byteLength(v) + OVERHEAD; }
  write(k, v) {
    const previous = this.entries.get(k);
    if (previous !== undefined) this.bytes -= this.#cost(k, previous);
    this.entries.set(k, v);
    this.bytes += this.#cost(k, v);
  }
  get count() { return this.entries.size; }
  snapshot() {                     // snapshot: all entries converted to text in a single pass
    let s = '';
    for (const [k, v] of this.entries) s += k + '\t' + v + '\n';
    return s;
  }
}

export function generator(seed) {     // deterministic pseudorandom generator
  let a = seed >>> 0;
  return () => {
    a = (a + 0x6d2b79f5) >>> 0;
    let t = a;
    t = Math.imul(t ^ (t >>> 15), t | 1);
    t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

export function generateLoad(count, seed, memberCount = 20000) {
  const r = generator(seed), list = [];
  for (let i = 0; i < count; i++) {
    const m = Math.floor(r() * memberCount);
    list.push([
      'loan:m' + String(m).padStart(5, '0'),
      `branch=${1 + (m % 8)};day=${[7, 14, 28][i % 3]};renewal=${i % 4}`,
    ]);
  }
  return list;
}

The load is the library’s loan traffic: sixty thousand loan writes spread across twenty thousand members. The generator is hand-written and its seed is visible, so every number below can be reproduced.

Interval, Data Loss Window, and Disk

In snapshot persistence, durability is set with a single dial: how many writes pass between snapshots. The run below replays the same sixty thousand writes across three intervals. After every write, the answer to “how many writes would be lost if the process stopped right now” is accumulated; at the end, its average and its worst case are printed.

// interval.mjs — data loss window and disk cost of three snapshot intervals
import { writeFileSync, statSync } from 'node:fs';
import { Store, generateLoad } from './store.mjs';

const LOAD = generateLoad(60000, 20260731);
console.log('load: 60000 writes, 20000 members, seed 20260731');
console.log('interval  snapshots  disk(MB)  average loss  worst loss');

for (const interval of [1000, 5000, 20000]) {
  const store = new Store();
  let snapshots = 0, diskBytes = 0, lossTotal = 0, worst = 0, lastSnapshot = 0;
  for (let i = 0; i < LOAD.length; i++) {
    store.write(LOAD[i][0], LOAD[i][1]);
    const loss = (i + 1) - lastSnapshot;        // writes that would be lost if the process stopped now
    lossTotal += loss;
    if (loss > worst) worst = loss;
    if ((i + 1) % interval === 0) {
      diskBytes += Buffer.byteLength(store.snapshot());
      snapshots++;
      lastSnapshot = i + 1;
    }
  }
  console.log(String(interval).padStart(6), String(snapshots).padStart(8),
    String(Math.round(diskBytes / 1048576)).padStart(10),
    String(Math.round(lossTotal / LOAD.length)).padStart(15), String(worst).padStart(14));
}

const final = new Store();
for (const [k, v] of LOAD) final.write(k, v);
writeFileSync('snapshot.txt', final.snapshot());
console.log('store:', final.count, 'entries,', final.bytes, 'bytes in memory |',
  'snapshot file', statSync('snapshot.txt').size, 'bytes');

load: 60000 writes, 20000 members, seed 20260731
interval  snapshots  disk(MB)  average loss  worst loss
  1000       60         30             501           1000
  5000       12          6            2501           5000
 20000        3          2           10001          20000
store: 19040 entries, 1593100 bytes in memory | snapshot file 717260 bytes

The data loss window is a direct function of the interval: its average is half the interval, its worst case is the interval itself. Compared to the twenty-thousand interval, the thousand-write interval saves an average of 9,500 writes. That is what is bought, and its cost sits in the same row: 30 MB of disk writes instead of 2 MB, a factor of fifteen. The cost grows this way because a snapshot writes the entire store every single time. Even for a change of a thousand writes, the whole store — around five hundred kilobytes — goes back to disk.

The last line gives a second distinction. Of the sixty thousand writes, 19,040 entries remain; the rest overwrote records of the same members. A snapshot’s size grows with the number of keys, not the number of writes — this is the fundamental difference from the log format in the next lesson. The same data holds 1,593,100 bytes in memory and 717,260 bytes in the file: the file is less than half of memory. The difference is overhead. Hash table slots, pointers, and length fields do not go to disk. Looking at the small size of the file on disk and concluding that the store will comfortably fit in memory accepts, from the start, an error of roughly double per entry.

The Cost of Taking the Snapshot Itself

The snapshot has to be consistent: recovery cannot work from a file that is half old and half new. There are two ways to guarantee consistency, and neither is free.

Pausing. Writes are held while the snapshot is taken. The memory cost is zero; clients pay the price in wait time.

Copying. Writes continue; when an entry that belongs in the snapshot is about to be changed, its old value is copied first, and the snapshot reads the copy. No one waits; the cost is paid in memory: the copies are held until the snapshot finishes.

PM2. Taking a snapshot is not instantaneous; every thousand entries scanned takes forty writes’ worth of time. Duration is counted in a unit tied to the entry count, not to the runtime environment.

// copy.mjs — two options while taking a snapshot: pausing or copying
import { Store, OVERHEAD, generateLoad } from './store.mjs';

const LOAD = generateLoad(60000, 20260731);
const SCAN = 40;                    // PM2: 40 writes of scan time per 1,000 entries
console.log('interval  snapshots  scan  pausing: waiting  copying: entries  peak bytes');

for (const interval of [1000, 5000, 20000]) {
  const store = new Store();
  let snapshots = 0, duration = 0, waiting = 0, copyCount = 0, peak = 0, remaining = 0, copy = null;
  for (let i = 0; i < LOAD.length; i++) {
    const [k, v] = LOAD[i];
    if (remaining > 0) {                  // snapshot in progress: the changing entry is copied first
      const previous = store.entries.get(k);
      if (previous !== undefined && !copy.has(k)) { copy.set(k, previous); copyCount++; }
      if (--remaining === 0) {
        let b = 0;
        for (const [ck, cv] of copy) b += Buffer.byteLength(ck) + Buffer.byteLength(cv) + OVERHEAD;
        if (b > peak) peak = b;
        copy = null;
      }
    }
    store.write(k, v);
    if ((i + 1) % interval === 0) {
      duration = Math.ceil((store.count / 1000) * SCAN);
      snapshots++; waiting += duration; remaining = duration; copy = new Map();
    }
  }
  console.log(String(interval).padStart(6), String(snapshots).padStart(8), String(duration).padStart(7),
    String(waiting).padStart(21), String(copyCount).padStart(18), String(peak).padStart(10));
}

interval  snapshots  scan  pausing: waiting  copying: entries  peak bytes
  1000       60     762                 33285              24985      59565
  5000       12     762                  6946               4710      58498
 20000        3     762                  1965                913      49266

The scan duration is the same in all three rows: 762 writes. Taking a snapshot depends on the entry count, not the interval, so shortening the interval does not make the scan faster — it only makes it more frequent.

At the thousand-write interval, the pausing option holds up 33,285 of the sixty thousand writes — more than half the load. At the twenty-thousand interval, that number drops to 1,965. The copying option holds up no writes at all; in exchange, it holds at most 59,565 extra bytes while the snapshot runs. That is roughly 3.7 percent of the store’s 1,593,100 bytes. The peak value coming out close across all three intervals has the same cause: what determines the peak is not the interval but how many distinct keys were touched during the scan duration. 762 writes could copy at most 762 distinct entries; because members repeat, the count stays around four hundred per snapshot. If writes had piled onto a few hot keys, the copy count would have been even lower; if every write had gone to a distinct key, the peak would have come close to double.

The total number of copied entries, though, grows with the interval: 24,985 against 913. Copying is cheap in peak memory, but as snapshots become more frequent it accumulates as work performed.

A hidden limit also becomes visible at the thousand-write interval. If the scan takes 762 writes, the store is taking a snapshot during 762 of every thousand writes. Once the interval is pushed below the scan duration, snapshots start overlapping; the new request is either skipped or races the previous one. Either way, the interval that actually occurs is larger than the one configured, and the data loss window written in the configuration no longer describes reality.

The Interval Decision

The three measurements come together in a single table. The numbers belong to the sixty-thousand-write run.

Interval	Average loss	Worst loss	Disk writes	Copied entries
1,000	501 writes	1,000 writes	30 MB	24,985
5,000	2,501 writes	5,000 writes	6 MB	4,710
20,000	10,001 writes	20,000 writes	2 MB	913

The decision is made by looking at what the lost write actually is. For the catalog cache, even the twenty-thousand interval is excessive protection: the entries that would be lost can be regenerated from their source anyway. For loan counters and the waiting list, an average loss of ten thousand writes can amount to a full day of the library’s transactions, and recovering it is done by hand.

This is where snapshot persistence hits its structural limit: no matter how short the interval gets, the data loss window never reaches zero, because every write after the last snapshot is unprotected. Pushing the interval below the scan duration does not help either. Actually closing the data loss window requires a format that records each write itself, not the entire store.

Summary

• Snapshot persistence writes the entire store to disk at intervals; the data loss window averages half the interval, with the worst case equal to the interval itself.
• In the measurement, the thousand-write interval brought average loss down from 10,001 to 501, and in exchange raised disk writes from 2 MB to 30 MB; a snapshot writes the entire store every time.
• A snapshot’s size grows with the key count, not the write count: 60,000 writes left 19,040 entries, and the file came out to 717,260 bytes.
• The same data holds 1,593,100 bytes in memory; the difference is overhead that never reaches disk, so memory requirements cannot be inferred from file size.
• While taking a snapshot, pausing held up 33,285 writes; copying held up none, but held 59,565 extra bytes at its peak; the peak is set by the number of distinct keys touched during the scan, not the interval.

Next Step

A snapshot writes the store’s state at one instant and records nothing that happens between two instants. The only way to close the data loss window is to record the writes that lead to a state, rather than the state itself. The next lesson takes this up: every write is appended to the end of a file as a record, and recovery replays those records. What gets measured is how many bytes a record costs per write, how much space the rewrite that trims an unboundedly growing file recovers, and how the choice of how often records are synced to disk sets the data loss window against the write cost.