Incremental Regeneration

One of static generation’s three limits was that freshness was tied to build frequency. A page may have been produced at build time, but it does not have to be treated as valid forever: it can be given a lifetime, and when that lifetime expires, the next request can receive the old copy while a new version is prepared in the background.

This mechanism is the server-side counterpart of the stale-while-revalidate behavior seen on the browser side in the Caching Strategies lesson. This lesson builds it as a rendering model and measures two quantities: how many requests see how much stale content, and how many regeneration jobs are born.

Three Response States

When a page request arrives, the store is checked, and one of three states occurs.

Miss: no copy of that page exists in the store. The request waits for production to finish and ends up paying server-side rendering’s cost. This state happens at most once per page.

Fresh hit: the copy in the store was produced within the freshness window. It is served directly; no production job is born.

Stale serve: a copy exists, but its window has expired. The copy is served as is, and regeneration is queued in the background. The request is not held up; the refresh is reflected in subsequent requests.

How these three states look on a server can be observed directly.

// incremental-server.mjs — server with a freshness window and background refresh
import { createServer } from "node:http";

const WINDOW = 3000;          // ms, how long a copy counts as fresh
const PRODUCTION_TIME = 1000; // ms, cost of producing the page

let sourceVersion = 0;
setInterval(() => { sourceVersion += 1; }, 1000).unref(); // the data changes every second

let record = null;       // { body, version, producedAt }
let refreshing = false;

const wait = (ms) => new Promise((c) => setTimeout(c, ms));

async function produce() {
  await wait(PRODUCTION_TIME);
  return { body: `measurement list version ${sourceVersion}\n`, version: sourceVersion, producedAt: Date.now() };
}

createServer(async (request, response) => {
  response.sendDate = false;
  if (request.url !== "/measurements") return response.writeHead(404).end();

  let state;
  if (!record) {
    state = "miss";                        // no copy: the request waits on production
    record = await produce();
  } else if (Date.now() - record.producedAt < WINDOW) {
    state = "fresh";                       // inside the window: served directly
  } else {
    state = "stale";                       // outside the window: the old copy is served
    if (!refreshing) {                     // refresh is queued only once
      refreshing = true;
      produce().then((next) => { record = next; refreshing = false; });
    }
  }
  response.setHeader("X-State", state);
  response.setHeader("Content-Type", "text/plain; charset=utf-8");
  response.setHeader("Content-Length", Buffer.byteLength(record.body));
  response.writeHead(200).end(record.body);
}).listen(8174, "127.0.0.1", () => console.log("listening: 127.0.0.1:8174"));

The script below runs in the same directory as this file. Port 8174 is chosen arbitrarily and must be free; if it is in use, change it in both files.

#!/usr/bin/env bash
# Starts incremental-server.mjs, makes requests at four different moments, then stops it.
node incremental-server.mjs > /dev/null &
server=$!
sleep 1

request() {
  printf '%-34s' "$1"
  curl -sS -D headers -w '   time %{time_total} s   ' http://127.0.0.1:8174/measurements | tr -d '\n'
  grep -i '^x-state' headers | tr -d '\r'
}

request "1) first request"
request "2) right after"
sleep 4
request "3) after the window fills"
sleep 2
request "4) after the refresh finishes"
rm -f headers

kill "$server"

1) first request                  measurement list version 2   time 1.008993 s   X-State: miss
2) right after                    measurement list version 2   time 0.000826 s   X-State: fresh
3) after the window fills         measurement list version 2   time 0.001321 s   X-State: stale
4) after the refresh finishes     measurement list version 7   time 0.001347 s   X-State: fresh

Duration fields depend on the machine. The decisive ratio is this: the miss takes about one second, while the other three requests stay under a few milliseconds — the gap between them is roughly a thousandfold. The third request serves a copy whose window has expired, and it is fast too; what waits is only the freshness, not the user.

The fourth line shows the mechanism’s characteristic behavior: the served version jumps from 2 to 7. Background production captures the source at the moment it finishes, not the moment it started; in the meantime, the source has advanced several versions.

The Freshness Window Does Not Tell the Real Age

The “fresh” stamp on a response reports not that the copy is identical to its source, but that it was produced within the window. The two are not the same thing. The simulation below exposes this distinction over a virtual clock; its output is independent of the machine.

// incremental.mjs — freshness window and background refresh queue model
// A virtual clock is used: the output is deterministic, not machine-dependent.

const DEFAULT_WINDOW = 60;      // seconds
const PRODUCTION_DURATION = 2;  // seconds, regenerating a page in the background
const DURATION = 1800;          // seconds, length of the simulation
const REQUEST_COUNT = 300;

// The data's version: the measurement list changes every 60 seconds, the archive never changes.
const sourceVersion = (path, t) => (path === "/measurements" ? Math.floor(t / 60) : 0);

// Deterministic pseudo-random sequence (linear congruential generator).
function generator(seed) {
  let x = seed;
  return () => (x = (x * 1103515245 + 12345) % 2147483648) / 2147483648;
}

// Request stream: paths are chosen by weight, times are produced in increasing order.
function requestStream() {
  const random = generator(20260728);
  const weight = [["/measurements", 6], ["/day-1", 2], ["/day-2", 1], ["/day-3", 1]];
  const pool = weight.flatMap(([path, n]) => Array(n).fill(path));
  const requests = [];
  for (let i = 0; i < REQUEST_COUNT; i++) {
    requests.push({
      t: Math.floor(random() * DURATION),
      path: pool[Math.floor(random() * pool.length)],
    });
  }
  return requests.sort((a, b) => a.t - b.t);
}

function run(window, keepLog) {
  const store = new Map();     // path -> { version, producedAt }
  const queue = new Map();     // path -> completion time
  const counts = { fresh: 0, stale: 0, miss: 0, production: 0 };
  const delays = [];
  const log = [];

  for (const { t, path } of requestStream()) {
    // If the queued refresh's duration has elapsed, write it to the store.
    const completesAt = queue.get(path);
    if (completesAt !== undefined && completesAt <= t) {
      store.set(path, { version: sourceVersion(path, completesAt), producedAt: completesAt });
      queue.delete(path);
      counts.production += 1;
    }

    const record = store.get(path);
    let outcome;
    if (!record) {
      // Miss: no copy, the request waits on production.
      store.set(path, { version: sourceVersion(path, t), producedAt: t });
      counts.miss += 1; counts.production += 1;
      outcome = "miss";
    } else if (t - record.producedAt < window) {
      counts.fresh += 1;
      outcome = "fresh";
    } else {
      // The old copy is served, the refresh is queued (a second one is not added if it is already queued).
      counts.stale += 1;
      if (!queue.has(path)) queue.set(path, t + PRODUCTION_DURATION);
      outcome = "stale";
    }

    const served = store.get(path);
    const delay = sourceVersion(path, t) - served.version;
    delays.push(delay);
    if (keepLog && path === "/measurements" && log.length < 14) {
      log.push(`${String(t).padStart(4)}s  ${outcome.padEnd(7)}` +
        `served version ${String(served.version).padStart(2)}   source version ` +
        `${String(sourceVersion(path, t)).padStart(2)}   delay ${delay}` +
        (queue.has(path) ? "   [queued]" : ""));
    }
  }
  const average = delays.reduce((a, b) => a + b, 0) / delays.length;
  return { counts, average, worst: Math.max(...delays), log };
}

const first = run(DEFAULT_WINDOW, true);
console.log(`--- /measurements requests, first 14 (window ${DEFAULT_WINDOW} s) ---`);
for (const line of first.log) console.log(line);

console.log(`\n--- ${REQUEST_COUNT} requests, ${DURATION} s, as the window changes ---`);
console.log("window".padStart(8) + "fresh".padStart(7) + "stale".padStart(7) + "miss".padStart(7) +
  "production".padStart(12) + "avg delay".padStart(14) + "worst".padStart(9));
for (const window of [10, 60, 300, 1800]) {
  const { counts, average, worst } = run(window, false);
  console.log(`${window} s`.padStart(8) + String(counts.fresh).padStart(7) +
    String(counts.stale).padStart(7) + String(counts.miss).padStart(7) +
    String(counts.production).padStart(12) + average.toFixed(2).padStart(14) +
    String(worst).padStart(9));
}

--- /measurements requests, first 14 (window 60 s) ---
   5s  miss   served version  0   source version  0   delay 0
   9s  fresh  served version  0   source version  0   delay 0
  20s  fresh  served version  0   source version  0   delay 0
  28s  fresh  served version  0   source version  0   delay 0
  37s  fresh  served version  0   source version  0   delay 0
  90s  stale  served version  0   source version  1   delay 1   [queued]
  92s  fresh  served version  1   source version  1   delay 0
  92s  fresh  served version  1   source version  1   delay 0
  98s  fresh  served version  1   source version  1   delay 0
 107s  fresh  served version  1   source version  1   delay 0
 116s  fresh  served version  1   source version  1   delay 0
 117s  fresh  served version  1   source version  1   delay 0
 146s  fresh  served version  1   source version  2   delay 1
 154s  stale  served version  1   source version  2   delay 1   [queued]

--- 300 requests, 1800 s, as the window changes ---
  window  fresh  stale   miss  production     avg delay    worst
    10 s     98    198      4         179          0.14        1
    60 s    216     80      4          73          0.38        2
   300 s    276     20      4          23          1.42        5
  1800 s    296      0      4           4          9.38       29

The log’s line at the 146th second is the lesson’s crucial point: the response is stamped fresh, but the delay is one. The copy was produced at the 92nd second; the 60-second window lasts until the 152nd second, yet the source advanced a version at the 120th second. The window measures the copy’s age, not the content’s correctness.

The line at the ninetieth second shows how the queue works: the old copy was served, the refresh was queued, and because it completed two seconds later, the request at the 92nd second saw the new version.

Window Length Is a Trade-off

The lower table shows the window length pulling two quantities in opposite directions. When the window is reduced to a tenth of a second, average delay drops to 0.14 versions, but the production count rises to 179. When the window is stretched to thirty minutes, production drops to four — only the first miss for each page — but average delay jumps to 9.38 versions, and the worst case to 29 versions.

The relationship between them is not linear. Stretching the window from 10 seconds to 60 seconds cuts production by 2.5 times while raising average delay by only 2.7 times; stretching it from 300 seconds to 1800 seconds cuts production by 5.8 times while raising delay by 6.6 times. The gain curve flattens: past a certain point, lengthening the window takes more from freshness than it saves.

It is also meaningful that the miss count is four in every row: there are four distinct paths, and each path experiences a miss only once. The count of requests that wait depends on the page count, not on request volume.

Points of Attention in Practice

The queue is deduplicated. In the model, a second refresh is not added while a path is already in the queue. Without this safeguard, a hundred concurrent requests to a popular page whose window has expired would spawn a hundred production jobs. If many copies’ windows expire at the same time, the production load spikes suddenly; adding a small random offset to the windows spreads out that spike.

The first miss is made predictable. The full production cost is paid the first time a page is requested. Pages with known traffic are prerendered during the build so that this cost is paid before launch; the rest are born on the first request.

Invalidation completes the window. The window is a time-based estimate. When it is known for certain that content has changed — a measurement is corrected, a page is taken down — directly invalidating the corresponding record removes the wait entirely. Time-based windows and event-based invalidation are used together.

Stale content is not hidden in the interface. A user looking at a measurement list should know when the value they see was produced. Writing the production time on the page does not remove the delay, but it stops the delay from being misleading.

Summary

• In incremental regeneration, a request falls into one of three states: a miss makes production wait, a fresh hit is served directly, and a stale serve gives the copy while queuing a refresh.
• The measurement shows a miss pays the production cost, while the other two states take on the order of a thousandth as long; what waits is the freshness, not the user.
• The freshness window measures the copy’s age, not how well it matches the source; a copy produced within the window can still be behind the source.
• Shortening the window reduces delay and increases production load; because the gain curve flattens, past a certain length the savings come at a higher cost to freshness.
• The miss count depends on the page count, not on request volume; deduplicating the queue keeps concurrent requests from multiplying the production load.

Next Step

The four models covered so far all argued about where the markup gets produced, and one thing stayed common to every one of them: the response is produced in a single piece, sent in a single piece, and is not interactive when it reaches the client. The gap measured in the Server-Side Rendering lesson — the interval between the moment content appears and the moment the button works — still stands. The next lesson names that gap and narrows it from two directions: sending the response piece by piece, and giving interactivity not to the whole page but only to the parts that genuinely need it.