Layer Cache

The previous lesson measured that a layer is born from every instruction and counted a single run line producing the bulk of the image. That measurement covered a single build. If a service is built several times a day, the same line does the same job every time — yet if its input has not changed, neither has its output, and layers are already named by their content digest.

This lesson builds the structure that removes that repetition. A layer cache stores a produced layer under its key; if the same key comes up on the next build, the layer is not rebuilt, it is taken from storage. What the key is derived from raises a decision: the instructions’ order. That is also what gets measured — how much does whether dependency install or source copy is written first move the hit rate, the count of regenerated layers, and the bytes written, across the same change sequence?

IM13. The cache is a model built with node; keys are kept in a map, and layers are really written under cache/<key>/. The tree is a simplified version of the previous lesson’s tree for this measurement; its byte values are not compared with that lesson’s numbers. IM14. The layer key is derived from the previous layer’s key and the instruction’s input digest. For copy, the input digest also covers the content of the files to be copied; for run, only the line itself. IM15. Duration is not measured. The quantities measured are run-independent: the count of regenerated layers and bytes written.

The Layer Key

The cache’s operation rests on one rule: a layer’s key must cover everything beneath it and its own input. If it does not, the layer above a changed lower layer hits by mistake and the image comes out inconsistent. The rule is the same as the previous lesson’s chain id; the difference is that the instruction’s input joins the chain too.

// measurement-network/cache.mjs — cached layer interpreter (model)
import { mkdirSync, writeFileSync, readFileSync, readdirSync, appendFileSync, statSync, rmSync } from "node:fs";
import { createHash } from "node:crypto";

const ROOT = "project";
const digest = (s) => createHash("sha256").update(s).digest("hex").slice(0, 12);
export const text = (name, n) => Array.from({ length: n }, (_, i) => `// ${name} ${i}\n`).join("");

const TREE = {
  "dependency-list": "csv 2.4\nqueue 1.9\n",
  "source/entry.js": text("entry", 20),
  "source/collect.js": text("collect", 140),
  "source/queue.js": text("queue", 95),
  "source/test/collect.test.js": text("test", 210),
};
export const OUTPUT = {
  "runtime-base": { "root/runtime.js": text("runtime", 420), "root/shell.js": text("shell", 60) },
  "install-deps": { "dependency/csv.js": text("csv", 900), "dependency/queue.js": text("queue-dep", 1400) },
};

export const ORDER_A = `base      runtime-base
copy      dependency-list  /app/list
run       install-deps
copy      source           /app/source`;

export const ORDER_B = `base      runtime-base
copy      source           /app/source
copy      dependency-list  /app/list
run       install-deps`;

// Change sequence: file touched before each build; empty means no change.
export const CHANGE = ["", "source/collect.js", "source/entry.js", "", "dependency-list",
  "source/collect.js", "source/test/collect.test.js", "source/queue.js"];

const writeFile = (path, content) => {
  mkdirSync(path.split("/").slice(0, -1).join("/"), { recursive: true });
  writeFileSync(path, content);
};
export const buildTree = () => {
  rmSync(ROOT, { recursive: true, force: true });
  rmSync("cache", { recursive: true, force: true });
  for (const [y, i] of Object.entries(TREE)) writeFile(`${ROOT}/${y}`, i);
};
export const touch = (path, no) => appendFileSync(`${ROOT}/${path}`, `// fix ${no}\n`);

const walk = (sub = "") => readdirSync(`${ROOT}/${sub}`, { withFileTypes: true }).flatMap((g) =>
  g.isDirectory() ? walk(`${sub}/${g.name}`) : [`${sub}/${g.name}`.replace(/^\//, "")]).sort();
const paths = (source) => statSync(`${ROOT}/${source}`).isDirectory() ? walk(source) : [source];

// Layer key: previous layer's key + the instruction's input digest.
// For "copy" the input digest also covers the content of the files to be copied; for "run" only the line itself.
const inputDigest = (line) => {
  const [instruction, source] = line.trim().split(/\s+/);
  if (instruction !== "copy") return digest(line);
  return digest(line + paths(source).map((y) => y + readFileSync(`${ROOT}/${y}`)).join());
};

export const build = (definition, cache) => {
  let key = "";
  const result = { hits: 0, builds: 0, bytes: 0, chain: [] };
  for (const line of definition.trim().split("\n")) {
    key = digest(key + inputDigest(line));
    result.chain.push(key);
    if (cache.has(key)) { result.hits++; continue; }
    const [instruction, source, target] = line.trim().split(/\s+/);
    const pairs = instruction === "copy"
      ? paths(source).map((y) => [`${target.slice(1)}/${y}`, readFileSync(`${ROOT}/${y}`)])
      : Object.entries(OUTPUT[source]);
    let bytes = 0;
    for (const [y, content] of pairs) { writeFile(`cache/${key}/${y}`, content); bytes += content.length; }
    cache.set(key, bytes);
    result.builds++; result.bytes += bytes;
  }
  return result;
};

A direct consequence of chaining the key is this: when one instruction misses, every instruction above it misses too, even if its own input never changed. The cache works bottom-up, not top-down, and everything after the first miss point is rebuilt in full.

Two Orders, the Same Change Sequence

Two definition files carry the same four instructions, only their order differs. Order A copies the dependency list and runs the install, leaving the source for last; order B moves the source to the front. Both definitions run over the same eight-build sequence; each step in the sequence adds one line to a file, in exactly the same order in both runs.

IM16. The change sequence represents the fictional reading collector service’s one-week fix flow: a first build with a cold cache, five source changes, one dependency-list change, and one build where nothing changes. IM17. Each order starts with its own empty cache; the cache is not shared between runs.

// measurement-network/run.mjs — runs two instruction orders over the same change sequence
import { buildTree, touch, build, CHANGE, ORDER_A, ORDER_B } from "./cache.mjs";

const runSequence = (label, definition) => {
  buildTree();
  const cache = new Map();
  const totals = { hits: 0, builds: 0, bytes: 0 };
  console.log(`\n${label}`);
  console.log(" no  change".padEnd(38) + "hits  builds     bytes");
  CHANGE.forEach((d, i) => {
    if (d) touch(d, i);
    const s = build(definition, cache);
    for (const k of Object.keys(totals)) totals[k] += s[k];
    console.log(`  ${i + 1}  ${d || (i ? "(no change)" : "(first build)")}`.padEnd(38) +
      String(s.hits).padStart(4) + String(s.builds).padStart(8) + String(s.bytes).padStart(9));
  });
  const attempts = CHANGE.length * definition.trim().split("\n").length;
  console.log(`  total: ${totals.hits}/${attempts} hits (%${((totals.hits * 100) / attempts).toFixed(1)}), ` +
    `layers built ${totals.builds}, ${totals.bytes} bytes written, ${cache.size} entries in cache ` +
    `${[...cache.values()].reduce((a, b) => a + b, 0)} bytes`);
  return totals;
};

const a = runSequence("A — dependency first, source last", ORDER_A);
const b = runSequence("B — source first, dependency last", ORDER_B);
console.log(`\nextra layers B builds over A: ${b.builds - a.builds}`);
console.log(`extra bytes B writes over A : ${b.bytes - a.bytes} (%${(((b.bytes - a.bytes) * 100) / a.bytes).toFixed(1)} more work)`);

A — dependency first, source last
 no  change                           hits  builds     bytes
  1  (first build)                       0       4    46558
  2  source/collect.js                   3       1     5769
  3  source/entry.js                     3       1     5778
  4  (no change)                         4       0        0
  5  dependency-list                     1       3    39685
  6  source/collect.js                   3       1     5787
  7  source/test/collect.test.js         3       1     5796
  8  source/queue.js                     3       1     5805
  total: 20/32 hits (%62.5), layers built 12, 115178 bytes written, 12 entries in cache 115178 bytes

B — source first, dependency last
 no  change                           hits  builds     bytes
  1  (first build)                       0       4    46558
  2  source/collect.js                   1       3    39667
  3  source/entry.js                     1       3    39676
  4  (no change)                         4       0        0
  5  dependency-list                     2       2    33907
  6  source/collect.js                   1       3    39694
  7  source/test/collect.test.js         1       3    39703
  8  source/queue.js                     1       3    39712
  total: 11/32 hits (%34.4), layers built 21, 278917 bytes written, 21 entries in cache 278917 bytes

extra layers B builds over A: 9
extra bytes B writes over A : 163739 (%142.2 more work)

What the Order Costs

Same four instructions, same eight changes, two different outcomes. Order A hits on 20 of 32 layer attempts — 62.5 percent — and builds 12 layers. Order B, over the same sequence, gets 11 hits, 34.4 percent, and produces 21 layers. Bytes written are 115,178 against 278,917: order B writes 142.2 percent more bytes to do the same job.

The entire difference collects in five lines — the builds with a source change. In order A, source copying sits on top; no layer beneath it is affected, and only a layer of around 5,800 bytes gets rebuilt. In order B, source copying sits at the bottom; the moment the source is touched, the list copy and dependency install above it invalidate too, and about 39,700 bytes get written. The roughly sevenfold difference does not come from the source files being small — it comes from what sits above them.

The fifth build shows the reverse direction. When the dependency list changes, order A builds three layers (39,685 bytes), order B builds two layers (33,907 bytes) — on this build, B is cheaper. The decision cannot be made by looking at a single build; it is made by looking at the sequence’s composition. Five of the eight builds change the source, one changes the dependency list, and in a real service tree the ratio is even sharper. The rule reads as follows: instructions are ordered by how often they change, the least-frequently-changing at the bottom. This is the build side’s counterpart to the previous lesson’s layer-order rule, and both rules come from the same measurement.

The fourth build counts too: when nothing changes, both orders give 4 hits and 0 bytes. The cache’s gain depends not only on the order but on whether a change exists at all — on an unchanged tree, both orders run for free.

The hit rate itself must be read carefully too. The denominator is the number of layer attempts: eight builds over a four-instruction definition make 32 attempts. If the instruction count grows, the denominator grows, and the same behavior can show a higher rate; the ratio alone is not enough to compare two different definition files. The two numbers that carry the decision are layers rebuilt and bytes written, because both are a direct measure of the work done and are independent of the definition’s line count. In this measurement all three point the same way, so the order decision is not in dispute; where they do not agree, the column to watch is the bytes column.

The cache’s own cost sits in the last column. Order A’s cache accumulates 12 entries and 115,178 bytes; order B’s accumulates 21 entries and 278,917 bytes. No entry is ever deleted: every miss writes a new layer to disk, and the old one stays in place. The cache is a store that buys build time at the price of bytes, and a bad order grows this store too, to about 2.4 times its size.

Where the store sits is also counted. The layer cache accumulates on the machine the build runs on; the 12- and 21-entry stores are that machine’s state. When the same definition file runs on a machine with an empty cache, both orders pay the first row’s cost: 4 layers, 46,558 bytes. The order decision’s gain shows up only from the second build onward. The difference measured under the name environment parity in the previous course takes a new shape here: a build’s result is independent of the machine, its cost is not. This is the item the isolation budget pays in this lesson — the repeated work was not pulled out of the environment, it was moved into a store, and that store still sits in the environment.

What the Cache Does Not Measure

The key covers only the line itself for the run instruction. As long as the line does not change, the layer is not rebuilt — the thing to be installed having changed on the outside does not enter the key. IM18. The outside world changing is modeled by growing the content in the install table; the list file is left untouched.

// measurement-network/stale.mjs — the input the cache key does not measure
import { buildTree, touch, build, text, OUTPUT, ORDER_A } from "./cache.mjs";

buildTree();
const cache = new Map();
const first = build(ORDER_A, cache);
const installKey = first.chain[2];
console.log(`first build: ${first.builds} layers, ${first.bytes} bytes`);

// The outside world changed: the content to install grew, the list file stayed the same.
OUTPUT["install-deps"]["dependency/csv.js"] = text("csv", 950);
const updated = Object.values(OUTPUT["install-deps"]).reduce((t, i) => t + i.length, 0);
const s = build(ORDER_A, cache);
console.log(`\nlist file unchanged, content to install changed:`);
console.log(`  hits ${s.hits}, layers rebuilt ${s.builds}, ${s.bytes} bytes written`);
console.log(`  install layer in cache ${cache.get(installKey)} bytes, updated content ${updated} bytes`);
console.log(`  difference ${updated - cache.get(installKey)} bytes, same key: ${s.chain[2] === installKey}`);

touch("dependency-list", 9);
const t = build(ORDER_A, cache);
console.log(`\nonce a line is added to the list file:`);
console.log(`  hits ${t.hits}, layers rebuilt ${t.builds}, ${t.bytes} bytes written`);
console.log(`  install layer ${cache.get(t.chain[2])} bytes`);

first build: 4 layers, 46558 bytes

list file unchanged, content to install changed:
  hits 4, layers rebuilt 0, 0 bytes written
  install layer in cache 33880 bytes, updated content 34430 bytes
  difference 550 bytes, same key: true

once a line is added to the list file:
  hits 1, layers rebuilt 3, 40217 bytes written
  install layer 34430 bytes

This is where isolation is pierced. Even though the content to install grew by 550 bytes, all four instructions hit, zero layers get built, and the image keeps carrying the old content. The key is the same, because the key measures the run line’s text, not its result. The cache is working correctly; what it measures is incomplete.

This is the cache-side shape of the reproducibility problem measured in the previous course. If a run line’s output does the work of something time-dependent — an install with no version pin written down, say — the same line produces two different layers at two different times, but the cache names both with the same key. The fix is not in the cache but in the instruction: if run’s input is written to a file, that file enters the context through copy, and its content joins the key. This is exactly what happens in the third block when a single line is added to the list file — the key changes, the install layer is rebuilt, and the layer in the store updates to 34,430 bytes.

Copy granularity is also an item of shortfall. In the seventh build, the file that changes is a test file; a file that will never run at runtime rebuilds a 5,796-byte layer, because the source directory is copied with a single instruction. The unneeded files the previous lesson measured extract a cost here a second time: first in image size, then in cache hits.

Summary

• The layer key is derived from the previous layer’s key and the instruction’s input digest. When one instruction misses, every instruction above it misses too; the cache rebuilds everything after the first miss point entirely.
• Same four instructions, same eight builds: with dependency install written first, 20 of 32 attempts hit (62.5 percent), 12 layers were built, and 115,178 bytes were written. Moving source copy first dropped hits to 11 (34.4 percent), layers rose to 21, and bytes written rose to 278,917 — 142.2 percent more work.
• The difference collects in the builds with a source change: around 5,800 bytes in order A, around 39,700 bytes in order B. When the dependency list changes, the direction reverses; the decision is made not by a single build but by the change sequence’s composition.
• The cache’s own cost is a store, and it only grows: order A accumulated 12 entries and 115,178 bytes, order B accumulated 21 entries and 278,917 bytes.
• The key measures only run’s line text. Even though the content to install grew by 550 bytes, all four instructions hit and zero layers were built; the image kept carrying the old content. Once the input is written to a file and pulled into the context, the key changes and the layer updates.

Next Step

The cache removed repeated work but never touched the image’s content. The image that emerges at the end of eight builds is identical to the stack the first build produced: everything the dependency install left behind, every source file copied, and every tool used during install sits in the final layer. Yet part of this was only needed during the build; the only thing running at runtime is the application itself. The next lesson measures this distinction: when the same application is built single-stage and multi-stage, how far apart are the final image’s size, layer count, and count of unneeded files left inside it, and what is multi-stage building’s cost on the cache side?