Volumes and Bind Mounts

The previous lesson measured that loss was unacceptable in two classes: job state and user data, 7,951 bytes in total. Both went away on deletion because both sat inside the writable layer. The fix fits in one sentence — put those two files outside the layer. But outside the layer is the host machine’s filesystem, and every path opened onto it spends an entry from the isolation budget.

This lesson builds two mount forms with real directories and counts three things separately: bytes remaining when the container is deleted, nodes that cannot be written when the container’s internal user identity does not match the directory’s owner, and which form breaks when the machine changes. The third measurement also names, in this course, the place where isolation is punctured most plainly.

Two Mount Forms

The filesystem a container sees is a mount table: every path inside the container corresponds to somewhere on the host machine. The writable layer is itself a row in this table, the only difference being that it is deleted with the container. There are two ways to get outside the layer, and the table distinguishes the two by how they are referred to.

A managed volume is referred to by name. The runtime resolves the name to a location under its own root directory; only the name sits in the config, not a path. A bind mount is referred to directly by an absolute path; the host directory path is written into the config, and there is no resolution step — the path is whatever it is.

The two share a side effect: if the mount point lands on a directory already populated in the image, the content there becomes invisible for as long as the mount lasts — mount shadowing. The bytes in the lower layer stay put, unerased; it is just that a different directory now sits at the end of the path the container sees. A default setting baked into the image thus silently disappears at run time; no error is raised, the file looks “gone”.

Which data gets moved out is also a budget decision. Of the five classes in lesson 02, only two were mounted; the log was left in the writable layer, and its 673,560 bytes went away again. Had it been mounted, it would not have been lost, but 673,560 bytes would have accumulated on persistent disk over thirty nights, adding the question of who deletes it. Every class moved out gains persistence, and in exchange loads a lifecycle responsibility.

• RT13 — The mount table is a model: three in-container paths correspond to three separate host machine targets. The directories and bytes are real.
• RT14 — The run from lesson 02 is repeated exactly: 30 nights, 480 readings a night, the same seed. The only thing that changes is where job state and corrections are written; the log stays in the writable layer.
• RT15 — The managed volume is referred to by the name reading-state under the runtime’s own root directory; the bind mount is referred to by an absolute path on the host machine.
• RT16 — The user identity inside the container does not match the file owner on the host machine; this is why the permission class applied is “other”. The mode bits are read from real files.
• RT17 — The mode distribution of the eight nodes is fictional; it represents the layout the field crew left behind.
• RT18 — The machine change is modeled by renaming the root directory.

// runtime/mounts.mjs — two mount forms: persistence, ownership conflict, portability.
// MODEL: the mount table and the container's user identity are modeled; directories, bytes, and mode bits are real.
import { mkdirSync, writeFileSync, appendFileSync, chmodSync, statSync, existsSync,
  readdirSync, renameSync, rmSync } from "node:fs";
import { join, dirname, resolve } from "node:path";

const ROOT = "root", RUNTIME = `${ROOT}/runtime`, UPPER = `${ROOT}/container/upper`, FIELD = `${ROOT}/field`;
const READINGS = 480, NIGHTS = 30, SEED = 20260801;             // same run as lesson 02
rmSync(ROOT, { recursive: true, force: true }); rmSync("root-b", { recursive: true, force: true });

const MOUNTS = {                                                 // RT13: in-container path -> host machine
  "/log": { kind: "writable layer", target: `${UPPER}/log` },
  "/data/state": { kind: "managed volume", name: "reading-state", target: `${RUNTIME}/volumes/reading-state` },
  "/data/corrections": { kind: "direct mount", hostPath: resolve(`${FIELD}/corrections`), target: `${FIELD}/corrections` },
};
const write = (mountPath, file, content, append) => {
  const p = join(MOUNTS[mountPath].target, file);
  mkdirSync(dirname(p), { recursive: true });
  (append ? appendFileSync : writeFileSync)(p, content);
};

let s = SEED % 2147483647;
const rand = () => (s = (s * 48271) % 2147483647) / 2147483647;
for (let g = 1; g <= NIGHTS; g += 1)
  for (let i = 1; i <= READINGS; i += 1) {
    const value = (100 + rand() * 900).toFixed(3);
    write("/log", "verification.log", `reading ${i} meter ${1000 + i} value ${value} status ok\n`, true);
    if (i % 60 === 0) write("/data/state", "progress.json", `{"processed":${i},"remaining":${READINGS - i}}`, false);
    if (i % 40 === 0) write("/data/corrections", "corrections.csv", `${1000 + i},operator,${value}\n`, true);
  }

const measure = (k) => { let b = 0, n = 0;
  const walk = (d) => { for (const e of readdirSync(d, { withFileTypes: true })) {
    const p = join(d, e.name); if (e.isDirectory()) walk(p); else { b += statSync(p).size; n += 1; } } };
  if (existsSync(k)) walk(k); return { bytes: b, files: n }; };

const before = Object.fromEntries(Object.entries(MOUNTS).map(([mountPath, t]) => [mountPath, measure(t.target)]));
rmSync(`${ROOT}/container`, { recursive: true, force: true });    // container is deleted
console.log(`${"mount".padEnd(22)}${"kind".padEnd(21)}${"before delete".padStart(14)}${"after delete".padStart(16)}`);
let remaining = 0, gone = 0;
for (const [mountPath, t] of Object.entries(MOUNTS)) {
  const after = measure(t.target).bytes;
  remaining += after; gone += before[mountPath].bytes - after;
  console.log(`${mountPath.padEnd(22)}${t.kind.padEnd(21)}${String(before[mountPath].bytes).padStart(14)}${String(after).padStart(16)}`);
}
console.log(`container deleted: gone ${gone} bytes, remaining ${remaining} bytes (lesson 02 had 0 bytes remaining)`);

// RT16: the identity inside the container does not match the host machine's owner; the applied class is "other".
const NODES = [[`${RUNTIME}/volumes/reading-state`, 0o755, "d"], [`${RUNTIME}/volumes/reading-state/progress.json`, 0o644, "f"],
  [`${RUNTIME}/volumes/reading-state/lock`, 0o644, "f"], [`${FIELD}/corrections`, 0o775, "d"],
  [`${FIELD}/corrections/corrections.csv`, 0o664, "f"], [`${FIELD}/archive/2026-07.csv`, 0o600, "f"],
  [`${FIELD}/share`, 0o777, "d"], [`${FIELD}/share/transfer.csv`, 0o666, "f"]];
for (const [p, m, kind] of NODES) {
  if (kind === "d") mkdirSync(p, { recursive: true });
  else if (!existsSync(p)) { mkdirSync(dirname(p), { recursive: true }); writeFileSync(p, "x"); }
  chmodSync(p, m);
}
console.log(`\n${"node".padEnd(46)}${"mode".padStart(6)}${"owner writable".padStart(17)}${"other writable".padStart(17)}`);
let ownerW = 0, otherW = 0;
for (const [p] of NODES) {
  const m = statSync(p).mode & 0o777;
  const s1 = (m & 0o200) !== 0, s2 = (m & 0o002) !== 0;
  ownerW += s1 ? 1 : 0; otherW += s2 ? 1 : 0;
  console.log(`${p.replace(`${ROOT}/`, "").padEnd(46)}${m.toString(8).padStart(6)}` +
    `${(s1 ? "yes" : "no").padStart(17)}${(s2 ? "yes" : "no").padStart(17)}`);
}
console.log(`when identities do not match, ${NODES.length - otherW} of ${NODES.length} nodes cannot be written; ` +
  `when identities are equal, ${NODES.length - ownerW} cannot be written`);

renameSync(ROOT, "root-b");                                        // machine changed: layout is different
console.log(`\n${"mount".padEnd(22)}${"reference in config".padEnd(30)}${"on new machine".padStart(14)}`);
let broken = 0, absolute = 0;
for (const [mountPath, t] of Object.entries(MOUNTS)) {
  if (t.kind === "writable layer") continue;
  const newPath = t.name ? join("root-b/runtime/volumes", t.name) : t.hostPath;
  const resolved = existsSync(newPath);
  if (!resolved) broken += 1;
  if (!t.name) absolute += 1;
  console.log(`${mountPath.padEnd(22)}${(t.name ? `name: ${t.name}` : "absolute path").padEnd(30)}${(resolved ? "resolved" : "broken").padStart(14)}`);
}
console.log(`${absolute} of the two mounts is tied to an absolute path; when the machine changed, ${broken} broke`);
rmSync("root-b", { recursive: true, force: true });

mount                 kind                  before delete    after delete
/log                  writable layer               673560               0
/data/state           managed volume                   31              31
/data/corrections     direct mount                   7920            7920
container deleted: gone 673560 bytes, remaining 7951 bytes (lesson 02 had 0 bytes remaining)

node                                            mode   owner writable   other writable
runtime/volumes/reading-state                    755              yes               no
runtime/volumes/reading-state/progress.json      644              yes               no
runtime/volumes/reading-state/lock               644              yes               no
field/corrections                                775              yes               no
field/corrections/corrections.csv                664              yes               no
field/archive/2026-07.csv                        600              yes               no
field/share                                      777              yes              yes
field/share/transfer.csv                         666              yes              yes
when identities do not match, 6 of 8 nodes cannot be written; when identities are equal, 0 cannot be written

mount                 reference in config           on new machine
/data/state           name: reading-state                 resolved
/data/corrections     absolute path                         broken
1 of the two mounts is tied to an absolute path; when the machine changed, 1 broke

These numbers are in the measurement class.

Persistence: A Deleted Container, Bytes That Remain

The first table shows that the problem is solved, and it also gives the boundary of that solution. After the container is deleted, the 673,560-byte log in the writable layer is gone; the 31 bytes in the managed volume and the 7,920 bytes in the bind mount are still there. 7,951 bytes remained in total — exactly the previous lesson’s unrecoverable number, and in that lesson this number was zero.

From a persistence standpoint, there is no difference at all between the two forms. Both sit outside the writable layer, and neither is tied to the container’s lifetime. This measure does not distinguish between them; the measures that do are the ones that follow.

Here the cost of isolation is measured not in run terms but in surface. In the previous lesson, the container’s write surface onto the host machine was a single directory, and it went away with the container. Now two more directories have been added, and both outlive the container. The container no longer starts from where it started before; its starting state depends on the 7,951 bytes sitting on disk. Statelessness, in the sense measured in lesson 02, is broken here — deliberately, and in exchange for something.

The second entry in the cost is not immediately visible. There is no longer a lifecycle that deletes the remaining 7,951 bytes. The container was deleted, the layer went away, the volume and the mount stayed put; what removes them is a separate decision, not the container’s death. Every directory moved outside isolation also demands its own retention rule — who deletes it, when, by what criterion. Left undecided, the measure becomes this: the container count stays constant while the number of directories sitting on disk keeps climbing, because every new name creates a new volume and nobody goes looking for the old ones.

Ownership: Where Isolation Is Punctured

The second table gives the place where isolation is most concretely punctured in containers.

The user running inside the container has an identity number. A directory opened through a bind mount, though, is on the host machine’s filesystem, and its files belong to a host machine user. Write permission is decided by comparing these two identities, and an identity mismatch between them is the default, not the exception. The namespace separates processes, the network, and the filesystem view; it does not separate the user identity’s number space. The identity inside the container and the owner on the host machine live in the same number space, and file permission looks only at that number.

In numbers: six of the eight nodes cannot be written. The two that can are the ones whose mode bits grant write to the “other” class — the 777 directory and the 666 file. The remaining six nodes are closed to the container even though their owner can write: 755, 644, 644, 775, 664, and 600. When the identities are made equal, the number drops to zero; all eight nodes become writable, because the class applied is no longer “other” but “owner”.

Two rows deserve a separate reading. field/corrections/corrections.csv is in mode 664; its owner and its group can write to it. This is usually assumed to be the fix — put the container user in the right group and it can write. But the group is also a number, and if the container’s internal group identity does not match the host machine’s, the class applied is again “other”; that row’s answer in the table is no. field/archive/2026-07.csv, meanwhile, is in mode 600 and open only to its owner: a past period’s archive, sitting inside the mounted directory, is both unwritable and unreadable for the container. A mount makes the directory visible; it does not grant access to what is inside it.

This table shows both wrong fixes as well. The first is loosening permissions: the 777 and 666 rows really are writable, but those two nodes are also open to every host machine user; the container’s write access costs write access for everyone with nothing to do with the container. The second is equating identities: the number drops to zero, but then the container behaves like a host machine user and can do anything that user can do in the mounted directory. From the isolation budget’s perspective, both spend the same entry — the authority surface opened onto the host machine’s filesystem.

There is a third way, and its measure can be read indirectly in the table: treat the mounted directory as read-only for the container. Then the six closed nodes stop being a problem, because writing was never wanted from them in the first place. In the measurement network this distinction is clean — the tariff table and the meter list are read, job state and corrections are written. Asked how many of the eight nodes genuinely need to be written, the number drops to two, and the conflict narrows to those two as well.

A managed volume does not eliminate this conflict, it only controls the starting state: because the volume is created by the runtime, its ownership can be set from the start to the identity the container expects. There is no such moment with a bind mount; the directory already exists, its owner is already fixed, and the container has to conform to it.

Portability: The Cost of an Absolute Path

The third table is the measure that separates the two forms. When the machine changes — the run models this by renaming the root directory — the managed volume resolved, and the bind mount broke.

The reason is the kind of reference in the config. The volume is referred to by name, and the name resolves under the runtime’s own root on every machine; wherever the root is, reading-state is found. The mount is referred to by an absolute path, and that path belongs to one machine’s directory layout. One of the two mounts is tied to an absolute path, and when the machine changed one broke — that is, the portability ratio is one in two.

The difference removed from the environment has a number here, and the table reads in two directions at once. Looked at from inside the container, the difference is removed entirely: the application writes to /data/state and /data/corrections on every machine, both paths are the same everywhere, both are embedded in the code, and neither changes with the machine. The difference has been pushed onto the host machine’s side, onto the mount table’s right-hand column, where one of the two references is tied to the machine. So the removed difference did not disappear, it collapsed into a single row — and that row now belongs to the config, not the application.

This is not a flaw of the bind mount, it is its definition: its purpose is precisely to open a specific host machine directory into the container. A development setup that mounts the directory holding the source code does this deliberately; being tied to the machine is the wanted behavior. The problem is the same mount carried over into the run environment. The number says this: if a machine-independent config is wanted, only one of the two forms makes that promise.

Summary

• Both mount forms sit outside the writable layer: after the container is deleted, 31 bytes remain in the managed volume and 7,920 bytes in the bind mount; 7,951 bytes in total, which was zero in the previous lesson. The 673,560-byte log in the writable layer went away again.
• The persistence measure does not distinguish the two forms; the measures that do are ownership and portability.
• Where isolation is punctured: the namespace does not separate the user identity’s number space. When the identity inside the container does not match the file owner on the host machine, six of the eight nodes cannot be written; when the identities are made equal, the number drops to zero.
• Both wrong fixes spend the same entry: loosening permissions (777, 666) opens the node to everyone on the host machine, and equating identities gives the container the host machine user’s authority.
• A managed volume does not remove the conflict, it controls the starting state: its ownership can be set the moment the volume is created; there is no such moment with a bind mount.
• The two forms diverge in portability: one of the two mounts is tied to an absolute path, and it broke when the machine changed; the volume referred to by name resolved.

Next Step

Persistent data has settled into place: job state and corrections now outlive the container, paid for with two directories opened onto the host machine’s filesystem. Everything measured here was within a single container’s own boundaries — its resources, its filesystem, its identity. But the measurement network is not a single program: the reading collector feeds the verifier, the verifier feeds billing, billing feeds the work order service, and the four run in separate containers. Once the filesystem is isolated, how do these four find each other, what name do they call each other by, and what happens when one is meant to be unreachable from another? The next lesson builds a name resolution and reachability matrix and counts this: how many paths close when four services are split across two networks, and how far the surface opened to the outside drops, from how many endpoints to how many.