---
title: 'Ephemeral Filesystem'
source: 'https://academia.sh/en/courses/containers/ephemeral-filesystem'
course: Containers
language: en
updated: '2026-08-23T16:55:07+00:00'
license: 'CC BY-SA 4.0'
---

# Ephemeral Filesystem

The fate of the data a container produces while running: building the writable layer with real directories, the distribution across data classes of the bytes accumulating over a thirty-night run, a one-byte change taking up an entire file's worth of space through copy-up, the number of records lost when the container is deleted, and which data class that loss is acceptable in.

The previous lesson measured two settings given to the container **from outside**: the resource
limit and the environment variables. Both were known before the container started. But a running
container does not sit idle. The verifier writes a log line for every reading, puts the tariff
value it computed into a cache, records where the nightly run has gotten to somewhere, appends the
operator's hand-entered correction to a file. None of this data was in the image, and none of it
came from the environment either — the container itself produced it.

The question is: where did it get written, how much space did it take up, and what happened to it
once the container went away. The answer is isolation's quietest cost, because it throws no error
and exceeds no limit. Data is written, read, used for a while, and then one day it is gone.

## The Writable Layer

The image is read-only, and this is not a preference but a consequence of the layer model: layers
are addressed by content digest, if the content changes the digest changes, and if the digest
changes the layer becomes a different layer. When a container needs to write, a **writable layer**
is added on top of this stack. Reads resolve from top to bottom; writes always land on top.

The run below builds this arrangement with real directories. The union filesystem is a **model**
— the read resolution and copy-up rule are hand-written — but the directories, files, and byte
counts are real; all of it is written to disk and measured with `node:fs`.

- **RT7** — The image's lower layers consist of four files: application code, rule set, tariff
  table, meter list. The sizes are fictional; the files are really written.
- **RT8** — The nightly run processes 480 meter readings.
- **RT9** — Every reading produces a log line; every fourth reading a cache write; every
  sixtieth reading a progress write; every fortieth an operator correction.
- **RT10** — At the end of every night, a one-byte change is made to the tariff table.
- **RT11** — The container runs for 30 nights without ever restarting, then is deleted. Deletion
  removes the writable layer and does not touch the lower layers.

```js
// runtime/writable-layer.mjs — counting what accumulates in the writable layer and what leaves when the container is deleted.
// MODEL: the union filesystem is modeled with node; directories, files, and byte counts are real.
import { mkdirSync, writeFileSync, appendFileSync, copyFileSync, existsSync, statSync,
  readdirSync, rmSync } from "node:fs";
import { join, dirname } from "node:path";

const LOWER = "root/lower", UPPER = "root/upper";        // read-only image layers / writable layer
const READINGS = 480, NIGHTS = 30, SEED = 20260801;       // RT8: nightly run, RT11: container lifetime
rmSync("root", { recursive: true, force: true });

const put = (root, path, content) => { mkdirSync(join(root, dirname(path)), { recursive: true });
  writeFileSync(join(root, path), content); };
for (const [path, n] of [["app/verifier.js", 12288], ["app/rules.json", 49152],
  ["config/tariff.csv", 98304], ["data/meter-list.csv", 204800]]) put(LOWER, path, "x".repeat(n));

let copiedBytes = 0, copiedCount = 0;
function appendToUpper(path, line) {                     // absent in upper, present in lower: copy-up
  const u = join(UPPER, path);
  if (!existsSync(u) && existsSync(join(LOWER, path))) {
    mkdirSync(dirname(u), { recursive: true });
    copyFileSync(join(LOWER, path), u);
    copiedBytes += statSync(u).size; copiedCount += 1;
  }
  mkdirSync(dirname(u), { recursive: true });
  appendFileSync(u, line);
}

let s = SEED % 2147483647;
const rand = () => (s = (s * 48271) % 2147483647) / 2147483647;
const counts = { log: 0, cache: 0, state: 0, user: 0, config: 0 };
function nightlyRun() {
  for (let i = 1; i <= READINGS; i += 1) {
    const value = (100 + rand() * 900).toFixed(3);
    appendToUpper("log/verification.log", `reading ${i} meter ${1000 + i} value ${value} status ok\n`);
    counts.log += 1;
    if (i % 4 === 0) { put(UPPER, `cache/tariff-${i / 4}.json`, `{"meter":${1000 + i},"unit":"${value}"}`); counts.cache += 1; }
    if (i % 60 === 0) { put(UPPER, "state/progress.json", `{"processed":${i},"remaining":${READINGS - i}}`); counts.state += 1; }
    if (i % 40 === 0) { appendToUpper("data/corrections.csv", `${1000 + i},operator,${value}\n`); counts.user += 1; }
  }
  appendToUpper("config/tariff.csv", "#");                // one-byte change, whole file gets copied
  counts.config += 1;
}

const measure = (root, on = "") => { let b = 0, n = 0;
  const walk = (d) => { for (const e of readdirSync(d, { withFileTypes: true })) {
    const p = join(d, e.name);
    if (e.isDirectory()) walk(p); else if (p.includes(on)) { b += statSync(p).size; n += 1; } } };
  if (existsSync(root)) walk(root); return { bytes: b, files: n }; };

const a = measure(LOWER);
const PREFIX = { log: "log/", cache: "cache/", state: "state/", user: "data/", config: "config/" };
console.log(`model: ${READINGS} readings/night, ${NIGHTS} nights (seed ${SEED}); lower layers ` +
  `${a.bytes} bytes / ${a.files} files, all read-only`);
console.log(`\n${"night".padEnd(6)}${"total bytes".padStart(12)}${"files".padStart(7)}` +
  Object.keys(PREFIX).map((k) => k.padStart(11)).join("") + `${"ratio to lower".padStart(19)}`);
for (let g = 1; g <= NIGHTS; g += 1) {
  nightlyRun();
  if (![1, 2, 7, 30].includes(g)) continue;
  const u = measure(UPPER);
  console.log(`${String(g).padEnd(6)}${String(u.bytes).padStart(12)}${String(u.files).padStart(7)}` +
    Object.values(PREFIX).map((o) => String(measure(UPPER, o).bytes).padStart(11)).join("") +
    `${(u.bytes / a.bytes).toFixed(2).padStart(19)}`);
}

const u = measure(UPPER);
console.log(`\ncopy-up ${copiedCount} files, ${copiedBytes} bytes; share of the layer first night ` +
  `${((100 * (copiedBytes + 1)) / 124772).toFixed(1)}%, night thirty ${((100 * copiedBytes) / u.bytes).toFixed(1)}%`);
rmSync(UPPER, { recursive: true, force: true });          // container is deleted
console.log(`container deleted: remaining ${measure(LOWER).bytes} bytes / ${measure(LOWER).files} files (lower layers), ` +
  `gone ${u.bytes} bytes / ${u.files} files`);
console.log(`records gone: ${counts.log} log lines, ${counts.cache} cache writes, ` +
  `${counts.state} progress writes, ${counts.user} operator corrections, ${counts.config} config changes`);
console.log(`if two containers ran from the same image: shared ${a.bytes + 2 * u.bytes} bytes, ` +
  `unshared ${2 * (a.bytes + u.bytes)} bytes; lower-layer sharing saves ${a.bytes} bytes`);
rmSync("root", { recursive: true, force: true });
```

```
model: 480 readings/night, 30 nights (seed 20260801); lower layers 364544 bytes / 4 files, all read-only

night  total bytes  files        log      cache      state       user     config     ratio to lower
1           124772    124      22452       3720         31        264      98305               0.34
2           147489    124      44904       3720         31        528      98306               0.40
7           261074    124     157164       3720         31       1848      98311               0.72
30          783565    124     673560       3720         31       7920      98334               2.15

copy-up 1 files, 98304 bytes; share of the layer first night 78.8%, night thirty 12.5%
container deleted: remaining 364544 bytes / 4 files (lower layers), gone 783565 bytes / 124 files
records gone: 14400 log lines, 3600 cache writes, 240 progress writes, 360 operator corrections, 30 config changes
if two containers ran from the same image: shared 1931674 bytes, unshared 2296218 bytes; lower-layer sharing saves 364544 bytes
```

These numbers are in the **measurement** class.

## Copy-Up: One Byte Costing 98,304 Bytes

The first row looks wrong at first glance: the first night's `config` column is 98,305 bytes. What
was changed was a single byte.

The reason lies in the write rule. The writable layer **cannot modify** a file in the lower layer,
because the lower layer is read-only and addressed by its digest. If a lower-layer file needs to
be written to, the whole file is first copied to the upper layer, and then the copy is modified.
This is called **copy-up**, and its cost is measured not by the size of the change but by the size
of **the file**.

The first night's makeup is skewed as a result. The genuine new data the run produces is 26,468
bytes — what is left once the 98,304 bytes copied are subtracted from the layer's total. What sits
in the layer is 124,772 bytes, that is, **4.7 times** as much. The entire difference is a single
copied file, and it fills 78.8% of the layer that night. The cost of isolation is stark here: what
giving up the right to modify a file in place costs is 98,304 bytes of space and that much copying
work, for a one-byte change.

The cost is paid once. The second night's `config` column is 98,306: the file is now in the upper
layer, the second byte is appended to its end, and nothing gets copied. The practical consequence
is this — in the writable layer, **which** file is touched for the first time is more expensive
than how much gets written to it afterward.

## Over Thirty Nights, the Layer's Shape Changes

The layer's makeup inverts as the table's rows go down. On the thirtieth night, the total is
783,565 bytes; the `config` column still sits at 98,334, and its share has dropped from 78.8% to
12.5%. Taking its place is the `log` column: 673,560 bytes, 86% of the layer. Log grows linearly
because it adds 22,452 bytes every night; copy-up, on the other hand, stays constant because it is
paid once.

The right-hand column gives the real fact. The writable layer's ratio to the lower layers is 0.34
on the first night and climbs to **2.15** by the thirtieth. By the end of thirty nights, more than
two-thirds of the space the container holds on disk is not the image's — it is data the run
produced. The phrase "the container is light" has a number behind it here, and the number that is
true on day one is false on day thirty.

Two columns do not grow at all, and their reasons are different. `cache` stays fixed at 3,720
bytes, because the same 120 tariff keys are overwritten every night. `state` stays fixed at 31
bytes, because the progress file is overwritten on every write — of the 240 writes, only the last
one remains. This is also why the file count stays at 124 over the thirty nights: the run does not
open new files, it grows the ones that already exist.

## What Leaves When the Container Is Deleted

After deletion, two numbers remain: 364,544 bytes / 4 files stand in the lower layers, and 783,565
bytes / 124 files left the upper layer. The lower layer is untouched — the image is whatever it
is. What left is what the container **produced itself**, and its breakdown by record is: 14,400
log lines, 3,600 cache writes, 240 progress writes, 360 operator corrections, 30 config changes.

The gap between record counts and byte counts is itself informative. The cache was written 3,600
times but left 120 files and 3,720 bytes in the layer: 97% of the writes were overwritten by a
later write. The writable layer is not a log, it is a **snapshot** — it holds not what was
written, but what was written last. This is why what gets deleted is not the sum of what was
written either, but its final state.

The progress row should also be read separately: 31 bytes sat in the layer, but the write count
was 240. The information on how far the nightly run had gotten was overwritten every time and went
away with the container. A new container starts that night from zero; all 480 readings are
processed again.

Which class of loss is acceptable is the measurement network's operating rule (**RT12**) and, with
the thirty-night numbers, gives the table below:

| Data class | Records | Bytes | Is loss acceptable | Rationale |
|---|---|---|---|---|
| Log | 14,400 lines | 673,560 | Conditional | Acceptable if lines are shipped out; if the only copy is on the container, an incident review goes with it too |
| Cache | 3,600 writes | 3,720 | Yes | Reproducible; the cost is slower initial requests |
| Job state | 240 writes | 31 | No | An unfinished run starts over, 480 readings get reprocessed |
| User data | 360 corrections | 7,920 | No | Not reproducible; the operator's hand entry is the only source |
| Config copy | 30 changes | 98,334 | Yes | Comes back from the image or the config source |

The table reads backward: the two unacceptable classes hold only 1% of the layer, 7,951 bytes. The
remaining 99% is reproducible data. The problem is not size, it is **mixture**: 7,951 bytes of
unrecoverable data sit inside 775,614 bytes of recoverable data, and both get deleted at the same
instant.

## The Difference Removed from the Environment

This loss also has a counterpart, and the course's measure requires asking about it: what
difference does the ephemeral layer remove from the environment.

The difference it removes is **cross-run residue**. A container deleted and recreated every night
runs every night from the same 364,544-byte, four-file starting point; its writable layer never
exceeds 124,772 bytes, and the difference between two nights' starting states is exactly zero
bytes. A half-finished log, a bloated cache, the previous night's hand-corrected config do not
carry over to the next night. This is the measure by which a container counts as "stateless": zero
difference between runs.

The container that lives thirty nights does not remove this difference. It walks into the
thirtieth night with its layer already 2.15 times the lower layers, and that night's behavior
depends on the residue of the previous twenty-nine nights. Where a problem needs to be looked for
is no longer just the image, it is that layer's contents too — in other words, the difference
thought to have been removed from the environment has come back.

The cost is the two rows in the table. Whatever brings the cross-run difference to zero also
deletes the 240 progress writes and the 360 operator corrections at the same time. **Statelessness
is not free: the price of zeroing the cross-run residue is also zeroing the 7,951 bytes of
unrecoverable data inside that residue.** This is exactly the problem the next lesson solves.

## Where Isolation Is Punctured

There are two holes, and one of them is deliberate.

The deliberate one is lower-layer sharing. If two containers run from the same image, total disk
usage is not 2,296,218 bytes, it is 1,931,674 bytes; the lower layers' 364,544 bytes are held once
and visible to both containers at the same time. Isolation is deliberately punctured here, because
without the hole the same content would be stored twice. Because it is read-only, sharing has no
effect on behavior — but the source is single.

The second hole is unintentional, and its name comes from a misunderstanding. **Ephemerality is
not a filesystem property, it is a lifecycle promise.** The writable layer is not in memory; it is
a directory on the host machine's disk. When the container is *stopped*, that directory keeps
standing; 783,565 bytes stay in place, and they come back if the container is restarted. Data goes
away only when the container is **deleted**. This has two consequences: a stopped container still
holds on disk the data it is assumed to have lost; and a container that is never deleted grows the
layer it assumes is temporary without limit — to 2.15 times over thirty nights.

## Summary

- The writable layer is a separate unit added on top of the read-only lower layers; reads resolve
  from top to bottom, writes always land on top.
- **Copy-up cost is measured by file size:** a one-byte change in the tariff table caused 98,304
  bytes to be copied and filled 78.8% of the layer on the first night. The genuine new data
  produced that night is 26,468 bytes; what sits in the layer is 124,772 bytes. The cost is paid
  once: no copying on the second night.
- Over thirty nights the layer's shape turns over: the total climbs to 783,565 bytes, the log's
  share rises to 86% while the copy-up share drops to 12.5%. The writable layer's ratio to the
  lower layers climbs from 0.34 to 2.15.
- When the container is deleted, 14,400 log lines, 3,600 cache writes, 240 progress writes, 360
  operator corrections, and 30 config changes go away; the lower layer's 364,544 bytes remain.
- The acceptability of loss depends on the data class, and mixture is dangerous: the two
  unacceptable classes hold only 1% of the layer, 7,951 bytes, but they get deleted at the same
  instant as the remaining 775,614 bytes.
- **Where isolation is punctured:** the lower layer is shared between two containers, and 364,544
  bytes are held once; also, ephemerality is not a filesystem property — the writable layer is a
  directory on the host machine's disk, and it goes away not when the container is stopped, but
  when it is deleted.

## Next Step

Loss was unacceptable in two classes: job state and user data. Together they hold 7,951 bytes, and
both get deleted because they sit inside the writable layer. The fix is obvious — put those two
files outside the layer, somewhere not tied to the container's lifecycle. But outside the layer is
the host machine's filesystem, and every path opened onto it takes something away from isolation.
The next lesson builds two mount forms with real directories and counts this: how many bytes
remain when the container is deleted, how many files cannot be written when the container's
internal user identity does not match the directory's owner, and which of the two forms breaks
when the machine changes.