Container Lifecycle

The previous lessons measured what isolation is, what it separates into how many bytes, and how the output is named and carried. All of it looked at a single moment: the moment the output exists. A container, though, has separate moments when it starts, runs, stops, and disappears, and what is held between these moments is not the same each time.

This lesson builds the lifecycle as a state machine. There are five states — nonexistent, created, running, stopped, cleaned up — and four transitions move between these states. What is measured is not the states themselves but the resource that keeps being held after every transition. The fictional regional measurement network suits this measurement: the nightly batch job starts and ends every night, services restart after a defect, and no run disappears on its own.

This is also where it becomes clear why the question needs measurement. The previous four lessons counted the cost of isolation at the moment the output exists: how many bytes are carried, how many criteria are separated, what order of magnitude of startup delay is demanded. None of those numbers had been multiplied by time. A container, though, is not a single object but one produced again and again; the same output is created dozens of times over a period, and every creation leaves its own trace. The axis this lesson adds is time, and adding it makes a single question measurable: is the cost paid once, or paid again on every run?

CC26. The lifecycle has five states, and no transition exists outside the transition table. CC27. The held resource is four line items: writable layer, network name, volume, process record. CC28. The writable layer’s growth is a model value; the first run adds 340 MB, the second run adds 96 MB.

Four Transitions and the Held Resource

// measurement-network/lifecycle.mjs — the container lifecycle as a state machine (model)
// Every transition changes the held resource. The held resource is four line items.

const ITEM = ["layerMB", "networkName", "volume", "processRecord"];

// Transition table: [transition, before, after, item changes]
const TRANSITION = [
  ["create", "nonexistent", "created", { layerMB: 0, networkName: 1, volume: 1 }],
  ["run",    "created",     "running", { processRecord: 1, layerMB: 340 }],
  ["stop",   "running",     "stopped", { processRecord: -1 }],
  ["run",    "stopped",     "running", { processRecord: 1, layerMB: 96 }],
  ["stop",   "running",     "stopped", { processRecord: -1 }],
  ["cleanup","stopped",     "cleaned", { layerMB: "reset", networkName: -1 }],
];

let state = "nonexistent", stoppedState = null;
const held = Object.fromEntries(ITEM.map((k) => [k, 0]));
const print = (label) => console.log(label.padEnd(13) + state.padEnd(15) +
  ITEM.map((k) => String(held[k]).padStart(k.length)).join("   "));

console.log("transition".padEnd(13) + "state".padEnd(15) + ITEM.join("   "));
print("(start)");
for (const [name, before, after, delta] of TRANSITION) {
  if (state !== before) throw new Error(`${name}: expected ${before}, found ${state}`);
  for (const [k, d] of Object.entries(delta)) held[k] = d === "reset" ? 0 : held[k] + d;
  state = after;
  if (state === "stopped") stoppedState = { ...held };
  print(name);
}

console.log("\nwhat a stopped-but-uncleaned container holds: " +
  ITEM.map((k) => `${k} ${stoppedState[k]}`).join(", "));
console.log("the only line item that drops on stopping is the process record; the other three stay in place");
console.log("the line item that is still not reset after cleanup: volume " + held.volume +
  " (a volume's lifecycle is separate from the container's)");

transition   state          layerMB   networkName   volume   processRecord
(start)      nonexistent          0             0        0               0
create       created              0             1        1               0
run          running            340             1        1               1
stop         stopped            340             1        1               0
run          running            436             1        1               1
stop         stopped            436             1        1               0
cleanup      cleaned              0             0        1               0

what a stopped-but-uncleaned container holds: layerMB 436, networkName 1, volume 1, processRecord 0
the only line item that drops on stopping is the process record; the other three stay in place
the line item that is still not reset after cleanup: volume 1 (a volume's lifecycle is separate from the container's)

The table separates two things. Creation and running are not the same step: creation reserves the network name and the volume but starts no process; the writable layer is zero bytes. Running adds the process record and grows the layer — 340 MB on the first run, 96 MB more on the second. The growth is permanent, because the writable layer carries over from run to run; on the second stop, what is held is 436 MB.

The real distinction is in the stop row. The only line item that drops on stopping is the process record. The writable layer, network name, and volume stay in place. A container being stopped does not mean what it left behind is freed — stopping is a process operation, not a resource operation. The transition that frees resources is cleanup, and even that does not free everything: the volume stays at 1 even after cleanup, because a volume’s lifecycle is separate from the container’s. In the fictional network this means the volume the nightly job writes its intermediate files to keeps standing on disk even after the container that ran the job is deleted.

The created-but-never-run state is a separate line item too, and it is easy to miss: the layer is zero bytes, the process record is zero, but the network name and the volume are already reserved. A container in this state is close to free in disk terms, but not in namespace terms — it holds its name, and a second container cannot be created under that name. This is exactly where measuring the cost with a single line item is misleading; unless all four line items are counted separately, the sentence “an idle container is not consuming anything” can be built, and the sentence is wrong on three of the four.

The state machine’s transition check also says something: there is no transition outside the table, and the model enforces the check. A stopped container can be run again, but a cleaned-up container cannot be run. The only one-way transition in the lifecycle is cleanup; there is no going back after it.

Accumulation Over a Period

A single container holding 436 MB is a small number. What the number means depends on how many containers are waiting in this state.

CC29. A period is 30 days; the services’ run frequencies are model inputs. CC30. The disk is 512,000 MB, the network name pool is 4,096 names. CC31. The cleanup rule takes the form “keep the last N periods”; partial cleanup is not modeled.

// measurement-network/accumulation.mjs — accumulation of stopped-but-uncleaned containers (model)
// A period is 30 days. Every run leaves behind one container: writable layer, network name, volume.

const PERIOD_DAYS = 30, DISK_MB = 512_000, NAME_POOL = 4_096, OUTPUT_MB = 84.11;
const SERVICE = {                        // fictional regional measurement network: frequency and remaining layer
  nightlyJob: { everyDays: 1,  layerMB: 436 },
  billing:    { everyDays: 3,  layerMB: 52 },
  collector:  { everyDays: 4,  layerMB: 26 },
  verifier:   { everyDays: 6,  layerMB: 14 },
  workOrder:  { everyDays: 10, layerMB: 8 },
};

console.log(`one period = ${PERIOD_DAYS} days; held at period end if no cleanup runs`);
console.log("service".padEnd(14) + "runs".padEnd(8) + "per run".padEnd(14) + "period accumulation");
let periodMB = 0, periodRuns = 0;
for (const [name, s] of Object.entries(SERVICE)) {
  const runs = Math.floor(PERIOD_DAYS / s.everyDays);
  periodMB += runs * s.layerMB; periodRuns += runs;
  console.log(name.padEnd(14) + String(runs).padEnd(8) + `${s.layerMB} MB`.padEnd(14) +
    `${runs * s.layerMB} MB`);
}
console.log(`total: ${periodRuns} containers, ${periodMB} MB, ${periodRuns} network names, ` +
  `${periodRuns} volumes, 0 process records`);
console.log(`the period's accumulation is ${Math.round(periodMB / OUTPUT_MB)} times a single output ` +
  `(${OUTPUT_MB} MB)`);

console.log("\ncleanup rule scanned (last N periods kept)");
console.log("N".padEnd(6) + "layer".padEnd(13) + "network names".padEnd(15) + "volumes".padEnd(9) +
  "percent of disk");
for (const N of [0, 1, 2, 3, 6, 12]) {
  const mb = N * periodMB, count = N * periodRuns;
  console.log(String(N).padEnd(6) + `${mb} MB`.padEnd(13) + String(count).padEnd(15) +
    String(count).padEnd(9) + ((mb / DISK_MB) * 100).toFixed(1) + "%");
}

const diskPeriods = DISK_MB / periodMB, namePeriods = NAME_POOL / periodRuns;
console.log(`\nwith no cleanup at all, the ${DISK_MB} MB disk fills in ${diskPeriods.toFixed(1)} periods ` +
  `(${Math.round(diskPeriods * PERIOD_DAYS)} days)`);
console.log(`the ${NAME_POOL}-name network name pool runs out in ${namePeriods.toFixed(1)} periods ` +
  `(${Math.round(namePeriods * PERIOD_DAYS)} days)`);
console.log(`runs out first: ${diskPeriods < namePeriods ? "disk" : "name pool"}; ` +
  `the ratio between the two limits is ${(namePeriods / diskPeriods).toFixed(1)}`);
console.log(`the N=1 rule keeps ${((periodMB / DISK_MB) * 100).toFixed(1)}% of the disk permanently occupied; ` +
  `the N=0 rule makes every run unrecoverable`);

one period = 30 days; held at period end if no cleanup runs
service       runs    per run       period accumulation
nightlyJob    30      436 MB        13080 MB
billing       10      52 MB         520 MB
collector     7       26 MB         182 MB
verifier      5       14 MB         70 MB
workOrder     3       8 MB          24 MB
total: 55 containers, 13876 MB, 55 network names, 55 volumes, 0 process records
the period's accumulation is 165 times a single output (84.11 MB)

cleanup rule scanned (last N periods kept)
N     layer        network names  volumes  percent of disk
0     0 MB         0              0        0.0%
1     13876 MB     55             55       2.7%
2     27752 MB     110            110      5.4%
3     41628 MB     165            165      8.1%
6     83256 MB     330            330      16.3%
12    166512 MB    660            660      32.5%

with no cleanup at all, the 512000 MB disk fills in 36.9 periods (1107 days)
the 4096-name network name pool runs out in 74.5 periods (2234 days)
runs out first: disk; the ratio between the two limits is 2.0
the N=1 rule keeps 2.7% of the disk permanently occupied; the N=0 rule makes every run unrecoverable

In one period, 55 containers are left behind and hold 13,876 MB. The size of the number comes from a single source: the nightly job alone produces 13,080 MB of the total, that is, 94%. The product of frequency and the layer left per run explains this — thirty runs, 436 MB per run. The other four services total 796 MB. Reading this product tells you where to look when writing a cleanup rule; it is not the number of services but the product of frequency and layer growth that decides. The work order service runs three times in the period and leaves 24 MB; a rule applied to it does not measurably change the total — it closes 24 of the 13,876 MB. The same rule applied to the nightly job, though, closes 13,080 MB, that is, 94% of the accumulation. The difference between writing a single rule for all five services and writing a separate rule for the biggest producer is the ratio of these two numbers.

The period’s accumulation is 165 times a single output. The 84.11 MB output measured in the first lesson looks small next to a month of uncleaned runs. This is the line item this lesson adds to the isolation budget: the cost of isolation is not only the output’s size, it is the trace that output leaves on every run. The first lesson’s question was “how much does the output grow if the difference is pulled out of the environment,” and the answer was 84.11 MB; this lesson asks the same question over a period, and the answer is 13,876 MB. The two numbers are two different scales of the same decision, and an accounting that looks only at the first leaves the second as a surprise.

The scan gives the rule’s cost directly. At N=1 the disk is kept permanently 2.7% full, at N=3 it is 8.1%, at N=12 it is 32.5%. With no cleanup at all, the disk fills in 36.9 periods — 1,107 days. The network name pool, meanwhile, runs out in 74.5 periods; the ratio between the two limits is 2.0, meaning the disk runs out twice as fast as the name pool. This ordering decides which resource a cleanup rule should be written against: a cleanup rule is put in place to protect the disk, not to protect the names.

The other end of the rule must be counted too. N=0 holds no disk at all, but it leaves nothing behind from a run — the writable layer is the only evidence left after a defective run, and a cleaned-up container’s layer cannot be recovered. When the fictional nightly job stops midway, the layer is exactly where you look: partial invoice files, the number of meters processed, the last record written. The N=0 rule erases this evidence the moment the run ends; the N=12 rule permanently holds a third of the disk. The number to write in between is not a preference, it is a balance between two measured quantities: how many days of evidence are kept versus what percentage of disk is held. Because the scan produces the table, the discussion stops being “should cleanup happen” and becomes “how many periods of evidence are worth what percentage of disk.”

Where Isolation Is Pierced

The lifecycle has three holes, and all three come from the difference between being stopped and not existing.

The first is the writable layer. A stopped container’s layer stands on disk, and everything in it — the nightly job’s intermediate files, partial invoice output, logs — remains readable. The delete marker measured in the previous lesson shows up here a second time: a file deleted from a layer kept standing in the lower layer; now the layer itself keeps standing after the container stops too. Isolation works while something is running; it does not work on the bytes left behind after the run ends.

The second is the network name. A stopped container’s name keeps holding a place in the namespace; a new container cannot be created under the same name. This is a direct source of defects for the fictional nightly job — if one night’s job is not cleaned up, the next night’s job cannot be started under the same name.

The third is the volume, and it is the quietest one: even the cleanup transition does not release it. It is the only line item that sits outside the lifecycle, and because it is the only one of the four line items that no single transition resets, its accumulation has to be measured by its own rule.

The common reason behind all three must also be written down: nothing performs the cleanup transition on its own. Creation, running, and stopping happen within the flow of a job — the nightly job starts, finishes, stops. Cleanup is not part of that flow; it is a transition that must be requested separately. Three of the lifecycle’s four transitions are triggered by the work itself, the fourth by a rule. When the rule is not written, the cycle runs with three transitions instead of four, and the scan above counts exactly the cost of that three-transition state.

Summary

• The lifecycle is a five-state state machine; creation and running are separate steps — creation reserves the network name and the volume, running adds the process record and grows the layer.
• The only line item that drops on stopping is the process record. The writable layer, network name, and volume stay in place; in the model, a stopped container holds 436 MB, 1 network name, and 1 volume.
• Cleanup does not release everything either: the volume keeps standing even after cleanup, because its lifecycle is separate from the container’s. Cleanup is the only one-way transition.
• 55 containers and 13,876 MB accumulate in one period; the nightly job alone produces 94% of it. The accumulation is 165 times a single output.
• The cleanup rule was scanned: at N=1, 2.7% of the disk is permanently held, at N=3 it is 8.1%, at N=12 it is 32.5%. With no cleanup at all the disk fills in 36.9 periods, the name pool runs out in 74.5 periods; the rule is written to protect the disk, not to protect the names.
• Isolation works while something is running, not on the bytes left behind afterward: the layer stays readable, the name holds its place, the volume never leaves. Nothing performs the cleanup transition on its own; only three of the four transitions happen within the flow of the work.

Next Step

This topic measured what isolation is: which criterion is separated, how many megabytes and what order of magnitude of startup delay separation demands, which mechanism sets which rule, how the output is named and carried, and what it leaves behind after it runs. All these measurements share one common gap: where the thing being run came from was never asked. The measured 84.11 MB output, the shadowed layers, and the digest chain were all taken as given — yet someone produced that stack.

The questions start here. How is that output produced; which step gives birth to which layer? Why a stack instead of a single file — what would be lost if it were a single file, what is gained by it being a stack? Who decides the layers’ order and by what, and how much of the chain has to be recomputed when the order changes? The next topic takes up the production of the output, and its first question is: what is the measurable payoff of splitting an output into layers?