--- title: 'Machine Images' source: 'https://academia.sh/en/courses/compute-and-networking-in-the-cloud/machine-images' course: 'Compute and Networking in the Cloud' language: en updated: '2026-09-13T05:07:37+00:00' license: 'CC BY-SA 4.0' objectives: - 'Separately counts the two sides of embedding an item in the image: net startup gain and monthly bake load' - 'Scans the embedding level from zero to eight to work out the trade-off rate between startup duration and stale item-days' - 'Counts the external dependency type and call count of the work remaining at startup, and shows whether startup stops when a dependency drops' - 'Separates the decisions on the image into kept and handed off, and names the three items that cannot be undone' --- # Machine Images Eight items done at startup are embedded into the image one at a time, and the seconds each item gains are counted alongside the staleness it costs; the embedding level is scanned from zero to eight, working out startup duration, external call count, dependency type, monthly bake load, and stale item-days; which dependency the work remaining at startup is bound to, and whether startup stops when that dependency drops, are measured. The previous lesson passed over one item in a single line: re-creation, 35 minutes. The real work inside that duration is an empty disk turning into a running machine, and that transformation starts from a **machine image**. An image is the ready-made file system copied onto the instance's root disk; whatever is inside it is not done at startup, and whatever is not inside it is done at startup. M22/K05 measured how this image is produced — the bake pipeline, its versioning, the rollout window; that measurement is not repeated here. This lesson's measure is a single axis: **startup duration**. The same question is asked for every item — if this item is embedded in the image, how many seconds does startup shorten by, how many more days does the image stay stale in exchange, and how many times a month does it need to be re-baked. The answer varies from item to item, and the size of that variation takes the choice between "embed everything in the image" and "embed nothing" out of the realm of taste. **CC7.** Ten items are done at the startup of the regional measurement network — the fictional software that collects readings from water meters; eight can be embedded in the image, two cannot. Durations, sizes, and change intervals are fictional. **CC8.** The base image is 900 MB, the pull speed is 60 MB/s, and kernel startup is 11 seconds on every path. **CC9.** Baking is done in a weekly batch; a change waits an average of 3.5 days to enter the image. **CC10.** A single bake, together with its test, takes 18 minutes. **CC11.** The embedding order is greedy: the item that gains the most seconds per bake is embedded first. **CC12.** An item being "critical" means the service does not start while that item is missing; the distinction is fictional. ## The Two Sides of an Item Every piece of work done at startup is an independent item, and embedding each item in the image is a separate decision. Once an item is embedded, its seconds at startup disappear, but the image's size grows — and pulling a larger image adds seconds to startup. Net gain is the difference between these two. The cost, though, is in a different unit: the more often an item changes, the more often embedding it in the image demands a re-bake. ```js // measurement-network/image-items.mjs — the startup gain and staleness cost of embedding an item in the image (model) export const SPEED = 60, BASE_MB = 900, KERNEL = 11; // MB/s, base image, kernel startup (s) export const BAKE_WAIT = 3.5, BAKE_MIN = 18; // average wait in a weekly batch bake // [name, seconds at startup, MB it adds to the image, days between changes, dependency, calls, critical?] export const ITEM = [ ["kernel packages", 46, 320, 45, "package repository", 6, true], ["runtime", 38, 210, 30, "package repository", 3, true], ["measurement agent", 14, 60, 14, "package repository", 2, false], ["log shipping", 8, 20, 21, "package repository", 1, false], ["certificate bundle", 9, 12, 7, "configuration server", 1, true], ["application binary", 22, 90, 2, "build repository", 1, true], ["network settings", 6, 2, 3, "configuration server", 1, true], ["environment variables", 3, 1, 1, "configuration server", 1, false], ]; // Items that cannot be embedded: always done at startup. export const NOT_EMBEDDABLE = [["node secret", 7, "secret store", 1, true], ["clock sync", 4, "time server", 1, false]]; export const measure = ([name, sec, mb, days, , calls]) => ({ name, gain: sec - mb / SPEED, bakes: 30 / days, calls, stale: (30 / days) * BAKE_WAIT, }); // Greedy order: the item that gains the most seconds per bake is embedded first. export const ORDER = [...ITEM].sort((a, b) => measure(b).gain / measure(b).bakes - measure(a).gain / measure(a).bakes); if (import.meta.url === `file://${process.argv[1]}`) { console.log("item".padEnd(23) + "at startup".padEnd(12) + "in image".padEnd(10) + "pull".padEnd(9) + "net gain".padEnd(12) + "bakes/month".padEnd(14) + "stale item-days".padEnd(17) + "ratio"); for (const k of ORDER) { const m = measure(k); console.log(m.name.padEnd(23) + `${k[1]} s`.padEnd(12) + `${k[2]} MB`.padEnd(10) + `${(k[2] / SPEED).toFixed(1)} s`.padEnd(9) + `${m.gain.toFixed(1)} s`.padEnd(12) + m.bakes.toFixed(2).padEnd(14) + m.stale.toFixed(1).padEnd(17) + (m.gain / m.bakes).toFixed(1)); } const g = NOT_EMBEDDABLE.reduce((t, x) => t + x[1], 0); console.log(`\nitems that cannot be embedded: ${NOT_EMBEDDABLE.map((x) => x[0]).join(", ")} — ` + `${g} s and ${NOT_EMBEDDABLE.length} calls stay at every startup`); } ``` ``` item at startup in image pull net gain bakes/month stale item-days ratio kernel packages 46 s 320 MB 5.3 s 40.7 s 0.67 2.3 61.0 runtime 38 s 210 MB 3.5 s 34.5 s 1.00 3.5 34.5 measurement agent 14 s 60 MB 1.0 s 13.0 s 2.14 7.5 6.1 log shipping 8 s 20 MB 0.3 s 7.7 s 1.43 5.0 5.4 certificate bundle 9 s 12 MB 0.2 s 8.8 s 4.29 15.0 2.1 application binary 22 s 90 MB 1.5 s 20.5 s 15.00 52.5 1.4 network settings 6 s 2 MB 0.0 s 6.0 s 10.00 35.0 0.6 environment variables 3 s 1 MB 0.0 s 3.0 s 30.00 105.0 0.1 items that cannot be embedded: node secret, clock sync — 11 s and 2 calls stay at every startup ``` The last column splits items into two groups. `kernel packages` gains 40.7 seconds and demands 0.67 bakes a month; its ratio is 61. `environment variables` gains 3 seconds and demands 30 bakes a month; its ratio is 0.1. There is a **six-hundredfold** difference between these two items, and the difference comes not from size but from **change frequency**. The item best suited for embedding in the image is not the one that takes the longest at startup; it is the one that takes long and changes rarely. The `application binary` row is where this distinction shows most clearly. It takes 22 seconds at startup — the third-longest item on the list — but changes every two days. Embedded, it triggers 15 bakes a month and accumulates 52.5 stale item-days. The `runtime` sells the same 20.5 seconds for only 3.5 stale item-days. Intuition says "embed the biggest item"; the ratio column says "embed the most stable item". The two items that cannot be embedded draw the measure's floor. The node secret cannot be written into the image, because anyone who can read the image can also read the secret; clock synchronization cannot come from the image, because the thing being measured is the moment of startup. These two items are 11 seconds and 2 calls, and no bake decision can remove them. ## Embedding Level Scan Individual items have been compared; the next question is how many of them to embed. The scan below takes the list sorted by the ratio column from the top and recomputes startup at every level from zero to eight. ```js // measurement-network/embedding-scan.mjs — what happens to startup as more items are embedded in the image (model) import { SPEED, BASE_MB, KERNEL, BAKE_MIN, ITEM, NOT_EMBEDDABLE, ORDER, measure } from "./image-items.mjs"; export const level = (k) => { const embedded = ORDER.slice(0, k), open = ORDER.slice(k); const mb = BASE_MB + embedded.reduce((t, x) => t + x[2], 0); const duration = KERNEL + mb / SPEED + open.reduce((t, x) => t + x[1], 0) + NOT_EMBEDDABLE.reduce((t, x) => t + x[1], 0); const calls = open.reduce((t, x) => t + x[5], 0) + NOT_EMBEDDABLE.reduce((t, x) => t + x[3], 0); const types = new Set([...open.map((x) => x[4]), ...NOT_EMBEDDABLE.map((x) => x[2])]).size; return { k, mb, duration, calls, types, bakes: embedded.reduce((t, x) => t + measure(x).bakes, 0), stale: embedded.reduce((t, x) => t + measure(x).stale, 0), last: k ? ORDER[k - 1][0] : "—" }; }; if (import.meta.url === `file://${process.argv[1]}`) { console.log("embedded".padEnd(10) + "last embedded".padEnd(23) + "image".padEnd(10) + "startup".padEnd(10) + "external calls".padEnd(16) + "dependency".padEnd(12) + "bakes/month".padEnd(14) + "baker minutes".padEnd(15) + "stale item-days"); for (let k = 0; k <= ITEM.length; k++) { const d = level(k); console.log(`${k}/${ITEM.length}`.padEnd(10) + d.last.padEnd(23) + `${d.mb} MB`.padEnd(10) + `${d.duration.toFixed(1)} s`.padEnd(10) + String(d.calls).padEnd(16) + `${d.types} types`.padEnd(12) + d.bakes.toFixed(2).padEnd(14) + Math.round(d.bakes * BAKE_MIN).toString().padEnd(15) + d.stale.toFixed(1)); } const s = level(0), t = level(4), h = level(ITEM.length); console.log(`\nfirst four items: startup shortens by ${(s.duration - t.duration).toFixed(1)} s, ` + `costing ${t.stale.toFixed(1)} stale item-days`); console.log(`last four items: startup shortens by ${(t.duration - h.duration).toFixed(1)} s, ` + `costing ${(h.stale - t.stale).toFixed(1)} stale item-days`); console.log(`staleness per second: first four ${(t.stale / (s.duration - t.duration)).toFixed(2)}, ` + `last four ${((h.stale - t.stale) / (t.duration - h.duration)).toFixed(2)} item-days/s`); } ``` ``` embedded last embedded image startup external calls dependency bakes/month baker minutes stale item-days 0/8 — 900 MB 183.0 s 18 5 types 0.00 0 0.0 1/8 kernel packages 1220 MB 142.3 s 12 5 types 0.67 12 2.3 2/8 runtime 1430 MB 107.8 s 9 5 types 1.67 30 5.8 3/8 measurement agent 1490 MB 94.8 s 7 5 types 3.81 69 13.3 4/8 log shipping 1510 MB 87.2 s 6 4 types 5.24 94 18.3 5/8 certificate bundle 1522 MB 78.4 s 5 4 types 9.52 171 33.3 6/8 application binary 1612 MB 57.9 s 4 3 types 24.52 441 85.8 7/8 network settings 1614 MB 51.9 s 3 3 types 34.52 621 120.8 8/8 environment variables 1615 MB 48.9 s 2 2 types 64.52 1161 225.8 first four items: startup shortens by 95.8 s, costing 18.3 stale item-days last four items: startup shortens by 38.3 s, costing 207.5 stale item-days staleness per second: first four 0.19, last four 5.42 item-days/s ``` The scan is not one curve but **two different curves**. Startup duration drops from 183 seconds to 48.9 seconds with diminishing returns: the first four items gain 95.8 seconds, the last four gain 38.3 seconds. Staleness runs the opposite way and accelerates: the first four items cost 18.3 stale item-days, the last four cost 207.5 stale item-days. The last line turns the two curves into a single number — the price of a gained second is 0.19 item-days in the first four, 5.42 item-days in the last four. **Twenty-eight times.** The baker minutes column writes the same thing from the labor side. With four items embedded, the image builder runs for 94 minutes a month; with eight items embedded, 1,161 minutes — roughly nineteen hours. This is a cost nobody writes into the plan but that gets paid every month, and it is why the bake pipeline itself is an infrastructure piece that needs monitoring. The dependency column, though, does not fall smoothly — it falls **step by step**: from 5 types to 4 at the fourth item, to 3 at the sixth, to 2 at the eighth. The reason is that all four of the first four items use the same dependency — the package repository. A dependency type is removed only once the **last** item bound to it is also embedded; embedding the first three items drops calls from 18 to 7 but does not change the type count at all. ## The Surface of What Remains at Startup Call count is a surface measure: every call is a separate thing that must be standing at the moment of startup. But not every call carries the same weight. When some items are missing, the service does not start at all; when others are missing, the machine boots and runs incomplete. ```js // measurement-network/startup-surface.mjs — the external surface of the work left at startup and the decision ledger (model) import { ITEM, NOT_EMBEDDABLE, ORDER } from "./image-items.mjs"; import { level } from "./embedding-scan.mjs"; const LEVELS = [0, 4, 8]; const open = (k) => [...ORDER.slice(k).map(([name, sec, , , dep, calls, crit]) => ({ name, sec, dep, calls, crit })), ...NOT_EMBEDDABLE.map(([name, sec, dep, calls, crit]) => ({ name, sec, dep, calls, crit }))]; const impact = (k, dep) => { const l = open(k).filter((x) => x.dep === dep); if (!l.length) return "no effect"; if (l.some((x) => x.crit)) return "startup fails"; return `-${l.reduce((t, x) => t + x.sec, 0)} s, ${l.length} item${l.length === 1 ? "" : "s"} missing`; }; // [item, holder, reversible?] const DECISIONS = [["which item goes into the image", "yours", "yes — new image"], ["base operating system version", "yours", "yes — new image"], ["bake frequency", "yours", "yes"], ["published image content", "yours", "no — does not change under the same identity"], ["secret embedded in the image", "yours", "no — deleting is not enough once copied"], ["startup script given to the instance", "yours", "no — does not change after creation"], ["image repository durability", "provider", "—"], ["image copy speed", "provider", "—"], ["virtualization beneath the base image", "provider", "—"]]; if (import.meta.url === `file://${process.argv[1]}`) { const DEPS = [...new Set([...ITEM.map((x) => x[4]), ...NOT_EMBEDDABLE.map((x) => x[2])])]; console.log("dropped dependency".padEnd(24) + LEVELS.map((k) => `${k}/8 embedded`.padEnd(22)).join("").trimEnd()); for (const dep of DEPS) console.log((dep.padEnd(24) + LEVELS.map((k) => impact(k, dep).padEnd(22)).join("")).trimEnd()); console.log(""); console.log("level".padEnd(9) + "startup".padEnd(10) + "open items".padEnd(13) + "critical open items".padEnd(22) + "dependency blocking startup"); for (const k of LEVELS) { const a = open(k); console.log(`${k}/8`.padEnd(9) + `${level(k).duration.toFixed(1)} s`.padEnd(10) + String(a.length).padEnd(13) + String(a.filter((x) => x.crit).length).padEnd(22) + String(new Set(a.filter((x) => x.crit).map((x) => x.dep)).size)); } const count = (v) => DECISIONS.filter((x) => x[1] === v).length; console.log(`\ndecision ledger: ${count("yours")} items kept, ${count("provider")} items with the provider`); for (const [name, , rev] of DECISIONS.filter((x) => x[2].startsWith("no"))) console.log(` cannot be undone — ${name.padEnd(38)} ${rev.slice(5)}`); } ``` ``` dropped dependency 0/8 embedded 4/8 embedded 8/8 embedded package repository startup fails no effect no effect configuration server startup fails startup fails no effect build repository startup fails startup fails no effect secret store startup fails startup fails startup fails time server -4 s, 1 item missing -4 s, 1 item missing -4 s, 1 item missing level startup open items critical open items dependency blocking startup 0/8 183.0 s 10 6 4 4/8 87.2 s 6 4 3 8/8 48.9 s 2 1 1 decision ledger: 6 items kept, 3 items with the provider cannot be undone — published image content does not change under the same identity cannot be undone — secret embedded in the image deleting is not enough once copied cannot be undone — startup script given to the instance does not change after creation ``` The number of dependencies that block startup drops from 4 to 1, and that is a gain independent of seconds. With no item embedded, four separate systems must be standing at the same time; with eight embedded, a single system — the secret store — remains. The secret store row is identical across all three columns and cannot be changed by an embedding decision. The clock server row is the one "soft" row: if it drops, startup still completes, and the machine begins taking work 4 seconds early and unsynchronized. This row is where a silent trap sits. The measurement network's readings are timestamped; the records an unsynchronized machine produces are harder to notice than a failed startup, because the machine looks healthy. A non-critical item starting up missing is not a bug, it is a decision. The decision ledger **leaves six of nine items with you**: what goes into the image, what the base version is, how often it gets baked, and three items that cannot be undone. The three that stay with the provider — image repository durability, copy speed, the virtualization beneath it — determine the pull seconds but cannot be chosen. The three items that cannot be undone are irreversible each in a different way. Published image content does not change under the same identity; fixing it means a new image, and that image does not delete the old one. A secret embedded in the image cannot be undone by deletion once the image has been copied; that is why the secret store call is a deliberate choice, not a gap. The startup script given to an instance, however, is set when the instance is created and does not change for as long as that instance lives — fixing the script does not fix the running machine, it only fixes machines **created from this point on**. ## Summary - An item's suitability for embedding is determined by change frequency, not size: `kernel packages` gains 61 seconds per bake, `environment variables` gains 0.1 — a six-hundredfold difference. - `application binary` takes 22 seconds at startup but changes every two days; embedded, it costs 15 bakes and 52.5 stale item-days a month, while `runtime` sells the same seconds for 3.5 item-days. - In the embedding level scan, startup drops from 183 seconds to 48.9 seconds; the price of a gained second is 0.19 stale item-days in the first four items, 5.42 in the last four. - External calls drop from 18 to 2, but dependency type count falls step by step: a type is removed only once the last item bound to it is also embedded; baker load climbs from 94 minutes to 1,161 minutes. - Dependencies that block startup drop from 4 to 1, not to zero; if the clock server drops, the machine boots and takes work unsynchronized — a condition harder to notice than a failed startup. - Six of nine items are kept, three are with the provider; the three that cannot be undone are published image content, the secret embedded in the image, and the startup script given to the instance. ## Next Step This lesson counted how long startup takes and what it depends on, but it never asked how long the machine will **live**. The entire calculation was built on the assumption that the instance stays up for as long as it is wanted: both 183 seconds and 48.9 seconds are paid once, and then the machine keeps doing work. There is also an interruptible form of renting — capacity is given cheaply, and in exchange the provider can take it back with a short notice. The previous course measured this form's price difference; here price becomes an input, and the question shifts to the type of job. The next lesson runs four job types on interruptible capacity and asks: as the notice period shortens, how much work does each job lose, what does restarting cost, and which job does not tolerate interruption at all?