--- title: 'Provisioning and Configuration Management' source: 'https://academia.sh/en/courses/infrastructure-as-code/provisioning-and-configuration-management' course: 'Infrastructure as Code' language: en updated: '2026-09-13T05:08:24+00:00' license: 'CC BY-SA 4.0' objectives: - "Splits an environment's items into two layers and measures the boundary with three numbers: overlap, gap, items applied twice" - 'Shows which layer wins on a conflicting item, and counts that the winner is not a decision but a run order' - 'Measures the drift window after reprovisioning, and the changes three separate boundary lines push onto the heavy path' --- # Provisioning and Configuration Management An environment's thirty-four items are split into two layers — the provisioning layer that creates the resource and the configuration layer that sets up the inside of the resource — and where the boundary is drawn gets counted. Overlapping items, the gap that falls into neither layer, which layer wins on a conflicting value, the drift window after reprovisioning, and the cost of three separate boundary lines are measured. The previous lesson mixed two jobs together. Which packages the image carries and what the settings on the machine are is one job; how many machines there are, which network they attach to, and which disk they carry is another. The hotfix getting lost happened exactly on the boundary between these two jobs. **Provisioning** creates the resource: it opens the machine, sets up the network, attaches the disk, creates the queue. **Configuration management** sets up the inside of the resource: it installs packages, writes files, starts the service. Both have a record, and the two records are written in different formats. This lesson splits an environment's items into two layers and counts **where the boundary is drawn**: how many items are defined in both layers, how many in neither, which layer wins on a conflicting item, and how many items get applied twice when the boundary shifts. The environment is again the staging environment of the **regional measurement network**, and it is **fictional**. The record files are genuinely written to disk; the split into layers, the application order, and the drift window are modeled with `node`. **IF19.** The staging environment is represented by thirty-four items, and each item's **natural layer** is written in the inventory: `resource` or `content`. **IF20.** The two layers' records are two separate formats — the provisioning record in blocks of the configuration language, the config record in the configuration agent's line format. **IF21.** If an item is defined in only one layer, that layer applies it; if it is defined in both, **whichever runs last** wins. **IF22.** The usual order is provisioning then config. When a reprovisioning is triggered, the order reverses: config has already run, provisioning runs after. **IF23.** The configuration agent runs every thirty minutes; there are fourteen machines in the environment. **IF24.** A quarter sees twenty-two changes — seven on resource items, fifteen on content items; six of the content changes are on items baked into the image. ## The Inventory and the Two Records' Formats ```bash #!/usr/bin/env bash # An environment's item inventory and the two layers' records: provisioning (creates the resource), config (inside the resource). set -e cat > inventory.txt <<'SON' network.measurement-network.block resource network.measurement-network.subnet_count resource network.measurement-network.egress_path resource machine.collector.count resource machine.collector.size resource machine.collector.disk_gb resource machine.collector.image resource machine.verifier.count resource machine.verifier.size resource queue.reading.partitions resource queue.reading.retention_hours resource data_store.measurement.size_gb resource data_store.measurement.backup_days resource object_store.archive.class resource object_store.archive.lifecycle_days resource security.collector.ingress resource security.collector.egress resource dns_record.collector.internal resource machine.collector.package.resolver content machine.collector.package.rule_package content machine.collector.config.log_level content machine.collector.config.concurrent_workers content machine.collector.config.batch_hour content machine.collector.dir.data content machine.collector.dir.log content machine.collector.service.autostart content machine.collector.timezone content machine.collector.log_rotation content machine.verifier.package.resolver content machine.verifier.config.log_level content machine.verifier.service.autostart content data_store.measurement.user.report content data_store.measurement.parameter.connection_count content object_store.archive.access_policy content SON cat > provisioning.infra <<'SON' resource network "measurement-network" { block = "10.40.0.0/16" subnet_count = 3 } resource machine "collector" { count = 3 size = "medium" disk_gb = 40 image = "base-7" } resource machine "collector" { config.log_level = "summary" timezone = "UTC" } resource machine "collector" { service.autostart = "open" package.resolver = "3.4.0" } resource machine "verifier" { count = 2 size = "small" } resource queue "reading" { partitions = 6 retention_hours = 72 } resource data_store "measurement" { size_gb = 200 backup_days = 14 } resource object_store "archive" { class = "sparse" lifecycle_days = 365 access_policy = "in-account" } resource security "collector" { ingress = "10.40.0.0/16" } SON cat > config.record <<'SON' machine/collector package.resolver = 3.5.0 machine/collector package.rule_package = 9.1.0 machine/collector config.log_level = summary machine/collector config.concurrent_workers = 8 machine/collector config.batch_hour = 02:30 machine/collector dir.data = /data/measurement machine/collector dir.log = /log/collector machine/collector service.autostart = open machine/collector timezone = UTC machine/collector disk_gb = 40 machine/verifier package.resolver = 3.5.0 machine/verifier config.log_level = detail machine/verifier service.autostart = open data_store/measurement user.report = read-only object_store/archive access_policy = in-account-plus-read SON ``` The inventory counts the items that **genuinely exist** in the environment; it is independent of what the records say. The two records fall on top of it. The provisioning record is written in blocks, and a single block can hold both attributes belonging to the resource's shell (`count`, `size`, `disk_gb`) and ones belonging to the inside of the resource (`package.resolver`, `timezone`). The config record is in line format and writes only to the inside of the resource. The boundary sits not between the two, but **at their intersection**. ## Where the Boundary Falls ```js // layer.mjs — matches the inventory against the two layers' records; counts overlap and gap. Model. import { readFileSync } from "node:fs"; const BLOCK = /^resource\s+(\w+)\s+"([\w-]+)"\s*\{(.*)\}$/; export const provisioning = new Map(); for (const s of readFileSync("provisioning.infra", "utf8").split("\n")) { const m = BLOCK.exec(s.trim()); if (m) for (const x of m[3].matchAll(/([\w.]+)\s*=\s*("[^"]*"|[\w./:-]+)/g)) provisioning.set(`${m[1]}.${m[2]}.${x[1]}`, x[2].replaceAll('"', "")); } export const config = new Map(); // the configuration agent's record: a different format for (const s of readFileSync("config.record", "utf8").split("\n")) { const m = /^(\w+)\/([\w-]+)\s+([\w.]+)\s*=\s*(\S+)$/.exec(s.trim()); if (m) config.set(`${m[1]}.${m[2]}.${m[3]}`, m[4]); } export const inventory = readFileSync("inventory.txt", "utf8").trim().split("\n") .map((s) => s.trim().split(/\s+/)).map(([k, natural]) => ({ k, natural })); export const split = (item, S, A) => { // every item falls into exactly one box const g = { provisioningOnly: [], configOnly: [], overlap: [], gap: [] }; for (const { k } of item) { const s = S.has(k), a = A.has(k); g[s && a ? "overlap" : s ? "provisioningOnly" : a ? "configOnly" : "gap"].push(k); } return g; }; if (process.argv[1].endsWith("layer.mjs")) { // writes when run directly, silent when imported const g = split(inventory, provisioning, config); console.log(`inventory ${inventory.length} items provisioning record ${provisioning.size}` + ` config record ${config.size}`); console.log(` provisioning only ${g.provisioningOnly.length} config only ${g.configOnly.length}` + ` overlap ${g.overlap.length} gap ${g.gap.length}`); console.log(`${g.overlap.length} overlapping items (defined in both layers):`); for (const k of g.overlap) console.log(` ${k.padEnd(40)} provisioning "${provisioning.get(k)}" config "${config.get(k)}"` + ` ${provisioning.get(k) === config.get(k) ? "same" : "CONFLICT"}`); console.log(`${g.gap.length} items in the gap (in neither layer):`); for (const k of g.gap) console.log(` ${k.padEnd(51)} natural layer: ${inventory.find((e) => e.k === k).natural}`); } ``` ``` inventory 34 items provisioning record 20 config record 15 provisioning only 14 config only 9 overlap 6 gap 5 6 overlapping items (defined in both layers): machine.collector.disk_gb provisioning "40" config "40" same machine.collector.package.resolver provisioning "3.4.0" config "3.5.0" CONFLICT machine.collector.config.log_level provisioning "summary" config "summary" same machine.collector.service.autostart provisioning "open" config "open" same machine.collector.timezone provisioning "UTC" config "UTC" same object_store.archive.access_policy provisioning "in-account" config "in-account-plus-read" CONFLICT 5 items in the gap (in neither layer): network.measurement-network.egress_path natural layer: resource security.collector.egress natural layer: resource dns_record.collector.internal natural layer: resource machine.collector.log_rotation natural layer: content data_store.measurement.parameter.connection_count natural layer: content ``` Thirty-four items fell into four boxes: fourteen only in provisioning, nine only in config, six in both, five in neither. The two records define thirty-five items in total, but the number of items they cover is twenty-nine — the difference of six is the **overlap**, and a definition written a second time gains no new item. Four of the six overlapping items carry the same value. These four look harmless, and the only condition under which they are harmless is that when one changes, the other is changed too; because the two are written in separate files, in separate formats, and often by separate people, this condition is not met automatically. The remaining two items are already **conflicting**: the resolver version is 3.4.0 in one record, 3.5.0 in the other; the archive's access policy is narrow in one record, wide in the other. Where the overlap comes from is also readable. Five of the six items sit on the machine resource, and all five have an image counterpart: because the provisioning layer opens the machine from an image, it **indirectly** defines the package version, timezone, and service state that image carries. The config layer defines the same values directly. The two records reach the same item by two different routes, and neither knows the other writes it; the overlap is not carelessness, it is the image carrying a definition that is invisible in the record. The five items in the gap are the same measure as the last line of lesson 01, and here they appear together with their natural layers: three are naturally resource, two naturally content. These stayed unwritten not because no one wrote a record, but because they **fell between** the two records — each layer's owner assumed the other one wrote it. The egress path, the security rule's egress direction, the DNS record, log rotation, and the connection count: all five are items no one claimed. ## Which Layer Wins ```js // boundary.mjs — application order, the post-reprovisioning drift window, and three boundary lines. import { provisioning, config, inventory, split } from "./layer.mjs"; const g = split(inventory, provisioning, config); const conflicting = g.overlap.filter((k) => provisioning.get(k) !== config.get(k)); console.log(`which layer wins on an overlapping item (last writer wins)`); for (const [name, order] of [["provisioning then config (usual)", ["provisioning", "config"]], ["config then reprovisioning", ["config", "provisioning"]]]) { const final = (k) => (order[1] === "config" ? config : provisioning).get(k); console.log(` ${name.padEnd(33)} conflicting ${conflicting.length} items -> ` + conflicting.map((k) => `${k.split(".").pop()}="${final(k)}"`).join(" ")); } // Reprovisioning: the machine is reborn from the image, and the item the config layer wrote reverts to it. const MACHINES = 14, AGENT_MIN = 30; const machineItems = [...config.keys()].filter((k) => k.startsWith("machine.")); const reverted = machineItems.filter((k) => (provisioning.get(k) ?? "default") !== config.get(k)); console.log(`\ndrift window after reprovisioning (agent runs every ${AGENT_MIN} minutes)`); console.log(` items the config layer writes on the machine ${machineItems.length}` + ` at the wrong value after reprovisioning ${reverted.length}`); console.log(` ${MACHINES} machines -> ${reverted.length * MACHINES} items` + ` average ${AGENT_MIN / 2} min, worst ${AGENT_MIN} min` + ` -> ${reverted.length * MACHINES * AGENT_MIN / 2} item-minutes of drift`); // Three boundary lines. Items baked into the image: package, service, and timezone. const bakedIntoImage = (k) => /package\.|service\.|timezone/.test(k); const boundary = { "C1 boundary at the resource shell": (k) => [provisioning.has(k), config.has(k)], "C2 boundary shifted to the image": (k) => [provisioning.has(k) || (config.has(k) && bakedIntoImage(k)), config.has(k) && !bakedIntoImage(k)], "C3 single layer (everything in provisioning)": (k) => [provisioning.has(k) || config.has(k), false], }; const HEAVY = { "C1 boundary at the resource shell": 0, "C2 boundary shifted to the image": 6, "C3 single layer (everything in provisioning)": 15 }; console.log(`\nwhat happens depending on where the boundary is drawn (22 changes in a quarter: 7 resource, 15 content)`); for (const [name, f] of Object.entries(boundary)) { const S = new Map(), A = new Map(); for (const { k } of inventory) { const [s, a] = f(k); if (s) S.set(k, 1); if (a) A.set(k, 1); } const b = split(inventory, S, A); console.log(` ${name.padEnd(46)} provisioning ${S.size} config ${A.size}` + ` overlap ${b.overlap.length} gap ${b.gap.length}` + ` applied twice ${b.overlap.length} changes falling to the heavy path ${HEAVY[name]}`); } ``` ``` which layer wins on an overlapping item (last writer wins) provisioning then config (usual) conflicting 2 items -> resolver="3.5.0" access_policy="in-account-plus-read" config then reprovisioning conflicting 2 items -> resolver="3.4.0" access_policy="in-account" drift window after reprovisioning (agent runs every 30 minutes) items the config layer writes on the machine 13 at the wrong value after reprovisioning 9 14 machines -> 126 items average 15 min, worst 30 min -> 1890 item-minutes of drift what happens depending on where the boundary is drawn (22 changes in a quarter: 7 resource, 15 content) C1 boundary at the resource shell provisioning 20 config 15 overlap 6 gap 5 applied twice 6 changes falling to the heavy path 0 C2 boundary shifted to the image provisioning 23 config 9 overlap 3 gap 5 applied twice 3 changes falling to the heavy path 6 C3 single layer (everything in provisioning) provisioning 29 config 0 overlap 0 gap 5 applied twice 0 changes falling to the heavy path 15 ``` The final value of the two conflicting items depends on the **run order**. In the usual order, the config layer runs last, so it wins; the resolver becomes 3.5.0, the access policy becomes wide. When a reprovisioning is triggered, the provisioning layer runs last, and the same two items revert to the opposite value. The winning layer is not a decision, it is a **timing outcome** — and what determines that timing is often an event nobody is watching: a machine dying, a scaling change, an image update. The second table gives its duration. After reprovisioning, nine of the thirteen items the config layer writes on the machine revert to the value the image carries; because the agent runs every thirty minutes, they stay at the wrong value for fifteen minutes on average. Across fourteen machines that is 126 items and 1,890 item-minutes. This duration is not an outage, it is a **drift window**: the environment is running, taking measurements, but nine items are not at the value the record says. ## When the Boundary Shifts The last table compares the boundary drawn in three separate places. `C1` keeps the boundary at the resource's shell: each layer writes its own record, overlap is six, items applied twice are six, but no change falls onto the heavy path — all fifteen content changes finish with an agent run. `C2` shifts the boundary to the image: package, service, and timezone items get baked into the image, the config layer drops from fifteen to nine, overlap falls from six to three — but six changes now require baking an image and rebuilding the machines. `C3` removes the boundary entirely: twenty-nine items in a single layer, overlap zero, items applied twice zero — and all fifteen content changes fall onto the heavy path. The provisioning record's item count also climbs across the three drawings, from twenty to twenty-three to twenty-nine. Dropping to a single layer does not simplify the record; it piles the sum of the two records into one file and confronts everyone who reads that file with items that do not concern them. The one number that does not change across all three drawings is the **gap**: five. Moving the boundary reduces or increases overlap, but it never brings into scope an item no layer claims. The gap has nothing to do with where the boundary is; it has to do with **the part of the inventory that falls outside the record**, and the only way to close it is to write an owner for the item. The overlap decreasing is not free either. Between `C1` and `C3`, as overlap falls from six to zero, the changes falling onto the heavy path climb from zero to fifteen. The two numbers move in opposite directions, and the decision is made between them: in a quarter, is an inconsistency six overlaps might miss more expensive, or is the duration of fifteen rebuilds more expensive. The question has no single right answer, but both answers are countable. ## Summary - When 34 items were split across two records, 14 came out only in provisioning, 9 only in config, 6 in both, 5 in neither; 35 definitions cover only 29 items. - 4 of the 6 overlapping items carry the same value, and their being harmless depends on the two records being updated together; 2 are already conflicting. - On a conflicting item, the winning layer is whichever ran last: config in the usual order, provisioning on reprovisioning; the winner is not a decision, it is a timing outcome. - After reprovisioning, 9 items per machine revert to the image's value and stay wrong for an average of 15 minutes until the agent runs; across 14 machines that is 1,890 item-minutes of drift. - When the boundary was drawn in three separate places, overlap was 6, 3, and 0; changes falling onto the heavy path climbed to 0, 6, and 15; the gap stayed at 5 across all three drawings. ## Next Step This lesson's last two numbers say the same thing from two places. After reprovisioning, nine items were not at the value the record said, and stayed that way for fifteen minutes; five items are always that way, because they fall inside no record's scope. Both share the same name: the record says one thing, reality says another. The next lesson closes this topic and measures that difference directly. A drift detector is written — the record is read, the actual state is read, the difference is worked out item by item — and four things are counted: the drift accumulated over a period, the **detection delay** by scan frequency, the source of the drift, and the cost of fixing it. The last is at least as important as the other three: undoing a drift sometimes breaks something that works.