--- title: 'Cost Estimation' source: 'https://academia.sh/en/courses/infrastructure-as-code/cost-estimation' course: 'Infrastructure as Code' language: en updated: '2026-09-13T05:08:24+00:00' license: 'CC BY-SA 4.0' objectives: - 'Produces a monthly cost estimate per change from the plan diff' - 'Separates estimable items from non-estimable ones and measures the distribution of error' - 'Runs a cost threshold rule over a known set of changes and counts changes held for nothing against costly changes that pass' --- # Cost Estimation Counting an infrastructure change's effect on the budget before it is applied: producing a monthly estimate per change from the plan diff, separating fixed-unit-price items from usage-dependent items, where the error comes from and how it is distributed when compared against what actually happens, how overall accuracy hides the deviation in individual changes, and how many changes a threshold rule holds for nothing versus how many costly changes it lets through. The previous lesson counted a validation run's own cost and the leaked cost of the leftovers it left behind. Both were counted after the fact — once the money had already been spent. The same question can be asked before applying too, because the plan is already in hand: which resource will be added, which will be grown, which will be deleted is written right there. A plan diff is at the same time a **bill diff**. Capacity and cost awareness were measured at system scale earlier; the unit here is **the change itself**: if this contribution is merged, how much does the monthly cost grow? This lesson produces an estimate per change from the plan, compares it against what actually happens, and counts three things: which item can be estimated, which cannot, how the error is distributed. Then a threshold rule — "stop if the estimate exceeds this much" — is run over a known set of changes, and the rule's two failure modes are both counted. The through-line is the **regional measurement network**; the price list and the amounts are both fictional. ## Estimable and Non-Estimable Items A cost item is one of two kinds. A **fixed-unit-price** item's amount is read directly from the plan: three machines, 120 gigabytes of disk, 500 gigabytes of storage. For these items, the estimate is a multiplication and does not change over the month. A **usage-dependent** item's amount, though, is not in the plan; the plan says the resource will exist but does not say how many requests it will take, how many gigabytes of data it will send out, how many messages it will carry. For these items, the estimate rests on the **expected usage** written in the record — that is, it is an estimate built on top of an estimate. ```js // estimate.mjs — produces a monthly cost estimate per change from the plan (model) import { writeFileSync } from "node:fs"; // IV19: the price list is fictional and has nothing to do with any provider's pricing; unit is "fu" (fictional unit) const FIXED = { "machine-medium": 30, "machine-large": 60, "disk-gb": 0.5, "store-gb": 0.8, "archive-gb": 0.1 }; // monthly fixed unit price per item // usage-dependent item: [monthly unit price, uncertainty multiplier r] — real amount is between 1/r and r const USAGE = { "request-million": [4, 4], "egress-gb": [0.9, 4], "message-million": [2, 2], "read-million": [3, 2] }; // IV20: twelve changes are read from the plan; each item's amount is the plan's resource // difference, and for usage-dependent items it is the expected usage written in the record (fictional) const CHANGE = [ ["G01", "three collector machines are added", [["machine-medium", 3], ["disk-gb", 120], ["egress-gb", 60]]], ["G02", "data store is grown by 500 GB", [["store-gb", 500], ["read-million", 20]]], ["G03", "archive stream is added", [["archive-gb", 2000], ["read-million", 30]]], ["G04", "measurement queue is added", [["message-million", 120]]], ["G05", "collector size is made large", [["machine-large", 3], ["machine-medium", -3]]], ["G06", "new region: network and two machines", [["machine-medium", 2], ["disk-gb", 80], ["egress-gb", 300]]], ["G07", "collector disk goes from 40 to 200 GB", [["disk-gb", 480], ["read-million", 10]]], ["G08", "backup window goes from 7 days to 30", [["store-gb", 1400]]], ["G09", "external endpoint is opened", [["request-million", 20], ["egress-gb", 800]]], ["G10", "archive class is moved to cold", [["archive-gb", -1500]]], ["G11", "collector count goes from 3 to 9", [["machine-medium", 6], ["disk-gb", 240], ["egress-gb", 120]]], ["G12", "collection frequency doubles", [["message-million", 40], ["read-million", 60]]], ]; const EST = CHANGE.map(([code, what, items]) => { let fixed = 0; const usage = []; for (const [item, amount] of items) { if (FIXED[item] !== undefined) fixed += FIXED[item] * amount; else usage.push([item, USAGE[item][0] * amount, USAGE[item][1]]); } const u = usage.reduce((s, [, t]) => s + t, 0); return { code, what, fixed, usage, estimate: fixed + u, fixedItems: items.length - usage.length }; }); writeFileSync("estimate.json", JSON.stringify(EST)); console.log(`${"code".padEnd(6)}${"change".padEnd(40)}${"fixed items".padStart(13)}` + `${"usage items".padStart(16)}${"fixed fu".padStart(10)}${"usage fu".padStart(13)}` + `${"estimate fu".padStart(13)}`); for (const d of EST) { const u = d.estimate - d.fixed; console.log(`${d.code.padEnd(6)}${d.what.padEnd(40)}${String(d.fixedItems).padStart(13)}` + `${String(d.usage.length).padStart(16)}${d.fixed.toFixed(1).padStart(10)}` + `${u.toFixed(1).padStart(13)}${d.estimate.toFixed(1).padStart(13)}`); } const fixedTotal = EST.reduce((s, d) => s + d.fixed, 0); const usageTotal = EST.reduce((s, d) => s + (d.estimate - d.fixed), 0); const itemCount = EST.reduce((s, d) => s + d.fixedItems + d.usage.length, 0); console.log(`\n${EST.length} changes, ${itemCount} items: ` + `${EST.reduce((s, d) => s + d.fixedItems, 0)} fixed-unit-price (estimable), ` + `${EST.reduce((s, d) => s + d.usage.length, 0)} usage-dependent (not estimable)`); console.log(`total estimate ${(fixedTotal + usageTotal).toFixed(1)} fu; ` + `fixed share ${fixedTotal.toFixed(1)} fu (${(100 * fixedTotal / (fixedTotal + usageTotal)).toFixed(0)}%), ` + `usage share ${usageTotal.toFixed(1)} fu (${(100 * usageTotal / (fixedTotal + usageTotal)).toFixed(0)}%)`); console.log(`changes carrying a usage-dependent item: ${EST.filter((d) => d.usage.length).length}/${EST.length}`); ``` ``` code change fixed items usage items fixed fu usage fu estimate fu G01 three collector machines are added 2 1 150.0 54.0 204.0 G02 data store is grown by 500 GB 1 1 400.0 60.0 460.0 G03 archive stream is added 1 1 200.0 90.0 290.0 G04 measurement queue is added 0 1 0.0 240.0 240.0 G05 collector size is made large 2 0 90.0 0.0 90.0 G06 new region: network and two machines 2 1 100.0 270.0 370.0 G07 collector disk goes from 40 to 200 GB 1 1 240.0 30.0 270.0 G08 backup window goes from 7 days to 30 1 0 1120.0 0.0 1120.0 G09 external endpoint is opened 0 2 0.0 800.0 800.0 G10 archive class is moved to cold 1 0 -150.0 0.0 -150.0 G11 collector count goes from 3 to 9 2 1 300.0 108.0 408.0 G12 collection frequency doubles 0 2 0.0 260.0 260.0 12 changes, 24 items: 13 fixed-unit-price (estimable), 11 usage-dependent (not estimable) total estimate 4362.0 fu; fixed share 2450.0 fu (56%), usage share 1912.0 fu (44%) changes carrying a usage-dependent item: 9/12 ``` Thirteen of the twenty-four items are fixed-unit-price, eleven are usage-dependent. By item count, the majority looks estimable, but the split is different by amount: fifty-six percent of the total is fixed, forty-four percent usage-dependent. More important is the distribution — nine of the twelve changes carry at least one usage-dependent item. Three changes (G05, G08, G10) consist only of fixed items, and their estimate is not a calculation but a **certainty**. ## Comparing Against Actuals **IV21 — a fixed-unit-price item does not change over the month, its error is exactly zero; a usage-dependent item's actual amount is between `1/r` and `r` times the expected amount.** **IV22 — twelve changes are applied within the same month, and the actual amount is read at month end (fictional).** The uncertainty multiplier varies by item: external requests and egress data are the most volatile items (`r = 4`), queue messages and store reads are calmer (`r = 2`). ```js // actual.mjs — the actual cost, the distribution of estimate error, and the threshold rule's result (model) import { readFileSync } from "node:fs"; // IV21: a fixed-unit-price item does not change over the month, its error is exactly zero; a // usage-dependent item's actual amount is between 1/r and r times the expected amount (r is the item's multiplier) const SEED = 20260401; let state = SEED; const random = () => (state = (state * 48271) % 2147483647) / 2147483647; const ACTUAL = JSON.parse(readFileSync("estimate.json", "utf8")).map((d) => { const actual = d.fixed + d.usage.reduce((s, [, amount, r]) => s + amount * Math.pow(r, 2 * random() - 1), 0); return { ...d, actual, error: actual - d.estimate }; }); console.log(`${"code".padEnd(6)}${"estimate fu".padStart(13)}${"actual fu".padStart(16)}` + `${"error fu".padStart(10)}${"error %".padStart(9)}${"source".padStart(20)}`); for (const d of ACTUAL) { const pct = d.estimate === 0 ? 0 : (100 * d.error) / Math.abs(d.estimate); console.log(`${d.code.padEnd(6)}${d.estimate.toFixed(1).padStart(13)}${d.actual.toFixed(1).padStart(16)}` + `${d.error.toFixed(1).padStart(10)}${pct.toFixed(0).padStart(9)}` + `${(d.usage.length ? `${d.usage.length} usage item${d.usage.length === 1 ? "" : "s"}` : "fixed only").padStart(20)}`); } const exact = ACTUAL.filter((d) => d.error === 0); const off = ACTUAL.filter((d) => d.error !== 0); const ratio = (d) => d.actual / d.estimate; console.log(`\n${exact.length} changes have exactly zero error (they carry only fixed-unit-price ` + `items); ${off.length} changes are off`); console.log(`deviation ranges from ${Math.min(...off.map(ratio)).toFixed(2)}x to ` + `${Math.max(...off.map(ratio)).toFixed(2)}x; ` + `total estimate ${ACTUAL.reduce((s, d) => s + d.estimate, 0).toFixed(1)} fu, ` + `actual ${ACTUAL.reduce((s, d) => s + d.actual, 0).toFixed(1)} fu ` + `(${(100 * (ACTUAL.reduce((s, d) => s + d.error, 0) / ACTUAL.reduce((s, d) => s + d.estimate, 0))).toFixed(0)}% overall)`); // IV23: the threshold rule looks at the plan's estimate: if the estimate exceeds the threshold, the change is held and goes to manual approval console.log(`\n${"threshold fu".padStart(14)}${"held".padStart(7)}${"held for nothing".padStart(18)}` + `${"costly miss".padStart(14)}${"correctly held".padStart(16)}${"budget to approval".padStart(20)}`); for (const E of [200, 300, 500, 800]) { const held = ACTUAL.filter((d) => d.estimate > E); const forNothing = held.filter((d) => d.actual <= E); const missed = ACTUAL.filter((d) => d.estimate <= E && d.actual > E); console.log(`${String(E).padStart(14)}${String(held.length).padStart(7)}` + `${String(forNothing.length).padStart(18)}${String(missed.length).padStart(14)}` + `${String(held.length - forNothing.length).padStart(16)}` + `${held.reduce((s, d) => s + d.estimate, 0).toFixed(0).padStart(20)}`); } for (const E of [200, 300]) { const ad = (f) => ACTUAL.filter(f).map((d) => d.code).join(", ") || "none"; console.log(`\nthreshold ${E} fu: held ${ad((d) => d.estimate > E)}; ` + `held for nothing ${ad((d) => d.estimate > E && d.actual <= E)}; ` + `costly miss ${ad((d) => d.estimate <= E && d.actual > E)}`); } ``` ``` code estimate fu actual fu error fu error % source G01 204.0 192.3 -11.7 -6 1 usage item G02 460.0 461.5 1.5 0 1 usage item G03 290.0 362.5 72.5 25 1 usage item G04 240.0 121.6 -118.4 -49 1 usage item G05 90.0 90.0 0.0 0 fixed only G06 370.0 535.2 165.2 45 1 usage item G07 270.0 288.3 18.3 7 1 usage item G08 1120.0 1120.0 0.0 0 fixed only G09 800.0 958.4 158.4 20 2 usage items G10 -150.0 -150.0 0.0 0 fixed only G11 408.0 368.8 -39.2 -10 1 usage item G12 260.0 316.2 56.2 22 2 usage items 3 changes have exactly zero error (they carry only fixed-unit-price items); 9 changes are off deviation ranges from 0.51x to 1.45x; total estimate 4362.0 fu, actual 4664.7 fu (7% overall) threshold fu held held for nothing costly miss correctly held budget to approval 200 10 2 0 8 4422 300 5 0 2 5 3158 500 2 0 1 2 1920 800 1 0 1 1 1120 threshold 200 fu: held G01, G02, G03, G04, G06, G07, G08, G09, G11, G12; held for nothing G01, G04; costly miss none threshold 300 fu: held G02, G06, G08, G09, G11; held for nothing none; costly miss G03, G12 ``` The first reading of the error table is misleading. In total, the estimate is 4,362, the actual is 4,664.7 fictional units — a seven percent deviation, which looks acceptable. But that seven percent comes from individual changes' deviations canceling each other out: G04 comes in about half low (0.51x), G06 comes in forty-five percent high (1.45x), G03 twenty-five percent high. **Overall accuracy gives the decision-maker nothing**, because decisions are made per change. Seven percent is a good number for a budget report; for a gate, the deviation's 0.51x–1.45x range is the number that matters. The second reading shows where the error comes from. Three changes have **exactly zero** error, and that is not luck but a structural outcome: those three carry only fixed-unit-price items. All nine of the rest are off, and the size of the deviation grows with the item's uncertainty multiplier. The way to bring down estimate error is not a better estimation tool; it is **knowing the share that is usage-dependent**. If a change's estimate does not come with the number "what percent of this amount is usage-dependent," the estimate alone does not say how reliable it is. The one negative item in the table follows the same rule: moving the archive class to cold is estimated as a monthly saving of 150 units, and it saves exactly 150 units. In a fixed-unit-price item, savings are as certain as spend; in a usage-dependent item, a promised reduction carries the same uncertainty and may not materialize. ## Threshold Rule The moment an estimate is tied to a gate, it starts producing decisions: if the estimate exceeds the threshold, the change is held and goes to manual approval. The rule has two failure modes, and both are counted in the table. **Held for nothing** is a change whose estimate exceeds the threshold but whose actual does not; its cost is a waiting contribution and spent approval time. **Costly miss** is a change whose estimate stays under the threshold but whose actual comes in above it; its cost is a surprise in the budget. A two-hundred-unit threshold holds ten of the twelve changes. At this threshold, no costly change slips through — but two of the held changes (G01 and G04) actually stay under the threshold, and the budget going to approval climbs to 4,422 units, nearly the entire total. At this point the gate stops being a gate: everything is held, and approval turns into a ceremony. At a three-hundred-unit threshold, the held count drops to five, and all five of those actually exceed the threshold — but this time two changes (G03 and G12) pass because their estimate stays under it, and they actually come in above it. At an eight-hundred-unit threshold, only one change is held and one slips through. No threshold zeroes out both at once, because a threshold looks at a single number, and that number is itself uncertain. The way to improve the rule is not to choose a better threshold; it is **to change the number the threshold looks at**: if the threshold looks at the estimate's usage-dependent share instead of the estimate itself, a change like G08, which carries only fixed items, can pass automatically despite its high amount, because its estimate is certain. There is one more limit. These estimates' input is the plan, and the plan does not fully predict the apply either: an item that never appears in the plan does not enter the estimate either, and an item that appears in the plan but does not materialize inflates the estimate. All the numbers here assume the plan is correct; the gap between the record and reality is a second error stacked on top of the estimate error. ## Summary - The estimate for twelve changes was produced from the plan diff: 13 of 24 items are fixed-unit-price (estimable), 11 are usage-dependent (not estimable); 56% of the amount is fixed, 44% usage-dependent, and 9 of 12 changes carry at least one usage-dependent item. - The 3 changes that carry only fixed items have exactly zero error; the remaining 9 deviate between 0.51x and 1.45x. - The overall deviation is 7%, and that number is deceptive: individual deviations cancel each other out, while decisions are made per change. - The threshold rule's two failure modes work in opposite directions: a 200-unit threshold holds 10 of 12 changes (2 for nothing, budget to approval 4,422 units) and misses no costly change; at 300 units, 5 are held and 2 costly changes pass; at 800 units, 1 is held and 1 costly change passes. - The way to bring down estimate error is not a better tool, it is reporting the estimate's **usage-dependent share**; if the threshold also looks at that share, a high-amount change with a certain estimate is not held for nothing. ## Next Step This topic's lessons so far have checked the record, run it through policy, tested it, and priced it. All four looked at the same thing: **the resource itself** — machine, disk, network, queue, store. But when a machine is created, its inside is empty; which package will be installed on it, which file will sit with which permission, which service will stay open is not written in the provisioning layer's record. The next lesson takes up the far side of that boundary: the **configuration management** layer that takes over once the machine is created, the difference between agent-based and agentless approaches, and where that layer's own record overlaps with the provisioning record and where it diverges.