--- title: 'Infrastructure as Code Scanning' source: 'https://academia.sh/en/courses/security-automation/infrastructure-as-code-scanning' course: 'Security Automation and the Supply Chain' language: en updated: '2026-09-26T03:02:40+00:00' license: 'CC BY-SA 4.0' objectives: - 'Splits template items by the decision a rule can make about them: certain, undeterminable, and ruleless' - 'Compares the cost of passing an undeterminable item through versus stopping it, with two numbers' - 'Runs the same rule set before and after deployment and measures stopped builds and exposure' --- # Infrastructure as Code Scanning Across 420 template items, the decision a rule can make before deployment is counted: of 100 real violations, 55 are caught with certainty, 26 stay undeterminable because their value is computed at deployment time, and 19 fall in a field class with no rule written at all. Passing the undeterminable through leaves 26 violations unseen, stopping them blocks 65 compliant items for nothing. The same rule set runs across 60 builds in four placements: a strict gate stops 34 builds and leaves 1,339 build-item exposure, a check that only scans after deployment drops to 979 without stopping a single build, and the two together drop to 697; the same 19 violations never close under any placement. The previous five checks looked at executable text: source code, running endpoints, dependency inventory, repository history, image layers. One more text sits in the repository and none of them read it — the templates that define **where, how many copies, with which network rule, and with which authority** the image will run. A single field here can open a wider path than every finding inside the image combined. **Infrastructure as code scanning** is the check that runs these templates through a rule set before deployment. What sets it apart is its timing: the rule runs before the resource ever exists, and for the item it stops there is not even a resource to fix. But this timing has its own limit, and this lesson counts it — the template text does not always say what will happen. - **AC30.** Resource definitions being written as text is **input** in this course; template authoring and rule authoring are not rebuilt here. The 420 items below are a **fictional** model of the fields across five templates. An **item** is a field in a resource definition. - **AC31.** Not every field class has a rule written for it. A ruleless item is never evaluated under any placement, and this number does not change with a setting. - **AC32.** A field's value cannot always be read from the template text: some of it is computed at deployment time. On an unreadable field the rule cannot make a certain decision; the result is **undeterminable**. - **AC33.** On a field whose value can be read, the rule makes a certain decision. This check has no false positive on a readable item; the only source of uncertainty is the unreadable value. - **AC34.** An item stopped before deployment never has its resource come into existence. An item found after deployment has a resource that exists, and fixing it takes one more deployment. - **AC35.** 2 model minutes per build for the template scan, 9 per run for the running-resource check, 20 per blocked build. **Exposure** is in build-item units. The durations are **data**. ## Certain, Undeterminable, and Ruleless ```js // template.mjs — resource definition templates are run through a rule set; what is // measured is the item a certain decision can be made about before deployment versus // the item that cannot. THIS IS A MODEL. export const SEED = 20260411, ITEM_COUNT = 420; const rng = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = rng(SEED); // Every item is a field in a resource definition. hasRule: whether a rule has been // written for that field class. readable: the probability that the field's value can // be read from the template text — a value written as a literal is readable, a value // produced at deployment is not. export const CLASSES = [ ["network access rule", 0.18, 0.26, 0.95, 0.86], ["storage visibility", 0.12, 0.31, 0.95, 0.62], ["identity permission", 0.16, 0.34, 0.95, 0.41], ["encryption setting", 0.14, 0.19, 0.95, 0.78], ["log collection", 0.11, 0.22, 0.95, 0.70], ["resource limit", 0.15, 0.15, 0.00, 0.88], ["tag and ownership", 0.14, 0.11, 0.00, 0.93], ].map(([name, share, violation, rule, readable]) => ({ name, share, violation, rule, readable })); const pick = (u) => { let t = 0; for (const s of CLASSES) { t += s.share; if (u < t) return s; } return CLASSES.at(-1); }; export const ITEMS = Array.from({ length: ITEM_COUNT }, (_, i) => { const s = pick(r()); return { i, cls: s, violation: r() < s.violation, readable: r() < s.readable, hasRule: r() < s.rule, born: Math.floor(r() * 60) }; }); // The decision made before deployment: silent if there is no rule, undeterminable if // the value cannot be read. export const decide = (k) => !k.hasRule ? "ruleless" : !k.readable ? "undeterminable" : k.violation ? "violation" : "compliant"; const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); if (import.meta.url === `file://${process.argv[1]}`) { const violations = ITEMS.filter((k) => k.violation); console.log(`model (seed ${SEED}): ${ITEM_COUNT} template items, ${violations.length} of ` + `them real violations; ${ITEMS.filter((k) => !k.hasRule).length} items have no rule ` + `written for them`); const y1 = gen([20, 8, 7, 10, 9, 17, 13]); console.log("\n" + y1(["field class", "items", "real", "ruleless", "flagged", "undeterminable", "und. real"])); for (const s of CLASSES) { const l = ITEMS.filter((k) => k.cls === s); const b = l.filter((k) => decide(k) === "undeterminable"); console.log(y1([s.name, l.length, l.filter((k) => k.violation).length, l.filter((k) => decide(k) === "ruleless").length, l.filter((k) => decide(k) === "violation").length, b.length, b.filter((k) => k.violation).length])); } const undeterminable = ITEMS.filter((k) => decide(k) === "undeterminable"); const ruleless = ITEMS.filter((k) => decide(k) === "ruleless"); const certainViolation = ITEMS.filter((k) => decide(k) === "violation"); const y2 = gen([26, 10, 9, 12, 14]); console.log("\nan undeterminable item can be handled in two different ways"); console.log(y2(["handling", "findings", "real", "false+", "missed viol."])); for (const [name, group] of [["let undeterminable pass", certainViolation], ["stop undeterminable", [...certainViolation, ...undeterminable]]]) { const g = group.filter((k) => k.violation).length; console.log(y2([name, group.length, g, group.length - g, violations.length - g])); } console.log(`\nwhen pass-through is chosen, ${undeterminable.filter((k) => k.violation).length} ` + `real violations go unseen before deployment; when stopping is chosen, ` + `${undeterminable.filter((k) => !k.violation).length} compliant items are stopped for ` + `nothing. There is no third value between the two options`); console.log(`under both options, ${ruleless.filter((k) => k.violation).length} violations ` + `are never evaluated at all because no rule was written for them; this number does not ` + `change with the handling`); } ``` ``` model (seed 20260411): 420 template items, 100 of them real violations; 132 items have no rule written for them field class items real ruleless flagged undeterminable und. real network access rule 85 22 3 18 8 3 storage visibility 48 16 1 8 22 8 identity permission 66 21 3 10 39 11 encryption setting 57 18 2 15 15 3 log collection 44 5 3 4 7 1 resource limit 59 10 59 0 0 0 tag and ownership 61 8 61 0 0 0 an undeterminable item can be handled in two different ways handling findings real false+ missed viol. let undeterminable pass 55 55 0 45 stop undeterminable 146 81 65 19 when pass-through is chosen, 26 real violations go unseen before deployment; when stopping is chosen, 65 compliant items are stopped for nothing. There is no third value between the two options under both options, 19 violations are never evaluated at all because no rule was written for them; this number does not change with the handling ``` The second table's first row writes down what sets this check apart from the previous ones: 55 of 55 findings are real, the false positive count is zero. Static analysis was guessing whether a code location was dangerous; here the rule reads a field's written value and compares it. On an item whose value can be read, there is no guessing. **This check's uncertainty does not come from how it produces findings, it comes from the field it cannot read.** That field sits in the first table's sixth column. 91 of the 420 items are undeterminable: the rule cannot decide because the value is computed at deployment time. The distribution splits sharply by class. 39 of the identity permission class's 66 items are undeterminable — who the permission is granted to is mostly a reference resolved at deployment. In the network access rule class the same number is only 8 out of 85 items, because address ranges are written into the template. **A template scan's coverage varies by field class, and the source of that variation is not the maturity of the rule set, it is when the value gets determined.** The second table measures the two ends of this uncertainty, and there is no option in between. Passing the undeterminable item through closes with 55 certain findings and never shows 26 real violations before deployment. Stopping it raises the finding count to 146, catches 81 real violations, but blocks 65 compliant items for nothing — 44.5 percent of the report turns into false positives. This is the same threshold trade-off as this course's first lesson, but here the knob is not a number, it is an assumption: is the unreadable value going to be treated as good, or as bad. The last number sits outside both. 132 items have no rule written for them, and 19 of those are real violations. The rule set never looks at the resource limit and tag and ownership classes at all. These 19 items are not undeterminable either — their values are readable, there is just no one asking. **A field class that reports "no violations" in a template scan is either compliant, or it was never asked about at all.** ## Gate, or the Next Run ```js // placement.mjs — the same rule set runs before and after deployment; what is // measured is the build blocked versus the exposure that accrues after the resource // comes into existence. THE DURATIONS ARE A MODEL. import { ITEMS, decide } from "./template.mjs"; const BUILDS = 60, AUDIT_EVERY = 5, FIX_TAKES = 3; const TEMPLATE_MIN = 2, RUNTIME_MIN = 9, BLOCK_MIN = 20; // gate: what it stops before deployment. A stopped item never has its resource come // into existence. after: the check that reads the running resource; there is no // undeterminable value there, but the item is already in production and fixing it // takes one more deployment. function measure(name, gate, after, scansTemplate) { const stopped = new Map(), inProduction = new Map(); for (const k of ITEMS) { if (gate(k)) stopped.set(k.born, [...(stopped.get(k.born) || []), k]); else if (k.violation) inProduction.set(k, null); } const blocked = stopped.size; const falseBlocked = [...stopped.values()].filter((l) => !l.some((k) => k.violation)).length; let exposure = 0, runs = 0; for (let y = 0; y < BUILDS; y++) { if (after && y % AUDIT_EVERY === AUDIT_EVERY - 1) { runs++; for (const [k, closesAt] of inProduction) if (closesAt === null && k.hasRule && k.born <= y) inProduction.set(k, y + FIX_TAKES); } for (const [k, closesAt] of inProduction) if (k.born <= y && (closesAt === null || y < closesAt)) exposure++; } return { name, blocked, falseBlocked, reached: inProduction.size, neverClosed: [...inProduction.values()].filter((v) => v === null).length, exposure, min: Math.round((scansTemplate ? BUILDS * TEMPLATE_MIN : 0) + runs * RUNTIME_MIN + blocked * BLOCK_MIN) }; } const PLACEMENTS = [ measure("before: certain stops", (k) => decide(k) === "violation", false, true), measure("before: undeterm. stops", (k) => decide(k) === "violation" || decide(k) === "undeterminable", false, true), measure("after deployment only", () => false, true, false), measure("before certain+after audit", (k) => decide(k) === "violation", true, true), ]; const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const y = gen([28, 8, 14, 9, 13, 12, 11]); console.log(`${BUILDS} builds; exposure counts the violation left open at each build after the ` + `resource comes into existence. The post-deployment audit runs every ${AUDIT_EVERY} builds, ` + `a fix takes ${FIX_TAKES} builds`); console.log(y(["placement", "blocked", "false blocked", "reached", "never closed", "exposure", "total min"])); for (const o of PLACEMENTS) { console.log(y([o.name, o.blocked, o.falseBlocked, o.reached, o.neverClosed, o.exposure, o.min])); } const [strict, wide, afterOnly, both] = PLACEMENTS; console.log(`\nthe gate that stops before deployment lets ${strict.reached} violations reach ` + `production and stops ${strict.blocked} builds; widening the gate to undeterminable items ` + `brings reached down to ${wide.reached}, but stops ${wide.falseBlocked} builds for nothing`); console.log(`a check that only scans after deployment stops no build at all and closes ` + `${afterOnly.reached - afterOnly.neverClosed} of ${afterOnly.reached} violations; its ` + `exposure is ${afterOnly.exposure}, ${(afterOnly.exposure / strict.exposure).toFixed(2)} ` + `times the strict gate's`); console.log(`running both together brings exposure down to ${both.exposure} build-items; the ` + `remaining ${both.neverClosed} violations are the same under every placement, because no ` + `rule was written for that field class`); ``` ``` 60 builds; exposure counts the violation left open at each build after the resource comes into existence. The post-deployment audit runs every 5 builds, a fix takes 3 builds placement blocked false blocked reached never closed exposure total min before: certain stops 34 0 45 45 1339 800 before: undeterm. stops 52 9 19 19 572 1160 after deployment only 0 0 100 19 979 108 before certain+after audit 34 0 45 19 697 908 the gate that stops before deployment lets 45 violations reach production and stops 34 builds; widening the gate to undeterminable items brings reached down to 19, but stops 9 builds for nothing a check that only scans after deployment stops no build at all and closes 81 of 100 violations; its exposure is 979, 0.73 times the strict gate's running both together brings exposure down to 697 build-items; the remaining 19 violations are the same under every placement, because no rule was written for that field class ``` The first row writes down the payoff of running before deployment: 34 builds stop, none of them for nothing, and the 55 stopped items never have their resource come into existence at all. But the same row's exposure column is 1,339 build-items. The 45 violations the gate lets through reach production and none of them close, because under this placement there is no other check looking for them. The third row breaks the ranking. A check that only reads the running resource after deployment stops no build, spends 108 model minutes — less than a seventh of the strict-gate placement — and brings exposure down to 979 build-items: **0.73 times** the strict gate's. The reason was written in the first section. There is no undeterminable field in a running resource; the value has already been computed and can be read. The rule that runs before deployment never saw 26 of the violations at all; the check that runs after deployment sees every one of them. **Running early does not always mean catching more; no early rule can say what the template text itself cannot say.** The second row tries to close this gap with the gate itself, and it gives the lowest exposure: 572. Its cost is written in two columns — 52 blocked builds, 9 of them for nothing, 1,160 minutes total. This is the pipeline's counterpart to the first section's trade-off: counting the undeterminable as a violation cuts exposure in half and makes every sixth stop unnecessary. The fourth row combines the two. Certain violations stop before deployment — 34 builds, zero for nothing — and the running-resource check, which runs every fifth build, closes the rest. Exposure is 697, total cost 908 minutes. Neither check sees the same 19 violations, and this number is identical across all four placements. **Running both checks does not make the rule set speak up about the field class it stays silent on.** ## Summary - Template scanning produces no false positive on a readable field: 55 of 55 certain findings are real. The only source of uncertainty is a field whose value is computed at deployment. - 91 of the 420 items are undeterminable, and the split is sharp by class: 39 of 66 items in identity permission, 8 of 85 in network access rule. - There is no middle ground between the two ways of handling an undeterminable item: passing it through makes 26 real violations invisible, stopping it blocks 65 compliant items for nothing and turns 44.5 percent of the report into false positives. - The 19 violations among the 132 items with no rule written show up under no handling and no placement; this number changes with neither the setting nor the placement. - Running early does not always mean catching more: a strict gate stops 34 builds and leaves 1,339 build-item exposure, while a check that only scans after deployment drops to 979 without stopping a single build, because a running resource has no undeterminable field. - Running both together brings exposure to 697 build-items, total cost to 908 minutes, and does so with zero false blocks. ## Next Step Six checks counted what gets scanned and what slips through with every change: code location, running endpoint, dependency record, repository history, image layer, template field. All of them shared one assumption — what got scanned was **text sitting in hand**. Yet these very measurements say the opposite. 75 of the image's 144 findings came from a base layer nobody wrote, and 186 of the dependency inventory's 210 components were transitive components whose name never even appears in the manifest file. Most of what ships to production was produced somewhere else, and the question to ask about those parts is not "what is inside it"; **it is who produced it, from what, in which environment**. The next topic makes this question measurable: what questions can the component list a build artifact carries actually answer, what does a signature prove and not prove, and which claim in a provenance document can actually be verified.