--- title: 'Governance, Risk and Compliance' source: 'https://academia.sh/en/courses/governance-risk-and-compliance' language: en updated: '2026-09-26T03:02:51+00:00' license: 'CC BY-SA 4.0' lessons: 17 outcomes: - 'Runs a risk assessment with qualitative and quantitative methods and shows where the ranking changes' - 'Turns security frameworks into a control and evidence mapping and counts the requirements without evidence' - 'Builds a vulnerability management program around remediation capacity and metric retention' --- # Governance, Risk and Compliance Ranking the defenses measured by nine courses on the same list and distributing a scarce budget: producing a risk list from asset, threat and impact analysis, how qualitative scoring and monetary modeling rank the same list differently, the risk reduced per unit by mitigation, transfer, acceptance and avoidance, the item missing from the inventory that can enter no ranking, the item delegated to a supplier but not measurable, the structure of function-based frameworks, certifiable management system models, the requirement left without evidence in control-to-requirement mapping, sector-specific compliance requirements, data subject rights and processing principles, the policy, standard and procedure hierarchy, independent assurance, the remediation capacity of a vulnerability management program, the balance of speed and risk in patch and change management, the retention of awareness training, business continuity exercises and security metrics. ## Risk Management Ranking the defenses measured one by one by nine courses on the same list: producing a risk list from asset, threat and impact items, how qualitative scoring and monetary modeling rank the same list differently, the risk reduced per unit by mitigation, transfer, acceptance and avoidance in the same budget, the item missing from the inventory that can enter no ranking, and the control delegated to a supplier but not verifiable. ### Risk Assessment Process Source: https://academia.sh/en/courses/governance-risk-and-compliance/risk-assessment-process The course's unit of measure is established: 70 pairs of ten assets and seven threats produce 47 risk items, a total annual risk of 5,768.61 fictional units is distributed across the asset classes, the 300-count class holds a 7.2 percent share while the three-count class reaches 17.9 percent, and ranking by decrease per unit of effort changes the position of 45 items in the same list and reduces 240.00 more units of risk on a 200-unit budget. The previous nine courses did the same work: each counted which path a defense closed, which item it set aside, which duration it shortened, and wrote down the legitimate work it stopped in return. At the end of every course, what remained was a measured list of defenses. The one thing never asked was what these lists were **relative to each other**: which one gets done first, which one a scarce budget goes to, and how its completion is proven to whom. This course ties those three questions to a measure, and its measure is **ranking**. The number of paths a defense closes does not by itself give a priority; priority only arises when it is compared against the other items standing in the same list. This lesson builds that list: the asset set, the threat set and the risk items that arise from their intersection. Risk being **the combination of likelihood and impact** was defined in the Introduction to Cybersecurity course; here that definition is the input, and the scale is not a single item but the **program**. #### From Asset to Item A risk item is not an asset by itself; "field laptop" is not a risk item but an asset class. An item arises from an asset being matched with **a specific threat**: malicious code on a field laptop is one item, physical access to the same machine is another. The two differ in frequency, impact, treatment path and cost. The assessment process therefore has three steps: classify the assets, mark which threats apply to each class, and write down the frequency and impact for every pair left at the intersection. The set below is a **model** of the fictional **regional measurement network**: no real inventory is scanned, no real incident log is read, no real currency is used. All values are in **fictional units**. - **RM1.** Ten asset classes, seven threat types and all coefficients are the model. - **RM2.** The applicability mask is taken as derived from the job description and is fixed in the model. In reality this mask is filled in by an assessment exercise and changes as the role changes. - **RM3.** Frequency is the product of the threat's base frequency and the asset's exposure coefficient; impact is the product of the asset class's value and the threat's impact multiplier. - **RM4.** Every item has a single treatment option; its cost is the product of the threat's base cost and the asset's cost coefficient, and it reduces risk by a fixed ratio tied to the threat. - **RM5.** Items are independent: treating one item does not change another item's risk. - **RM6.** The budget is spent in order; an item that does not fit is skipped, and the order is not disturbed. ```js // risk/list.mjs — this IS A MODEL of the fictional regional measurement network's risk items: there // is no real inventory, no real incident log and no real currency; values are in "fictional units". // RM1: threat row: name | annual base frequency | impact multiplier | treatment cost | reduction ratio export const THREAT = ` credential-theft 0.30 0.35 40 0.70 malicious-code 0.50 0.25 30 0.60 misconfiguration 0.80 0.15 12 0.75 insider-misuse 0.10 0.40 55 0.40 service-disruption 0.40 0.20 25 0.50 physical-access 0.06 0.50 35 0.65 supply-chain 0.08 0.45 60 0.30` .trim().split("\n").map((r) => { const [name, baseFreq, mult, cost, reduction] = r.trim().split(/\s+/); return { name, baseFreq: +baseFreq, mult: +mult, cost: +cost, reduction: +reduction }; }); // RM2: asset row: name | count | class value | exposure coefficient | cost coefficient | // threat mask (seven digits in THREAT order; 1 applicable, 0 not) export const ASSET = ` field-laptop 120 900 1.6 1.0 1111011 branch-endpoint 40 600 1.2 1.0 1111110 measurement-collector 300 750 1.4 1.5 0111110 processing-server 12 1800 0.6 0.8 1111101 perimeter-network 4 1200 1.8 0.6 0011100 identity-directory 1 2400 1.0 0.7 1011100 cloud-account 3 2000 1.5 1.4 1011100 subscriber-database 1 3000 0.4 0.9 1011100 field-application 1 1400 1.1 0.7 1111001 supplier-connection 3 1000 1.3 1.2 1001101` .trim().split("\n").map((r) => { const [name, count, value, exposure, costMult, mask] = r.trim().split(/\s+/); return { name, count: +count, value: +value, exposure: +exposure, costMult: +costMult, mask }; }); // RM3: an item is an applicable (asset, threat) pair. Frequency derives from the asset and the // threat, impact derives from the asset's value; risk = frequency x impact (the previous course's // definition, here it is input). export const ITEM = []; for (const v of ASSET) for (const [j, t] of THREAT.entries()) if (v.mask[j] === "1") { const frequency = +(t.baseFreq * v.exposure).toFixed(3), impact = +(v.value * t.mult).toFixed(1); const risk = +(frequency * impact).toFixed(2), cost = +(t.cost * v.costMult).toFixed(1); // RM4: treatment reduces the item's risk by a ratio tied to the threat; the decrease // per unit of effort = risk reduced / cost spent. ITEM.push({ v, t, frequency, impact, risk, cost, reduction: t.reduction, decrease: +(risk * t.reduction).toFixed(2), ratio: +((risk * t.reduction) / cost).toFixed(3) }); } export const TOTAL = +ITEM.reduce((a, k) => a + k.risk, 0).toFixed(2); // Ties keep the generation order (stable sort). export const sortBy = (f) => ITEM.map((k, i) => ({ k, i })) .sort((a, b) => f(b.k) - f(a.k) || a.i - b.i).map((x) => x.k); export const position = (L, k) => L.indexOf(k) + 1; ``` #### Counting the List ```js // risk/generate.mjs — the item list, program scale and the course's unit of measure from the pair set import { ASSET, THREAT, ITEM, TOTAL, sortBy, position } from "./list.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const BY_RISK = sortBy((k) => k.risk); // risk from largest to smallest const BY_RATIO = sortBy((k) => k.ratio); // decrease per unit of effort console.log(`${ASSET.length} assets x ${THREAT.length} threats = ${ASSET.length * THREAT.length}` + ` possible pairs; applicable ${ITEM.length} items; total annual risk ${TOTAL} fictional units`); const A = [-22, 6, 7, 10, 9, -22]; console.log("\n1. per asset"); y(A, "asset", "count", "items", "risk", "share %", " largest item"); for (const v of ASSET) { const ks = ITEM.filter((k) => k.v === v); const top = ks.reduce((a, k) => (k.risk > a.risk ? k : a)); y(A, v.name, v.count, ks.length, ks.reduce((a, k) => a + k.risk, 0).toFixed(2), ((ks.reduce((a, k) => a + k.risk, 0) / TOTAL) * 100).toFixed(1), " " + top.t.name); } const B = [-22, -21, 10, 8, 9, 7, 7, 5, 5]; console.log("\n2. first ten items by risk order"); y(B, "asset", "threat", "frequency", "impact", "risk", "cost", "ratio", "R", "O"); for (const k of BY_RISK.slice(0, 10)) y(B, k.v.name, k.t.name, k.frequency.toFixed(2), k.impact.toFixed(1), k.risk.toFixed(2), k.cost.toFixed(1), k.ratio.toFixed(3), position(BY_RISK, k), position(BY_RATIO, k)); const C = [-22, -21, 10, 8, 9, 7, 7, 5, 5]; console.log("\n3. first five items by ratio order"); y(C, "asset", "threat", "frequency", "impact", "risk", "cost", "ratio", "R", "O"); for (const k of BY_RATIO.slice(0, 5)) y(C, k.v.name, k.t.name, k.frequency.toFixed(2), k.impact.toFixed(1), k.risk.toFixed(2), k.cost.toFixed(1), k.ratio.toFixed(3), position(BY_RISK, k), position(BY_RATIO, k)); const D = [-38, 6]; console.log("\n4. difference between the two rankings"); const shift = ITEM.map((k) => Math.abs(position(BY_RISK, k) - position(BY_RATIO, k))); y(D, "items with a changed rank", shift.filter((x) => x > 0).length); y(D, "largest rank shift", Math.max(...shift)); y(D, "shared items in the top ten", BY_RATIO.slice(0, 10).filter((k) => BY_RISK.slice(0, 10).includes(k)).length); y(D, "ratio rank of the riskiest item", position(BY_RATIO, BY_RISK[0])); y(D, "risk rank of the highest-ratio item", position(BY_RISK, BY_RATIO[0])); const BUDGET = 200; const E = [-14, 7, 10, 9, 12]; console.log(`\n5. spending a ${BUDGET}-unit budget according to the two rankings`); y(E, "ranking", "item", "spent", "decrease", "remaining %"); const spend = (L) => L.reduce((s, k) => (s.h + k.cost > BUDGET ? s : { h: s.h + k.cost, a: s.a + k.decrease, n: s.n + 1 }), { h: 0, a: 0, n: 0 }); const [R, O] = [spend(BY_RISK), spend(BY_RATIO)]; for (const [name, s] of [["risk", R], ["ratio", O]]) y(E, name, s.n, s.h.toFixed(1), s.a.toFixed(2), (((TOTAL - s.a) / TOTAL) * 100).toFixed(1)); console.log(`the ratio ranking gives ${(O.a - R.a).toFixed(2)} more units of decrease ` + `(${(((O.a - R.a) / R.a) * 100).toFixed(1)}%) and ${(R.h - O.h).toFixed(1)} fewer units of spending`); ``` ``` 10 assets x 7 threats = 70 possible pairs; applicable 47 items; total annual risk 5768.61 fictional units 1. per asset asset count items risk share % largest item field-laptop 120 6 656.64 11.4 malicious-code branch-endpoint 40 6 360.00 6.2 malicious-code measurement-collector 300 5 414.75 7.2 malicious-code processing-server 12 6 546.48 9.5 malicious-code perimeter-network 4 3 518.40 9.0 misconfiguration identity-directory 1 4 828.00 14.4 misconfiguration cloud-account 3 4 1035.00 17.9 misconfiguration subscriber-database 1 4 414.00 7.2 misconfiguration field-application 1 5 656.04 11.4 malicious-code supplier-connection 3 4 339.30 5.9 credential-theft 2. first ten items by risk order asset threat frequency impact risk cost ratio R O cloud-account misconfiguration 1.20 300.0 360.00 16.8 16.071 1 4 cloud-account credential-theft 0.45 700.0 315.00 56.0 3.938 2 15 identity-directory misconfiguration 0.80 360.0 288.00 8.4 25.714 3 2 perimeter-network misconfiguration 1.44 180.0 259.20 7.2 27.000 4 1 identity-directory credential-theft 0.30 840.0 252.00 28.0 6.300 5 8 cloud-account service-disruption 0.60 400.0 240.00 35.0 3.429 6 17 field-application malicious-code 0.55 350.0 192.50 21.0 5.500 7 10 identity-directory service-disruption 0.40 480.0 192.00 17.5 5.486 8 11 field-application misconfiguration 0.88 210.0 184.80 8.4 16.500 9 3 field-laptop malicious-code 0.80 225.0 180.00 30.0 3.600 10 16 3. first five items by ratio order asset threat frequency impact risk cost ratio R O perimeter-network misconfiguration 1.44 180.0 259.20 7.2 27.000 4 1 identity-directory misconfiguration 0.80 360.0 288.00 8.4 25.714 3 2 field-application misconfiguration 0.88 210.0 184.80 8.4 16.500 9 3 cloud-account misconfiguration 1.20 300.0 360.00 16.8 16.071 1 4 field-laptop misconfiguration 1.28 135.0 172.80 12.0 10.800 11 5 4. difference between the two rankings items with a changed rank 45 largest rank shift 16 shared items in the top ten 6 ratio rank of the riskiest item 4 risk rank of the highest-ratio item 4 5. spending a 200-unit budget according to the two rankings ranking item spent decrease remaining % risk 9 198.3 1547.40 73.2 ratio 13 184.7 1787.40 69.0 the ratio ranking gives 240.00 more units of decrease (15.5%) and 13.6 fewer units of spending ``` The numbers are in the **measurement** class; its inputs are RM1–RM6. #### The Filter Itself Is a Decision The first line gives two numbers: 70 possible pairs and 47 applicable items. The 23 pairs in between never enter the list at all, because the applicability mask filters them out. The perimeter network has no credential theft marked against it, the measurement collector has no credential theft marked against it, the subscriber database has no malicious code marked against it. Each is a defensible decision, but none of them is a measured decision. A filtered-out pair's risk is not written as zero; it is **not written at all**. It does not appear in the ranking, it does not take up room in the budget, it has no share in the total. This is the quietest step of the assessment process: an item that is never added to the list stays outside every measurement that follows. If one of the 23 exclusions is wrong, that wrong decision appears in none of this lesson's tables. #### Count Is Not Risk The first table gives the program scale's first result. The measurement collector is the most numerous asset class with 300 units and carries **7.2 percent** of the total risk. The cloud account has **three** units and carries **17.9 percent**. The identity directory has **one** unit and carries 14.4 percent. Together the two classes account for a third of the program's risk and take up four rows in the inventory. The reason for this divergence is that value is carried by function, not by count. When one measurement collector goes down, a small part of the region cannot be read; when the identity directory goes down, every access decision goes down with it. A priority list built on count gives exactly the reverse order, and that is the easiest decision to make while the inventory sheet is open. The second divergence is between the asset total and the item ranking. The field laptop is **third** in class total (656.64 units), but this class's largest item only makes it to **tenth** place in the top ten. The class climbs up on the sum of six items, while each item on its own stays middling. In the opposite direction, the perimeter network carries only three items, and one of them is in fourth place. Treating an asset and treating an item are not the same job: the first means six separate pieces of work, the second means one. #### The Course's Unit of Measure Every ranking so far has gone by the size of the risk itself and never asks one thing: how much effort it costs to reduce that risk. The item list's fourth column writes this down. **The risk decrease per unit of effort** is the reduced risk divided by the cost spent, and it is this course's unit of measure. The second table's last two columns place the two rankings side by side. The credential theft item on the cloud account is **second** in risk order and **fifteenth** in ratio order. Its risk is 315.00 units, but its treatment cost of 56.0 units is one of the six highest costs among the forty-seven items: changing the identity setup on three cloud accounts is both the broadest piece of work and one whose reduction ratio stays at 70 percent. Its neighbor at the same rank — misconfiguration on the identity directory — carries 288.00 units of risk and is treated for 8.4 units. The fourth table counts the difference in aggregate. Across the same 47 items, **45 change rank**, the largest shift is **16 places**, and only **six** of the top ten are shared. The riskiest item drops to fourth place in ratio order, and the highest-ratio item drops to fourth place in risk order. Neither falls from the top of the list; the real shift happens in the middle band, and the middle band is where most of the budget gets spent. #### When Ranking Turns Into Budget The fifth table turns the difference into a decision. When a two-hundred-unit budget is spent in risk order, **nine items** are treated, 198.3 units are spent, and total risk drops by 1,547.40 units. When the same budget is spent in ratio order, **thirteen items** are treated, 184.7 units are spent, and risk drops by 1,787.40 units. The difference is **240.00 units**, that is, 15.5 percent more decrease — and with 13.6 units less spending, too. The risk ranking exhausting the budget on nine items is not a flaw; it is a consequence of the ranking's own logic: the biggest risks are also the most expensive to treat, because both come from the same place — the asset's centrality. When the priority list says "biggest first," it ties the budget up in a small number of expensive items and never touches the remaining thirty-eight. This is the first case of the ranking **putting the wrong item first**, and its cost can be counted. The third table writes down the ratio ranking's own blind spot: **all five** of the top five items are the misconfiguration threat. The threat type with the lowest treatment cost entirely occupies the top of the list. When the program follows this order, it goes deep on a single threat type while the remaining risk piles up in the other six types. What both rankings leave behind is **between 69 percent and 73 percent**; the four-point difference between them is a real gain, but the real number is that the budget is not enough to finish the list. #### Summary - A risk item is not an asset but a pair of an asset and a threat; of the 70 pairs of ten assets and seven threats, 47 are applicable and carry a total annual risk of 5,768.61 fictional units. - The 23 filtered-out pairs are not written as zero; they are not written at all. An item that never enters the list stays outside every ranking, budget and total that follows. - Count and risk give different rankings: the 300-count class carries 7.2 percent, the three-count class carries 17.9 percent, and the one-count class carries 14.4 percent. The class that is third in asset total has its largest item reach only tenth place in the item ranking. - The course's unit of measure is the risk decrease per unit of effort; in the same list, 45 items change rank, the largest shift is 16 places, and six of the top ten stay shared. - A two-hundred-unit budget gives nine items and a 1,547.40-unit decrease in risk order, and thirteen items and a 1,787.40-unit decrease in ratio order: a 240.00-unit difference, with 13.6 units less spending. - The ratio ranking's top five all come from a single threat type; both rankings leave between 69 and 73 percent of the risk unaddressed. #### Next Step Every number in this lesson rests on a single assumption: that frequency and impact are known numbers. Yet no row's 0.80 frequency or 300.0 impact anywhere in the list came from a measurement; it came from an estimate. Most organizations therefore use two separate paths: qualitative scoring that places items into bands instead of numbers, or a monetary model that splits the loss into asset value and exposure ratio. The next lesson runs both paths on the **same 47 items**, calculates single-occurrence and annual loss expectations in fictional units, and answers a single question: how many rankings do the two methods write differently from the same data, and which items does it drop into ties. ### Qualitative and Quantitative Approaches Source: https://academia.sh/en/courses/governance-risk-and-compliance/qualitative-and-quantitative-approaches The same 47 items are run through two scales: the monetary model computes single and annualized loss expectancy in fictional units, while the five-by-five qualitative matrix reduces the items to three levels, ties 479 of the 1,081 item pairs and reverses 52 of them, writes the 184.80-unit item as medium and the 96.00-unit item as high, and opens a 745.20-unit gap in the outcome of the same ten items chosen with the same effort. The previous lesson produced and ranked the 47-item list, but the frequency and impact in each row came not from a measurement but from an estimate. There are two common ways to estimate: the **qualitative approach**, which places items into bands instead of numbers, and the **quantitative approach**, which splits the loss into asset value and loss ratio. This lesson runs both on the same list. The question is not which one is correct — both work from the same uncertain input — but **how many places apart the same data ranks things** and how many units that difference costs in the budget. The priority list is not a product of the data; it is a product of the scale applied to the data. #### Two Scales The quantitative approach splits the loss into two parts. **Single loss expectancy** is what is lost when the event happens once, and it is the product of asset value and the **exposure ratio**: a 35 percent exposure ratio on a 3,000-fictional-unit database writes a single loss of 1,050 units. **Annualized loss expectancy** is the product of the single loss and the annual rate of occurrence. The previous lesson's "impact" and "risk" columns are exactly these; the name changes, the number does not. The qualitative approach reduces the same two inputs to bands and feeds the band pair into a **matrix**. Each cell gives an ordinal level. Levels are an order, not a scale: the distance between "high" and "medium" is undefined, and two "medium"s cannot be added together. - **RM7.** Seven threats, ten assets and all the coefficients are the model; the unit is the **fictional unit**. - **RM8.** Asset value is assumed to be derived from the business description and is fixed in the model. - **RM9.** Single loss expectancy is the product of asset value and exposure ratio; annualized loss expectancy is the product of single loss and annual frequency. - **RM10.** The qualitative matrix is five by five and carries four levels; the levels are ordinal, they cannot be summed. - **RM11.** Baseline frequency is an estimate; the drift generator gives each threat a coefficient between 0.75 and 1.25, the seed is visible and the output does not change from run to run. ```js // risk/scale.mjs — the same 47 items, two scales: the qualitative band matrix and the monetary // model. THIS IS A MODEL; there is no real inventory, no real incident log and no real currency, // the unit is the "fictional unit". // RM7: threat row: name | annual rate of occurrence (baseline) | exposure ratio const THREAT = ` credential-theft 0.30 0.35 malicious-code 0.50 0.25 misconfiguration 0.80 0.15 insider-misuse 0.10 0.40 service-disruption 0.40 0.20 physical-access 0.06 0.50 supply-chain 0.08 0.45` .trim().split("\n").map((r) => { const [name, s, m] = r.trim().split(/\s+/); return { name, baseFreq: +s, exposureRatio: +m }; }); // RM8: asset row: name | asset value | exposure multiplier | threat mask (seven digits) const ASSET = ` field-laptop 900 1.6 1111011 branch-endpoint 600 1.2 1111110 measurement-collector 750 1.4 0111110 processing-server 1800 0.6 1111101 perimeter-network 1200 1.8 0011100 identity-directory 2400 1.0 1011100 cloud-account 2000 1.5 1011100 subscriber-database 3000 0.4 1011100 field-application 1400 1.1 1111001 supplier-connection 1000 1.3 1001101` .trim().split("\n").map((r) => { const [name, v, k, mask] = r.trim().split(/\s+/); return { name, value: +v, exposureMult: +k, mask }; }); // RM9: monetary model. Single loss expectancy (SLE) = asset value x exposure ratio; annualized // loss expectancy (ALE) = SLE x annual frequency. These are the previous lesson's "impact" and // "risk" columns; the name changed, the number did not. export const ITEM = []; for (const v of ASSET) for (const [j, t] of THREAT.entries()) if (v.mask[j] === "1") { const freq = +(t.baseFreq * v.exposureMult).toFixed(3), sle = +(v.value * t.exposureRatio).toFixed(1); ITEM.push({ v, t, j, freq, sle, ale: +(freq * sle).toFixed(2) }); } // RM10: qualitative scale. Frequency and SLE are each reduced to a band; the band pair looks up // a matrix cell; the cell gives one of four levels. Levels are ordinal, they cannot be summed. const band = (x, s) => s.findIndex((u) => x < u) + 1 || s.length + 1; export const FREQ_BOUNDS = [0.10, 0.25, 0.50, 1.00], SLE_BOUNDS = [180, 315, 450, 700]; export const LEVEL = ["low", "medium", "high", "critical"]; // Row = SLE band (5 to 1), column = frequency band (1 to 5). The digit is the LEVEL index. export const MATRIX = ["12233", "11223", "01122", "00112", "00011"]; export const freqBand = (k) => band(k.freq, FREQ_BOUNDS); export const sleBand = (k) => band(k.sle, SLE_BOUNDS); export const level = (k) => +MATRIX[5 - sleBand(k)][freqBand(k) - 1]; for (const k of ITEM) { k.freqBand = freqBand(k); k.sleBand = sleBand(k); k.level = level(k); } export const TOTAL = +ITEM.reduce((a, k) => a + k.ale, 0).toFixed(2); export const sortBy = (f) => ITEM.map((k, i) => ({ k, i })) .sort((a, b) => f(b.k) - f(a.k) || a.i - b.i).map((x) => x.k); export const position = (L, k) => L.indexOf(k) + 1; // RM11: estimation drift. Baseline frequency is an estimate, not a measurement; the generator // below gives each threat a coefficient between 0.75 and 1.25. The seed is visible, the output // does not change. let seed = 20250401; const rand = () => (seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648; export const DRIFT = THREAT.map(() => +(0.75 + rand() * 0.5).toFixed(3)); export const drifted = (k) => { const freq = +(k.freq * DRIFT[k.j]).toFixed(3); return { freq, ale: +(freq * k.sle).toFixed(2), level: +MATRIX[5 - k.sleBand][band(freq, FREQ_BOUNDS) - 1] }; }; ``` #### Two Scales on the Same List ```js // risk/compare.mjs — the qualitative level and the annualized loss expectancy race on the same list import { ITEM, TOTAL, LEVEL, MATRIX, FREQ_BOUNDS, SLE_BOUNDS, DRIFT, drifted, sortBy, position } from "./scale.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const BY_ALE = sortBy((k) => k.ale); // monetary: annualized loss expectancy const BY_LEVEL = sortBy((k) => k.level); // qualitative: level, ties keep list order console.log(`${ITEM.length} items, total annualized loss expectancy ${TOTAL} fictional units; ` + `band boundaries ${FREQ_BOUNDS.join("/")} and ${SLE_BOUNDS.join("/")}`); const A = [-9, 9, 9, 9, 9, 9]; console.log("\n1. qualitative matrix (row: single-loss band 5..1, column: frequency band 1..5)"); y(A, "sle\\freq", "1", "2", "3", "4", "5"); MATRIX.forEach((r, i) => y(A, String(5 - i), ...[...r].map((c) => LEVEL[+c]))); const freq5 = ITEM.filter((k) => k.freqBand === 5), sle5 = ITEM.filter((k) => k.sleBand === 5); console.log(`${new Set(ITEM.map((k) => `${k.sleBand}${k.freqBand}`)).size} of 25 cells are filled; ` + `of the ${freq5.length} items with frequency band 5, the highest single-loss band is ` + `${Math.max(...freq5.map((k) => k.sleBand))}, of the ${sle5.length} items with single-loss band 5, ` + `the highest frequency band is ${Math.max(...sle5.map((k) => k.freqBand))}`); const B = [-8, 7, 10, 10, 6, 12, 8]; console.log("\n2. per level"); y(B, "level", "items", "smallest", "largest", "ratio", "total ALE", "share %"); for (const [i, name] of LEVEL.entries()) { const g = ITEM.filter((k) => k.level === i); if (g.length === 0) { y(B, name, 0, "-", "-", "-", "0.00", "0.0"); continue; } const lo = Math.min(...g.map((k) => k.ale)), hi = Math.max(...g.map((k) => k.ale)); const t = g.reduce((a, k) => a + k.ale, 0); y(B, name, g.length, lo.toFixed(2), hi.toFixed(2), (hi / lo).toFixed(1), t.toFixed(2), ((t / TOTAL) * 100).toFixed(1)); } const HIGH = ITEM.filter((k) => k.level === 2).sort((a, b) => b.ale - a.ale); const C = [-22, -21, 5, 5, 10, 5]; console.log("\n3. the part of the 'high' level that falls back in the monetary order"); y(C, "asset", "threat", "f-b", "s-b", "ALE", "N"); for (const k of HIGH.slice(8)) y(C, k.v.name, k.t.name, k.freqBand, k.sleBand, k.ale.toFixed(2), position(BY_ALE, k)); console.log(`the first ${HIGH.filter((k, i) => position(BY_ALE, k) === i + 1).length} items of this level ` + `are in the same place in the monetary order too`); const midTop = ITEM.filter((k) => k.level === 1).reduce((a, k) => (k.ale > a.ale ? k : a)); console.log(`the "medium" item with the highest ALE: ${midTop.v.name} / ${midTop.t.name} ` + `${midTop.ale.toFixed(2)} units, ALE rank ${position(BY_ALE, midTop)}`); const D = [-42, 8]; console.log("\n4. the gap between the two scales"); const pair = (f) => { let n = 0; for (let a = 0; a < ITEM.length; a++) for (let b = a + 1; b < ITEM.length; b++) if (f(ITEM[a], ITEM[b])) n++; return n; }; y(D, "total item pairs", pair(() => true)); y(D, "pairs tied in the qualitative scale", pair((a, b) => a.level === b.level)); y(D, "pairs the two scales rank in reverse", pair((a, b) => (a.level - b.level) * (a.ale - b.ale) < 0)); y(D, "levels used in the qualitative scale", new Set(ITEM.map((k) => k.level)).size + "/4"); y(D, "how many of the top ten are shared", BY_LEVEL.slice(0, 10).filter((k) => BY_ALE.slice(0, 10).includes(k)).length); const E = [-42, 8]; const top = (L) => +L.reduce((a, k) => a + k.ale, 0).toFixed(2); console.log("\n5. if the same effort picks ten items (the arbitrariness of a tie)"); y(E, "top ten by monetary order", top(BY_ALE.slice(0, 10))); y(E, "qualitative: best ten from high level", top(HIGH.slice(0, 10))); y(E, "qualitative: worst ten from high level", top(HIGH.slice(-10))); y(E, "gap from arbitrariness", +(top(HIGH.slice(0, 10)) - top(HIGH.slice(-10))).toFixed(2)); const F = [-47, 8]; const BY_ALE_DRIFT = sortBy((k) => drifted(k).ale); console.log(`\n6. estimation drift (coefficient per threat: ${DRIFT.join(" ")})`); y(F, "items whose ALE rank changes", ITEM.filter((k) => position(BY_ALE, k) !== position(BY_ALE_DRIFT, k)).length); y(F, "largest rank shift", Math.max(...ITEM.map((k) => Math.abs(position(BY_ALE, k) - position(BY_ALE_DRIFT, k))))); y(F, "how many of the top ten by money stay the same", BY_ALE_DRIFT.slice(0, 10).filter((k) => BY_ALE.slice(0, 10).includes(k)).length); y(F, "items whose qualitative level changes", ITEM.filter((k) => drifted(k).level !== k.level).length); ``` ``` 47 items, total annualized loss expectancy 5768.61 fictional units; band boundaries 0.1/0.25/0.5/1 and 180/315/450/700 1. qualitative matrix (row: single-loss band 5..1, column: frequency band 1..5) sle\freq 1 2 3 4 5 5 medium high high critical critical 4 medium medium high high critical 3 low medium medium high high 2 low low medium medium high 1 low low low medium medium 18 of 25 cells are filled; of the 4 items with frequency band 5, the highest single-loss band is 2, of the 8 items with single-loss band 5, the highest frequency band is 3 2. per level level items smallest largest ratio total ALE share % low 5 21.60 57.60 2.7 181.50 3.1 medium 28 38.88 184.80 4.8 2705.71 46.9 high 14 96.00 360.00 3.8 2881.40 49.9 critical 0 - - - 0.00 0.0 3. the part of the 'high' level that falls back in the monetary order asset threat f-b s-b ALE N field-application credential-theft 3 4 161.70 13 subscriber-database misconfiguration 3 4 144.00 15 processing-server malicious-code 3 4 135.00 17 subscriber-database credential-theft 2 5 126.00 21 cloud-account insider-misuse 2 5 120.00 22 identity-directory insider-misuse 2 5 96.00 25 the first 8 items of this level are in the same place in the monetary order too the "medium" item with the highest ALE: field-application / misconfiguration 184.80 units, ALE rank 9 4. the gap between the two scales total item pairs 1081 pairs tied in the qualitative scale 479 pairs the two scales rank in reverse 52 levels used in the qualitative scale 3/4 how many of the top ten are shared 7 5. if the same effort picks ten items (the arbitrariness of a tie) top ten by monetary order 2463.5 qualitative: best ten from high level 2404.4 qualitative: worst ten from high level 1659.2 gap from arbitrariness 745.2 6. estimation drift (coefficient per threat: 1.116 0.896 1.167 0.756 0.814 0.759 1.191) items whose ALE rank changes 37 largest rank shift 7 how many of the top ten by money stay the same 8 items whose qualitative level changes 8 ``` The numbers are in the **measurement** class; its inputs are RM7–RM11. #### The Corner That Never Fills The matrix defines four levels and the program uses **three** of them. The "critical" cells stay empty, because those cells demand both high frequency and high single loss, and the two conditions never meet in the same item: of the four items with frequency band 5, the highest single-loss band is **2**; of the eight items with single-loss band 5, the highest frequency band is **3**. Eighteen of twenty-five cells are filled, but the filled cells spread out on both sides of the diagonal. This shows that the scale's **resolution** narrows relative to the data: 47 items are described with three words, and the most crowded word is "medium" — 28 items, sixty percent of the list, sit in a single box. #### The Spread Inside the Same Level The second table gives this lesson's main finding. The annualized loss expectancy of the 14 items in the "high" level ranges **between 96.00 and 360.00 units** — a **3.8-fold** spread. The spread is even wider in the "medium" level: between 38.88 and 184.80, a **4.8-fold** spread. The real result is where the two levels intersect. The "medium" item with the highest annualized loss expectancy is the misconfiguration on the field application: 184.80 units, and **ninth** in the monetary order. In the same list, the insider-misuse item on the identity directory is written as "high" and carries 96.00 units, and it is **twenty-fifth** in the monetary order. The qualitative scale ranks the second one ahead; the monetary model ranks the first one ahead by sixteen places. The reversal is not a coincidence. The qualitative matrix bands single loss and frequency **separately**; when an item goes to an extreme in a single dimension, the cell carries it upward without looking at what the product gives. The third table separates out these items: the first eight of the "high" level are also the first eight in the monetary order; the remaining six are scattered between the 13th and 25th places, and three of them have single-loss band 5 and frequency band 2. The fourth table measures the same phenomenon by pair count. 47 items produce 1,081 pairs; the qualitative scale drops **479 of them into a tie**, and in **52 pairs** the two scales say the opposite of each other. A tie is a deficiency, a reversal is a contradiction; the first defers the decision, the second delivers the wrong one. #### When a Tie Turns Into Budget The fifth table writes the cost of the tie. If the program can handle ten items, the monetary order picks the top ten and targets **2,463.50 units**. The qualitative order picks ten of the 14 items in the "high" level, but it does not say which ten: the best choice comes to **2,404.40**, the worst to **1,659.20 units**. The **745.20-unit** gap between them is the arbitrariness the scale leaves behind. The same list, the same effort and the same number of items; what determines the outcome is the order in which the list was written. The good end of the qualitative approach comes within 59.10 units of the monetary model's result; the bad end is 804.30 units away. The scale is not wrong, it is **incomplete**: it carries the decision so far and leaves the last step to an unnamed choice. #### Facing Estimation Drift The sixth table measures the monetary model's own weakness. When the baseline frequencies are shaken within a 25 percent band — the generator is written into the lesson, the seed is visible — **37 items change place** in the monetary ranking and the largest shift is seven positions. Eight of the top ten stay in place; the band that moves is the middle one. The same drift changes only **8 items' level** in the qualitative scale. The band boundaries absorb the movement in the input: a frequency between 0.36 and 0.44 falls into the same band. The loss of resolution turns into a gain here — a small number of boxes reacts less to input error. When the two tables are read together, the choice becomes clear. The monetary model makes a finer distinction, and part of that distinction is not real but estimation noise; the 37-item shift is proof of it. The qualitative model withstands the noise, and in exchange it says nothing in 479 pairs. **Both put the same uncertainty in a different place:** one in the digits of the number, the other inside the box. Unless which approach was used is written down, how much of the ranking came from the data cannot be known. #### Summary - The monetary model splits the loss in two: single loss expectancy is the product of asset value and exposure ratio, annualized loss expectancy is the product of single loss and annual frequency; the total is 5,768.61 fictional units. - The five-by-five qualitative matrix defines four levels and the program uses three of them; 18 of 25 cells are filled and "critical" stays empty, because high frequency and high single loss never meet in the same item. - The spread inside the same level runs between 3.8-fold and 4.8-fold; the 184.80-unit item is written as "medium," the 96.00-unit item as "high," and sixteen places separate them. - 479 of the 1,081 item pairs fall into a tie in the qualitative scale, and in 52 the two scales say the opposite of each other; only seven of the top ten are shared. - The same ten-item effort targets 2,463.50 units by the monetary order and between 1,659.20 and 2,404.40 units by the qualitative order; the 745.20-unit gap is the choice the scale does not name. - When the frequency estimate is shaken within a 25 percent band, 37 items change place in the monetary order and only 8 items' level changes in the qualitative scale: the loss of resolution gives resilience against input error. #### Next Step Both lessons carry the same silent assumption: there was a single way to handle each item, and that way reduced risk at a fixed rate. In reality, a risk item is answered in four separate ways — adding a control to **mitigate** it, **transferring** its cost through a contract, **accepting** it as it is, and **avoiding** it by dropping the work that creates the risk. All four are spent from the same budget, and the risk they reduce per unit differs. The next lesson runs all four on the same list, and separately counts which items the transfer fails to hand off. ### Risk Treatment Options Source: https://academia.sh/en/courses/governance-risk-and-compliance/risk-treatment-options Mitigation, transfer, acceptance and avoidance are all run under the same annual budget of 300 units: mitigation reduces risk by 2,746.71 units with 22 decisions while transfer and avoidance stay at 243.00 and 656.04 units, the transfer agreement takes over 76 of 235 component items and leaves 3,507.70 units outside, avoidance is an asset-level decision so it can never enter the ranking even under a 1,200-unit budget, and 1,699.81 of the 3,021.90 accepted units come from 25 items that were never opened. The previous two lessons built the list and ranked it on two scales, but recognized only a single response for each item. There are actually four responses. **Mitigation** adds a control and reduces the probability or the impact, **transfer** loads part of the loss onto someone else through a transfer agreement, **acceptance** carries the item as it is, **avoidance** gives up the work that creates the risk. All four are spent from the same budget but consume it in different units. This lesson converts them to a common unit, runs all four against the same list, and separately counts two things: what transfer **does not** hand over, and avoidance's **inability** to enter the ranking. #### Four Options, One Unit Mitigation's cost is a setup and a maintenance fee; the setup is paid once, the maintenance every year. Transfer's cost is an annual premium, and the premium is the transferred risk multiplied by a **loading** coefficient. Avoidance's cost is the annual value of the work given up; acceptance's cost, and the risk it reduces, are both zero. The common unit is the **annual fictional unit**: the setup cost is divided over three years and the maintenance is added; the premium and the given-up work value are already annual. Without this conversion, the comparison breaks silently. - **RM12.** The coefficients, costs, transfer loadings and component distributions are all a model; the unit is the **fictional unit**. - **RM13.** The avoidance cost is the annual work-value loss from giving up the work; an asset marked `-` cannot have its work given up. - **RM14.** The transfer agreement covers the first two of the five components and does not cover the other three. - **RM15.** All costs are converted to the annual unit: mitigation's annual cost is the setup divided by three plus the annual maintenance. - **RM16.** The budget is spent in order of ratio; an item that has already been taken is not handled a second time. Avoidance is an asset-level decision and closes all of an asset's items at once. ```js // risk/option.mjs — same 47 items, four treatment options. THIS IS A MODEL; there is no real // inventory, no real contract and no real currency, every cost is an annual "fictional unit". // RM12: threat row: name | frequency | exposure ratio | ratio mitigation reduces | // setup cost | annual maintenance | transfer loading ('-' cannot be transferred) | // component distribution (percent): direct / interruption / remediation / notification / reputation export const THREAT = ` credential-theft 0.30 0.35 0.70 40 9 1.35 30 15 20 20 15 malicious-code 0.50 0.25 0.60 30 8 1.50 25 35 25 5 10 misconfiguration 0.80 0.15 0.75 12 5 - 20 45 25 0 10 insider-misuse 0.10 0.40 0.40 55 12 1.80 35 10 15 20 20 service-disruption 0.40 0.20 0.50 25 6 1.20 10 65 20 0 5 physical-access 0.06 0.50 0.65 35 7 1.45 45 15 15 15 10 supply-chain 0.08 0.45 0.30 60 14 1.60 30 25 20 10 15` .trim().split("\n").map((r) => { const [name, freq, exp, red, setup, maint, load, ...dist] = r.trim().split(/\s+/); return { name, baseFreq: +freq, exposureRatio: +exp, reduction: +red, setupCost: +setup, maint: +maint, transferLoading: load === "-" ? null : +load, componentDist: dist.map(Number) }; }); // RM13: asset row: name | value | exposure multiplier | cost multiplier | // annual avoidance cost ('-' the work cannot be given up) | threat mask export const ASSET = ` field-laptop 900 1.6 1.0 - 1111011 branch-endpoint 600 1.2 1.0 - 1111110 measurement-collector 750 1.4 1.5 - 0111110 processing-server 1800 0.6 0.8 - 1111101 perimeter-network 1200 1.8 0.6 - 0011100 identity-directory 2400 1.0 0.7 - 1011100 cloud-account 2000 1.5 1.4 620 1011100 subscriber-database 3000 0.4 0.9 - 1011100 field-application 1400 1.1 0.7 300 1111001 supplier-connection 1000 1.3 1.2 260 1001101` .trim().split("\n").map((r) => { const [name, v, m, bk, kac, mask] = r.trim().split(/\s+/); return { name, value: +v, exposureMult: +m, costMult: +bk, avoidCost: kac === "-" ? null : +kac, mask }; }); export const COMPONENT = ["direct", "interruption", "remediation", "notification", "reputation"]; // RM14: the transfer agreement covers the first two components and does not cover the other three. export const COVERAGE = [1, 1, 0, 0, 0]; export const ITEM = []; for (const v of ASSET) for (const t of THREAT.filter((_, j) => v.mask[j] === "1")) { const risk = +(t.baseFreq * v.exposureMult * v.value * t.exposureRatio).toFixed(2); // RM15: all costs are converted to the annual unit. Mitigation's annual cost is the setup // divided over three years plus the annual maintenance; the transfer premium is the // transferred risk times the loading. const mitCost = +(((t.setupCost / 3) + t.maint) * v.costMult).toFixed(2); const share = t.componentDist.reduce((a, x, i) => a + x * COVERAGE[i], 0) / 100; ITEM.push({ v, t, risk, mitigation: { cost: mitCost, reduction: +(risk * t.reduction).toFixed(2) }, transfer: t.transferLoading === null ? null : { cost: +(risk * share * t.transferLoading).toFixed(2), reduction: +(risk * share).toFixed(2), share }, }); } for (const k of ITEM) for (const s of ["mitigation", "transfer"]) if (k[s]) k[s].ratio = +(k[s].reduction / k[s].cost).toFixed(3); // Avoidance is an asset-level decision: it removes all of an asset's items from the list. export const AVOIDANCE = ASSET.filter((v) => v.avoidCost !== null).map((v) => { const items = ITEM.filter((k) => k.v === v); const reduction = +items.reduce((a, k) => a + k.risk, 0).toFixed(2); return { v, items, cost: v.avoidCost, reduction, ratio: +(reduction / v.avoidCost).toFixed(3) }; }); export const TOTAL = +ITEM.reduce((a, k) => a + k.risk, 0).toFixed(2); ``` #### The Same Budget, Four Strategies ```js // risk/race.mjs — four options run under the same budget; the accepted remainder is counted separately import { ITEM, AVOIDANCE, TOTAL, COMPONENT, COVERAGE } from "./option.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const TRANSFERABLE = ITEM.filter((k) => k.transfer); const candidate = (L, field) => L.map((k) => ({ ...k[field], option: field, items: [k] })); const BEST = ITEM.map((k) => (k.transfer && k.transfer.ratio > k.mitigation.ratio ? { ...k.transfer, option: "transfer", items: [k] } : { ...k.mitigation, option: "mitigation", items: [k] })); const MIXED = [...BEST, ...AVOIDANCE.map((x) => ({ ...x, option: "avoidance" }))]; // The budget is spent in order of ratio; an item that has been taken is not handled a second time. const run = (candidates, budget) => { const taken = new Set(); const chosen = []; let spent = 0, reduced = 0; for (const a of [...candidates].sort((x, z) => z.ratio - x.ratio)) if (!a.items.some((k) => taken.has(k)) && spent + a.cost <= budget) { a.items.forEach((k) => taken.add(k)); spent += a.cost; reduced += a.reduction; chosen.push(a); } return { chosen, taken, n: chosen.length, spent: +spent.toFixed(2), reduced: +reduced.toFixed(2), accepted: +(TOTAL - reduced).toFixed(2) }; }; console.log(`${ITEM.length} items, total annual risk ${TOTAL} fictional units`); const A = [-10, 9, 10, 10, 10, 9]; console.log("\n1. applicability and ratio per option (risk reduced / annual cost)"); y(A, "option", "units", "lowest", "highest", "average", "decision"); for (const [option, L, f, unit] of [["mitigation", ITEM, (k) => k.mitigation.ratio, "item"], ["transfer", TRANSFERABLE, (k) => k.transfer.ratio, "item"], ["avoidance", AVOIDANCE, (x) => x.ratio, "asset"]]) { const o = L.map(f); y(A, option, L.length, Math.min(...o).toFixed(3), Math.max(...o).toFixed(3), (o.reduce((a, x) => a + x, 0) / o.length).toFixed(3), unit); } console.log("acceptance 47 items, cost 0, risk reduced 0: it is the default for every item not chosen"); console.log("items where transfer's ratio beats mitigation's: " + `${TRANSFERABLE.filter((k) => k.transfer.ratio > k.mitigation.ratio).length}`); const B = [-22, 7, 10, 13, 13, 12]; console.log("\n2. 300-unit annual budget, four strategies"); y(B, "strategy", "chosen", "spent", "risk reduced", "accepted", "remaining %"); for (const [label, L] of [["mitigation only", candidate(ITEM, "mitigation")], ["transfer only", candidate(TRANSFERABLE, "transfer")], ["avoidance only", AVOIDANCE], ["mixed", MIXED]]) { const r = run(L, 300); y(B, label, r.n, r.spent.toFixed(2), r.reduced.toFixed(2), r.accepted.toFixed(2), ((r.accepted / TOTAL) * 100).toFixed(1)); } const C = [-13, 12, 8, 15, 8]; console.log("\n3. what transfer takes over and does not, by component (38 transferable items)"); y(C, "component", "coverage", "items", "units carried", "share %"); const carried = (i) => +TRANSFERABLE.reduce((a, k) => a + k.risk * k.t.componentDist[i] / 100, 0).toFixed(2); for (const [i, name] of COMPONENT.entries()) y(C, name, COVERAGE[i] ? "transferred" : "retained", TRANSFERABLE.length, carried(i).toFixed(2), ((carried(i) / TOTAL) * 100).toFixed(1)); const transferred = COMPONENT.reduce((a, _, i) => a + COVERAGE[i] * carried(i), 0); const total = ITEM.length * COMPONENT.length, moved = TRANSFERABLE.length * COVERAGE.filter(Boolean).length; console.log(`${total} component items: ${moved} are transferred, ${total - moved} remain; ` + `transferred ${transferred.toFixed(2)}, remaining outside ${(TOTAL - transferred).toFixed(2)} units`); const D = [-8, 7, 10, 13, 11, 9, 10]; console.log("\n4. as the mixed strategy's budget grows: which option enters the list"); y(D, "budget", "chosen", "spent", "risk reduced", "mitigation", "transfer", "avoidance"); for (const b of [300, 600, 900, 1200]) { const r = run(MIXED, b); const count = (option) => r.chosen.filter((a) => a.option === option).length; y(D, b, r.n, r.spent.toFixed(2), r.reduced.toFixed(2), count("mitigation"), count("transfer"), count("avoidance")); } const r300 = run(MIXED, 300); const unopened = ITEM.filter((k) => !r300.taken.has(k)); const unopenedRisk = +unopened.reduce((a, k) => a + k.risk, 0).toFixed(2); const largestSingle = unopened.reduce((a, k) => (k.v.value * k.t.exposureRatio > a.v.value * a.t.exposureRatio ? k : a)); console.log(`\n5. accepted under the 300-unit mixed strategy: ${r300.accepted.toFixed(2)} units`); console.log(`${unopened.length} items never opened, ${unopenedRisk} units; the ${r300.n} items handled ` + `have a remainder of ${(r300.accepted - unopenedRisk).toFixed(2)} units`); console.log(`largest single loss among the unopened: ${largestSingle.v.name} / ${largestSingle.t.name}, ` + `annual ${largestSingle.risk.toFixed(2)}, single ${(largestSingle.v.value * largestSingle.t.exposureRatio).toFixed(1)} units`); const F = [-22, 8, 7, 10, 9]; console.log("\n6. why avoidance is never chosen (1,200-unit budget)"); y(F, "asset", "ratio", "items", "preempted", "cost"); const r1200 = run(MIXED, 1200); for (const x of AVOIDANCE) y(F, x.v.name, x.ratio.toFixed(3), x.items.length, x.items.filter((k) => r1200.chosen.some((a) => a.ratio > x.ratio && a.items.includes(k))).length, x.cost); ``` ``` 47 items, total annual risk 5768.61 fictional units 1. applicability and ratio per option (risk reduced / annual cost) option units lowest highest average decision mitigation 47 0.344 36.000 6.286 item transfer 38 0.556 0.833 0.686 item avoidance 3 1.305 2.187 1.720 asset acceptance 47 items, cost 0, risk reduced 0: it is the default for every item not chosen items where transfer's ratio beats mitigation's: 5 2. 300-unit annual budget, four strategies strategy chosen spent risk reduced accepted remaining % mitigation only 22 295.40 2746.71 3021.90 52.4 transfer only 4 291.60 243.00 5525.61 95.8 avoidance only 1 300.00 656.04 5112.57 88.6 mixed 22 295.40 2746.71 3021.90 52.4 3. what transfer takes over and does not, by component (38 transferable items) component coverage items units carried share % direct transferred 38 1008.57 17.5 interruption transferred 38 1252.34 21.7 remediation retained 38 803.40 13.9 notification retained 38 463.58 8.0 reputation retained 38 489.92 8.5 235 component items: 76 are transferred, 159 remain; transferred 2260.91, remaining outside 3507.70 units 4. as the mixed strategy's budget grows: which option enters the list budget chosen spent risk reduced mitigation transfer avoidance 300 22 295.40 2746.71 22 0 0 600 36 595.93 3386.45 36 0 0 900 45 887.04 3579.51 42 3 0 1200 47 944.39 3611.37 42 5 0 5. accepted under the 300-unit mixed strategy: 3021.90 units 25 items never opened, 1699.81 units; the 22 items handled have a remainder of 1322.09 units largest single loss among the unopened: subscriber-database / insider-misuse, annual 48.00, single 1200.0 units 6. why avoidance is never chosen (1,200-unit budget) asset ratio items preempted cost cloud-account 1.669 4 3 620 field-application 2.187 5 3 300 supplier-connection 1.305 4 2 260 ``` The numbers are in the **measurement** class; its inputs are RM12–RM16. #### Where the Ratios Split The first table compares the four options in a single column. Mitigation's ratio ranges from 0.344 to 36.000, averaging 6.286. Transfer's ratio is between 0.556 and 0.833, and **none of it ever passes 1**. This is not a flaw, it is the definition itself: the premium is the transferred risk times a loading, and the loading is greater than one. In an expected-value sense, transfer inevitably costs more than it returns. The same row also shows what transfer is actually for: transfer buys **variance**, not the average, and the ratio column does not measure variance. Avoidance's ratio is between 1.305 and 2.187 and applies to only three assets. Acceptance's ratio is undefined; acceptance looks like an option, but it is **the default for every item that is not chosen**. #### Same Budget, Four Outcomes The second table spends the 300-unit annual budget on four strategies. Mitigation alone makes **22 decisions** and reduces risk by **2,746.71 units**; 52.4 percent of the total remains accepted. Transfer alone makes **4 decisions** with the same budget and reduces 243.00 units. Avoidance alone makes a single decision, spends all 300 units on one item, and reduces 656.04 units. The mixed strategy takes the best-ratio option for every item, and its outcome is **identical** to mitigation alone. Transfer's ratio beats mitigation's in five items, but since the highest of those five is 0.833, the budget never reaches them. **An option that is better for a given item never shows up in the budget unless it reaches the top of the list.** The fourth table looks for the threshold: at 600 units, all 36 decisions are still mitigation; at 900 units, the first **three transfer decisions** enter; at 1,200 units, there are five. As the decision count rises from 45 to 47, the risk reduced grows by only 31.86 units. #### What Transfer Does Not Hand Over The third table counts transfer's coverage as component items. An incident's loss splits into five parts: direct loss, interruption, remediation effort, notification obligation and reputation. The transfer agreement takes over the first two and does not take the other three. The count comes out in two forms. As items, 47 items across five components produce **235 component items**; the agreement takes over **76 of them**, and **159** stay with the organization. As units, the transferred amount is 2,260.91, and the amount left outside is **3,507.70 units**. The remaining three components share something in common: none of them closes by paying money. Remediation effort is the organization's own time, notification is the organization's own task, reputation is not bought back with a price. **Transfer hands over a loss, not a task.** #### Avoidance's Inability to Enter the Ranking The sixth table gives this lesson's structural finding. Avoidance's ratio, across three assets, is between 1.305 and 2.187 and is higher than every transfer ratio; yet even under a 1,200-unit budget, **no avoidance decision is ever chosen**. The reason is in the count: avoidance is a single decision that closes all of an asset's items at once, but the ranking is done at the item level. **Three** of field-application's five items are already taken by item-level decisions with a ratio higher than avoidance's 2.187; the same holds for three of four in cloud-account and two of four in supplier-connection. The ranking does not eliminate avoidance, it makes it **unreachable**. What is wrong is not the ranking itself but **at which level** it is done: an asset-level option can only be compared against an asset-level list. #### Acceptance's Silent Share The fifth line writes out what is left after the budget runs out. Of the 3,021.90 accepted units, **1,699.81 comes from 25 items that were never opened**, and 1,322.09 comes from the remainder of 22 items that were handled. The two numbers are written separately because they are separate questions: the first is "we did not look", the second is "we looked and it was not enough". Among the items never opened, the one carrying the largest single loss is insider misuse on the subscriber database: an annual **48.00 units**, with a single loss of **1,200.0 units**. Because its annual value is small, both rankings push it to the back; yet it is exactly the item where the transfer agreement would do the most good in terms of variance. This is why the choice cannot be made from a single column: the ratio carries the average, transfer carries the tail, and acceptance carries whatever is left outside both. #### Summary - The four options cannot be compared until they are converted to a common annual unit: the setup cost is divided by three and the maintenance is added, the premium and the given-up work value are already annual, and acceptance's cost is zero. - Transfer's ratio is between 0.556 and 0.833 and never passes 1; transfer buys variance, not the average. - Under a 300-unit budget, mitigation alone reduces 2,746.71 units with 22 decisions, transfer alone reduces 243.00 with 4 decisions, and avoidance alone reduces 656.04 with a single decision; the mixed strategy produces the same outcome as mitigation alone. - The transfer agreement takes over 76 of 235 component items, leaving 159 with the organization; 2,260.91 units are transferred, 3,507.70 remain outside, and the remaining three components do not close by paying money. - Avoidance is an asset-level decision, and even though its ratio is high, it is never chosen even under a 1,200-unit budget: several of the asset's items have already been taken by higher-ratio item-level decisions. - Of the 3,021.90 accepted units, 1,699.81 comes from 25 items that were never opened; the largest among the unopened items is the one recording an annual 48.00 units with a single loss of 1,200.0 units. #### Next Step Across three lessons, the list stayed fixed: 47 items, ten assets, seven threats. Every ranking and every budget assumed this list was **complete**. Yet every item was derived from an asset record, and something kept those records: an inventory. A server that is not in the inventory, or a service a team stood up on its own, never enters a single pairing, is never written into a ranking, and takes no place in any budget — it does not even count as accepted, because acceptance requires at least a record. The next lesson counts the items left out of the inventory and measures a single question: how many positions in the ranking would change if these items entered the list. ### Asset and Inventory Management Source: https://academia.sh/en/courses/governance-risk-and-compliance/asset-and-inventory-management When six off-inventory assets are added to the list, the 47-item list grows to 69 items and the 5,768.61-unit total rises to 8,045.45 fictional units: off-inventory items carry 28.3 percent of the true risk, 41 recorded items change rank, the largest shift is 22 positions, six shadow items enter the top twenty, 8 of the 30 shadow items can never enter any ranking because their value was never written, and the shadow item that rises to the highest rank comes from an asset no discovery source has ever seen. Across three lessons the same list was ranked, measured and budgeted: 47 items, ten assets, seven threats. Every calculation assumed this list was complete. Yet the list was derived from an inventory, and an inventory is a set of records — an asset that was never recorded cannot enter the list. An unrecorded asset's risk is not zero; it is unwritten. It is invisible in the ranking, takes no place in the budget, is not even counted as accepted — because acceptance requires at least a record. This lesson counts that gap: how many assets sit outside the inventory, how many items they produce, how many ranking positions would change if they entered the list, and what the unit cost of finding them is. #### Left Outside the Inventory An off-inventory asset is not usually a rule violation; more often it is the residue of ordinary operations: a server spun up for a test and never torn down, a cloud account a team opened on its own budget, a transfer service the field team set up itself, a branch endpoint that was decommissioned but never unplugged, a second supplier connection, a backup data copy. Each one exists because it served some purpose, and none of them was set up with updating the inventory in mind. The way to find them is discovery sources, and each source looks from a different angle. Network scanning sees what is addressable, identity records see what has an account, billing records see what generates a cost, the configuration repository sees what is defined, the endpoint agent sees what has software running on it. An asset is as findable as the number of sources it appears in; if it appears in none, it cannot be found. - **RM17.** The seven threats and their coefficients are the same as in the previous lessons; the unit is the **fictional unit**. - **RM18.** The five discovery sources, their costs and source masks are a model; no real scan is performed. - **RM19.** The six off-inventory assets are a model. The `?` mark shows that the asset has been found but its value has not been written. - **RM20.** An item's risk can only be calculated if the asset's value has been written; an asset whose value is unwritten cannot enter any ranking even if it is discovered. - **RM21.** Discovery is one-time and puts the asset it finds into the inventory; entering the inventory does not mean the value has been written. - **RM22.** Off-inventory assets are written with higher exposure multipliers than recorded assets; the model's rationale is that maintenance and hardening work is applied only to recorded assets. ```js // risk/inventory.mjs — assets in the inventory and off-inventory assets. THIS IS A MODEL; no real // scan is performed, no real account is opened, every value is in "fictional units". // RM17: threat row: name | annual baseline frequency | exposure ratio export const THREAT = ` credential-theft 0.30 0.35 malicious-code 0.50 0.25 misconfiguration 0.80 0.15 insider-misuse 0.10 0.40 service-disruption 0.40 0.20 physical-access 0.06 0.50 supply-chain 0.08 0.45` .trim().split("\n").map((r) => { const [name, s, m] = r.trim().split(/\s+/); return { name, baseFreq: +s, exposureRatio: +m }; }); // RM18: discovery source: name | annual cost. Every asset carries a source mask. export const SOURCE = ` network-scan 35 identity-record 15 billing-record 10 config-repository 25 endpoint-agent 45` .trim().split("\n").map((r) => { const [name, b] = r.trim().split(/\s+/); return { name, cost: +b }; }); // RM19: asset row: name | value ('?' value not written) | exposure | threat mask | // source mask | in inventory. The ten assets in the inventory are already recorded. const ROW = ` field-laptop 900 1.6 1111011 11001 E branch-endpoint 600 1.2 1111110 10011 E measurement-collector 750 1.4 0111110 10010 E processing-server 1800 0.6 1111101 11011 E perimeter-network 1200 1.8 0011100 10010 E identity-directory 2400 1.0 1011100 01010 E cloud-account 2000 1.5 1011100 01100 E subscriber-database 3000 0.4 1011100 00011 E field-application 1400 1.1 1111001 00110 E supplier-connection 1000 1.3 1001101 01100 E temporary-test-server 800 1.7 1111101 11010 H team-cloud-account ? 1.6 1011100 01100 H field-transfer-service 700 1.9 1111011 10001 H retired-branch-endpoint 500 1.5 1111110 10010 H third-supplier-endpoint ? 1.4 1001101 01100 H backup-data-copy 2200 0.8 1011100 00000 H`; export const ASSET = ROW.trim().split("\n").map((r) => { const [name, v, m, mask, km, inv] = r.trim().split(/\s+/); return { name, value: v === "?" ? null : +v, exposureMult: +m, mask, sourceMask: km, inInventory: inv === "E", visible: km.includes("1") }; }); export const RECORDED = ASSET.filter((v) => v.inInventory); export const SHADOW = ASSET.filter((v) => !v.inInventory); // RM20: an item's risk can only be calculated if the asset's value has been written; an asset // whose value is unwritten cannot enter any ranking even if it is discovered. export const ALL = ASSET.flatMap((v) => THREAT .filter((_, j) => v.mask[j] === "1") .map((t) => ({ v, t, risk: v.value === null ? null : +(t.baseFreq * v.exposureMult * v.value * t.exposureRatio).toFixed(2) }))); export const REC = ALL.filter((k) => k.v.inInventory); export const SHA = ALL.filter((k) => !k.v.inInventory); export const rank = (L) => L.filter((k) => k.risk !== null) .map((k, i) => ({ k, i })).sort((a, b) => b.k.risk - a.k.risk || a.i - b.i).map((x) => x.k); export const sum = (L) => +L.reduce((a, k) => a + (k.risk || 0), 0).toFixed(2); ``` #### Counting the Two Lists ```js // risk/missing.mjs — how the ranking would change if off-inventory items entered the list import { RECORDED, SHADOW, SOURCE, ALL, REC, SHA, rank, sum } from "./inventory.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const S1 = rank(REC), S2 = rank(ALL); const position = (L, k) => L.indexOf(k) + 1; console.log(`${RECORDED.length} assets / ${REC.length} items / ${sum(REC)} fictional units are in the inventory; ` + `${SHADOW.length} off-inventory assets / ${SHA.length} items / ${sum(SHA)} units ` + `(${((sum(SHA) / sum(ALL)) * 100).toFixed(1)} percent of the true total)`); const A = [-23, 7, 7, 10, 10, 10]; console.log("\n1. off-inventory assets"); y(A, "asset", "value", "items", "risk", "sources", "status"); for (const v of SHADOW) { const ks = SHA.filter((k) => k.v === v); y(A, v.name, v.value ?? "?", ks.length, v.value === null ? "-" : sum(ks).toFixed(2), v.sourceMask.split("").filter((c) => c === "1").length, v.value === null ? "valueless" : v.visible ? "visible" : "unseen"); } const scorable = SHA.filter((k) => k.risk !== null); console.log(`${scorable.length} of the ${SHA.length} off-inventory items are scorable ` + `(${sum(scorable).toFixed(2)} units), ${SHA.length - scorable.length} cannot enter any ranking ` + `because their value was never written`); const B = [-38, 9]; console.log("\n2. what changes when the full list is ranked"); const shift = REC.map((k) => Math.abs(position(S1, k) - position(S2, k))); y(B, "inventory list / full list items", `${S1.length} / ${S2.length}`); y(B, "total risk", `${sum(REC).toFixed(2)} / ${sum(ALL).toFixed(2)}`); y(B, "recorded items with a changed rank", shift.filter((x) => x > 0).length); y(B, "largest rank shift", Math.max(...shift)); y(B, "off-inventory items in top ten", S2.slice(0, 10).filter((k) => !k.v.inInventory).length); y(B, "highest off-inventory item's rank", Math.min(...scorable.map((k) => position(S2, k)))); y(B, "off-inventory items in top twenty", S2.slice(0, 20).filter((k) => !k.v.inInventory).length); const C = [-22, -21, 10, 9, 6]; console.log("\n3. the full list's top ten"); y(C, "asset", "threat", "risk", "source", "rank"); for (const k of S2.slice(0, 10)) y(C, k.v.name, k.t.name, k.risk.toFixed(2), k.v.inInventory ? "recorded" : "shadow", position(S1, k) || "-"); const D = [-21, 7, 8, 13, 8, 9]; console.log("\n4. per discovery source (off-inventory assets only)"); y(D, "source", "cost", "assets", "risk opened", "items", "ratio"); const opened = (f) => { const vs = SHADOW.filter(f); const ks = SHA.filter((k) => vs.includes(k.v) && k.risk !== null); return { vs, ks, r: sum(ks) }; }; for (const [i, s] of SOURCE.entries()) { const a = opened((v) => v.sourceMask[i] === "1"); y(D, s.name, s.cost, a.vs.length, a.r.toFixed(2), a.ks.length, (a.r / s.cost).toFixed(3)); } const all5 = opened((v) => v.visible); const cost = SOURCE.reduce((a, s) => a + s.cost, 0); y(D, "all five combined", cost, all5.vs.length, all5.r.toFixed(2), all5.ks.length, (all5.r / cost).toFixed(3)); const blind = SHADOW.filter((v) => !v.visible); console.log(`assets no source ever sees: ${blind.length}, items ` + `${SHA.filter((k) => blind.includes(k.v)).length}, risk ` + `${sum(SHA.filter((k) => blind.includes(k.v))).toFixed(2)} units`); ``` ``` 10 assets / 47 items / 5768.61 fictional units are in the inventory; 6 off-inventory assets / 30 items / 2276.84 units (28.3 percent of the true total) 1. off-inventory assets asset value items risk sources status temporary-test-server 800 6 688.16 3 visible team-cloud-account ? 4 - 2 valueless field-transfer-service 700 6 606.48 2 visible retired-branch-endpoint 500 6 375.00 2 visible third-supplier-endpoint ? 4 - 2 valueless backup-data-copy 2200 4 607.20 0 unseen 22 of the 30 off-inventory items are scorable (2276.84 units), 8 cannot enter any ranking because their value was never written 2. what changes when the full list is ranked inventory list / full list items 47 / 69 total risk 5768.61 / 8045.45 recorded items with a changed rank 41 largest rank shift 22 off-inventory items in top ten 1 highest off-inventory item's rank 7 off-inventory items in top twenty 6 3. the full list's top ten asset threat risk source rank cloud-account misconfiguration 360.00 recorded 1 cloud-account credential-theft 315.00 recorded 2 identity-directory misconfiguration 288.00 recorded 3 perimeter-network misconfiguration 259.20 recorded 4 identity-directory credential-theft 252.00 recorded 5 cloud-account service-disruption 240.00 recorded 6 backup-data-copy misconfiguration 211.20 shadow - field-application malicious-code 192.50 recorded 7 identity-directory service-disruption 192.00 recorded 8 field-application misconfiguration 184.80 recorded 9 4. per discovery source (off-inventory assets only) source cost assets risk opened items ratio network-scan 35 3 1669.64 18 47.704 identity-record 15 3 688.16 6 45.877 billing-record 10 2 0.00 0 0.000 config-repository 25 2 1063.16 12 42.526 endpoint-agent 45 1 606.48 6 13.477 all five combined 130 5 1669.64 18 12.843 assets no source ever sees: 1, items 4, risk 607.20 units ``` The numbers are in the **measurement** class; its inputs are RM17–RM21. #### The Missing List's Share Six off-inventory assets produce 30 risk items, and the risk they carry is 2,276.84 fictional units. The true total is not 5,768.61 but **8,045.45 units**; the off-inventory share is **28.3 percent**. Every ranking, every budget calculation and every acceptance decision made across three lessons was given under the assumption that this share did not exist. The number matters not only because it is large but because of its **distribution**. The backup data copy alone carries 607.20 units and outweighs six of the ten assets in the inventory. The temporary test server, at 688.16 units, outweighs eight of those ten. What they have in common is a low value — but a low value multiplied by a high exposure does not produce a small number. The exposure multiplier column explains this height, and it is a deliberate choice in the model (RM22): off-inventory assets take values between 0.8 and 1.9, recorded assets between 0.4 and 1.8. The rationale is simple and self-reinforcing — hardening, patching and monitoring work is applied to the list in the inventory, and an asset that is not on the list receives none of it. Being unrecorded means not only invisibility but also **neglect**; the two effects multiply in the same direction. #### Where the Ranking Shifts The second table answers the real question. The incomplete list ranks 47 items, the full list ranks 69; **41 recorded items change rank** and the largest shift is **22 positions**. Six shadow items enter the top twenty, and one of them rises to **seventh place**. The third table names this item: misconfiguration on the backup data copy, seventh on the list at 211.20 units, and it does not exist in the inventory list **at all**. When it enters, the malicious-code item on the field application drops from seventh to eighth, and every item below it moves down one place. If the budget handles the first ten items, a program working from the incomplete list never opens this item at all — and it does not even know this is a gap, because the item is not on the list and so does not appear in the "not handled" column either. The size of the shift is tied to how flat the list's distribution is. When the risk distribution is flat, 22 new items scatter throughout it and change nearly every item's position. If there had been a sharp peak, the new items would pile up at the bottom and the top ten would not change. Here the top ten does change, which means the gap reaches the region where **decisions are made**. #### Being Found and Being Counted Are Not the Same The first table's last column writes a second distinction. Two of the six assets — the cloud account the team opened and the third supplier endpoint — appear in two separate sources, but their values were never written. These two assets produce 8 risk items, and none of those items can be scored: no value means no impact, and no impact means no risk can be calculated. **22 of the 30 shadow items are scorable, 8 are not.** This is the half of inventory work that gets missed most often. Discovery records that an asset **exists**; the inventory records **what it is** — who owns it, what work it carries, what happens if it goes down. The first can be automated, the second requires a person to make a judgment. An asset that has been found but whose value was never written ends up in the same place as an asset that was never found at all: **neither enters any ranking.** The only difference between them is that the first one's name is on a list somewhere. #### Discovery's Unit Cost The fourth table tests discovery against the course's unit of measure. Network scanning opens three assets and 1,669.64 units of risk for 35 units; its ratio of **47.704** is the highest on the list. Identity records open 688.16 units for 15 units (45.877). Billing records find two assets for 10 units and open **0.00 units** — the two assets they find have no value written, so the scorable risk they open is zero. The last row tests the sum. All five sources together find five assets for 130 units, but the risk they open is still **1,669.64 units** — the number network scanning opens on its own. The remaining four sources spend 95 more units and add nothing to the scorable risk; the two assets they add stay valueless. The return per unit falls from 47.704 to **12.843**. Adding a discovery source always finds more items, but if the item it finds cannot be scored, it does not enter the ranking and takes no place in the budget. The final line writes the lesson's harshest number: there is an asset **no source ever sees**, carrying 4 items and 607.20 units. Even if all five sources were bought, it would not be found. And the item in seventh place on the full list comes from exactly this asset. Discovery coverage is not a lid, it is a **threshold**: everything that falls below it goes unmeasured, and everything unmeasured is counted as accepted by default. #### Summary - Six off-inventory assets carry 30 items and 2,276.84 fictional units; the true total is 8,045.45 units and the off-inventory share is 28.3 percent. - When the full list is ranked, 41 recorded items change position, the largest shift is 22 places; six shadow items enter the top twenty and one rises to seventh. - A program working from the incomplete list cannot even see that it has not handled this item: because the item is not on the list, it is not in the "not handled" column either. - Being found and being counted are not the same thing. Two shadow assets appear in two sources but their value was never written; 22 of the 30 shadow items are scorable, 8 can never enter any ranking. - Network scanning gives a ratio of 47.704 for 35 units; all five sources together open the same risk for 130 units and the ratio falls to 12.843. Adding sources finds more assets, not more scorable risk. - An asset no source ever sees carries 4 items and 607.20 units; the item in seventh place on the full list comes from this asset. #### Next Step Every discovery source assumes one thing in common: the asset being sought leaves a trace in the organization's own network, its own identity system, or its own billing. A server sitting in a supplier's own systems leaves none of these traces. Network scanning never reaches it, the endpoint agent never runs on it, the configuration repository does not recognize it. All that stands in the organization's place for those assets is a contract clause and an assessment questionnaire. The next lesson measures this across a set of three suppliers: how many controls the contract states, how many the questionnaire actually proves, and how many items remain that look delegated but cannot be verified. ### Third-Party Risk Source: https://academia.sh/en/courses/governance-risk-and-compliance/third-party-risk Across three suppliers, 3 of the 16 controls written into the contract are never asked about in the survey, 6 are answered but not verified, and only 7 are verified by evidence: the 9 controls that appear delegated but cannot be verified carry 555.10 fictional units, 25.4 percent of supplier risk; when the 62-item program list is ranked by evidence instead of declaration, 51 items change position and the number of supplier items in the top ten rises from one to three. The previous lesson measured discovery sources, and all of them assumed the same thing: the asset being searched for leaves a trace in the institution's own network, its own identity system, or its own billing. A server on the supplier's own system leaves none of these traces. The network scan never reaches it, the endpoint agent never runs there, the configuration repository never recognizes it. In place of those assets, what the institution holds is two pieces of paper: a **contract clause** and an **assessment questionnaire**. The shared-responsibility split established in the Cloud Foundations course goes one step further here — there, the question was who held the responsibility; here, the question is **how it is known that the responsibility was met**. This lesson measures that across a set of three suppliers. #### Three Sets, Three Separate Counts A supplier control can exist in three separate sets, and the three are not the same thing. A control **stated in the contract** is an obligation; one of the parties has agreed to perform it. A control **answered in the survey** is a declaration; the supplier has said that it does it. A control **verified by evidence** is an observation; the institution has seen an output showing that it works. The three sets are expected to nest inside one another: the verified set is a subset of the survey set, and the survey set is a subset of the contract set. The two gaps between them are counted separately, because they are separate problems. A control that is stated in the contract but never asked about in the survey is a **scope gap**: a commitment was obtained, but it was never looked at. A control that is answered but not verified is an **evidence gap**: it was looked at, an answer was obtained, but the truth of that answer was never shown. - **RM23.** The seven threats and their coefficients are the same as in the previous lessons; the unit is the **fictional unit**. - **RM24.** The eight controls, the ratio by which each reduces risk, the threats they touch, and the verification costs are a model; there is no real contract or audit. - **RM25.** The institutional assets are the same ten-asset set from the previous lessons. - **RM26.** The three suppliers and the three masks are a model; the evidence set is written as a subset of the survey set, and the survey set as a subset of the contract set. - **RM27.** Controls apply independently: an item's remaining risk drops through the product of the controls that touch that threat. - **RM28.** Verification does not lower the risk, it **sharpens the measurement**; a verified control enters the evidence set, and the measured remaining risk is recalculated accordingly. ```js // risk/supplier.mjs — three suppliers, eight contract controls and three separate evidence // levels. MODEL; there is no real contract, no real audit and no real currency, the unit is // "fictional unit". // RM23: threat row: name | annual baseline frequency | exposure ratio export const THREAT = ` credential-theft 0.30 0.35 malicious-code 0.50 0.25 misconfiguration 0.80 0.15 insider-misuse 0.10 0.40 service-disruption 0.40 0.20 physical-access 0.06 0.50 supply-chain 0.08 0.45` .trim().split("\n").map((r) => { const [name, s, m] = r.trim().split(/\s+/); return { name, baseFreq: +s, exposureRatio: +m }; }); // RM24: control row: name | reduction ratio | affected threat mask | // evidence-request cost | on-site verification cost export const CONTROL = ` access-log-retention 0.10 1001100 6 22 multi-factor-authentication 0.35 1000000 4 18 patch-time-commitment 0.30 0110000 9 26 data-encryption 0.25 1000010 7 24 subcontractor-notification 0.15 0000001 5 30 incident-notification-time 0.12 1111111 8 20 backup-and-recovery-test 0.28 0000100 11 28 personnel-security-screening 0.20 0001001 6 34` .trim().split("\n").map((r) => { const [name, d, tm, kb, yb] = r.trim().split(/\s+/); return { name, reductionRatio: +d, threatMask: tm, evidenceCost: +kb, verificationCost: +yb }; }); // RM25: institutional asset: name | value | exposure | threat mask export const ASSET = ` field-laptop 900 1.6 1111011 branch-endpoint 600 1.2 1111110 measurement-collector 750 1.4 0111110 processing-server 1800 0.6 1111101 perimeter-network 1200 1.8 0011100 identity-directory 2400 1.0 1011100 cloud-account 2000 1.5 1011100 subscriber-database 3000 0.4 1011100 field-application 1400 1.1 1111001 supplier-connection 1000 1.3 1001101` .trim().split("\n").map((r) => { const [name, v, m, mask] = r.trim().split(/\s+/); return { name, value: +v, exposureMult: +m, mask, isSupplier: false }; }); // RM26: supplier row: name | value | exposure | threat mask | contract-stated control | // survey-answered control | evidence-verified control (eight digits, in CONTROL order) export const SUPPLIER = ` meter-maintenance-firm 1400 1.5 1101101 11101100 11101000 10001000 cloud-hosting 2600 1.3 1011101 11111110 11011110 11010010 field-workforce-firm 900 1.7 1101011 10100101 10100001 00100000` .trim().split("\n").map((r) => { const [name, v, m, mask, contract, survey, evidence] = r.trim().split(/\s+/); return { name, value: +v, exposureMult: +m, mask, contract, survey, evidence, isSupplier: true }; }); // RM27: an item's remaining risk drops through the independent application of the controls that // touch that threat. export const ALL = [...ASSET, ...SUPPLIER].flatMap((v) => THREAT .map((t, j) => ({ v, t, j })) .filter(({ j }) => v.mask[j] === "1") .map(({ t, j }) => ({ v, t, j, raw: +(t.baseFreq * v.exposureMult * v.value * t.exposureRatio).toFixed(2) }))); // level: 'contract' declared/committed control set, 'survey' survey-answered control set, // 'evidence' evidence-verified control set. export const remaining = (k, level) => { if (!k.v.isSupplier) return k.raw; const p = CONTROL.reduce((a, c, i) => (k.v[level][i] === "1" && c.threatMask[k.j] === "1" ? a * (1 - c.reductionRatio) : a), 1); return +(k.raw * p).toFixed(2); }; export const rank = (level) => ALL.map((k, i) => ({ k, i })) .sort((a, b) => remaining(b.k, level) - remaining(a.k, level) || a.i - b.i).map((x) => x.k); export const sum = (L, level) => +L.reduce((a, k) => a + remaining(k, level), 0).toFixed(2); export const position = (L, k) => L.indexOf(k) + 1; ``` #### Counting the Three Evidence Levels ```js // risk/verify.mjs — controls stated in the contract, answered in the survey and verified by // evidence are counted import { ALL, SUPPLIER, CONTROL, remaining, rank, sum, position } from "./supplier.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const count = (m) => [...m].filter((c) => c === "1").length; const SUP = ALL.filter((k) => k.v.isSupplier); const S = { contract: rank("contract"), survey: rank("survey"), evidence: rank("evidence") }; console.log(`${ALL.length} items; ${SUP.length} of them across ${SUPPLIER.length} suppliers. ` + `Control set is ${CONTROL.length} items, fictional unit.`); const A = [-23, 8, 7, 9, 15, 10, 12]; console.log("\n1. controls and remaining risk per supplier"); y(A, "supplier", "contract", "survey", "evidence", "by declaration", "by survey", "by evidence"); for (const v of SUPPLIER) { const ks = SUP.filter((k) => k.v === v); y(A, v.name, count(v.contract), count(v.survey), count(v.evidence), sum(ks, "contract").toFixed(2), sum(ks, "survey").toFixed(2), sum(ks, "evidence").toFixed(2)); } const t3 = (d) => sum(SUP, d); y(A, "total", SUPPLIER.reduce((a, v) => a + count(v.contract), 0), SUPPLIER.reduce((a, v) => a + count(v.survey), 0), SUPPLIER.reduce((a, v) => a + count(v.evidence), 0), t3("contract").toFixed(2), t3("survey").toFixed(2), t3("evidence").toFixed(2)); const B = [-40, 8]; console.log("\n2. delegated but unmeasurable items"); const contractT = SUPPLIER.reduce((a, v) => a + count(v.contract), 0); const surveyT = SUPPLIER.reduce((a, v) => a + count(v.survey), 0); const evidenceT = SUPPLIER.reduce((a, v) => a + count(v.evidence), 0); y(B, "contract-stated controls", contractT); y(B, "never asked in the survey", contractT - surveyT); y(B, "answered but without evidence", surveyT - evidenceT); y(B, "verified by evidence", evidenceT); y(B, "delegated but unverifiable", contractT - evidenceT); y(B, "declaration-to-evidence risk gap", (t3("evidence") - t3("contract")).toFixed(2)); y(B, "evidence gap's share of risk %", (((t3("evidence") - t3("contract")) / t3("evidence")) * 100).toFixed(1)); const C = [-40, 8]; console.log("\n3. ranking within the same program list"); const shift = ALL.map((k) => Math.abs(position(S.contract, k) - position(S.evidence, k))); y(C, "items with a changed rank", shift.filter((x) => x > 0).length); y(C, "largest rank shift", Math.max(...shift)); y(C, "supplier items in top ten (declaration)", S.contract.slice(0, 10).filter((k) => k.v.isSupplier).length); y(C, "supplier items in top ten (evidence)", S.evidence.slice(0, 10).filter((k) => k.v.isSupplier).length); y(C, "program total, declaration / evidence", `${sum(ALL, "contract").toFixed(2)} / ${sum(ALL, "evidence").toFixed(2)}`); const D = [-23, -20, 9, 9, 7, 7]; console.log("\n4. the five items that rise the most (declaration to evidence)"); for (const k of [...ALL].sort((a, b) => (position(S.contract, b) - position(S.evidence, b)) - (position(S.contract, a) - position(S.evidence, a))).slice(0, 5)) y(D, k.v.name, k.t.name, remaining(k, "contract").toFixed(2), remaining(k, "evidence").toFixed(2), position(S.contract, k), position(S.evidence, k)); const E = [-24, 8, 9, 10, 9]; console.log("\n5. the uncertainty verification closes (risk does not drop, the measurement sharpens)"); y(E, "option", "cost", "controls", "closed", "ratio"); const noEvidence = SUPPLIER.flatMap((v) => CONTROL .map((c, i) => ({ v, c, i })).filter(({ v: t, i }) => t.contract[i] === "1" && t.evidence[i] !== "1")); const surveyOnly = noEvidence.filter(({ v, i }) => v.survey[i] === "1"); // If a group is verified, those controls enter the evidence set; the measured remaining risk is // recalculated. const measure = (group) => { const before = SUPPLIER.map((v) => v.evidence); SUPPLIER.forEach((v) => { v.evidence = [...v.evidence].map((c, i) => (group.some((g) => g.v === v && g.i === i) ? "1" : c)).join(""); }); const r = sum(SUP, "evidence"); SUPPLIER.forEach((v, n) => { v.evidence = before[n]; }); return r; }; const baseline = t3("evidence"); for (const [name, group, cost] of [["renewing the survey", [], 12], ["requesting evidence", surveyOnly, surveyOnly.reduce((a, g) => a + g.c.evidenceCost, 0)], ["verifying on-site", noEvidence, noEvidence.reduce((a, g) => a + g.c.verificationCost, 0)]]) { const closed = +(baseline - measure(group)).toFixed(2); y(E, name, cost, group.length, closed.toFixed(2), (closed / cost).toFixed(3)); } ``` ``` 62 items; 15 of them across 3 suppliers. Control set is 8 items, fictional unit. 1. controls and remaining risk per supplier supplier contract survey evidence by declaration by survey by evidence meter-maintenance-firm 5 4 2 531.35 603.80 752.01 cloud-hosting 7 6 4 739.17 846.25 979.89 field-workforce-firm 4 3 1 362.99 412.49 456.71 total 16 13 7 1633.51 1862.54 2188.61 2. delegated but unmeasurable items contract-stated controls 16 never asked in the survey 3 answered but without evidence 6 verified by evidence 7 delegated but unverifiable 9 declaration-to-evidence risk gap 555.10 evidence gap's share of risk % 25.4 3. ranking within the same program list items with a changed rank 51 largest rank shift 21 supplier items in top ten (declaration) 1 supplier items in top ten (evidence) 3 program total, declaration / evidence 7402.12 / 7957.22 4. the five items that rise the most (declaration to evidence) meter-maintenance-firm credential-theft 113.51 198.45 30 9 field-workforce-firm insider-misuse 38.78 61.20 58 46 meter-maintenance-firm malicious-code 161.70 262.50 15 5 field-workforce-firm supply-chain 38.78 55.08 59 50 field-workforce-firm credential-theft 127.23 160.65 25 18 5. the uncertainty verification closes (risk does not drop, the measurement sharpens) option cost controls closed ratio renewing the survey 12 0 0.00 0.000 requesting evidence 38 6 326.07 8.581 verifying on-site 216 9 555.10 2.570 ``` The numbers are in the **measurement** class; its inputs are RM23–RM28. #### Between the Three Sets The first table places three numbers side by side: across the three suppliers there are **16** controls in the contract, **13** in the survey, and **7** in the evidence. The narrowing is not uniform. At the meter maintenance firm, two of five controls are verified; at cloud hosting, four of seven; at the field workforce firm, one of four. The supplier that commits to the most controls also provides the most evidence and has the highest ratio — four of its seven contract items. At the other two suppliers, the ratio stays below half: two in five and one in four. The second table splits the narrowing into two. **3 controls are never asked about in the survey** — they are written into the contract but fall outside the scope of the assessment questions. **6 controls are answered but without evidence** — asked, the supplier said "yes," and no output showing that "yes" was ever requested. In total there are **9 controls that appear delegated but cannot be verified**, and that is more than half of what is written into the contract. #### The Number the Unverifiable Carries The control count alone gives no magnitude; the last two lines of the second table convert it into a unit. If every contract control is assumed to work, the suppliers' remaining risk is 1,633.51 fictional units. If only the verified controls are taken to work, it is **2,188.61 units**. The **555.10-unit** gap between them, **25.4 percent** of supplier risk, is the share carried by the nine controls that cannot be verified. This number is not a loss but an **uncertainty**, and it can resolve in either direction. If the controls really work, the true risk is close to 1,633.51 units, and the institution is writing its own list higher than it needs to. If they do not, the true risk is close to 2,188.61 units, and the institution is carrying a portion it believes it has delegated. The measurement cannot tell the two cases apart, because the data that would distinguish them was never collected. This is also why the survey column stands apart: 1,862.54 units, between the two. Counting a declaration as if it were evidence does not lower the risk by 326.07 units; it only makes the table easier to read. #### Ranking by Declaration, Ranking by Evidence The third table returns to this course's central measure. When the same 62-item program list is ranked by declaration and ranked by evidence, **51 items change position**, and the largest shift is **21 places**. The top ten contains **1** supplier item by declaration and **3** by evidence. The fourth table shows which items climb. The meter maintenance firm's credential-theft item is **thirtieth** in the declaration ranking and **ninth** in the evidence ranking; its number rises from 113.51 to 198.45 units. The same firm's malicious-code item climbs from fifteenth to fifth. What the two have in common is that the controls behind the drop were answered in the survey but never verified. The result is the supplier-side counterpart of this course's rule. When the priority list is built not on who committed to what but on what was shown, **the top ten changes** — that is, where the budget goes changes. A program that runs on declaration writes supplier items into the middle of the list and spends its budget on its own endpoints instead; this is the third case in which ranking promotes the wrong item, and the cause is neither scale nor level — it is **the data source**. #### Verification's Unit Cost The fifth table tests verification against the course's unit of measure; what it measures here is not the risk that drops but the **uncertainty that closes**. Renewing the survey costs 12 units and verifies **zero** controls: asking the same question a second time produces no new observation. Requesting evidence costs 38 units, closes **six** controls, and removes 326.07 units of uncertainty; its ratio is **8.581**. Verifying on-site costs 216 units and closes **nine** controls — the entire uncertainty — but its ratio falls to **2.570**. The difference between the two lines is the management decision itself. Requesting evidence closes 58.7 percent of the uncertainty at a sixth of the cost; verifying on-site asks for 178 more units to close the remaining three controls. Those remaining three controls are exactly **the ones never asked about in the survey**, and the only way to close them is to look on-site — because the supplier has declared nothing about them, there is not even an answer to refute. #### Summary - A supplier control can exist in three separate sets: an obligation stated in the contract, a declaration answered in the survey, an observation verified by evidence. Across the three suppliers the numbers are 16, 13, and 7. - The narrowing comes from two separate gaps: 3 controls are never asked about in the survey (the scope gap), and 6 controls are answered but not verified (the evidence gap). In total, 9 controls appear delegated but cannot be verified. - The nine unverifiable controls carry 555.10 fictional units, 25.4 percent of supplier risk. This is not a loss but an uncertainty, and it can resolve in either direction; the measurement cannot tell the two cases apart. - When the same 62-item list is ranked by declaration and by evidence, 51 items change position, the largest shift is 21 places, and the number of supplier items in the top ten rises from one to three. - One item climbs from thirtieth place to ninth; the controls behind the drop were answered in the survey but never verified. What changes the ranking is not scale or level but the data source. - Renewing the survey verifies zero controls at 12 units; requesting evidence closes six controls and 58.7 percent of the uncertainty at 38 units (ratio 8.581); verifying on-site closes all of it at 216 units (ratio 2.570). #### Next Step Across five lessons, a single list was built, ranked on two scales, raced against four options, completed with off-inventory items, and finally reordered by evidence level. What exists now is a countable priority list, a risk reduction per unit of effort, and a cost for every decision. The one thing missing is a language that can explain all of this **to someone looking in from outside**. This lesson demonstrates that itself. The institution asked its three suppliers for evidence and defined for itself what would count as evidence: eight controls, three masks, one cost list. The same question is put to the institution as well — whose is the subscriber? whose is the auditor? — and the institution has no shared vocabulary to give them. Every institution that tries to prove its own ranking is correct first invents its own control names, its own maturity levels, and its own form of evidence; the result is a table in which no two institutions say the same thing. The next topic, **Frameworks and Compliance**, takes up that shared language: how control sets are structured, which control satisfies a given requirement, and how many items the evidence for that satisfaction must produce. ## Frameworks Translating an organization's own measurement into an externally readable common language, and the cost of that language in terms of evidence production: the maturity calculation of a function-based framework, the certifiable management system cycle and the items a narrowed scope leaves out, the requirements left without evidence and the controls that satisfy more than one requirement in a control-to-requirement matrix, the extra controls brought in by sector-specific requirements tied to data type, the per-system search cost of data subject rights, which level counts as evidence in an audit within the policy/standard/procedure hierarchy, and the findings that independent assurance's separation of interests brings to light. ### Security Frameworks Source: https://academia.sh/en/courses/governance-risk-and-compliance/security-frameworks The structure of a function-based framework and a maturity calculation on a fictional control set: an overall maturity of 1.35 across 5 functions, 14 categories, and 31 control items; 5 controls sitting at level 0 and one category with no working control at all; 15 of 31 controls producing evidence for 25 evidence items; a 45 person-day budget where gap priority and yield priority rank all five functions differently; and a yield-first order that raises the average by 0.07 while leaving the weakest function 0.33 behind. The previous topic left behind a list: items were ranked, options were pitted against each other by cost, and the case for spending a scarce budget on one item over another was made with numbers. That list has a flaw invisible from inside the organization — it is written entirely in the organization's own language. The organization chose the metrics, the organization assigned the weights, the organization produced the ranking. When an outsider — a buyer, an insurer, or another organization purchasing a service — says "show me your security posture," this list is not an answer: there is no common axis on which the other party can lay it next to their own list. A **security framework** is that axis. It is a common classification that breaks down the security work an organization does into countable headings, independent of who does it or with what tool. This lesson treats the framework by its **structure** and ties it to a single question: when the same control set is placed into this structure, what number comes out, what does that number show, and what does it hide. No framework, standard, or organization name appears anywhere in this lesson; what is measured is not a name but the structure itself. #### Function, Category, Control A **function-based framework** has three layers. At the top sits the **function**, which splits security work into five sequential headings: **identify** (what do I have, what risk am I carrying), **protect** (what narrows access), **detect** (how do I see what is happening), **respond** (what do I do once I see it), and **recover** (how do I get back on my feet afterward). Each function is split into **categories**, and each category into individual **controls**. The point of this breakdown is that the work two different organizations do with two different tools falls under the same five headings, so the two organizations become comparable in the same table. The second layer is the **maturity level**, and it measures not whether a control exists but **how institutionalized it is**. This distinction is the lesson's central concept: a control existing and a control operating in an auditable way are not the same thing. The model below uses five levels — nonexistent, performed without documentation, documented, measured, and continuously improved. - **FW1.** The example organization is the fictional **regional measurement network** carried through nine courses: endpoints, network, identity, leased accounts, and three suppliers. The control set and levels are **fictional**. - **FW2.** The framework consists of five functions, fourteen categories, and thirty-one control items. - **FW3.** The maturity level ranges from 0 to 4: 0 nonexistent, 1 performed without documentation, 2 documented, 3 measured, 4 continuously improved. - **FW4.** Category maturity is the **unweighted average** of its controls' maturity, and function maturity is the average of its categories; overall maturity is the average of the five functions. - **FW5.** Raising a control by one level costs `2 + 3 × current level` person-days: climbing out of a low level is cheap, carrying a high level is expensive. - **FW6.** Evidence production depends on the level: from level 2 onward, one document is produced; from level 3 onward, two more measurement log entries are produced. Levels 0 and 1 produce no evidence item at all. ```js // framework/maturity.mjs — calculating the maturity level in a function-based framework. // MODEL: regional measurement network (fictional); the control set and levels are fictional. export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); export const gen = (B) => (...h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); // function | category | maturity levels of the controls in the category (0-4) export const CATEGORY = row(` identify|asset inventory|3 2 1 identify|business context|2 2 identify|risk assessment|3 2 protect|access control|3 3 2 protect|data security|3 2 2 protect|maintenance and patching|2 1 detect|continuous monitoring|2 1 1 detect|event detection|1 0 respond|response plan|2 1 respond|notification|1 0 respond|mitigation|1 1 recover|recovery plan|1 0 recover|improvement|0 0 recover|communications|1`) .map(([fn, name, d]) => ({ fn, name, level: d.split(" ").map(Number) })); export const FUNCTION = ["identify", "protect", "detect", "respond", "recover"]; export const categoriesOf = (i) => CATEGORY.filter((k) => k.fn === i); export const catAvg = (k) => k.level.reduce((a, b) => a + b, 0) / k.level.length; export const fnAvg = (i) => categoriesOf(i).reduce((a, k) => a + catAvg(k), 0) / categoriesOf(i).length; // Raising a control by one level costs 2 + 3 x current level person-days (fictional). export const days = (d) => 2 + 3 * d; // Evidence: one more document from level 2 onward, two more measurement log entries from level 3 onward. export const evidence = (d) => (d >= 2 ? 1 : 0) + (d >= 3 ? 2 : 0); if (import.meta.url === `file://${process.argv[1]}`) { const control = CATEGORY.flatMap((k) => k.level); console.log(`model: ${FUNCTION.length} functions, ${CATEGORY.length} categories, ` + `${control.length} control items`); const y = gen([10, 9, 8, 10, 30, 9]); console.log("\n" + y("function", "category", "control", "maturity", "weakest", "level 0")); for (const i of FUNCTION) { const c = categoriesOf(i), d = c.flatMap((k) => k.level); const weakest = c.reduce((a, k) => (catAvg(k) < catAvg(a) ? k : a)); console.log(y(i, c.length, d.length, fnAvg(i).toFixed(2), `${weakest.name} ${catAvg(weakest).toFixed(1)}`, d.filter((x) => x === 0).length)); } const all = CATEGORY.flatMap((k) => k.level); const overallAvg = FUNCTION.reduce((a, i) => a + fnAvg(i), 0) / FUNCTION.length; const noWorking = CATEGORY.filter((k) => k.level.every((d) => d === 0)); console.log(`\noverall maturity ${overallAvg.toFixed(2)}; level 0 controls ` + `${all.filter((d) => d === 0).length}/${all.length}; ` + `categories with no working control ${noWorking.length} ` + `(${noWorking.map((k) => k.name).join(", ")})`); console.log(`controls producing evidence ${all.filter((d) => d >= 2).length}/${all.length}; ` + `evidence items produced ${all.reduce((a, d) => a + evidence(d), 0)}`); // Gain from raising one level toward function maturity: 1 / (categories x controls). const z = gen([10, 9, 14, 33, 11]); console.log("\n" + z("function", "maturity", "maturity rank", "cheapest +1", "yield rank")); const yieldList = FUNCTION.map((i) => { const c = categoriesOf(i); const best = c.flatMap((k) => k.level.map((d) => ({ name: k.name, gain: 1 / (c.length * k.level.length) / days(d), d }))) .reduce((a, b) => (b.gain > a.gain ? b : a)); return { i, avg: fnAvg(i), best }; }); const sM = [...yieldList].sort((a, b) => a.avg - b.avg).map((v) => v.i); const sY = [...yieldList].sort((a, b) => b.best.gain - a.best.gain).map((v) => v.i); for (const v of yieldList) { console.log(z(v.i, v.avg.toFixed(2), sM.indexOf(v.i) + 1, `${v.best.name} ${days(v.best.d)} days`, sY.indexOf(v.i) + 1)); } const same = sM.filter((i, k) => i === sY[k]).length; console.log(`\nmaturity rank: ${sM.join(" > ")}`); console.log(`yield rank : ${sY.join(" > ")}`); console.log(`functions in the same rank: ${same}/${FUNCTION.length}`); } ``` ``` model: 5 functions, 14 categories, 31 control items function category control maturity weakest level 0 identify 3 7 2.17 asset inventory 2.0 0 protect 3 8 2.17 maintenance and patching 1.5 0 detect 2 5 0.92 event detection 0.5 1 respond 3 6 1.00 notification 0.5 1 recover 3 5 0.50 improvement 0.0 3 overall maturity 1.35; level 0 controls 5/31; categories with no working control 1 (improvement) controls producing evidence 15/31; evidence items produced 25 function maturity maturity rank cheapest +1 yield rank identify 2.17 4 asset inventory 5 days 5 protect 2.17 5 maintenance and patching 5 days 4 detect 0.92 2 event detection 2 days 1 respond 1.00 3 notification 2 days 2 recover 0.50 1 recovery plan 2 days 3 maturity rank: recover > detect > respond > identify > protect yield rank : detect > respond > recover > protect > identify functions in the same rank: 0/5 ``` The first table does the framework's real job: thirty-one scattered control items collapse into five numbers, and the five numbers are **comparable**. The organization's distribution is skewed — protect and identify sit at 2.17, recover at 0.50. This is exactly what nine courses have left behind: a great deal of work has gone into narrowing access, changing what data shows when it is read, and drawing the boundary, and very little work into getting back on your feet after an event. This skew is only visible once the five headings are written out separately. Overall maturity is 1.35, and this number is misleading when it stands alone. For two reasons: first, **the average swallows the weakest link** — the recover function sits at about a third of the average, while the overall number lands somewhere in the middle. Second, the average does not distinguish between a control that is **entirely absent** and one that is **present at a low level**. The last column makes that distinction: five of the thirty-one controls are at level 0, and one category — improvement — has no working control at all. That single line produces a different decision than the sentence "our recovery maturity is 0.50," because a category being completely empty is not a slowdown, it is a **gap**. The third number opens up the evidence side and feeds the next two lessons. Fifteen of the thirty-one controls produce evidence, and twenty-five evidence items are produced in total. The sixteen controls at level 0 and 1, on the other hand, **can show nothing** in an audit: the work may be happening, but there is no item that demonstrates it happened. The maturity level being so tightly bound to evidence is no accident — the level definitions themselves are written in terms of documents and measurement log entries. The second table carries the course's measurement axis into the framework. The left side ranks the functions by maturity: recover, the weakest, comes first. The right side ranks the same functions by **maturity gain per unit of effort**. The two rankings do not agree on a single function. The reason is structural: a contribution to a function's maturity is divided by that function's number of categories and the number of controls in the category. Detect has two categories, so one upgrade there moves the needle more than one upgrade in recover — which is why recover, the lowest function, ranks third in the yield order. #### The Same Budget, Two Orders The real difference between the two rankings shows up once a fixed budget is spent. The file below raises the same control set under two separate rules: **gap first** takes, at every step, the cheapest item belonging to the lowest-maturity function; **yield first** takes the item that contributes the most to overall maturity per day. The budget is forty-five person-days and is fictional. ```js // framework/budget.mjs — the same framework raised by two different orders (fictional budget). import { CATEGORY, FUNCTION, gen, days, evidence } from "./maturity.mjs"; const BUDGET = 45; // person-days (fictional unit) const clone = () => CATEGORY.map((k) => ({ ...k, level: [...k.level] })); const avg = (K, i) => { const c = K.filter((k) => k.fn === i); return c.reduce((a, k) => a + k.level.reduce((x, d) => x + d, 0) / k.level.length, 0) / c.length; }; const overall = (K) => FUNCTION.reduce((a, i) => a + avg(K, i), 0) / FUNCTION.length; const evidenceCount = (K) => K.flatMap((k) => k.level).reduce((a, d) => a + evidence(d), 0); const zeroCount = (K) => K.flatMap((k) => k.level).filter((d) => d === 0).length; // Upgrade candidate: category, control, cost, and gain toward overall maturity. const candidates = (K) => K.flatMap((k, ki) => k.level.map((d, di) => ({ ki, di, d, k, cost: days(d), gain: 1 / (FUNCTION.length * K.filter((x) => x.fn === k.fn).length * k.level.length) })).filter((a) => a.d < 4)); function run(pick) { const K = clone(); let left = BUDGET, steps = 0; for (;;) { const eligible = candidates(K).filter((a) => a.cost <= left); if (!eligible.length) break; const a = pick(eligible, K); K[a.ki].level[a.di]++; left -= a.cost; steps++; } return { K, left, steps }; } const ROUTE = [ ["gap first", (u, K) => { const lowest = [...FUNCTION].sort((a, b) => avg(K, a) - avg(K, b))[0]; const within = u.filter((a) => a.k.fn === lowest); return (within.length ? within : u).sort((a, b) => a.cost - b.cost || a.ki - b.ki)[0]; }], ["yield first", (u) => [...u].sort((a, b) => b.gain / b.cost - a.gain / a.cost || a.ki - b.ki)[0]], ]; const y = gen([13, 9, 7, 8, 9, 9, 17]); console.log(`budget ${BUDGET} person-days; starting overall maturity ` + `${overall(clone()).toFixed(2)}, level 0 controls ${zeroCount(clone())}, ` + `evidence items ${evidenceCount(clone())}`); console.log("\n" + y("run", "raised", "left", "overall", "level 0", "evidence", "weakest function")); const result = {}; for (const [name, pick] of ROUTE) { const { K, left, steps } = run(pick); const weakest = [...FUNCTION].sort((a, b) => avg(K, a) - avg(K, b))[0]; result[name] = { K, weakest }; console.log(y(name, steps, left, overall(K).toFixed(2), zeroCount(K), evidenceCount(K), `${weakest} ${avg(K, weakest).toFixed(2)}`)); } const z = gen([11, 12, 14, 14, 12]); console.log("\n" + z("function", "baseline", "gap first", "yield first", "diff")); for (const i of FUNCTION) { const a = avg(result["gap first"].K, i), b = avg(result["yield first"].K, i); console.log(z(i, avg(clone(), i).toFixed(2), a.toFixed(2), b.toFixed(2), (b - a).toFixed(2))); } const v = result["yield first"].K; const untouched = FUNCTION.filter((i) => avg(v, i) === avg(clone(), i)); console.log(`\nyield first never raised ${untouched.length} function(s): ` + `${untouched.join(", ")}`); console.log(`overall maturity difference ${(overall(v) - overall(result["gap first"].K)).toFixed(2)}; ` + `weakest function difference ${(avg(v, result["yield first"].weakest) - avg(result["gap first"].K, result["gap first"].weakest)).toFixed(2)}`); ``` ``` budget 45 person-days; starting overall maturity 1.35, level 0 controls 5, evidence items 25 run raised left overall level 0 evidence weakest function gap first 12 0 1.77 0 32 detect 1.50 yield first 11 2 1.83 0 32 respond 1.17 function baseline gap first yield first diff identify 2.17 2.17 2.17 0.00 protect 2.17 2.17 2.33 0.17 detect 0.92 1.50 1.83 0.33 respond 1.00 1.50 1.17 -0.33 recover 0.50 1.50 1.67 0.17 yield first never raised 1 function(s): identify overall maturity difference 0.07; weakest function difference -0.33 ``` Both rows spend the same forty-five days, and both bring the five level-0 controls up out of zero; evidence items rise from twenty-five to thirty-two in both. Where they diverge is the last two columns. The yield order carries overall maturity to 1.83, while the gap order stays at 1.77 — a difference of 0.07, and read on its own, the yield order looks like the winner. But look at the weakest function and the picture reverses: the gap order brings the weakest function up to 1.50, the yield order leaves it at 1.17. The difference is 0.33 — roughly five times what was gained on average piles up as a loss in the weakest link. The table below shows why. The yield order leaves the respond function 0.33 behind the gap order and never touches the identify function at all. The rule does this knowingly: an upgrade in identify costs five days and moves little, one in detect costs two days and moves a lot. **The yield measure does not ask how critical an item is, only how much it moves the metric.** For this lesson, the conclusion is structural: once a maturity number becomes a target, the order that raises the number most cheaply and the order that closes the gap diverge, and the same budget produces two different organizations. A third observation follows from this, and it concerns the framework itself. The unweighted average (FW4) is a choice, not an inevitability: a function with fewer categories moves more per item. The rule that constructs the metric also decides which item the metric favors. If a framework report shows only five numbers, how those five numbers were assembled must also be asked. #### Summary - A function-based framework has three layers — function, category, control — and reduces thirty-one scattered items to five comparable numbers: identify and protect at 2.17, respond at 1.00, detect at 0.92, recover at 0.50. - Overall maturity is 1.35 and hides two things: it swallows the weakest function, and it does not distinguish "absent" from "low." Five of the thirty-one controls are at level 0, and one category has no working control at all. - Evidence production depends on the level: only fifteen of the thirty-one controls produce evidence, yielding twenty-five items in total; the sixteen controls at level 0 and 1 can show nothing in an audit. - Ranking the functions by maturity and ranking them by gain per unit of effort do not agree on a single one of the five functions; recover, the lowest, ranks third in the yield order. - On a forty-five person-day budget, the yield order raises overall maturity by an extra 0.07 but leaves the weakest function 0.33 behind and never touches one function at all: raising the number and closing the gap are not the same job. #### Next Step The framework has established a common axis and reduced scattered controls to comparable numbers. All of these numbers share one property: **the organization itself calculated them.** The organization assigned the levels, the organization took the average, the organization decided which item to raise. For an outsider, this is only one step ahead of the list from the previous topic — the language became shared, but the party producing the number did not change. The next lesson takes up the structure that takes that step: a management system model with a declared scope, a written record of which control is applied and why or why not, an internal audit, and **certification by an outside party**. What gets measured is not the certificate itself but the scope statement: narrowing the scope makes the certificate cheaper, and, by the same move, leaves a certain number of items outside it. ### Management System Standards Source: https://academia.sh/en/courses/governance-risk-and-compliance/management-system-standards Running the certifiable management system model through its scope statement, statement of applicability, internal audit, and external certification cycle: in a 23-item inventory, a narrow scope leaves 25 of 100 control items unapplied, keeps 76 percent of total risk out of the certificate, and turns 15 items into an external party; a correction capacity of 20 nonconformities per round blocks certification at broad scope; and the absolute-cost order comes out the exact reverse of the cost-per-unit-of-risk order. The previous lesson reduced scattered controls to five comparable numbers, but the organization itself calculated those numbers. The organization assigned the levels, the organization took the average. To an outside observer, this is a well-organized self-declaration; a self-declaration's evidentiary value depends on the interest of the party making it. This lesson's subject is the structure that breaks that link: the **certifiable management system standard**. It defines a four-step cycle — scope statement, statement of applicability, internal audit, external certification — and at the end of the cycle a party outside the organization makes a decision. Because the decision itself is binary ("granted" or "not granted"), the cycle's real measure is not the decision; it is the **scope**. A certificate is always issued for a scope, and the organization chooses that scope itself. This lesson counts what the scope choice hides. No standard, certification body, or program name appears. #### Scope Statement and Statement of Applicability The **scope statement** is the definition that writes down which units, which locations, and which assets the management system covers. The **statement of applicability**, in turn, lists item by item which of the standard's control items are applied and on what grounds the ones left out are excluded. Read alone, the document says nothing until the two are read together: the scope gives what is audited, and the statement of applicability gives, even within the scope, **what is not audited**. - **FW7.** The inventory is the fictional **regional measurement network** from the previous topic: 23 items, seven units, four regions. Risk scores are taken from the previous topic's ranking and are **fictional**. - **FW8.** The standard's control set is 100 items, split into 12 feature groups. A group counts as **applicable** only if at least one in-scope item carries that feature. - **FW9.** Three scope statements are compared: narrow (billing unit only), medium (the non-field items of the billing, measurement, and identity units), broad (the entire inventory). - **FW10.** An item that sits outside the scope but shares a feature with an in-scope item appears in the document as an **external party**: it has an interface, but no audit. - **FW11.** The internal audit scans all applied controls; each group has a known nonconformity rate (fictional). - **FW12.** Correction capacity is 20 nonconformities per round. The external certification audit examines 40 percent of the applied controls and denies the certificate if it finds more than two nonconformities. - **FW13.** Audit duration is `⌈items/2⌉ + ⌈applied controls/15⌉` days: a fixed scope-walking load plus a per-item review load that grows with each item. ```js // system/scope.mjs — the scope statement in a certifiable management system. // MODEL: regional measurement network (fictional); items, risk scores, and control counts are fictional. export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); export const gen = (B) => (...h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); // name | unit | region | risk score (carried from the previous topic) | features export const ITEM = row(` billing-application|billing|central|8|personal-data payment-data development backup billing-database|billing|central|9|personal-data payment-data cryptography backup payment-gateway-link|billing|leased|7|payment-data vendor cryptography subscriber-portal|billing|leased|8|personal-data remote-access cloud-account billing-report-service|billing|leased|4|personal-data cloud-account collector-service|measurement|central|7|cloud-account backup development validator-service|measurement|central|6|cloud-account development field-meter-network|measurement|field|9|field-device wireless cryptography field-handheld-terminal|measurement|field|6|field-device wireless remote-access branch-endpoint|measurement|field|5|field-device physical-facility identity-provider|identity|central|9|remote-access cryptography incident-response admin-jump-host|identity|central|8|remote-access cryptography directory-service|identity|central|7|personal-data remote-access work-order-service|field-operations|central|5|personal-data development field-crew-application|field-operations|field|4|personal-data remote-access field-device vehicle-tracking-link|field-operations|vendor|5|personal-data vendor data-center-space|facilities|central|6|physical-facility backup branch-storage|facilities|field|3|physical-facility log-collection-pipeline|security|central|7|incident-response cloud-account backup on-call-and-response|security|central|6|incident-response maintenance-vendor-access|procurement|vendor|8|vendor remote-access cloud-vendor-account|procurement|vendor|7|vendor cloud-account consultant-development-access|procurement|vendor|6|vendor development`) .map(([name, unit, region, risk, feat]) => ({ name, unit, region, risk: +risk, feat: feat.split(" ") })); // Control set: each feature triggers a control group. // feature | control count | rate found nonconforming in internal audit (percent) export const GROUP = row(` personal-data|12|25 payment-data|9|22 cryptography|8|25 remote-access|11|18 cloud-account|10|30 field-device|7|43 wireless|5|40 physical-facility|6|17 backup|8|38 incident-response|7|29 development|9|33 vendor|8|50`) .map(([feature, n, rate]) => ({ feature, n: +n, rate: +rate })); export const TOTAL_CONTROL = GROUP.reduce((a, o) => a + o.n, 0); export const TOTAL_RISK = ITEM.reduce((a, k) => a + k.risk, 0); // Three scope statements. A scope is a predicate that selects a set of items. export const SCOPE = [ ["narrow", (k) => k.unit === "billing"], ["medium", (k) => ["billing", "measurement", "identity"].includes(k.unit) && k.region !== "field"], ["broad", () => true], ]; export const within = (predicate) => ITEM.filter(predicate); export const features = (set) => new Set(set.flatMap((k) => k.feat)); export const applicable = (set) => { const f = features(set); return GROUP.filter((b) => f.has(b.feature)); }; // Boundary interface: an item outside the scope that shares a feature with an // in-scope item appears in the document as an "external party" (model assumption). export const externalParty = (set) => ITEM.filter((b) => !set.includes(b) && b.feat.some((f) => set.some((a) => a.feat.includes(f)))); if (import.meta.url === `file://${process.argv[1]}`) { console.log(`model: ${ITEM.length} items, total risk ${TOTAL_RISK}, ` + `${GROUP.length} feature groups, ${TOTAL_CONTROL} control items`); const y = gen([8, 8, 10, 12, 14, 17, 15]); console.log("\n" + y("scope", "items", "scope %", "applied", "not applied", "risk outside %", "external party")); for (const [name, predicate] of SCOPE) { const set = within(predicate); const u = applicable(set).reduce((a, b) => a + b.n, 0); const risk = set.reduce((a, k) => a + k.risk, 0); console.log(y(name, set.length, (100 * set.length / ITEM.length).toFixed(0), u, TOTAL_CONTROL - u, (100 * (TOTAL_RISK - risk) / TOTAL_RISK).toFixed(1), externalParty(set).length)); } // Groups not applicable in narrow scope, and the risk items left outside it const narrow = within(SCOPE[0][1]); const missing = GROUP.filter((b) => !features(narrow).has(b.feature)); console.log(`\ngroups not applicable in narrow scope: ` + `${missing.map((b) => `${b.feature} (${b.n})`).join(", ")}`); const outsideRisk = ITEM.filter((k) => !narrow.includes(k)).sort((a, b) => b.risk - a.risk).slice(0, 4); console.log(`the four highest-risk items outside narrow scope: ` + `${outsideRisk.map((k) => `${k.name} (${k.risk})`).join(", ")}`); } ``` ``` model: 23 items, total risk 150, 12 feature groups, 100 control items scope items scope % applied not applied risk outside % external party narrow 5 22 75 25 76.0 15 medium 10 43 82 18 51.3 11 broad 23 100 100 0 0.0 0 groups not applicable in narrow scope: field-device (7), wireless (5), physical-facility (6), incident-response (7) the four highest-risk items outside narrow scope: field-meter-network (9), identity-provider (9), admin-jump-host (8), maintenance-vendor-access (8) ``` The place to read the table is between the third and fifth columns. Narrow scope takes 22 percent of the inventory and applies **75** of the 100 control items. This disproportion is the most often misread part of a scope statement: scope drops to a fifth, yet control items shrink by only a quarter, because control items scale not with the number of items but with **feature diversity**. The billing unit alone carries personal data, payment data, cryptography, remote access, a cloud account, development, backup, and a vendor connection; it triggers eight of the twelve groups. For the party reading the document, the result is this: the sentence "seventy-five of one hundred items are applied" gives the impression of a broad scope, when in fact 78 percent of the inventory and 76 percent of the total risk sit outside it. What is left outside is not random either — two of the four highest-risk items are the field meter network and the identity provider, both with a risk score of 9. The four unapplied groups confirm this: field device, wireless, physical facility, and incident response. These four groups get stamped "not applicable" because the billing unit carries no field device and no facility at all; the stamp's grounds are formally correct, and the result is still misleading. Medium scope is not a midpoint but a different choice: it doubles the item count and moves the applied item count only from 75 to 82. The seven added items come from the incident-response group, because the identity provider is the first in-scope item to carry that feature. The control-side cost of widening the scope depends on whether the added items **bring a new feature**; adding an item that carries features already present changes the statement of applicability not at all. The last column names this fallacy. In narrow scope, **15 items count as an external party**: they have an interface with the system, but no audit. A document gathers the interfaces at the scope boundary under the heading "external party" and expects that party to prove its own controls. If the organization's own items show up as an external party in its own document, the scope boundary has split the organization in two. This is why the statement of applicability is required alongside the certificate: only that document lists the 25 items left outside and the grounds for it. #### Cycle and Correction Capacity Once the scope is chosen, the rest of the cycle runs: the internal audit scans the applied controls, what is found gets corrected, the external certification audit takes a sample, and a decision is made. ```js // system/cycle.mjs — the internal audit and external certification cycle run on the same three scopes. import { ITEM, SCOPE, TOTAL_CONTROL, TOTAL_RISK, within, applicable, externalParty, gen } from "./scope.mjs"; const CORRECTION = 20; // number of nonconformities correctable in one round (fictional capacity) const SAMPLE = 0.4; // external certification audit examines 40% of the applied controls const THRESHOLD = 2; // certificate is denied if external audit finds more than 2 nonconformities const dayCalc = (items, controls) => Math.ceil(items / 2) + Math.ceil(controls / 15); const y = gen([8, 11, 18, 12, 11, 17, 11, 13]); console.log(`${TOTAL_CONTROL} control items, ${ITEM.length} items, total risk ` + `${TOTAL_RISK}; correction capacity ${CORRECTION} nonconformities/round (fictional)`); console.log("\n" + y("scope", "applied", "internal finding", "corrected", "remaining", "external sample", "caught", "certificate")); const results = []; for (const [name, predicate] of SCOPE) { const set = within(predicate), groups = applicable(set); const applied = groups.reduce((a, b) => a + b.n, 0); const findings = groups.reduce((a, b) => a + Math.round(b.n * b.rate / 100), 0); const remaining = Math.max(0, findings - CORRECTION); const sample = Math.round(applied * SAMPLE); const caught = Math.round(remaining * SAMPLE); const risk = set.reduce((a, k) => a + k.risk, 0); const days = dayCalc(set.length, applied); results.push({ name, set, applied, findings, remaining, caught, risk, days }); console.log(y(name, applied, findings, Math.min(findings, CORRECTION), remaining, sample, caught, caught > THRESHOLD ? "not issued" : "issued")); } const z = gen([8, 11, 11, 9, 12, 15, 16]); console.log("\n" + z("scope", "covered", "outside", "days", "days / risk", "items outside", "external party")); for (const s of results) { console.log(z(s.name, s.risk, TOTAL_RISK - s.risk, s.days, (s.days / s.risk).toFixed(3), ITEM.length - s.set.length, externalParty(s.set).length)); } const cheapest = [...results].sort((a, b) => a.days - b.days).map((s) => s.name); const efficient = [...results].sort((a, b) => a.days / a.risk - b.days / b.risk).map((s) => s.name); console.log(`\nabsolute cost order : ${cheapest.join(" < ")}`); console.log(`cost per unit of risk order: ${efficient.join(" < ")}`); const [narrow, , broad] = results; console.log(`\nnarrow scope saves ${broad.days - narrow.days} audit days and ` + `${broad.findings - narrow.findings} internal findings; in exchange, ` + `${TOTAL_RISK - narrow.risk} risk points and ${TOTAL_CONTROL - narrow.applied} control items ` + `stay outside the certificate`); const external = externalParty(narrow.set).filter((k) => k.risk >= 7); console.log(`of the ${externalParty(narrow.set).length} items counted as an external party ` + `in narrow scope, ${external.length} carry a risk score of 7 or higher: ` + `${external.map((k) => k.name).join(", ")}`); console.log(`remaining nonconformities in broad scope ${broad.remaining}; for the certificate ` + `to be issued, correction capacity would have had to be ` + `${Math.ceil(broad.findings - THRESHOLD / SAMPLE)} (currently ${CORRECTION})`); ``` ``` 100 control items, 23 items, total risk 150; correction capacity 20 nonconformities/round (fictional) scope applied internal finding corrected remaining external sample caught certificate narrow 75 22 20 2 30 1 issued medium 82 24 20 4 33 2 issued broad 100 30 20 10 40 4 not issued scope covered outside days days / risk items outside external party narrow 36 114 8 0.222 18 15 medium 73 77 11 0.151 13 11 broad 150 0 19 0.127 0 0 absolute cost order : narrow < medium < broad cost per unit of risk order: broad < medium < narrow narrow scope saves 11 audit days and 8 internal findings; in exchange, 114 risk points and 25 control items stay outside the certificate of the 15 items counted as an external party in narrow scope, 8 carry a risk score of 7 or higher: collector-service, field-meter-network, identity-provider, admin-jump-host, directory-service, log-collection-pipeline, maintenance-vendor-access, cloud-vendor-account remaining nonconformities in broad scope 10; for the certificate to be issued, correction capacity would have had to be 25 (currently 20) ``` The first table gives the cycle's harshest result: **widening the scope loses the certificate.** In narrow and medium scope, correction capacity swallows the internal findings, leaving two and four nonconformities behind; the external audit's sample sees one and two of these, the threshold is not crossed, and the certificate is granted. In broad scope, internal findings rise to 30, capacity stays at 20, 10 remain, the sample turns up four nonconformities, and the certificate is not granted. The last line converts this into capacity: for broad scope to get the certificate, correction capacity would have had to be 25, not 20. This explains the most widely misunderstood part of compliance work. Narrowing the scope is not a trick; where capacity falls short, it is **the only way to obtain the certificate** — when the organization chooses to audit more items with the same capacity, the nonconformity count grows and the decision flips. The certificate measures not the level of security but **the ratio between scope and correction capacity**. The same organization, without changing a single control, becomes certified merely by shortening its scope statement. The second table ties the course's ranking axis to this decision. The absolute-cost order is narrow, medium, broad — narrow scope takes 8 days, broad scope 19. The cost-per-unit-of-risk order is exactly the reverse: narrow scope spends 0.222 days per risk point it covers, broad scope 0.127. The reason lies in the audit's fixed load; as the scope shrinks, the fixed load is divided across less risk. **The cheapest certificate is the most expensive certificate per unit of risk it covers.** The same data yields two rankings, and both are correct; which one to use depends on whether the question is "what does this certificate cost" or "how much risk does this money bring under audit." #### Summary - A certifiable management system is a four-step cycle — scope statement, statement of applicability, internal audit, external certification — and a party outside the organization makes the decision; but the organization chooses the scope. - Control items scale with feature diversity, not item count: narrow scope takes 22 percent of the inventory but applies 75 of the 100 items. The sentence "seventy-five of one hundred items" gives the impression of a broad scope. - Narrow scope's cost is three numbers: 25 control items not applied, 76 percent of total risk outside the certificate, and 15 items counted as an external party — eight of which carry a risk score of 7 or higher. - The ratio between scope and correction capacity determines the certificate decision: with a capacity of 20 nonconformities per round, narrow and medium scope get the certificate and broad scope does not; getting it would have required a capacity of 25. - The absolute-cost order (narrow, medium, broad) and the cost-per-unit-of-risk order (broad, medium, narrow) are exact reverses of each other: the cheapest certificate is the most expensive certificate per unit of risk it covers. #### Next Step Two lessons have set up two measures: the maturity level says how institutionalized a control is, the scope statement says how far the audit reaches. Both are about the control **itself**. The question asked by the party reading the certificate or the maturity report is a different one: my requirement is this — which control satisfies it, and where is the item that shows it does. This question is not a one-to-one mapping. One control can satisfy more than one requirement, one requirement can call for more than one control, and some requirements are tied to no control at all. The next lesson is the topic's most important part: it runs the matrix between control and requirement, separates the requirement satisfied by no control from the requirement that looks satisfied but has no evidence, and counts how many items it takes to prove a single control. ### Audit Control Sets Source: https://academia.sh/en/courses/governance-risk-and-compliance/audit-control-sets Running the matrix between control and requirement: in 28 controls, 25 requirements, and 33 links, only 19 requirements carry operating evidence, 2 are mapped to no control at all, 2 are mapped but produce no evidence, and 2 rest on provider attestation only; 13 requirements stand on a single operating control; 12.3 percent of the 3,205 evidence items produced in the period are reviewed; and the control-count and evidence-load rankings share only three common members in their top five. The previous two lessons measured the control itself: how institutionalized it is, and how far the audit reaches. The question asked by the party reading the certificate is different — it brings its own requirement and asks which control satisfies it. The structure that answers this question is the **audit control set**: a set of controls grouped by trust principle, proven to have operated **over a specific period**. The phrase "over a specific period" is decisive for this lesson. Showing that a control exists as a **design** is done with a document; showing that it **operated** can only be done by sampling records produced over the period. M23/K05 counted where evidence comes from in the leased layer; that distinction is an input here. The provider's document shows the design, not that it operated over the period — and that may not be enough to count a requirement as "satisfied." - **FW14.** The control set has 28 items and is split across five trust principles: security, availability, processing integrity, confidentiality, privacy. The set and the numbers are **fictional**. - **FW15.** Every control has an **evidence source**: its own record, a provider's document, or none. - **FW16.** Every control produces a certain number of **evidence items** in a period; a control producing zero cannot be sampled in an audit. - **FW17.** The requirement set has 25 items; each requirement maps to zero or more controls. The mapping is a **claim**, not evidence. - **FW18.** A requirement's status falls into four categories: **operating evidence** if it has a control carrying at least one piece of operating evidence; **attestation only** if it rests solely on a provider's document; **no evidence** if the controls it maps to produce no evidence at all; **unmapped** if it has no control at all. - **FW19.** A shared control's evidence load is divided equally among the requirements it satisfies. - **FW20.** An auditor examines at most 25 items per control and spends 0.25 person-hours compiling per item examined (fictional). ```js // matrix/mapping.mjs — the control-requirement matrix in an audit control set. // MODEL: regional measurement network (fictional); controls, requirements, and evidence counts are fictional. export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); export const gen = (B) => (...h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); // code | trust principle | evidence source (own|provider|none) | evidence items produced in the period export const CONTROL = Object.fromEntries(row(` C01|security|own|240 C02|security|own|180 C03|security|own|60 C04|security|own|48 C05|security|own|365 C06|security|own|52 C07|security|own|12 C08|security|provider|1 C09|security|provider|1 C10|security|own|4 C11|security|own|3 C12|security|own|2 C13|availability|own|365 C14|availability|own|4 C15|availability|own|365 C16|availability|provider|1 C17|availability|own|1 C18|processing-integrity|own|365 C19|processing-integrity|own|12 C20|processing-integrity|own|52 C21|processing-integrity|own|310 C22|confidentiality|own|1 C23|confidentiality|own|365 C24|confidentiality|own|365 C25|confidentiality|provider|1 C26|privacy|none|0 C27|privacy|own|30 C28|privacy|none|0`) .map(([code, principle, source, item]) => [code, { code, principle, source, item: +item }])); // requirement | controls claimed to satisfy it export const REQUIREMENT = row(` R01 blocking of unauthorized access|C01 C03 C04 R02 monitoring of privileged operations|C01 C02 R03 reflection of personnel turnover|C04 R04 continuity of endpoint protection|C05 C06 R05 closing of a vulnerability within a reasonable time|C06 R06 review of network boundary rules|C07 R07 audit of physical access|C08 R08 secure disposal of hardware|C09 R09 awareness training of personnel|C10 R10 evaluation of third parties|C11 R11 incident readiness|C12 R12 prevention of data loss|C13 C14 R13 continuity of service without interruption|C15 C16 R14 recovery after an outage|C14 C17 R15 validation of input|C18 R16 completeness of processing|C19 C20 R17 authorization of change|C21 R18 classification of data|C22 R19 compliance with the retention period|C23 R20 encryption of data|C24 C25 R21 visibility of where data flows|C26 R22 fulfillment of deletion requests|C26 C27 R23 basis for processing|C28 R24 visibility of the subprocessor chain| R25 immutability of records|`) .map(([name, k]) => ({ name, code: name.split(" ")[0], controls: k ? k.split(" ") : [] })); export const PRINCIPLE = ["security", "availability", "processing-integrity", "confidentiality", "privacy"]; // Operating evidence: an item produced from its own record over a period. A provider's // document shows the design, not that it operated over the period (model assumption). export const operating = (k) => CONTROL[k].source === "own" && CONTROL[k].item > 0; export const status = (g) => !g.controls.length ? "unmapped" : g.controls.some(operating) ? "operating evidence" : g.controls.some((k) => CONTROL[k].source === "provider") ? "attestation only" : "no evidence"; if (import.meta.url === `file://${process.argv[1]}`) { const links = REQUIREMENT.reduce((a, g) => a + g.controls.length, 0); console.log(`model: ${Object.keys(CONTROL).length} controls, ${REQUIREMENT.length} requirements, ` + `${links} links, ${PRINCIPLE.length} trust principles`); const y = gen([22, 10, 11, 15, 14, 12]); console.log("\n" + y("trust principle", "controls", "operating", "reqs. covered", "period items", "items/req.")); for (const i of PRINCIPLE) { const set = Object.values(CONTROL).filter((k) => k.principle === i); const codes = set.map((k) => k.code); const reqs = REQUIREMENT.filter((g) => g.controls.some((k) => codes.includes(k))); const item = set.reduce((a, k) => a + k.item, 0); console.log(y(i, set.length, set.filter((k) => operating(k.code)).length, reqs.length, item, reqs.length ? (item / reqs.length).toFixed(1) : "-")); } const z = gen([20, 8, 10, 11]); const STATUS = ["operating evidence", "attestation only", "no evidence", "unmapped"]; console.log("\n" + z("requirement status", "count", "percent", "controls")); for (const d of STATUS) { const k = REQUIREMENT.filter((g) => status(g) === d); console.log(z(d, k.length, (100 * k.length / REQUIREMENT.length).toFixed(0), k.reduce((a, g) => a + g.controls.length, 0))); } for (const d of STATUS.slice(1)) { console.log(`${d}: ${REQUIREMENT.filter((g) => status(g) === d) .map((g) => g.code).join(", ")}`); } // Leverage: how many requirements a control satisfies. const leverage = Object.values(CONTROL).map((k) => ({ ...k, g: REQUIREMENT.filter((x) => x.controls.includes(k.code)).length })); const many = leverage.filter((k) => k.g > 1), single = leverage.filter((k) => k.g === 1); console.log(`\ncontrols satisfying more than one requirement ${many.length}/${leverage.length}: ` + `${many.map((k) => `${k.code} (${k.g})`).join(", ")}`); console.log(`controls carrying a single requirement ${single.length}; controls tied to no ` + `requirement ${leverage.filter((k) => k.g === 0).length}`); const total = Object.values(CONTROL).reduce((a, k) => a + k.item, 0); console.log(`evidence items produced in the period ${total}; per control ` + `${(total / leverage.length).toFixed(1)}, per satisfied requirement ` + `${(total / REQUIREMENT.filter((g) => status(g) === "operating evidence").length).toFixed(1)}`); } ``` ``` model: 28 controls, 25 requirements, 33 links, 5 trust principles trust principle controls operating reqs. covered period items items/req. security 12 10 11 968 88.0 availability 5 4 3 736 245.3 processing-integrity 4 4 3 739 246.3 confidentiality 4 3 3 732 244.0 privacy 3 1 3 30 10.0 requirement status count percent controls operating evidence 19 76 29 attestation only 2 8 2 no evidence 2 8 2 unmapped 2 8 0 attestation only: R07, R08 no evidence: R21, R23 unmapped: R24, R25 controls satisfying more than one requirement 5/28: C01 (2), C04 (2), C06 (2), C14 (2), C26 (2) controls carrying a single requirement 23; controls tied to no requirement 0 evidence items produced in the period 3205; per control 114.5, per satisfied requirement 168.7 ``` The second table is the lesson's main result. Of the twenty-five requirements, only **nineteen** are in a state where operating over a period can be proven; the remaining six are unevidenced for three separate reasons, and the three reasons are nothing alike. R24 and R25 are **unmapped**: they have rows in the matrix, but the cells are empty — nobody has proposed a control. R21 and R23 are mapped, but the controls they map to produce no evidence items at all; they look filled in the matrix and stand bare in an audit. R07 and R08, meanwhile, rest on a provider's document: the design can be shown, that it operated over the period cannot. This three-way split says why reading the matrix as "how many cells are filled" is misleading. Three of the thirty-three links — R21's and R22's link to C26, and R23's link to C28 — produce no evidence at all. Going by fill rate, twenty-three of the twenty-five requirements are mapped (92 percent); going by operating evidence, nineteen are satisfied (76 percent). **A mapping is a claim; the item that turns it into evidence must be counted separately.** The leverage row gives the second observation: of the twenty-eight controls, only five satisfy more than one requirement, and twenty-three carry a single requirement. Leveraged controls buy scale in evidence production — C01 alone is enough for two requirements. But the fifth leveraged control is C26, and it produces no evidence at all: of the two requirements it satisfies, it leaves one (R21) **without evidence**, while its backup control C27 saves the other (R22). Leverage is valuable only on the condition that the control operates; a leveraged control that does not operate is a multiplier for the spread of failure. #### Evidence Load and Critical Control ```js // matrix/evidence.mjs — evidence load, critical control, and two rankings are derived from the matrix. import { CONTROL, REQUIREMENT, PRINCIPLE, operating, status, gen } from "./mapping.mjs"; const SAMPLE_CAP = 25; // an auditor examines at most 25 items per control (fictional) const COMPILE = 0.25; // 0.25 person-hours to compile per item examined (fictional) // If a control drops, which requirements lose their operating evidence const dropImpact = (code) => REQUIREMENT.filter((g) => status(g) === "operating evidence" && g.controls.filter(operating).every((k) => k === code)); const critical = Object.keys(CONTROL).map((k) => ({ k, g: dropImpact(k) })) .filter((x) => x.g.length > 0); // A shared control's evidence load is divided among the requirements it satisfies. const reqCount = (code) => REQUIREMENT.filter((g) => g.controls.includes(code)).length || 1; const load = (g) => g.controls.filter(operating).reduce((a, k) => a + CONTROL[k].item / reqCount(k), 0); const TOTAL = REQUIREMENT.reduce((a, g) => a + load(g), 0); const y = gen([48, 10, 7, 9, 10]); console.log(`${REQUIREMENT.length} requirements, total evidence load ${TOTAL.toFixed(0)} items/period`); console.log("\n" + y("highest evidence-load requirement", "controls", "items", "share %", "cumul. %")); let cumulative = 0; for (const g of [...REQUIREMENT].sort((a, b) => load(b) - load(a)).slice(0, 5)) { cumulative += 100 * load(g) / TOTAL; console.log(y(g.name, g.controls.length, load(g).toFixed(0), (100 * load(g) / TOTAL).toFixed(1), cumulative.toFixed(1))); } const zero = REQUIREMENT.filter((g) => load(g) === 0); console.log(`\nrequirements producing no evidence load ${zero.length}: ` + `${zero.map((g) => g.code).join(", ")}`); // Two rankings: by control count and by evidence load const byControlCount = [...REQUIREMENT].sort((a, b) => b.controls.length - a.controls.length || a.code.localeCompare(b.code)).map((g) => g.code); const byLoad = [...REQUIREMENT].sort((a, b) => load(b) - load(a) || a.code.localeCompare(b.code)) .map((g) => g.code); const common = byLoad.slice(0, 5).filter((k) => byControlCount.slice(0, 5).includes(k)); console.log(`top five by control count: ${byControlCount.slice(0, 5).join(", ")}`); console.log(`top five by evidence load : ${byLoad.slice(0, 5).join(", ")}`); console.log(`common: ${common.length}/5`); // Critical control and auditor workload console.log(`\ncontrols whose drop leaves at least one requirement without evidence ${critical.length}/` + `${Object.keys(CONTROL).length}; ${critical.filter((x) => reqCount(x.k) === 1).length} of these ` + `are already single-requirement controls`); const singleBacked = REQUIREMENT.filter((g) => g.controls.filter(operating).length === 1); console.log(`requirements standing on a single operating control ${singleBacked.length}/` + `${REQUIREMENT.length}: ${singleBacked.map((g) => g.code).join(", ")}`); const z = gen([22, 10, 10, 10, 12]); console.log("\n" + z("trust principle", "produced", "reviewed", "review %", "person-hrs")); for (const i of PRINCIPLE) { const set = Object.values(CONTROL).filter((k) => k.principle === i); const produced = set.reduce((a, k) => a + k.item, 0); const reviewed = set.reduce((a, k) => a + Math.min(SAMPLE_CAP, k.item), 0); console.log(z(i, produced, reviewed, produced ? (100 * reviewed / produced).toFixed(1) : "-", (reviewed * COMPILE).toFixed(1))); } const produced = Object.values(CONTROL).reduce((a, k) => a + k.item, 0); const reviewed = Object.values(CONTROL).reduce((a, k) => a + Math.min(SAMPLE_CAP, k.item), 0); console.log(`total: ${produced} items produced, ${reviewed} items reviewed (` + `${(100 * reviewed / produced).toFixed(1)}%), ${(reviewed * COMPILE).toFixed(1)} person-hours ` + `to compile`); ``` ``` 25 requirements, total evidence load 3201 items/period highest evidence-load requirement controls items share % cumul. % R04 continuity of endpoint protection 2 391 12.2 12.2 R12 prevention of data loss 2 367 11.5 23.7 R13 continuity of service without interruption 2 365 11.4 35.1 R15 validation of input 1 365 11.4 46.5 R19 compliance with the retention period 1 365 11.4 57.9 requirements producing no evidence load 6: R07, R08, R21, R23, R24, R25 top five by control count: R01, R02, R04, R12, R13 top five by evidence load : R04, R12, R13, R15, R19 common: 3/5 controls whose drop leaves at least one requirement without evidence 13/28; 11 of these are already single-requirement controls requirements standing on a single operating control 13/25: R03, R05, R06, R09, R10, R11, R13, R15, R17, R18, R19, R20, R22 trust principle produced reviewed review % person-hrs security 968 173 17.9 43.3 availability 736 56 7.6 14.0 processing-integrity 739 87 11.8 21.8 confidentiality 732 52 7.1 13.0 privacy 30 25 83.3 6.3 total: 3205 items produced, 393 items reviewed (12.3%), 98.3 person-hours to compile ``` Evidence load is concentrated in five requirements: five of the twenty-five requirements carry 57.9 percent of the total load. What they have in common is that they depend on a control that runs daily — endpoint status, backup, capacity monitoring, input validation, and the retention-period run each produce 365 items a year. By contrast, the six requirements that produce no load at all are exactly the six unevidenced requirements from above. **A requirement with zero evidence load is not cheap; it is unsatisfied.** If a compliance budget is allocated by looking only at the number of items produced, these six rows never show up at all. The top five of the two rankings overlap in only three members. Sorting by control count puts R01 and R02 first — with three and two controls, they look the most "covered." Sorting by evidence load puts R15 and R19 first instead; both have a single control, but that single control produces 365 items in the period. Control count is a measure of **coverage**, evidence load is a measure of **cost**, and the two order the same list differently. The critical-control count gives the third result: if any of thirteen of the twenty-eight controls drops, at least one requirement is left without evidence, and eleven of these are already single-requirement controls. So the fragility is concentrated not in leveraged controls but in **unbacked** ones: thirteen requirements stand on a single operating control. Losing a leveraged control usually does not drop a requirement, because another control stands beside it. The last table gives the audit's workload. 3,205 evidence items are produced in the period, the auditor reviews 12.3 percent of them — 393 items — and this requires 98.3 person-hours to compile. Because of the sample cap, high-volume controls are reviewed at a proportionally lower rate: 7.6 percent for availability, 83.3 percent for privacy. This shows that the real cost of producing evidence is not **being audited** — 2,812 of the 3,205 items produced are stored without ever being read, but they still have to be produced and stored so that they can be shown when needed. #### Summary - An audit control set proves operation over a period, not design; a provider's document shows the design, not that it operated over the period. - Nineteen of the twenty-five requirements carry operating evidence; the remaining six are unevidenced for three separate reasons: two are not mapped at all, two are mapped but produce no evidence, and two rest on attestation only. - The fill rate is 92 percent, the operating-evidence rate is 76 percent: a mapping is a claim, and the item that turns it into evidence must be counted separately. - Five of the twenty-eight controls satisfy more than one requirement; but because one of them produces no evidence at all, it leaves one of its two requirements without evidence — leverage is valuable only if the control operates. - Fragility lies in unbacked controls: if any of thirteen controls drops, a requirement is left without evidence, and thirteen requirements stand on a single operating control. - 57.9 percent of the evidence load is concentrated in five requirements; the six requirements producing no load at all are exactly the unevidenced ones. 3,205 items are produced in the period, and 12.3 percent are reviewed. #### Next Step This lesson's matrix took the requirements independent of their sources: each row was a sentence, and where it came from was never asked. Part of the requirements, though, arise not from what the organization does but from **what data it touches**. A system that processes payment card data, a system that holds health data, and a system that stores personal data carry different control lists even if they have no other difference at all. The next lesson ties the requirement to the data type: how many extra controls each type brings, how the lists stack up when a single system holds two types at once, and how many controls the harshest form of narrowing the scope — not holding the data at all — eliminates. The remaining controls that cannot be eliminated will be counted too. ### Sector-Specific Compliance Requirements Source: https://academia.sh/en/courses/governance-risk-and-compliance/sector-specific-compliance-requirements Tying compliance requirements to the data type, and the control-side counterpart of narrowing the scope: in 16 systems and 4 data types, the extra controls per type add up to 29 when summed separately and to 20 in the union; 9 multi-type systems carry a control load of 129; removing a type from every system drops 4 and 3 controls respectively, while removing it partially drops no controls at all; residual controls come back; and three separate measures order the same four actions in three different sequences. The previous lesson's matrix took the requirements independent of their sources: each row was a sentence. But what determines the length of a requirement list is often not what the organization does, but **what data it touches**. If one of two systems carrying the same architecture processes payment card data and the other does not, the two do not have equal control lists. This lesson ties the requirement to the data type. The types are referred to here **as data types** — payment card data, health data, personal data, measurement data — and no regulation, institution, or sector-association name is written down. Three things are to be measured: how many extra controls a type brings, how the lists accumulate when a single system holds two types at once, and how many controls the harshest form of narrowing the scope — not holding the data at all — actually eliminates. - **FW21.** The inventory is 16 systems taken from a fictional **regional measurement network**. The systems, data types, and control counts are **fictional**. - **FW22.** There are four data types, and each type brings a set of extra controls. The sets intersect: a control can appear in more than one type. - **FW23.** A system's control list is the **union** of the control sets of the types it carries; if the same control comes from two types, it is counted once. - **FW24.** Every control produces a certain number of evidence items in the period, and it produces this **separately for each system**; the total evidence volume is summed over system-control pairs. - **FW25.** A control disappears only once **no system carrying it** remains. Removing a type from some systems does not shorten the control list. - **FW26.** Removing the data can have a **residual control**: the boundary itself that carries the data out, or the assessment of the party it was outsourced to, comes back as a new control. ```js // sector/data.mjs — sector-specific compliance requirements are tied to the data type. // MODEL: regional measurement network (fictional); systems, controls, and counts are fictional. export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); export const gen = (B) => (...h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); // code | control | evidence items produced in the period export const CONTROL = Object.fromEntries(row(` C01|encryption at rest|365 C02|encryption in transit|365 C03|network segmentation|12 C04|validation of the tokenization boundary|52 C05|access logging|365 C06|privileged access approval|48 C07|retention-period run|365 C08|deletion request flow|30 C09|data inventory refresh|4 C10|data flow map|2 C11|consent record|120 C12|purpose-limitation review|4 C13|additional access approval for special-category data|24 C14|personnel confidentiality commitment|1 C15|additional awareness training|2 C16|vendor assessment|3 C17|periodic penetration test|1 C18|log integrity|365 C19|measurement-accuracy reconciliation|12 C20|test data masking|12`) .map(([code, name, item]) => [code, { code, name, item: +item }])); // data type | extra controls brought by that type. No regulation name is written; the type name is. export const TYPE = Object.fromEntries(row(` payment-card|C01 C02 C03 C04 C05 C06 C16 C17 C20 health|C01 C05 C07 C11 C12 C13 C14 C15 personal|C01 C02 C05 C07 C08 C09 C10 C11 C12 measurement|C05 C18 C19`).map(([t, k]) => [t, k.split(" ")])); // system | data types it carries export const SYSTEM = row(` billing-application|payment-card personal billing-database|payment-card personal payment-gateway-link|payment-card subscriber-portal|personal call-center-recording|personal health field-crew-app|personal health occupational-health-record|health aggregator-service|measurement validator-service|measurement field-meter-network|measurement reporting-warehouse|payment-card personal measurement backup-store|payment-card personal health measurement log-collection-pipeline|personal measurement identity-provider|personal archive-store|personal health test-environment|payment-card personal`).map(([name, t]) => ({ name, type: t.split(" ") })); export const controlsFor = (types) => new Set(types.flatMap((t) => TYPE[t])); export const required = (systems) => controlsFor([...new Set(systems.flatMap((s) => s.type))]); if (import.meta.url === `file://${process.argv[1]}`) { const typeNames = Object.keys(TYPE); console.log(`model: ${SYSTEM.length} systems, ${typeNames.length} data types, ` + `${Object.keys(CONTROL).length} controls`); const y = gen([14, 9, 16, 8, 8, 16]); console.log("\n" + y("data type", "systems", "extra controls", "unique", "shared", "evidence items")); for (const t of typeNames) { const s = SYSTEM.filter((x) => x.type.includes(t)); const unique = TYPE[t].filter((k) => !typeNames.some((o) => o !== t && TYPE[o].includes(k))); console.log(y(t, s.length, TYPE[t].length, unique.length, TYPE[t].length - unique.length, TYPE[t].reduce((a, k) => a + CONTROL[k].item, 0))); } const sum = typeNames.reduce((a, t) => a + TYPE[t].length, 0); const union = required(SYSTEM); console.log(`\ncontrols per type summed separately ${sum}; union ${union.size} — ` + `${sum - union.size} controls repeat across more than one type`); const z = gen([21, 7, 10, 16, 13]); console.log("\n" + z("highest-load system", "types", "controls", "evidence items", "single-type")); const topLoaded = [...SYSTEM].sort((a, b) => controlsFor(b.type).size - controlsFor(a.type).size || b.type.length - a.type.length).slice(0, 4); for (const s of topLoaded) { const k = [...controlsFor(s.type)]; const single = k.filter((x) => s.type.filter((t) => TYPE[t].includes(x)).length === 1).length; console.log(z(s.name, s.type.length, k.length, k.reduce((a, x) => a + CONTROL[x].item, 0), single)); } const multi = SYSTEM.filter((s) => s.type.length > 1); console.log(`\nmulti-type systems ${multi.length}/${SYSTEM.length}; their control load ` + `${multi.reduce((a, s) => a + controlsFor(s.type).size, 0)} items, single-type ` + `${SYSTEM.length - multi.length} systems' load ` + `${SYSTEM.filter((s) => s.type.length === 1).reduce((a, s) => a + TYPE[s.type[0]].length, 0)}`); } ``` ``` model: 16 systems, 4 data types, 20 controls data type systems extra controls unique shared evidence items payment-card 6 9 6 3 1223 health 5 8 3 5 1246 personal 11 9 3 6 1620 measurement 6 3 2 1 742 controls per type summed separately 29; union 20 — 9 controls repeat across more than one type highest-load system types controls evidence items single-type backup-store 4 20 2152 14 reporting-warehouse 3 17 2125 14 billing-application 2 15 1748 12 billing-database 2 15 1748 12 multi-type systems 9/16; their control load 129 items, single-type 7 systems' load 44 ``` The place to read in the first table is the fourth and fifth columns. Payment card data brings nine extra controls, and six of them are unique to that type alone; health data brings eight controls, but only three are unique — five are shared with other types. This is the item most often miscalculated in compliance work: when the types are summed separately, 29 controls come out; the union is 20. Nine controls repeat across more than one type, meaning **the control cost of taking on a second data type is lower than the first.** Encryption at rest appears in three types at once; set up once, it answers three obligations at the same time. The reverse side of sharing must also be counted. If a control comes from more than one type, the cost of letting that control slip is also spread across more than one obligation: access logging appears in all four of the four types, so a single logging stream breaking leaves four separate requirements without evidence at the same time. The previous lesson's leverage measure is reproduced here by data type — a shared control is the source of both the saving and the fragility. The second table shows this accumulation at the system level. Because the backup store carries all four types, it takes on **all** twenty controls; no system in the inventory carries this much load, and the backup store itself does no work of its own — it only holds other systems' data. The reporting warehouse and the test environment are in the same class. Data-type lists accumulate in these three places because all three collect data **independent of its source**. The last line says the same thing in aggregate: nine multi-type systems carry a control load of 129, while seven single-type systems carry 44. #### Not Holding the Data at All The harshest way to shorten the control list is not to hold the data at all: removing the type from the system, outsourcing it, or aggregating it in a way that strips the identifiers. The file below runs four actions on the same inventory and counts how many controls each one actually drops. ```js // sector/narrowing.mjs — how many controls not holding the data at all eliminates. import { CONTROL, TYPE, SYSTEM, controlsFor, required, gen } from "./data.mjs"; // Evidence items a system produces in the period: the controls of the types it carries const volume = (set) => set.reduce((a, s) => a + [...controlsFor(s.type)].reduce((x, k) => x + CONTROL[k].item, 0), 0); // action | data type removed | systems touched ("*" = all) | residual control | days const ACTION = [ ["token and gateway outsourcing", "payment-card", "*", ["C04", "C16"], 30], ["health record outsourcing", "health", "*", ["C16"], 14], ["aggregation in reporting", "personal", ["reporting-warehouse", "backup-store", "test-environment", "archive-store"], ["C09"], 20], ["field summarization of measurement", "measurement", ["reporting-warehouse", "backup-store", "log-collection-pipeline"], [], 10], ]; const INIT_CONTROLS = required(SYSTEM), INIT_VOLUME = volume(SYSTEM); const apply = (type, target, residual) => { const next = SYSTEM.map((s) => (target === "*" || target.includes(s.name)) ? { ...s, type: s.type.filter((t) => t !== type) } : s); const k = required(next); for (const a of residual) k.add(a); return { next, k }; }; console.log(`start: ${INIT_CONTROLS.size} controls, ${INIT_VOLUME.toLocaleString("en-US")} ` + `evidence items in the period, ${SYSTEM.length} systems`); const y = gen([36, 9, 10, 9, 16, 6, 11]); console.log("\n" + y("action", "systems", "controls", "dropped", "evidence items", "days", "items/day")); const result = []; for (const [name, type, target, residual, days] of ACTION) { const { next, k } = apply(type, target, residual); const h = volume(next) + residual.reduce((a, x) => a + CONTROL[x].item, 0); const touched = SYSTEM.filter((s) => (target === "*" || target.includes(s.name)) && s.type.includes(type)).length; result.push({ name, dropped: INIT_CONTROLS.size - k.size, fell: INIT_VOLUME - h, days }); console.log(y(name, touched, k.size, INIT_CONTROLS.size - k.size, h.toLocaleString("en-US"), days, ((INIT_VOLUME - h) / days).toFixed(0))); } const order = (f) => [...result].sort((a, b) => f(b) - f(a)).map((s) => s.name); const byDropped = order((s) => s.dropped), byVolume = order((s) => s.fell), byRate = order((s) => s.fell / s.days); console.log(`\nby controls dropped : ${byDropped.join(" > ")}`); console.log(`by items fallen : ${byVolume.join(" > ")}`); console.log(`by rate per day : ${byRate.join(" > ")}`); // The difference between removing a type from every system and removing it partially const partial = result.filter((s) => s.dropped === 0); console.log(`\n${result.length - partial.length} actions removing a type from every system drop ` + `controls; ${partial.length} actions removing it partially drop no controls at all but reduce ` + `${partial.reduce((a, s) => a + s.fell, 0).toLocaleString("en-US")} evidence items`); const RESIDUAL = ["C04", "C16"]; const both = apply("payment-card", "*", RESIDUAL).next .map((s) => ({ ...s, type: s.type.filter((t) => t !== "health") })); const afterBoth = required(both); for (const a of RESIDUAL) afterBoth.add(a); console.log(`removing two types at once: ${INIT_CONTROLS.size - afterBoth.size} of ` + `${INIT_CONTROLS.size} controls disappear, ${afterBoth.size} remain; ` + `${RESIDUAL.map((k) => CONTROL[k].name).join(" and ")} come back as residual controls`); console.log(`of the remaining ${afterBoth.size} controls, ${TYPE["personal"].length} come from ` + `the personal data type; evidence items in the period ` + `${[...afterBoth].reduce((a, k) => a + CONTROL[k].item, 0).toLocaleString("en-US")}`); ``` ``` start: 20 controls, 24,394 evidence items in the period, 16 systems action systems controls dropped evidence items days items/day token and gateway outsourcing 6 16 4 22,586 30 60 health record outsourcing 5 17 3 23,043 14 97 aggregation in reporting 4 20 0 22,911 20 74 field summarization of measurement 3 20 0 23,263 10 113 by controls dropped : token and gateway outsourcing > health record outsourcing > aggregation in reporting > field summarization of measurement by items fallen : token and gateway outsourcing > aggregation in reporting > health record outsourcing > field summarization of measurement by rate per day : field summarization of measurement > health record outsourcing > aggregation in reporting > token and gateway outsourcing 2 actions removing a type from every system drop controls; 2 actions removing it partially drop no controls at all but reduce 2,614 evidence items removing two types at once: 7 of 20 controls disappear, 13 remain; validation of the tokenization boundary and vendor assessment come back as residual controls of the remaining 13 controls, 9 come from the personal data type; evidence items in the period 2,052 ``` The fourth column is the lesson's harsh result. The actions that remove personal data from four systems and measurement data from three systems **eliminate no control at all.** The reason is FW25: a control disappears only once no system that requires it remains; since personal data appears in eleven systems, removing it from four of them does not shorten the list. These two actions still pay off — they reduce evidence items by 2,614 — but what they gain is not the control list, it is the **evidence volume**. Two different gains and two different decisions; folding them into one heading produces an unmeasurable sentence like "we reduced the data, so our compliance load went down." The two actions that remove a type from every system really do drop controls: four for payment card data, three for health data. But the number dropped is less than the number the type brought. Payment card data had brought nine controls; removing it drops six unique controls, and then two come back as **residual controls** — validation of the boundary that keeps the data out, and assessment of the party it was outsourced to. Outsourcing the data does not zero out the obligation; it moves it to **the boundary itself**. The three rankings give three different lists, and none of them is wrong. By controls dropped, the token action comes first; by rate per day, it comes last, and field summarization of measurement moves to the front. The choice between dropping four controls with a thirty-day effort and dropping no controls but reducing 1,131 evidence items with a ten-day effort depends on what the organization experiences as the real constraint: the length of the audit list, or the labor of compiling evidence. The last line writes the limit — even after removing two types at once, thirteen of the twenty controls remain standing, and nine of them come from the personal data type. #### Summary - Part of the compliance requirements arises not from what the organization does but from which data type it touches; the four types bring 9, 8, 9, and 3 extra controls respectively. - Summed separately, the types produce 29 controls; the union produces 20: because nine controls repeat across more than one type, the control cost of a second data type is lower than the first. - Control load accumulates in systems that collect data independent of its source: the backup store takes on all twenty controls because it carries all four types; nine multi-type systems' load is 129, seven single-type systems' load is 44. - A control disappears only once no system that requires it remains: removing personal data from four systems and measurement data from three eliminates no control at all, only reducing 2,614 evidence items. - Removing a type entirely drops four and three controls, but residual controls come back: outsourcing the data does not zero out the obligation, it moves it to the boundary itself. When two types are removed at once, thirteen of the twenty controls remain standing. - Controls dropped, evidence items reduced, and gain per unit day produce three different rankings; the action that drops the most controls comes last per unit of effort. #### Next Step One of this lesson's four types behaved differently from the others: personal data appears in eleven systems, does not disappear through any narrowing, and even after the removal attempts, it alone carries nine of the thirteen remaining controls. The reason is not just its prevalence. For the other types, the obligation is written **to the party holding the data**; for personal data, part of the obligation is given **to the person the data is about**, as a right, and that right is exercised the moment it is requested. The next lesson counts these rights as items: how many separate request types there are, how many systems are searched while a request is being fulfilled, and how many requests are answered incompletely because of a system missing from the inventory. Processing principles are handled the same way — each principle is tied to a control and to an evidence item. ### Personal Data Protection Source: https://academia.sh/en/courses/governance-risk-and-compliance/personal-data-protection Counting data subject rights and processing principles as items: because 3 of 15 systems are not in the inventory, 6 of 7 right-request types are answered incompletely, and 28 percent of the 121 monthly requests get caught on shadow systems; a request takes between 55 and 240 minutes, and the monthly workload of 381 person-hours exceeds the 260-person-hour capacity; completing the inventory raises the load to 494.8 person-hours, data minimization lowers it to 324.9, and all 7 processing principles depend on the completeness of the inventory. One of the previous lesson's four data types did not behave like the others: personal data appears in eleven systems, does not disappear through any narrowing, and even after the removal attempts, it alone carried nine of the remaining thirteen controls. The source of the difference is not prevalence. For the other types, the obligation is written to the party holding the data; for personal data, part of the obligation is given **to the person the data is about, as a right**. The right is exercised not on the organization's schedule but the moment its holder wants it, and it demands an **action** from the organization. This lesson counts rights and processing principles as items. No law, article, institution, or country name is written down; what is measured is not the name of the obligation but how many systems a request travels through in the organization and how many minutes it takes. The inventory from the previous lessons becomes decisive here: **a request can only be searched for in the systems that are in the inventory.** - **FW27.** The fictional institution has 15 systems carrying personal data; three of them are not registered in the inventory. The systems, record counts, and durations are **fictional**. - **FW28.** Every system holds one or more **data classes**: identity, contact, contract, transaction, location, complaint, profile. - **FW29.** Seven right-request types are defined; each type searches specific data classes and touches only the systems holding those classes. - **FW30.** A request's duration is the sum of the search times of the systems it touches **in the inventory**, plus a fixed processing time. A system not in the inventory is not searched, and it does not enter the duration either. - **FW31.** The probability that the data subject's record exists in a system that is not in the inventory is the ratio of that system's record count to the subscriber count. If a request has left a record in any of these systems, it has been **answered incompletely**. - **FW32.** The request team's capacity is 260 person-hours per month (fictional). ```js // personal/request.mjs — counting data subject rights as items. // MODEL: regional measurement network (fictional); systems, record counts, and durations are fictional. export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); export const gen = (B) => (...h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); export const SUBSCRIBERS = 420000; // fictional subscriber count; the coverage ratio is divided by this // system | in inventory? | records | search minutes | data classes it holds export const SYSTEM = row(` subscriber-portal|yes|420000|4|identity contact contract billing-application|yes|380000|6|identity contract transaction billing-database|yes|380000|8|identity contract transaction call-center-recording|yes|95000|12|identity contact complaint field-crew-app|yes|22000|9|identity location work-order-service|yes|140000|7|location contract identity-provider|yes|31000|3|identity directory-service|yes|31000|3|identity contact reporting-warehouse|yes|1200000|25|transaction profile backup-store|yes|2400000|60|identity contact contract transaction archive-store|yes|900000|40|contract log-collection-pipeline|yes|5400000|18|transaction marketing-list|no|60000|10|contact profile field-spreadsheet-files|no|8000|35|location complaint old-call-backup|no|75000|30|contact complaint`) .map(([name, inv, records, minutes, cls]) => ({ name, inventoried: inv === "yes", records: +records, minutes: +minutes, class: cls.split(" "), coverage: Math.min(1, +records / SUBSCRIBERS) })); // right | data classes searched | extra processing minutes per item | monthly requests (fictional) export const RIGHT = row(` R1 copy of record|identity contact contract transaction location complaint profile|30|40 R2 correction|identity contact|20|25 R3 deletion|identity contact contract transaction location complaint profile|45|30 R4 restriction of processing|transaction profile|25|6 R5 objection to processing|contact profile|20|12 R6 portability|contract transaction|40|5 R7 objection to automated decision-making|profile|30|3`) .map(([name, s, extra, monthly]) => ({ name, code: name.split(" ")[0], class: s.split(" "), extra: +extra, monthly: +monthly })); export const touched = (h, set = SYSTEM) => set.filter((s) => s.class.some((x) => h.class.includes(x))); export const duration = (h, set = SYSTEM) => h.extra + touched(h, set).filter((s) => s.inventoried).reduce((a, s) => a + s.minutes, 0); // The probability that the data subject's record exists in a system outside the inventory is the coverage ratio. export const missingRate = (h, set = SYSTEM) => 1 - touched(h, set) .filter((s) => !s.inventoried).reduce((a, s) => a * (1 - s.coverage), 1); if (import.meta.url === `file://${process.argv[1]}`) { const outside = SYSTEM.filter((s) => !s.inventoried); console.log(`model: ${SYSTEM.length} systems (${outside.length} not in inventory), ` + `${RIGHT.length} right-request types, ${RIGHT.reduce((a, h) => a + h.monthly, 0)} ` + `requests/month (fictional)`); const y = gen([43, 10, 9, 14, 18, 13, 11]); console.log("\n" + y("right request type", "searched", "skipped", "duration min", "records searched", "requests/mo", "missing %")); for (const h of RIGHT) { const d = touched(h), searched = d.filter((s) => s.inventoried); console.log(y(h.name, searched.length, d.length - searched.length, duration(h), searched.reduce((a, s) => a + s.records, 0).toLocaleString("en-US"), h.monthly, (100 * missingRate(h)).toFixed(1))); } const missingRequests = RIGHT.reduce((a, h) => a + h.monthly * missingRate(h), 0); const total = RIGHT.reduce((a, h) => a + h.monthly, 0); console.log(`\nright types touching a system outside the inventory ` + `${RIGHT.filter((h) => missingRate(h) > 0).length}/${RIGHT.length}; per month ` + `${missingRequests.toFixed(1)}/${total} requests (${(100 * missingRequests / total).toFixed(1)}%) ` + `are answered incompletely`); console.log(`${outside.length} systems outside the inventory: ` + `${outside.map((s) => `${s.name} (${(100 * s.coverage).toFixed(1)}%)`).join(", ")}`); const costliest = [...SYSTEM].filter((s) => s.inventoried).sort((a, b) => b.minutes - a.minutes) .slice(0, 3); console.log(`three systems that set the search duration: ` + `${costliest.map((s) => `${s.name} ${s.minutes} min`).join(", ")}; total workload ` + `${(RIGHT.reduce((a, h) => a + h.monthly * duration(h), 0) / 60).toFixed(1)} person-hours/month`); } ``` ``` model: 15 systems (3 not in inventory), 7 right-request types, 121 requests/month (fictional) right request type searched skipped duration min records searched requests/mo missing % R1 copy of record 12 3 225 11,399,000 40 30.9 R2 correction 8 2 125 3,759,000 25 29.6 R3 deletion 12 3 240 11,399,000 30 30.9 R4 restriction of processing 5 1 142 9,760,000 6 14.3 R5 objection to processing 5 2 124 4,146,000 12 29.6 R6 portability 8 0 208 11,220,000 5 0.0 R7 objection to automated decision-making 1 1 55 1,200,000 3 14.3 right types touching a system outside the inventory 6/7; per month 33.9/121 requests (28.0%) are answered incompletely 3 systems outside the inventory: marketing-list (14.3%), field-spreadsheet-files (1.9%), old-call-backup (17.9%) three systems that set the search duration: backup-store 60 min, archive-store 40 min, reporting-warehouse 25 min; total workload 381.2 person-hours/month ``` The table turns rights from a single sentence into a set of items. The seven right types do not do the same work: copy of record and deletion search all seven data classes and travel through twelve systems, objection to automated decision-making only looks at the profile class and ends at a single system. The duration ranges from 55 minutes to 240 minutes — **for the same organization, for the same person, doing work that varies more than fourfold depending on which right is exercised.** What sets the duration is not the request itself but the shape of the inventory: the backup store alone takes 60 minutes and is searched by six right types at once. The second column is the lesson's main measure. Six of the seven right types touch a system that is not in the inventory, and that system is never searched. A system that is not searched does not enter the duration; that is why **an incomplete answer gives no sign at all in the response time.** The request closes on time, its holder is told "your record was found in these three systems," and the record in the shadow system is never mentioned. 33.9 of the 121 monthly requests — 28 percent — are in this state. This is a defect different from what the previous three lessons measured. An unevidenced requirement shows up in an audit, an out-of-scope item is written into an applicability statement — both are **recorded** somewhere. The gap the shadow system produces is recorded nowhere: it leaves no trace in the request record, in the duration measurement, or in the evidence file. The only way to measure it is to search separately for what is outside the inventory; it cannot be found by the inventory itself. The rate varies by right type, and the reason is the coverage ratio of the shadow systems: the old call backup contains 17.9 percent of subscribers, the marketing list 14.3 percent, the field spreadsheet files 1.9 percent. Rights touching the contact class have two shadow systems active at once, and the incomplete-answer rate rises to 29.6 percent. Portability, on the other hand, is the one clean right: the contract and transaction classes are held in no shadow system at all. #### Two Options, One Capacity There are two ways to lower the incomplete-answer rate, and they work in opposite directions: completing the inventory (registering the shadow systems and including them in the search), or reducing the data (not holding a class at all, emptying a store whose retention period has expired). ```js // personal/scenario.mjs — the effect of completing the inventory and reducing data on the request load. import { SYSTEM, RIGHT, row, gen, duration, missingRate } from "./request.mjs"; const CAPACITY = 260; // person-hours/month (two people, fictional) const workload = (k) => RIGHT.reduce((a, h) => a + h.monthly * duration(h, k), 0) / 60; const missing = (k) => RIGHT.reduce((a, h) => a + h.monthly * missingRate(h, k), 0); // The scenarios are transformations defined on the same system set. const completeInventory = (k) => k.map((s) => ({ ...s, inventoried: true })); const minimizeData = (k) => k.filter((s) => s.name !== "archive-store") .map((s) => s.name === "reporting-warehouse" ? { ...s, class: s.class.filter((x) => x !== "profile") } : s); const SCENARIO = [["baseline", (k) => k], ["inventory completed", completeInventory], ["data minimization", minimizeData], ["both combined", (k) => minimizeData(completeInventory(k))]]; const y = gen([21, 9, 12, 11, 15, 12]); console.log(`${RIGHT.reduce((a, h) => a + h.monthly, 0)} requests/month, capacity ` + `${CAPACITY} person-hours/month (fictional)`); console.log("\n" + y("scenario", "systems", "missing/mo", "missing %", "person-hrs/mo", "capacity %")); for (const [name, transform] of SCENARIO) { const k = transform(SYSTEM), w = workload(k), m = missing(k); const total = RIGHT.reduce((a, h) => a + h.monthly, 0); console.log(y(name, k.length, m.toFixed(1), (100 * m / total).toFixed(1), w.toFixed(1), (100 * w / CAPACITY).toFixed(0))); } // Processing principles: each principle is tied to an evidence item and a set of data classes. // principle | data classes it covers | evidence items produced in the period const ALL_CLASSES = "identity contact contract transaction location complaint profile"; const PRINCIPLE = row(` lawful basis|contact profile|120 purpose limitation|transaction profile|4 data minimization|${ALL_CLASSES}|4 accuracy|identity contact|12 retention limit|contract transaction complaint|365 integrity and confidentiality|${ALL_CLASSES}|365 accountability|${ALL_CLASSES}|4`) .map(([name, s, item]) => ({ name, class: s.split(" "), item: +item })); const z = gen([31, 14, 16, 26, 12]); console.log("\n" + z("processing principle", "data classes", "evidence items", "system outside inventory", "verifiable")); for (const i of PRINCIPLE) { const outside = SYSTEM.filter((s) => !s.inventoried && s.class.some((x) => i.class.includes(x))); console.log(z(i.name, i.class.length, i.item, outside.length, outside.length ? "no" : "yes")); } const dependent = PRINCIPLE.filter((i) => SYSTEM.some((s) => !s.inventoried && s.class.some((x) => i.class.includes(x)))); console.log(`\n${dependent.length} of ${PRINCIPLE.length} processing principles depend on the ` + `completeness of the inventory; once the inventory is completed, all ${PRINCIPLE.length} ` + `principles become verifiable, and the workload rises by ` + `${(workload(completeInventory(SYSTEM)) - workload(SYSTEM)).toFixed(1)} person-hours/month`); console.log(`evidence items the principles produce in the period ` + `${PRINCIPLE.reduce((a, i) => a + i.item, 0)}; annual load of fulfilling requests ` + `${(workload(SYSTEM) * 12).toFixed(0)} person-hours`); ``` ``` 121 requests/month, capacity 260 person-hours/month (fictional) scenario systems missing/mo missing % person-hrs/mo capacity % baseline 15 33.9 28.0 381.2 147 inventory completed 15 0.0 0.0 494.8 190 data minimization 14 33.9 28.0 324.9 125 both combined 14 0.0 0.0 438.6 169 processing principle data classes evidence items system outside inventory verifiable lawful basis 2 120 2 no purpose limitation 2 4 1 no data minimization 7 4 3 no accuracy 2 12 2 no retention limit 3 365 2 no integrity and confidentiality 7 365 3 no accountability 7 4 3 no 7 of 7 processing principles depend on the completeness of the inventory; once the inventory is completed, all 7 principles become verifiable, and the workload rises by 113.7 person-hours/month evidence items the principles produce in the period 874; annual load of fulfilling requests 4574 person-hours ``` The first table shows the trade-off in its bare form. In the baseline, there is 147 percent of the team's capacity worth of work, and requests are already falling behind — and this backlog is happening while answering 28 percent of requests incompletely. Once the inventory is completed, the incomplete-answer rate goes to zero, but the workload rises to 494.8 person-hours — 190 percent of capacity. **Answering correctly is 30 percent more expensive than answering incorrectly**, and this difference never shows up anywhere as a line-item cost. Placing the second and third rows side by side is the lesson's result. Data minimization lowers the workload to 324.9 and brings it closer to capacity, but it does not change the incomplete-answer rate at all: the items removed were already in the inventory, the shadow systems stay right where they are. The two options solve **different problems.** Choosing only one leaves either a correct but backlogged process, or a fast but incomplete one; applying both at once zeroes out the incomplete answers and brings the load down to 438.6 — still above capacity. There is no situation resolved without growing the capacity, and that too is a decision. The second table shows the same gap on the evidence side. **All seven** of the seven processing principles touch at least one system that is not in the inventory, and none of them are in a verifiable state. Purpose limitation is caught on a single shadow system, data minimization and accountability on three at once. This does not mean the principles are not applied; it means **that their being applied cannot be shown.** The third lesson's distinction applies here exactly the same way: a principle can have a control, it can even be producing an evidence item, but the scope of the evidence is bounded by the scope of the inventory. That is why, for personal data, the first item is not a control but **the inventory itself**; everything counted before it has been counted on top of an incomplete set. #### Summary - Data subject rights are seven separate request types and do not do the same work: copy of record and deletion travel through twelve systems and take 225–240 minutes, while objection to automated decision-making finishes at a single system in 55 minutes. - A system that is not in the inventory is not searched and does not enter the duration; that is why an incomplete answer gives no sign at all in the response time. Six of the seven right types are in this state, and 28 percent of the 121 monthly requests are answered incompletely. - The coverage ratio of the shadow systems determines the incomplete-answer rate: rights touching the contact class have two shadow systems active at once, and the rate rises to 29.6 percent; portability is the one clean right. - Answering correctly is expensive: once the inventory is completed, the incomplete-answer rate goes to zero, but the monthly workload rises from 381.2 to 494.8 person-hours — 190 percent of the 260-person-hour capacity. - Completing the inventory and reducing the data solve different problems: minimization lowers the load to 324.9 but does not change the incomplete-answer rate at all; even with both applied at once, the load stays above capacity. - All seven of the seven processing principles are unverifiable because they touch a system that is not in the inventory: the issue is not that the principle is not applied, but that its being applied cannot be shown. #### Next Step What the last two tables have in common is that everything measured rests on a **document**: the inventory is a document, the retention period is written in a document, the steps of the request flow are defined in a document. Up to this lesson, documents were treated as a single mass — yet there is a sharp difference between them from an audit standpoint. One document says "we protect personal data" and proves nothing; another says "the retention period is 1,095 days" and gives a measurable threshold; a third defines a procedure step by step and produces a record every time it runs. The next lesson separates these three levels — policy, standard, and procedure — along the axis of evidence production: which document level counts as evidence in an audit, which merely states intent, and how many links of the chain need to be complete to show that a requirement has been satisfied. ### Policy, Standard, and Procedure Source: https://academia.sh/en/courses/governance-risk-and-compliance/policy-standard-and-procedure Separating the document hierarchy along the axis of evidence production: only 10 of 26 documents produce evidence in an audit and 9 do not carry a measurable threshold, 9 of 15 requirement chains are complete and 11 are provable, two policy-less chains remain provable thanks to a record-producing procedure, completing the missing links gains evidence for 104 person-days while 14 person-days only fix the appearance, and the rankings by document count and by evidence production diverge. Everything measured in the previous lesson's last two tables rested on a document: the inventory is a document, the retention period is written in a document, the steps of the request flow are defined in a document. Up to this point, documents were treated as a single mass. From an audit standpoint, however, there is a sharp difference between them, and this lesson measures that difference. The document hierarchy has three levels. A **policy** states intent and responsibility; it says who owns what, and gives no measurable threshold. A **standard** gives the threshold: a number, a duration, a configuration value. A **procedure** defines a task step by step and produces a **record** every time it runs. Evidence comes only from the last level. The cost of turning an architectural rule into a runnable check was measured in M20/K04; the question here is different — which document level counts as an **item** in front of an auditor, and which one is merely read and passed over. - **FW33.** The document set is 26 items and is spread across eight subject domains. The documents, record counts, and ages are **fictional**. - **FW34.** Every document has a level (policy, standard, procedure), whether or not it carries a measurable threshold, the number of records it produced in the period, and the number of days since its last update. - **FW35.** A requirement is written as a **chain** linking the three levels together: policy → standard → procedure. An empty cell is a missing link. - **FW36.** Evidence rule: a requirement is considered provable in an audit only if it has a **record-producing procedure**. A policy and a standard are not evidence on their own. - **FW37.** The cost of producing a missing link varies by level: policy 3, standard 8, procedure 15, the procedure's record flow 10 person-days (fictional). - **FW38.** A procedure can appear in more than one chain; in that case the record item is shared too. ```js // document/hierarchy.mjs — separating policy, standard, and procedure levels along the axis of evidence production. // MODEL: regional measurement network (fictional); documents, record counts, and ages are fictional. export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); export const gen = (B) => (...h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); // code | level | domain | measurable threshold | records produced in period | days since last update export const DOCUMENT = Object.fromEntries(row(` P1|policy|access|no|0|420 P2|policy|data|no|0|180 P3|policy|change|no|0|900 P4|policy|incident|no|0|150 P5|policy|vendor|no|0|640 P6|policy|awareness|no|0|300 S1|standard|access|yes|0|120 S2|standard|access|yes|0|540 S3|standard|data|yes|0|90 S4|standard|data|no|0|380 S5|standard|change|yes|0|75 S6|standard|incident|yes|0|200 S7|standard|vendor|no|0|700 S8|standard|continuity|yes|0|260 S9|standard|awareness|yes|0|340 Y1|procedure|access|yes|240|60 Y2|procedure|access|yes|48|110 Y3|procedure|data|yes|365|30 Y4|procedure|data|yes|30|45 Y5|procedure|change|yes|310|20 Y6|procedure|incident|yes|18|95 Y7|procedure|incident|no|0|410 Y8|procedure|vendor|yes|3|280 Y9|procedure|continuity|yes|4|150 Y10|procedure|awareness|yes|4|85 Y11|procedure|asset|yes|12|40`) .map(([code, level, domain, threshold, records, age]) => [code, { code, level, domain, threshold: threshold === "yes", records: +records, age: +age }])); // requirement | policy | standard | procedure (an empty cell is a missing link) export const CHAIN = row(` Z01 access approval|P1|S1|Y1 Z02 privileged access|P1|S2|Y2 Z03 authentication rules|P1|S2| Z04 data classification|P2|S3|Y3 Z05 deletion request|P2|S4|Y4 Z06 retention period|P2|S3|Y3 Z07 change approval|P3|S5|Y5 Z08 incident notification|P4|S6|Y6 Z09 post-incident review|P4|S6|Y7 Z10 vendor assessment|P5|S7|Y8 Z11 vendor offboarding|P5|| Z12 continuity exercise||S8|Y9 Z13 awareness training|P6|S9|Y10 Z14 asset inventory|||Y11 Z15 record integrity|||`) .map(([name, p, s, y]) => ({ name, code: name.split(" ")[0], p, s, y })); export const LEVEL = ["policy", "standard", "procedure"]; // Evidence rule: a requirement is provable only with a record-producing procedure. export const provable = (z) => Boolean(z.y) && DOCUMENT[z.y].records > 0; export const breakPoint = (z) => !z.p ? "no policy" : !z.s ? "no standard" : !z.y ? "no procedure" : DOCUMENT[z.y].records === 0 ? "produces no records" : "complete"; if (import.meta.url === `file://${process.argv[1]}`) { const all = Object.values(DOCUMENT); console.log(`model: ${all.length} documents, ${CHAIN.length} requirement chains, ` + `${new Set(all.map((b) => b.domain)).size} subject domains`); const y = gen([11, 6, 11, 11, 16, 14]); console.log("\n" + y("level", "docs", "threshold", "producing", "period records", "avg age days")); for (const d of LEVEL) { const k = all.filter((b) => b.level === d); console.log(y(d, k.length, k.filter((b) => b.threshold).length, k.filter((b) => b.records > 0).length, k.reduce((a, b) => a + b.records, 0), (k.reduce((a, b) => a + b.age, 0) / k.length).toFixed(0))); } console.log(`of the ${all.length} documents in total, ${all.filter((b) => b.records > 0).length} ` + `produce evidence in an audit; ${all.filter((b) => !b.threshold).length} do not carry a ` + `measurable threshold`); const z = gen([21, 7, 9, 37]); console.log("\n" + z("chain status", "count", "percent", "requirements")); const STATUS = ["complete", "produces no records", "no procedure", "no standard", "no policy"]; for (const d of STATUS) { const k = CHAIN.filter((x) => breakPoint(x) === d); console.log(z(d, k.length, (100 * k.length / CHAIN.length).toFixed(0), k.map((x) => x.code).join(" ") || "-")); } console.log(`\nprovable requirement ${CHAIN.filter(provable).length}/${CHAIN.length}; ` + `with a policy ${CHAIN.filter((x) => x.p).length}, with a standard ` + `${CHAIN.filter((x) => x.s).length}, with a procedure ${CHAIN.filter((x) => x.y).length}`); const stale = CHAIN.filter((x) => x.p && x.y && DOCUMENT[x.p].age > DOCUMENT[x.y].age + 180); console.log(`chains where the policy is more than 180 days older than its procedure ${stale.length}: ` + `${stale.map((x) => `${x.code} (${DOCUMENT[x.p].age}/${DOCUMENT[x.y].age} days)`).join(", ")}`); } ``` ``` model: 26 documents, 15 requirement chains, 8 subject domains level docs threshold producing period records avg age days policy 6 0 0 0 432 standard 9 7 0 0 301 procedure 11 10 10 1034 120 of the 26 documents in total, 10 produce evidence in an audit; 9 do not carry a measurable threshold chain status count percent requirements complete 9 60 Z01 Z02 Z04 Z05 Z06 Z07 Z08 Z10 Z13 produces no records 1 7 Z09 no procedure 1 7 Z03 no standard 1 7 Z11 no policy 3 20 Z12 Z14 Z15 provable requirement 11/15; with a policy 12, with a standard 12, with a procedure 12 chains where the policy is more than 180 days older than its procedure 5: Z01 (420/60 days), Z02 (420/110 days), Z07 (900/20 days), Z10 (640/280 days), Z13 (300/85 days) ``` The first table separates the hierarchy level by level, and reading it requires taking in three columns at once. **None** of the six policies carry a measurable threshold, and none produce records; seven of the nine standards carry a threshold, but again none of them produce records; ten of the eleven procedures both carry a threshold and produce records, and put out 1,034 items in the period. Ten of the twenty-six documents produce evidence in an audit. The remaining sixteen can be placed in the audit file, read, and accepted — but they do not show that work was done, only that it should be done. The age column explains why this is so. The average age of policies is 432 days, of standards 301, of procedures 120. The direction of this ranking is not a coincidence: a record-producing document meets reality on every run and changes when it does not hold up; a document that never produces a record faces no pressure of its own accord. The last row gives this its sharpest form — in five chains, the policy is more than 180 days older than its own procedure; in the change domain, the policy is 900 days old and the procedure 20. In these chains, two documents may be describing the same work differently, and the contradiction only becomes visible if a reader happens to notice it. The second table runs the chains. Nine of the fifteen requirements are complete chains. Where the remaining six chains break is quite different from one another and produces four separate outcomes: Z09 has a procedure but it produces no records, Z03 has a standard but no procedure, Z11 has only a policy, and Z12, Z14, and Z15 have no policy. The number just before the last line gives this lesson's main distinction: **the provable requirement count is 11, while the complete-chain count is 9.** The two requirements in between — the continuity exercise and the asset inventory — have no policy, but they do have a record-producing procedure. Both are proven in an audit. In other words, evidence production does not depend on the upper levels of the hierarchy; a record-producing procedure shows that work was done even with no policy sitting above it. The reverse is not true: twelve chains have a policy, but none of the policies alone prove any requirement. #### The Cost of the Missing Link ```js // document/completion.mjs — the cost of completing a broken link, and evidence production per domain. import { DOCUMENT, CHAIN, LEVEL, gen, provable, breakPoint } from "./hierarchy.mjs"; // The cost of producing a missing link (person-days, fictional) const DAYS = { policy: 3, standard: 8, procedure: 15, records: 10 }; const missingPieces = (z) => [...(!z.p ? ["policy"] : []), ...(!z.s ? ["standard"] : []), ...(!z.y ? ["procedure", "records"] : DOCUMENT[z.y].records === 0 ? ["records"] : [])]; const cost = (z) => missingPieces(z).reduce((a, e) => a + DAYS[e], 0); const incompleteChain = CHAIN.filter((z) => missingPieces(z).length > 0); const y = gen([26, 35, 6, 15, 13]); console.log(`${incompleteChain.length} chains have a missing link; provable requirement ` + `${CHAIN.filter(provable).length}/${CHAIN.length}`); console.log("\n" + y("requirement", "missing link", "days", "evidence gain", "days / gain")); for (const z of [...incompleteChain].sort((a, b) => cost(a) - cost(b))) { const gain = provable(z) ? 0 : 1; console.log(y(z.name, missingPieces(z).join("+"), cost(z), gain, gain ? cost(z) : "no gain")); } const noEvidence = incompleteChain.filter((z) => !provable(z)); const appearanceOnly = incompleteChain.filter((z) => provable(z)); console.log(`\ntotal cost of the ${noEvidence.length} chains that gain evidence ` + `${noEvidence.reduce((a, z) => a + cost(z), 0)} person-days; cost of the ` + `${appearanceOnly.length} chains that gain no evidence at all ` + `${appearanceOnly.reduce((a, z) => a + cost(z), 0)} person-days`); console.log(`cheapest to costliest: ` + `${[...incompleteChain].sort((a, b) => cost(a) - cost(b)).map((z) => z.code).join(" < ")}`); console.log(`ranked by evidence gain: ` + `${[...incompleteChain].sort((a, b) => (provable(a) ? 1 : 0) - (provable(b) ? 1 : 0) || cost(a) - cost(b)).map((z) => z.code).join(" < ")}`); // Document count vs. evidence production per domain const domains = [...new Set(Object.values(DOCUMENT).map((b) => b.domain))]; const docsIn = (a) => Object.values(DOCUMENT).filter((b) => b.domain === a); const z2 = gen([12, 6, 8, 10, 11, 16]); console.log("\n" + z2("domain", "docs", ...LEVEL, "period records")); for (const a of domains) { const k = docsIn(a); console.log(z2(a, k.length, ...LEVEL.map((d) => k.filter((b) => b.level === d).length), k.reduce((x, b) => x + b.records, 0))); } const byDocCount = [...domains].sort((a, b) => docsIn(b).length - docsIn(a).length || a.localeCompare(b)); const byRecordCount = [...domains].sort((a, b) => docsIn(b).reduce((x, i) => x + i.records, 0) - docsIn(a).reduce((x, i) => x + i.records, 0) || a.localeCompare(b)); console.log(`\ntop three by document count : ${byDocCount.slice(0, 3).join(", ")}`); console.log(`top three by evidence production: ${byRecordCount.slice(0, 3).join(", ")}`); console.log(`shared: ${byRecordCount.slice(0, 3).filter((a) => byDocCount.slice(0, 3).includes(a)).length}/3; ` + `the single-document asset domain produces ${docsIn("asset").reduce((x, b) => x + b.records, 0)} records, ` + `the four-document incident domain produces ${docsIn("incident").reduce((x, b) => x + b.records, 0)} records`); console.log(`break points: ` + `${incompleteChain.map((z) => `${z.code} ${breakPoint(z)}`).join("; ")}`); ``` ``` 6 chains have a missing link; provable requirement 11/15 requirement missing link days evidence gain days / gain Z12 continuity exercise policy 3 0 no gain Z09 post-incident review records 10 1 10 Z14 asset inventory policy+standard 11 0 no gain Z03 authentication rules procedure+records 25 1 25 Z11 vendor offboarding standard+procedure+records 33 1 33 Z15 record integrity policy+standard+procedure+records 36 1 36 total cost of the 4 chains that gain evidence 104 person-days; cost of the 2 chains that gain no evidence at all 14 person-days cheapest to costliest: Z12 < Z09 < Z14 < Z03 < Z11 < Z15 ranked by evidence gain: Z09 < Z03 < Z11 < Z15 < Z12 < Z14 domain docs policy standard procedure period records access 5 1 2 2 288 data 5 1 2 2 395 change 3 1 1 1 310 incident 4 1 1 2 18 vendor 3 1 1 1 3 awareness 3 1 1 1 4 continuity 2 0 1 1 4 asset 1 0 0 1 12 top three by document count : access, data, incident top three by evidence production: data, change, access shared: 2/3; the single-document asset domain produces 12 records, the four-document incident domain produces 18 records break points: Z03 no procedure; Z09 produces no records; Z11 no standard; Z12 no policy; Z14 no policy; Z15 no policy ``` When sorted from cheapest to costliest, the first table carries two traps right at the top. The continuity exercise becomes a "complete chain" at 3 person-days, the asset inventory at 11; **neither produces any new evidence**, because both are already in a provable state. The total of 14 person-days fixes the table's appearance in the audit report and does not change the number of items shown in the audit. Setting up the post-incident review's record flow, by contrast, costs 10 person-days and turns an unprovable requirement into a provable one. This is exactly where the two rankings diverge. The cheapest-to-costliest order starts with Z12; the ranking by evidence gain starts with Z09, and Z12 and Z14 fall to the very end. Completing missing links by looking at the level — "the policies should be written first, so the hierarchy is complete" — means spending 14 person-days on **appearance alone**. The total cost of the four chains that gain evidence is 104 person-days, and 36 of that goes to a single chain, record integrity, which has no level at all. The last table shows the same divergence at the domain level. The top three by document count are the access, data, and incident domains; the top three by evidence production are data, change, and access. The incident domain carries four documents and produces 18 records in the period; the asset domain has a single document and produces 12. **How many documents a domain has says nothing about how much evidence that domain produces.** Counting documents is a measure of effort; counting records is a measure of outcome, and there is no structural link between these two numbers. #### Summary - The document hierarchy has three levels: a policy states intent, a standard gives a measurable threshold, a procedure produces a record every time it runs. In an audit, evidence comes only from the last level. - Ten of the twenty-six documents produce evidence; none of the six policies carry a threshold or produce records, and seven of the nine standards carry a threshold but none of them produce records. - A record-producing document is fresher: the average age of policies is 432 days, of procedures 120. In five chains, the policy is more than 180 days older than its own procedure. - The provable requirement count is 11, the complete-chain count 9: two requirements have no policy, yet are proven thanks to their record-producing procedure. The reverse is not true — no policy proves a requirement on its own. - Completing missing links by level is misleading: 14 person-days make two chains look "complete" and produce no new evidence at all; the cost of the four chains that gain evidence is 104 person-days, and the cheapest of them is 10 days. - The number of documents in a subject domain says nothing about that domain's evidence production: the four-document incident domain produces 18 records, the single-document asset domain 12. #### Next Step Every number in this lesson rests on an assumption: that the cells said to be filled in the chain really are filled, that the procedure said to produce records really does produce them, that a 900-day-old policy is still valid. The source of this assumption is clear — the party filling in the table, writing the documents, and reporting that the records were produced is the same party. Everything measured across five lessons has been run on the institution's own **self-declaration**. The next lesson closes out the topic and removes this assumption: how many findings never surface at all if the same person both builds the control and audits it, how many items separating the builder from the auditor uncovers, and how many items of extra work that very separation brings. Independent assurance is not an adjective; it is an arrangement that can be counted. ### The Function of Auditor and Compliance Roles Source: https://academia.sh/en/courses/governance-risk-and-compliance/function-of-auditor-and-compliance-roles Measuring independent assurance in terms of separation of duties: of the 17 real findings in 24 controls, only 3 are visible under self-review, and 12 of the 14 invisible findings stay hidden because of independence; the compliance function sees 8, independent internal audit 17, and sampled external audit 9 findings; full separation surfaces 14 findings while bringing 72 audit-days, 19.2 extra work-days, and 48 extra evidence items; and the rankings by findings seen and by days per finding swap places at the two ends. Every number in the previous lesson rested on an assumption: the cells said to be filled really are filled, the procedure said to produce records really does produce them. The source of this assumption is clear — the party filling in the table, writing the documents, and reporting that the records were produced is the **same party**. Everything measured throughout the topic ran on the institution's own self-declaration. This lesson removes that assumption and measures **independent assurance**. Independent assurance is not an adjective; it is a countable arrangement with two dimensions. The first is **depth**: how much technical review is needed for a finding to be seen — is looking at the record enough, does it need to be cross-checked against a second source, or does the work need to be independently rerun. The second is **independence**: the distance between the person who will report the finding and the owner of the finding. A finding becomes visible only when **both conditions** are met at once. - **FW39.** The control set is 24 items spread across six teams; 17 of them have a genuine nonconformity. The set and the findings are **fictional**. - **FW40.** Every finding has a **depth** requirement (1 visible from the record, 2 requires a cross-check, 3 requires an independent rerun) and an **independence** requirement (0 whoever looks sees it, 1 requires a separate team, 2 requires a separate reporting line). - **FW41.** Four assurance arrangements are compared: self-review (depth 1, independence 0), the compliance function (2, 1), independent internal audit (3, 2), external audit (3, 2 but sampled). - **FW42.** External audit looks at 40 percent of the controls and picks its sample from the controls with the **most evidence items**. - **FW43.** Audit-days per control range from 0.5 to 4.0 depending on the arrangement (fictional). - **FW44.** When separation is applied, every control brings one extra approval step (0.4 days) and one handoff cost per independence level (0.2 days); every independent review produces two extra evidence items in the period. - **FW45.** Partial separation means applying independent audit to the `n` controls with the highest evidence volume, and self-review to the rest. ```js // role/assurance.mjs — the findings each independent assurance arrangement sees. // MODEL: regional measurement network (fictional); controls, findings, and days are fictional. export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|")); export const gen = (B) => (...h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); // code | owning team | has finding? | depth (1-3) | independence (0-2) | evidence items export const CONTROL = row(` C01|infrastructure|yes|1|0|240 C02|infrastructure|yes|2|1|180 C03|infrastructure|no|1|0|60 C04|infrastructure|yes|3|2|48 C05|infrastructure|yes|1|1|365 C06|application|yes|2|0|52 C07|application|no|1|0|12 C08|application|yes|3|1|310 C09|application|yes|1|2|30 C10|application|no|2|1|24 C11|identity|yes|2|2|120 C12|identity|yes|1|0|48 C13|identity|no|3|1|4 C14|identity|yes|3|2|365 C15|data|yes|1|1|365 C16|data|no|2|0|30 C17|data|yes|2|2|12 C18|data|yes|3|0|4 C19|field|yes|1|2|18 C20|field|no|1|1|9 C21|field|yes|2|1|3 C22|vendor|yes|3|2|3 C23|vendor|no|2|2|2 C24|vendor|yes|1|0|1`) .map(([code, team, finding, d, b, item]) => ({ code, team, finding: finding === "yes", d: +d, b: +b, item: +item })); // arrangement | depth it can see | independence it carries | days per control | sample ratio export const ARRANGEMENT = row(` self-review|1|0|0.5|1 compliance function|2|1|1.5|1 independent internal audit|3|2|3.5|1 external audit|3|2|4.0|0.4`) .map(([name, d, b, days, sample]) => ({ name, d: +d, b: +b, days: +days, sample: +sample })); export const FINDING = CONTROL.filter((k) => k.finding); // External audit's sample: the controls with the most evidence items are selected (model assumption). export const scope = (z) => z.sample === 1 ? CONTROL : [...CONTROL].sort((a, b) => b.item - a.item).slice(0, Math.round(CONTROL.length * z.sample)); export const seen = (z) => scope(z).filter((k) => k.finding && k.d <= z.d && k.b <= z.b); if (import.meta.url === `file://${process.argv[1]}`) { console.log(`model: ${CONTROL.length} controls, ${FINDING.length} real findings, ` + `${ARRANGEMENT.length} assurance arrangements`); const y = gen([28, 10, 10, 12, 12, 9, 16]); console.log("\n" + y("assurance arrangement", "scope", "seen", "unseen", "seen %", "days", "days / finding")); for (const z of ARRANGEMENT) { const g = seen(z), days = scope(z).length * z.days; console.log(y(z.name, scope(z).length, g.length, FINDING.length - g.length, (100 * g.length / FINDING.length).toFixed(1), days.toFixed(1), g.length ? (days / g.length).toFixed(2) : "-")); } const z2 = gen([14, 10, 13, 12, 16]); console.log("\n" + z2("independence", "findings", "self-review", "compliance", "internal audit")); for (const b of [0, 1, 2]) { const group = FINDING.filter((k) => k.b === b); console.log(z2(`level ${b}`, group.length, ...ARRANGEMENT.slice(0, 3).map((z) => group.filter((k) => k.d <= z.d && k.b <= z.b).length))); } const hidden = FINDING.filter((k) => !seen(ARRANGEMENT[0]).includes(k)); console.log(`\nof the ${hidden.length} findings not visible under self-review, ` + `${hidden.filter((k) => k.b > 0).length} stay hidden because of independence, ` + `${hidden.filter((k) => k.b === 0).length} because of depth`); const sG = [...ARRANGEMENT].sort((a, b) => seen(b).length - seen(a).length).map((z) => z.name); const sV = [...ARRANGEMENT].sort((a, b) => (scope(a).length * a.days) / (seen(a).length || 1) - (scope(b).length * b.days) / (seen(b).length || 1)).map((z) => z.name); console.log(`by findings seen : ${sG.join(" > ")}`); console.log(`by day per finding: ${sV.join(" > ")}`); } ``` ``` model: 24 controls, 17 real findings, 4 assurance arrangements assurance arrangement scope seen unseen seen % days days / finding self-review 24 3 14 17.6 12.0 4.00 compliance function 24 8 9 47.1 36.0 4.50 independent internal audit 24 17 0 100.0 84.0 4.94 external audit 10 9 8 52.9 40.0 4.44 independence findings self-review compliance internal audit level 0 5 3 4 5 level 1 5 0 4 5 level 2 7 0 0 7 of the 14 findings not visible under self-review, 12 stay hidden because of independence, 2 because of depth by findings seen : independent internal audit > external audit > compliance function > self-review by day per finding: self-review > external audit > compliance function > independent internal audit ``` The first row of the first table is the lesson's headline number. When the team that built a control audits its own control, only **three** of the 17 real findings are visible; 14 are never reported. This requires no assumption of bad faith: the same team looks with the same assumptions, reads the same record the same way, and never asks a question outside its own design. The second table breaks this down by cause — of the 14 invisible findings, 12 stay hidden because of independence, two because of depth. **The problem is overwhelmingly one of position, not competence.** The rows of the second table spell this out further. Of the five findings that require no independence at all, three are visible under self-review; **none** of the level-1 or level-2 findings are visible. The compliance function — a separate team, but on the same management line — opens up level 0 and level 1, yet still cannot see any of the seven level-2 findings. Seven findings surface only with a separate reporting line. What determines the independence a finding needs is not the size of the finding, but who it would make look bad. The last two lines carry the topic's measurement axis into this lesson, and the two rankings **swap places at their two ends**; the relative order of the two middle arrangements stays the same in both. By findings seen, independent internal audit ranks first; by day per finding, it ranks last. Self-review spends 4.00 days per finding it sees, independent internal audit 4.94. Read on its own, this number makes self-review look efficient; but self-review's cheapness is a byproduct of the fact that **the three findings it sees are the easiest ones to see**. An arrangement that finds the easy findings and never looks for the hard ones will always win on cost per finding. Choosing an assurance arrangement by unit cost alone rewards blindness. #### The Cost of Separation and Partial Separation ```js // role/separation.mjs — the findings separation of duties surfaces and the extra work it brings. import { CONTROL, ARRANGEMENT, FINDING, gen, seen, scope } from "./assurance.mjs"; const APPROVAL = 0.4; // extra approval step per control after separation (days, fictional) const HANDOFF = 0.2; // handoff between teams (days/control, fictional) const EVIDENCE = 2; // every independent review produces 2 extra evidence items per period (fictional) const days = (z) => scope(z).length * z.days; const extraWork = (z) => (z.b === 0 ? 0 : scope(z).length * (APPROVAL + HANDOFF * z.b)); const extraEvidence = (z) => (z.b === 0 ? 0 : scope(z).length * EVIDENCE); const y = gen([28, 6, 10, 12, 17, 22]); console.log(`${CONTROL.length} controls, ${FINDING.length} real findings; under self-review ` + `${seen(ARRANGEMENT[0]).length} are visible`); console.log("\n" + y("assurance arrangement", "seen", "surfaced", "audit days", "extra work days", "extra evidence items")); for (const z of ARRANGEMENT) { console.log(y(z.name, seen(z).length, seen(z).length - seen(ARRANGEMENT[0]).length, days(z).toFixed(1), extraWork(z).toFixed(1), extraEvidence(z))); } // Hybrid arrangement: independent audit on the N controls with the highest evidence volume, self-review on the rest. const sorted = [...CONTROL].sort((a, b) => b.item - a.item); const z2 = gen([18, 13, 6, 8, 12, 16]); console.log("\n" + z2("hybrid threshold", "independent", "seen", "missed", "audit days", "days / finding")); for (const n of [0, 6, 12, 18, 24]) { const independent = new Set(sorted.slice(0, n).map((k) => k.code)); const g = CONTROL.filter((k) => k.finding && (independent.has(k.code) ? k.d <= ARRANGEMENT[2].d && k.b <= ARRANGEMENT[2].b : k.d <= ARRANGEMENT[0].d && k.b <= ARRANGEMENT[0].b)); const total = n * ARRANGEMENT[2].days + (CONTROL.length - n) * ARRANGEMENT[0].days; console.log(z2(`${n} controls`, n, g.length, FINDING.length - g.length, total.toFixed(1), (total / g.length).toFixed(2))); } // Findings that stay hidden, per team const teams = [...new Set(CONTROL.map((k) => k.team))]; const visible = new Set(seen(ARRANGEMENT[0]).map((k) => k.code)); const z3 = gen([16, 10, 10, 9, 8]); console.log("\n" + z3("team", "controls", "findings", "visible", "hidden")); for (const e of teams) { const k = CONTROL.filter((x) => x.team === e), b = k.filter((x) => x.finding); console.log(z3(e, k.length, b.length, b.filter((x) => visible.has(x.code)).length, b.filter((x) => !visible.has(x.code)).length)); } const mostHidden = teams.map((e) => ({ e, n: CONTROL.filter((x) => x.team === e && x.finding && !visible.has(x.code)).length })).sort((a, b) => b.n - a.n)[0]; console.log(`\nteam with the most findings hidden under self-review: ${mostHidden.e} (${mostHidden.n} findings)`); console.log(`full separation surfaces ${seen(ARRANGEMENT[2]).length - seen(ARRANGEMENT[0]).length} ` + `findings; its cost is ${(days(ARRANGEMENT[2]) - days(ARRANGEMENT[0])).toFixed(1)} audit-days, ` + `${extraWork(ARRANGEMENT[2]).toFixed(1)} extra work-days, and ${extraEvidence(ARRANGEMENT[2])} ` + `extra evidence items`); ``` ``` 24 controls, 17 real findings; under self-review 3 are visible assurance arrangement seen surfaced audit days extra work days extra evidence items self-review 3 0 12.0 0.0 0 compliance function 8 5 36.0 14.4 48 independent internal audit 17 14 84.0 19.2 48 external audit 9 6 40.0 8.0 20 hybrid threshold independent seen missed audit days days / finding 0 controls 0 3 14 12.0 4.00 6 controls 6 8 9 30.0 3.75 12 controls 12 12 5 48.0 4.00 18 controls 18 14 3 66.0 4.71 24 controls 24 17 0 84.0 4.94 team controls findings visible hidden infrastructure 5 4 1 3 application 5 3 0 3 identity 4 3 1 2 data 4 3 0 3 field 3 2 0 2 vendor 3 2 1 1 team with the most findings hidden under self-review: infrastructure (3 findings) full separation surfaces 14 findings; its cost is 72.0 audit-days, 19.2 extra work-days, and 48 extra evidence items ``` The first table splits the cost of separation into three distinct items, and the three come from different places. Full separation surfaces 14 findings; in exchange it brings 72 audit-days, 19.2 extra work-days, and 48 extra evidence items. Audit-days are the auditor's work; extra work-days belong to **the party being audited** — the extra approval steps and the handoffs between teams fall on the team running the control, not on the auditor. The extra evidence items feed directly into the load counted in the third lesson: an independent review produces its own record, and that record also has to be stored, compiled, and sampled. **Independence is not free, and two-thirds of its bill goes to the audited party.** The external audit row offers an interesting comparison alongside this. It carries the same depth and the same independence as independent internal audit, but its sample is 40 percent of the controls, and it sees nine of the 17 findings. Because the sample is selected by evidence volume, it carries a systematic bias: controls that produce a lot of records get seen, controls that produce few do not. Since there is no link between evidence volume and the importance of a finding, this is the quietest way of narrowing scope. The second table maps out partial separation, and the curve is not flat. Applying independent audit to six controls raises findings from three to eight and **lowers** the cost per finding from 4.00 to 3.75 — better than both full self-review and full separation. Once the threshold reaches 12, the unit cost returns to its starting value, and it climbs further at 18 and 24. The reason is plain: the first controls added are the ones with the highest evidence volume, and finding density is higher among them; each subsequent addition brings in fewer and fewer findings. **Independence shows diminishing returns**, but the difference between the five, three, and zero findings left uncovered is also real; which one to accept is a decision, and that decision's number should be written down. The last table shows where separation does the most good. Under self-review, the infrastructure, application, and data teams each have three findings hidden; the vendor team has only one hidden finding. Separation does not need to be applied to the whole institution at once — starting with the teams that have the highest finding density and the highest secrecy surfaces more findings for the same number of days. Knowing this also requires a measurement, and that measurement is only available once independent audit has been run at least once. #### Summary - Independent assurance is a two-dimensional arrangement: depth gives how much review a finding demands, independence gives the distance between the reporter and the owner. A finding becomes visible only when both conditions are met at once. - When the team that built a control audits its own control, only three of the 17 real findings are visible; of the 14 invisible findings, 12 stay hidden because of independence, two because of depth. - The compliance function opens up the level-0 and level-1 findings but cannot see any of the seven level-2 findings: what determines the independence a finding needs is not its size, but who it would make look bad. - The ranking by findings seen and the ranking by day per finding swap places at their two ends: self-review spends 4.00 days per finding, independent internal audit 4.94. Self-review's cheapness is a byproduct of seeing only the easy findings. - Full separation surfaces 14 findings and brings a three-part cost: 72 audit-days, 19.2 extra work-days, and 48 extra evidence items. The extra work-days fall on the team running the control, not on the auditor. - Partial separation shows diminishing returns: independent audit on six controls raises findings to 8 and lowers the unit cost to 3.75; on 24 controls findings rise to 17, but the unit cost climbs to 4.94. #### Next Step The topic closes here. It opened with a problem: the previous topic's numbering was written in the institution's own language and could not be read from the outside. Seven lessons built a common language and counted that language's cost — a maturity measure split across five functions, a documentation cycle with a written scope, a matrix binding controls to requirements, extra controls tied to data type, rights counted item by item, document levels that do and do not produce evidence, and a separation of duties that makes findings visible. Each one had its number written down, and each one showed the same thing: **mapping is a claim, evidence is a separate item.** This common language has a limit, and it stood silently throughout the whole topic. Maturity level was calculated on a date, a document was issued on a date, the control set was sampled for a period, the evidence file was compiled for one audit. All of it is a photograph that is correct **once a year**. Every day in between opens a new gap, a patch window slips, behavior corrected after training drifts back to its old state, and a recovery plan that was never rehearsed quietly goes stale. The photograph being correct says nothing about the 364 days between two photographs. The next topic, `Program Management`, measures those 364 days: the gap between the speed of exposure inflow and remediation capacity, how a shrinking patch window trades off against rising outage caused by that same narrowing, which week an awareness metric drifts back to its old value, the plan item that fails to hold up in a drill, and the metric that never changes behavior at all. ## Program Management Five lessons that measure every day between the annual snapshots: the gap between the incoming finding stream and remediation capacity in a vulnerability management program and the growth of the queue, the point where the exploited vulnerability and the unplanned outage intersect as the patch window narrows, the decay of awareness training gains by group and the retraining week, the gap between the exercise item that failed to hold and the target time versus the measured time, and finally the separation between the metric that changes behavior and the metric that only reaches the report. ### Vulnerability Management Program Source: https://academia.sh/en/courses/governance-risk-and-compliance/vulnerability-management-program Against 884 findings arriving over 52 weeks, a weekly remediation capacity of 84 person-hours: a 13.2-hour gap leaves a 684-hour queue by year's end, four prioritization rules close between 709 and 843 items on the same set, and risk exposure ranges from 6,681 to 18,511 point-weeks. The rule's value depends on capacity: the queue empties at 101 hours/week, and the rules' order reverses at 109 hours/week. The previous topic built a common language: controls were mapped to requirements, the requirement left without evidence was counted, and what an outside observer would see was measured. All of these measurements share one property — they are true once a year. The certification snapshot is valid the moment it is taken; the next morning a new finding drops, a patch falls behind, and a user slips back into an old habit. This topic's five lessons measure the days between the two snapshots. The first lesson starts with the finding itself. A scanning regime produces new items every week, the remediation team's weekly capacity is fixed, and the two are not equal. The question asked at program scale is not "how severe is this finding"; that question was measured at the evidenced-finding level in the Security Tooling and Penetration Testing course and enters here as a single score column. The question here is: **when the incoming stream is larger than capacity, which item waits, and what is the cost of waiting?** The unit of measure is **point-weeks**: an item's score multiplied by the number of weeks it stays open. The number of closed items is an output measure; the total point-weeks — in this lesson, **risk exposure** — is what the program actually reduces, and it is meant to be low. - **SP1.** The 52-week horizon, the six asset classes, and the weekly finding stream belong to a fictional regional measurement network and are a **model**. No real scan runs, and no real asset is scanned. - **SP2.** The score column is an external input. How a finding is evidenced and how it is scored is not re-derived in this lesson. - **SP3.** Effort is in person-hours and does not change over the course of remediation. In reality the first fix in a class makes subsequent ones cheaper; this learning effect is not modeled. - **SP4.** Capacity is fixed per week and divisible: an item left half-finished continues where it left off the following week. The cost of context switching is ignored. - **SP5.** A closed item is not reopened, and the remediation itself does not produce a new finding. - **SP6.** The randomness generator is written inline in the lesson, and the seed is visible. ```js // vuln/model.mjs — MODEL of the vulnerability management program: weekly finding stream, fixed // remediation capacity and queue in a fictional regional measurement network. No real scan // runs; the score column is an external input. export const SEED = 20260803, WEEKS = 52, CAPACITY = 84, AGED = 12, HIGH = 7; const rng = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; export const CLASS = [ // name, weekly new findings, score tendency (0-10), effort tendency (person-hours) { name: 'field-endpoint', weekly: 6, score: 5.2, effort: 3 }, { name: 'branch-endpoint', weekly: 3, score: 5.8, effort: 4 }, { name: 'collector', weekly: 2, score: 6.6, effort: 9 }, { name: 'processing-server', weekly: 2, score: 7.1, effort: 14 }, { name: 'network-device', weekly: 1, score: 6.0, effort: 7 }, { name: 'cloud-account', weekly: 3, score: 6.3, effort: 5 }, ]; export const STREAM = (() => { const r = rng(SEED); const list = []; let id = 0; for (let w = 1; w <= WEEKS; w += 1) for (const c of CLASS) for (let k = 0; k < c.weekly; k += 1) { const p = Math.min(10, Math.max(1, c.score + (r() - 0.5) * 4.4)); list.push({ id: (id += 1), arrival: w, class: c.name, score: +p.toFixed(1), effort: Math.max(1, Math.round(c.effort * (0.5 + r()))) }); } return list; })(); export const arriving = (w) => STREAM.filter((b) => b.arrival === w); export const DEMAND = STREAM.reduce((a, b) => a + b.effort, 0) / WEEKS; export const RULE = { 'score': (b) => b.score, 'yield': (b) => b.score / b.effort, 'arrival': (b) => -b.arrival, 'score+age': (b, w) => b.score + 0.25 * (w - b.arrival), }; // Queue: each week's arrivals are added, the rule order is applied, and capacity is spent from // the top. An item that does not fit is left partly done and continues from where it left off // the following week. export function run(name, capacity = CAPACITY) { const rule = RULE[name]; const queue = []; const closed = []; for (let w = 1; w <= WEEKS; w += 1) { for (const b of arriving(w)) queue.push({ ...b, remaining: b.effort }); queue.sort((a, b) => rule(b, w) - rule(a, w) || a.id - b.id); let free = capacity; while (free > 0 && queue.length) { const b = queue[0]; if (b.remaining <= free) { free -= b.remaining; closed.push({ ...b, closedAt: w }); queue.shift(); } else { b.remaining -= free; free = 0; } } } const openWeeks = (b) => (b.closedAt ?? WEEKS + 1) - b.arrival; const all = [...closed, ...queue]; const high = closed.filter((b) => b.score >= HIGH); return { name, closed: closed.length, score: closed.reduce((a, b) => a + b.score, 0), exposure: all.reduce((a, b) => a + b.score * openWeeks(b), 0), queue: queue.length, hours: queue.reduce((a, b) => a + b.remaining, 0), highDuration: high.reduce((a, b) => a + openWeeks(b), 0) / high.length, aged: queue.filter((b) => WEEKS - b.arrival + 1 > AGED).length, highOpen: queue.filter((b) => b.score >= HIGH).length }; } ``` #### The Gap Between Flow and Capacity ```js // vuln/flow.mjs — the incoming finding stream, remediation capacity and the result of four // prioritization rules on the same set import { STREAM, CLASS, WEEKS, CAPACITY, DEMAND, AGED, HIGH, RULE, run } from './model.mjs'; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); console.log(`${WEEKS} weeks, ${STREAM.length} findings, ${CLASS.length} asset classes; ` + `weekly demand ${DEMAND.toFixed(1)} person-hours, capacity ${CAPACITY} person-hours`); console.log(`gap ${(DEMAND - CAPACITY).toFixed(1)} person-hours/week; ` + `${STREAM.filter((b) => b.score >= HIGH).length} items scoring ${HIGH} or higher`); const A = [-19, 8, 10, 11, 12, 13, 13]; console.log('\n1. flow per class'); y(A, 'class', 'items', 'avg score', 'avg effort', 'hours/week'); for (const c of CLASS) { const l = STREAM.filter((b) => b.class === c.name); y(A, c.name, l.length, (l.reduce((a, b) => a + b.score, 0) / l.length).toFixed(1), (l.reduce((a, b) => a + b.effort, 0) / l.length).toFixed(1), (l.reduce((a, b) => a + b.effort, 0) / WEEKS).toFixed(1)); } console.log('\n2. four prioritization rules, same set, same capacity'); y(A, 'rule', 'closed', 'closed', 'exposure', 'backlog', `${AGED}+ weeks`, `open score>=${HIGH}`); y(A, '', 'items', 'points', 'point-weeks', 'items', 'aged', 'items'); for (const k of Object.keys(RULE)) { const x = run(k); y(A, k, x.closed, x.score.toFixed(0), Math.round(x.exposure), x.queue, x.aged, x.highOpen); } ``` ``` 52 weeks, 884 findings, 6 asset classes; weekly demand 97.2 person-hours, capacity 84 person-hours gap 13.2 person-hours/week; 228 items scoring 7 or higher 1. flow per class class items avg score avg effort hours/week field-endpoint 312 5.3 3.0 18.2 branch-endpoint 156 5.8 4.0 11.9 collector 104 6.6 8.9 17.8 processing-server 104 7.0 13.8 27.7 network-device 52 5.7 7.0 7.0 cloud-account 156 6.5 4.9 14.6 2. four prioritization rules, same set, same capacity rule closed closed exposure backlog 12+ weeksopen score>=7 items pointspoint-weeks items aged items score 709 4579 18219 175 122 0 yield 843 5050 6681 41 27 6 arrival 763 4564 18511 121 0 33 score+age 747 4593 18208 137 12 9 ``` The first table gives the lesson's starting condition, and reading a single line is enough: weekly demand is 97.2 person-hours, capacity is 84. The 13.2-hour gap between them is reborn every week and leaves a 684-hour queue by the end of fifty-two weeks. This is a gap that no prioritization rule can close; a rule only decides **which item** waits. The class rows show where the gap comes from. Field endpoints produce the most items (312) but each one takes only 3 hours; processing servers produce three times fewer items (104) and take the largest share of weekly demand (27.7 hours). The same class also carries the highest average score. Expensive, high-scoring items piling up in the same class is the source of all the tension in the next table. #### Four Rules, Two Different Answers The second table runs the same 884 items through the same capacity in four different orders. The pure-score order closes 709 items; the order that divides score by effort closes 843. The second is ahead not only in item count but also in **total points closed** (5,050 against 4,579). This can look surprising, because the pure-score order by definition takes the highest-scoring item first. The reason is in the table: the highest-scoring class is also the most expensive one, and the time spent on one expensive item keeps four and a half cheap items waiting that could have been closed in the same time. The gap in risk exposure is larger: 18,219 point-weeks against 6,681, roughly 2.7 times. The queue that runs on arrival order is the worst (18,511), and the reason shows up in the columns — under this rule there is **not a single item** aged over twelve weeks, but the 33 items left open score 7 or higher. A young queue does not mean low risk; it only means nobody has waited long. The last two columns reverse the rules' ranking. The pure-score order leaves 122 items waiting longer than twelve weeks, but the number of high-scoring items it leaves open is **zero**: this rule never starves a high-scoring item; instead it indefinitely postpones everything low-scoring. The rule that divides score by effort, on the other hand, leaves six high-scoring items open, and these are exactly the expensive ones. **The same data yields two different priorities**: a manager who watches exposure picks the yield rule; a manager who adopts the rule "no high-scoring item stays open" picks the pure-score rule. Both decisions find support in the same table. #### The Rule's Value Depends on Capacity ```js // vuln/capacity.mjs — as capacity changes, the value of the rule: at what point the queue // empties and at what point the rules' order changes import { DEMAND, RULE, run } from './model.mjs'; const AD = Object.keys(RULE); const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const A = [-10, 8, 11, 11, 13, 11, 13]; y(A, 'capacity', 'demand', ...AD, 'lowest'); y(A, 'hours/wk', 'ratio', 'exposure', 'exposure', 'exposure', 'exposure', 'exposure'); for (let c = 84; c <= 120; c += 6) { const s = AD.map((k) => run(k, c)); const m = Math.round(Math.min(...s.map((x) => x.exposure))); const en = AD.filter((_, i) => Math.round(s[i].exposure) === m); y(A, c, (c / DEMAND).toFixed(2), ...s.map((x) => Math.round(x.exposure)), en.length === 1 ? en[0] : 'tie'); } const empty = (() => { for (let c = 84; c <= 160; c += 1) if (AD.every((k) => run(k, c).queue === 0)) return c; })(); const flip = (() => { for (let c = 84; c <= 160; c += 1) { const s = AD.map((k) => run(k, c).exposure); if (s[1] > s[0]) return c; } })(); console.log(`\nfirst capacity where the queue empties under all four rules: ${empty} hours/week ` + `(${(empty / DEMAND).toFixed(2)}x of demand)`); console.log(`capacity where the rule order changes: ${flip} hours/week ` + `(${(flip / DEMAND).toFixed(2)}x of demand); below it yield leads, above it score leads`); for (const c of [84, empty, flip]) { const s = AD.map((k) => run(k, c).exposure); console.log(` ${String(c).padStart(3)} hours/week: gap between best and worst rule ` + `${(Math.max(...s) - Math.min(...s)).toFixed(0)} point-weeks ` + `(${(Math.max(...s) / Math.max(1, Math.min(...s))).toFixed(1)}x)`); } ``` ``` capacity demand score yield arrival score+age lowest hours/wk ratio exposure exposure exposure exposure exposure 84 0.86 18219 6681 18511 18208 yield 90 0.93 10663 3671 9881 9884 yield 96 0.99 1615 831 1578 1689 yield 102 1.05 281 181 353 297 yield 108 1.11 74 55 89 74 yield 114 1.17 11 14 11 11 tie 120 1.24 0 0 0 0 tie first capacity where the queue empties under all four rules: 101 hours/week (1.04x of demand) capacity where the rule order changes: 109 hours/week (1.12x of demand); below it yield leads, above it score leads 84 hours/week: gap between best and worst rule 11831 point-weeks (2.8x) 101 hours/week: gap between best and worst rule 233 point-weeks (2.0x) 109 hours/week: gap between best and worst rule 34 point-weeks (1.6x) ``` The third table holds the rule fixed and changes capacity, yielding two separate thresholds. The first is 101 hours/week: once capacity reaches 1.04 times demand, the queue empties under all four rules, meaning zero items fall behind. The second is 109 hours/week, and it is this lesson's central proof — at that point **the rules' ranking reverses**. As long as capacity is tight, the rule that divides score by effort gives the lowest exposure; once capacity passes 1.12 times demand, the pure-score order takes the lead. With abundance, no item goes hungry, so the payoff from favoring cheap items ends, and touching the high-scoring item first reduces exposure more. The last three lines measure how valuable the rule is. At 84 hours/week there is a gap of 11,831 point-weeks between the best rule and the worst; at 101 this gap drops to 233, at 109 to 34. A rule is a tool for scarcity, and its value erodes as scarcity eases. The same numbers make both interventions comparable. With capacity held fixed at 84, changing the rule drops exposure from 18,219 to 6,681. Keeping the rule at its best choice and raising capacity from 84 to 101 — twenty percent — drops 6,681 to 233. The second move divides exposure by roughly twenty-nine, the first by only two point seven. **Arguing over the prioritization rule without writing down the capacity gap is an argument without a scale**; once the two numbers are written side by side, it is clear which decision is larger. #### Summary - Over 52 weeks, 884 findings produce a weekly demand of 97.2 person-hours against a capacity of 84 person-hours: a 13.2-hour gap leaves a 684-hour queue by year's end, and no rule closes this gap. - The rule that divides score by effort closes both more items (843 against 709) and more points (5,050 against 4,579) than the pure-score order; risk exposure is 6,681 against 18,219 point-weeks. - The same table yields two different priorities: whoever watches exposure picks the yield rule, and whoever adopts the "no high-scoring item stays open" criterion picks the pure-score rule (0 against 6 open high-scoring items). - The queue that runs on arrival order has no item aged over twelve weeks, but the 33 items left open score 7 or higher: a young queue does not mean low risk. - The queue empties at 101 hours/week (1.04 times demand); at 109 hours/week the rules' ranking reverses and pure score takes the lead. A rule is a tool for scarcity. - Raising capacity by twenty percent drops exposure from 6,681 to 233; choosing the best rule drops it from 18,219 to 6,681. The second move divides exposure by twenty-nine, the first by two point seven. #### Next Step This lesson treated remediation as a one-dimensional operation: person-hours are spent on an item and the item closes. In reality every remediation is a **change**, and a change touches the production environment. Closing an item fast means applying it without testing; testing, in turn, extends how long it stays open. The next lesson measures this trade-off: as the patch window narrows, the number of vulnerabilities caught before they become exploitable rises, but the same narrowing lowers test coverage and produces unplanned outages. Both curves are plotted on the same axis, and the point where they intersect is calculated — that point is the window length where speed and risk are equal. ### Patch and Change Management Source: https://academia.sh/en/courses/governance-risk-and-compliance/patch-and-change-management In a set of forty announcements, the window length is run as a decision variable: as the window shortens, the caught open findings rise from 23 to 30, but unplanned outages climb from 9 to 50; the two curves cross at day nine, and when the outage cost is raised from 900 to 14,000 risk units, the best window shifts from 1 day to 17 days. The approval gate shortens the window by lowering the error multiplier; the two-path regime cuts the single window's total from 207,637 to 130,159 risk units, and a twelve-announcement misclassification error gives back nine-tenths of that gain. The previous lesson treated remediation as a one-dimensional operation: person-hours are spent on an item and the item closes. In reality every remediation is a **change**, and a change touches the production environment. Closing an item fast means applying it under-tested; testing it fully keeps the item open longer. This lesson measures that trade-off on a single decision variable: the **patch window** — the time between the day an advisory is published and the day the patch is applied. The fleet-side mechanics of the window — the maintenance calendar, the machine that misses its cycle, point-days — were measured in the Endpoint and System Hardening course and are taken here as input. The question here is at program scale, and it sits between two curves: as the window shortens, the number of open findings caught before the exploit becomes accessible rises, and that same narrowing lowers test coverage and produces unplanned outages. The two curves are drawn on the same axis, and the point where they cross is calculated. - **SP7.** The forty advisories, the affected-asset counts and the day the exploit becomes accessible belong to the fictional regional measurement network and are a **model**. No real patch is downloaded. - **SP8.** Test coverage grows linearly with window length and completes at twenty-one days; the error rate grows with the square of the missing coverage. In reality the curve is jumpy. - **SP9.** The risk unit (ru) is **fictional**; it is not a real currency. One unplanned outage was counted as 3,600 ru. This is an assumption, and the second block measures what happens when it changes. - **SP10.** The day the exploit becomes accessible is known in the model. In reality it is estimated; the last block measures the cost of that estimate being wrong. - **SP11.** The approval gate's error multiplier is fixed, and the gate itself introduces no defects. - **SP12.** Faulty patches, rollbacks and staged rollout were not modeled; all three are held constant. ```js // patch/model.mjs — MODEL of window length: fictional advisory set, the day the // exploit becomes accessible, test coverage and the change failure rate. // No real patch is downloaded; fleet mechanics are taken as input. export const SEED = 20260811, ANNOUNCEMENT_COUNT = 40, ROLLOUT = 6; export const FULL_TEST = 21, ERROR_BASE = 0.01, ERROR_RISE = 0.34; export const OUTAGE_RU = 3600; // cost of one unplanned outage (risk units, fictional) const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; export const ANNOUNCEMENT = (() => { const r = generator(SEED); return Array.from({ length: ANNOUNCEMENT_COUNT }, (_, i) => ({ no: i + 1, score: +(4 + r() * 5).toFixed(1), affected: 20 + Math.round(r() * 280), // the day the exploit becomes usable: some are ready on the advisory day exploit: Math.round(Math.max(0, -6 + r() * 46)), })); })(); export const coverage = (W) => Math.min(1, W / FULL_TEST); export const errorRate = (W) => ERROR_BASE + ERROR_RISE * (1 - coverage(W)) ** 2; export const outages = (W) => Math.round(ANNOUNCEMENT_COUNT * ROLLOUT * errorRate(W)); export const caught = (W) => ANNOUNCEMENT.filter((d) => W <= d.exploit).length; export const exposure = (W) => ANNOUNCEMENT.reduce((a, d) => a + d.score * d.affected * Math.max(0, W - d.exploit), 0); export const total = (W) => exposure(W) + outages(W) * OUTAGE_RU; // Regime: a function that assigns a window to each advisory. Outage count is computed per advisory. export const REGIME = { 'no gate': { delay: 0, multiplier: 1.00 }, 'light approval': { delay: 2, multiplier: 0.60 }, 'full approval': { delay: 5, multiplier: 0.35 } }; export function measure(window, regimeName = 'no gate') { const d = REGIME[regimeName]; let exp = 0, out = 0, caughtCount = 0; for (const x of ANNOUNCEMENT) { const W = Math.max(window(x), d.delay); exp += x.score * x.affected * Math.max(0, W - x.exploit); out += ROLLOUT * errorRate(W) * d.multiplier; if (W <= x.exploit) caughtCount += 1; } return { exposure: exp, outages: Math.round(out), caught: caughtCount, total: exp + Math.round(out) * OUTAGE_RU }; } export const bestSingle = (regimeName) => { let b = 1; for (let W = 1; W <= 40; W += 1) if (measure(() => W, regimeName).total < measure(() => b, regimeName).total) b = W; return b; }; ``` #### Where the Two Curves Cross ```js // patch/window.mjs — window length sweep: caught open findings, exposure, outages and // the gain and cost of shortening by one day import { ANNOUNCEMENT, ROLLOUT, OUTAGE_RU, coverage, errorRate, measure } from './model.mjs'; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const single = (W) => measure(() => W); console.log(`${ANNOUNCEMENT.length} announcements, ${ROLLOUT} rollout events per announcement; ` + `one unplanned outage costs ${OUTAGE_RU} risk units (fictional)`); const A = [-9, 9, 11, 12, 10, 11, 11, 18]; y(A, 'window', 'test', 'caught', 'exposure', 'outage', 'outage', 'total', '1-day cut'); y(A, 'days', 'coverage', 'open', 'ru', 'events', 'ru', 'ru', 'gain - cost'); for (let W = 5; W <= 15; W += 1) { const r = single(W), previous = single(W - 1); const gain = r.exposure - previous.exposure; // gain of shortening by 1 day const cost = (previous.outages - r.outages) * OUTAGE_RU; // cost of shortening by 1 day y(A, W, coverage(W).toFixed(2), r.caught, Math.round(r.exposure), r.outages, r.outages * OUTAGE_RU, Math.round(r.total), `${Math.round(gain)} - ${Math.round(cost)}`); } const bestAt = (k) => { let b = 1; const t = (W) => single(W).exposure + single(W).outages * k; for (let W = 1; W <= 40; W += 1) if (t(W) < t(b)) b = W; return b; }; const cut = bestAt(OUTAGE_RU); const step = (W) => `${W} -> ${W - 1}: gain ` + `${Math.round(single(W).exposure - single(W - 1).exposure)} ru, cost ` + `${(single(W - 1).outages - single(W).outages) * OUTAGE_RU} ru`; console.log(`\ncrossing at day ${cut}: ${step(cut + 1)} | ${step(cut)}`); console.log('best window as outage cost changes:'); for (const k of [900, 3600, 14000]) { const W = bestAt(k); console.log(` ${String(k).padStart(5)} ru/outage -> ${String(W).padStart(2)} days; ` + `caught ${single(W).caught}/${ANNOUNCEMENT.length}, error rate ${errorRate(W).toFixed(3)}`); } ``` ``` 40 announcements, 6 rollout events per announcement; one unplanned outage costs 3600 risk units (fictional) window test caught exposure outage outage total 1-day cut days coverage open ru events ru ru gain - cost 5 0.24 30 48357 50 180000 228357 10800 - 21600 6 0.29 28 60960 44 158400 219360 12602 - 21600 7 0.33 27 74746 39 140400 215146 13787 - 18000 8 0.38 27 88533 34 122400 210933 13787 - 18000 9 0.43 26 103237 29 104400 207637 14704 - 18000 10 0.48 25 119524 25 90000 209524 16286 - 14400 11 0.52 24 137399 21 75600 212999 17876 - 14400 12 0.57 23 156171 17 61200 217371 18772 - 14400 13 0.62 23 174942 14 50400 225342 18771 - 10800 14 0.67 23 193714 11 39600 233314 18772 - 10800 15 0.71 23 212485 9 32400 244885 18772 - 7200 crossing at day 9: 10 -> 9: gain 16286 ru, cost 14400 ru | 9 -> 8: gain 14704 ru, cost 18000 ru best window as outage cost changes: 900 ru/outage -> 1 days; caught 34/40, error rate 0.318 3600 ru/outage -> 9 days; caught 26/40, error rate 0.121 14000 ru/outage -> 17 days; caught 22/40, error rate 0.022 ``` Two columns in the table move in opposite directions. As the window stretches from five days to fifteen, exposure climbs from 48,357 risk units to 212,485, while outage events drop from 50 to 9. Neither carries the decision alone: a program that watches only exposure shrinks the window to one day and ends up above fifty unplanned outages a year; a program that watches only outages stretches the window and never sees this. The last column makes the decision. Dropping from ten to nine gains 16,286 ru of exposure and adds 14,400 ru of outage cost — the gain is larger, so the cut is made. Dropping from nine to eight gains 14,704 ru but costs 18,000 ru — this step is reversed. **The two curves cross at day nine**, and the total takes its lowest value there (207,637 ru). The caught-open column carries a separate warning. It holds steady at 23 from day twelve through day fifteen, but exposure rises thirty-six percent over that same span. The caught-open count is a discrete metric and loses sensitivity as the window lengthens; exposure is continuous and counts every day. The question "how many open findings did we close in time" is the least informative form of the same data. The bottom block shows where the window comes from. When the outage cost is counted as 900 ru, the best window is one day; at 14,000 ru it is seventeen days — a seventeen-fold difference for the same forty advisories. **Window length is not a property of the finding; it is a consequence of the outage's cost.** When two organizations look at the same set of advisories and choose different windows, that is not an inconsistency — as long as the cost is written down. If it is not written down, the window is argued as a bare number, and the argument never closes. #### Approval Gate and the Two-Path Window ```js // patch/regime.mjs — change regime: the cost of the approval gate, the two-path window // and the cost of a classification error import { ANNOUNCEMENT, ROLLOUT, REGIME, measure, bestSingle } from './model.mjs'; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const APPROVAL_HOURS = { 'no gate': 0, 'light approval': 1.5, 'full approval': 5 }; // approval hours per rollout event const EVENTS = ANNOUNCEMENT.length * ROLLOUT; const A = [-16, 9, 9, 9, 11, 9, 11, 12]; console.log('1. three change regimes, each at its own best window'); y(A, 'regime', 'delay', 'error', 'window', 'caught', 'outage', 'total', 'approval'); y(A, '', 'days', 'factor', 'days', 'open', 'events', 'ru', 'hours'); const rows = []; for (const name of Object.keys(REGIME)) { const W = Math.max(bestSingle(name), REGIME[name].delay); const r = measure(() => W, name), hours = EVENTS * APPROVAL_HOURS[name]; rows.push({ name, W, r, hours }); y(A, name, REGIME[name].delay, REGIME[name].multiplier.toFixed(2), W, r.caught, r.outages, Math.round(r.total), hours); } console.log('\n2. risk reduction per unit of effort when moving up one regime'); for (let i = 1; i < rows.length; i += 1) { const d = rows[i - 1].r.total - rows[i].r.total, s = rows[i].hours - rows[i - 1].hours; console.log(` ${rows[i - 1].name} -> ${rows[i].name}: ${Math.round(d)} ru / ${s} hours = ` + `${(d / s).toFixed(0)} ru/hour`); } // Two-path regime: advisories whose exploit is ready early get a short window, the rest a long one. let best = null; for (let threshold = 2; threshold <= 20; threshold += 1) for (let s = 1; s <= 12; s += 1) for (let l = s; l <= 30; l += 1) { const r = measure((x) => (x.exploit <= threshold ? s : l)); if (!best || r.total < best.r.total) best = { threshold, s, l, r }; } const B = [-38, 11, 10, 11, 11]; console.log('\n3. two-path regime instead of a single window (no gate)'); y(B, 'regime', 'caught', 'outage', 'total ru'); const t9 = measure(() => bestSingle('no gate')); y(B, `single window ${bestSingle('no gate')} days`, t9.caught, t9.outages, Math.round(t9.total)); y(B, `two paths: <=${best.threshold} days -> ${best.s}, other -> ${best.l}`, best.r.caught, best.r.outages, Math.round(best.r.total)); // Classification error: the n advisories closest to the threshold are placed on the flipped path. const sorted = [...ANNOUNCEMENT].sort((a, b) => Math.abs(a.exploit - best.threshold) - Math.abs(b.exploit - best.threshold) || a.no - b.no); console.log('\n4. if the decision separating the two paths is wrong'); y(B, 'advisories placed on the flipped path', 'caught', 'outage', 'total ru'); for (const n of [0, 4, 12]) { const flipped = new Set(sorted.slice(0, n).map((x) => x.no)); const r = measure((x) => ((x.exploit <= best.threshold) !== flipped.has(x.no) ? best.s : best.l)); y(B, `${n} advisories`, r.caught, r.outages, Math.round(r.total)); } ``` ``` 1. three change regimes, each at its own best window regime delay error window caught outage total approval days factor days open events ru hours no gate 0 1.00 9 26 29 207637 0 light approval 2 0.60 6 28 26 154560 360 full approval 5 0.35 5 30 17 109557 1200 2. risk reduction per unit of effort when moving up one regime no gate -> light approval: 53078 ru / 360 hours = 147 ru/hour light approval -> full approval: 45002 ru / 840 hours = 54 ru/hour 3. two-path regime instead of a single window (no gate) regime caught outage total ru single window 9 days 26 29 207637 two paths: <=11 days -> 3, other -> 19 30 28 130159 4. if the decision separating the two paths is wrong advisories placed on the flipped path caught outage total ru 0 advisories 30 28 130159 4 advisories 26 22 155943 12 advisories 23 19 199971 ``` The first table reads in an unexpected direction. The full-approval gate adds a five-day delay to every change, yet its best window is **shorter** than the no-gate regime's (5 versus 9), and it catches more open findings (30 versus 26). The reason is in the second column: because the gate multiplies the error rate by 0.35, the outage cost of shortening by one day falls, so shortening stays profitable for longer. The approval gate's cost is paid **in hours, not in days**. The second block counts those hours. Moving from no gate to light approval gains 53,078 ru and costs 360 hours: 147 ru per hour. Moving from light approval to full approval gains 45,002 ru and costs 840 hours: 54 ru/hour. A ranking based on total risk puts full approval first; a ranking based on risk reduction per unit of effort stops at light approval. **The ranking changes when the metric changes**, and which one is right depends on where else those 840 hours could be used — the remediation-capacity gap from the previous lesson is one place those hours could go. The third block shows a gain that costs no hours. Splitting the advisories in two — a three-day window for those whose exploit is ready within eleven days, a nineteen-day window for the rest — drops the total from 207,637 to 130,159 ru. The gain comes from the sorting itself: the expensive testing cost of a short window is paid only for the advisories that need it. The fourth block measures what that gain depends on. When the four advisories closest to the threshold are placed on the flipped path, the total rises to 155,943 — a third of the gain is gone. When twelve advisories are misclassified, it settles at 199,971, and the gain over the single window is only ten percent. A second thing happens in the same rows: outage events **drop** from 28 to 19. A program that watches only the unplanned-outage count reads this deterioration as an improvement. #### Summary - As the window stretches from five days to fifteen, exposure climbs from 48,357 ru to 212,485, and outage events drop from 50 to 9; because the two columns move in opposite directions, neither carries the decision alone. - The two curves cross at day nine: dropping from 10 to 9 gains 16,286 ru at a cost of 14,400 ru; dropping from 9 to 8 gains 14,704 ru at a cost of 18,000 ru. The total is lowest there, at 207,637 ru. - The caught-open count holds steady between days 12 and 15 while exposure rises 36 percent: a discrete metric loses sensitivity as the window lengthens. - The best window shifts along with the outage's cost: 1 day at 900 ru, 17 days at 14,000 ru. The window is a consequence of the cost, not of the finding. - The full-approval gate adds a five-day delay, yet its best window is shorter than the no-gate regime's (5 versus 9) and it catches more open findings (30 versus 26); its cost is 1,200 approval hours. - By total risk, full approval leads; by gain per hour, light approval leads (147 versus 54 ru/hour). The two-path window drops the total to 130,159, but a twelve-advisory classification error gives back nine-tenths of that gain, and it does so while lowering the outage count. #### Next Step Both lessons worked on a queue and a calendar: items entered the queue, windows were tuned, and in both cases what changed was a configuration. A configuration stays changed once it is changed. The next lesson's subject is the one item that does not stay where it is tuned: human behavior. A training session raises a metric, then the metric drifts back to its old value within weeks, and the program has to spend the same effort again. What must be measured is not how good the training was, but **in which week the gain decays** — and because that decay does not run at the same speed in every group, which group gets the scarce training hours is once again a ranking question. ### Security Awareness Training Source: https://academia.sh/en/courses/governance-risk-and-compliance/security-awareness-training Retention is measured separately for five groups: the return week ranges from 5 to 29, and at a thirteen-week retraining interval the same data produces three different rankings by items, risk units and gain per hour, with all five groups changing place. The same 1,512-hour budget yields 51,066 risk units when distributed to everyone as a single interval and 64,391 when distributed group by group; the most populous group gets no training at all under the first two budgets. The previous two lessons worked on a queue and a calendar; in both, what changed was a configuration, and it stopped changing once the configuration did. This lesson's subject is the one item that never stops. A training session raises a metric, then the metric drifts back to its old value over the following weeks, and the program has to spend the same money all over again. The shape of decay on a single population, and the fact that a single training session lasts less than a quarter, were measured in the Attack Techniques and the Threat Landscape course; here that curve is **input** and is not re-derived. What is added at program scale is this: decay is not the same speed in every group, each group's event count and the weight an event carries differ, and training hours are scarce. Retention thus turns into a **ranking** question — which group the scarce hours go to, and how often. - **SP13.** Five groups, their headcounts, baseline rates, effect points and decay coefficients belong to the fictional regional measurement network and are a **model**. No real exercise is run. - **SP14.** Decay is exponential: a fixed fraction of the gain is retained every week. In reality decay is not smooth; the exponential form only approximates the return week. - **SP15.** Within-group distribution is ignored; each group is represented by its average. This hides the segment that never learns and makes the gain look smoother than it is. - **SP16.** The risk unit (ru) is **fictional**. The weight of an item passed within a group is fixed; in reality items differ even within the same group. - **SP17.** Training time is 1.5 hours per person, and session content is treated as identical across groups. The teaching effect of the measurement runs themselves is not modeled. - **SP18.** No one changes groups, leaves or joins; headcounts are constant across the fifty-two weeks. ```js // training/model.mjs — the group MODEL of the awareness program: on the fictional // regional measurement network, five groups, each with its baseline rate, training // effect, weekly decay and event flow. // The decay shape is taken as input from the Attack Techniques and the Threat Landscape course. export const WEEKS = 52, TRAINING_HOURS = 1.5, THRESHOLD = 1; // session hours per person export const GROUP = [ // name, people, baseline pass rate %, points the training removes, gain remaining per week, // annual suspicious events, risk units per item passed (fictional) { name: 'field team', people: 120, baseline: 34, effect: 20, decay: 0.86, events: 2400, weight: 40 }, { name: 'subscriber team', people: 60, baseline: 28, effect: 18, decay: 0.78, events: 5200, weight: 55 }, { name: 'billing', people: 35, baseline: 22, effect: 15, decay: 0.72, events: 1600, weight: 120 }, { name: 'technical team', people: 25, baseline: 16, effect: 9, decay: 0.62, events: 900, weight: 150 }, { name: 'management', people: 12, baseline: 30, effect: 21, decay: 0.90, events: 400, weight: 300 }, ]; export const gain = (g, h) => g.effect * g.decay ** h; export const rate = (g, h) => g.baseline - gain(g, h); export const returnWeek = (g) => { let h = 0; while (gain(g, h) >= THRESHOLD) h += 1; return h; }; // Items passed per year at interval T: the rate is computed for each week and multiplied by the event flow. export function passed(g, T) { let s = 0; for (let h = 0; h < WEEKS; h += 1) s += (g.events / WEEKS) * (T ? rate(g, h % T) : g.baseline) / 100; return s; } export const sessions = (T) => (T ? Math.ceil(WEEKS / T) : 0); export const hours = (g, T) => sessions(T) * g.people * TRAINING_HOURS; export const benefit = (g, T) => (passed(g, 0) - passed(g, T)) * g.weight; // risk units export const INTERVALS = [26, 13, 8, 4, 2]; ``` #### The Return Week Varies Fivefold by Group ```js // training/group.mjs — the retention of groups and three different rankings at the same retraining interval import { GROUP, WEEKS, THRESHOLD, gain, returnWeek, passed, hours, benefit } from './model.mjs'; const T = 13; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const rank = (f) => { const s = [...GROUP].sort((a, b) => f(b) - f(a)); return new Map(s.map((g, i) => [g.name, i + 1])); }; const A = [-17, 7, 9, 9, 9, 9, 10, 10]; console.log(`${GROUP.length} groups, ${GROUP.reduce((a, g) => a + g.people, 0)} people; ` + `return is counted once gain drops below ${THRESHOLD} point`); console.log('\n1. retention: gain remaining per week and the return week'); y(A, 'group', 'people', 'base %', 'eff pt', 'decay', 'h=4 pt', 'h=12 pt', 'return wk'); for (const g of GROUP) y(A, g.name, g.people, g.baseline, g.effect, g.decay.toFixed(2), gain(g, 4).toFixed(1), gain(g, 12).toFixed(1), returnWeek(g)); const B = [-17, 11, 11, 11, 9, 9, 9, 9]; console.log(`\n2. retraining every ${T} weeks: the same data, three different rankings`); y(B, 'group', 'saved', 'benefit', 'spent', 'ru/hour', 'rank', 'rank', 'rank'); y(B, '', 'items/yr', 'ru/yr', 'hours/yr', '', 'items', 'ru', 'ru/hr'); const sK = rank((g) => passed(g, 0) - passed(g, T)); const sR = rank((g) => benefit(g, T)); const sV = rank((g) => benefit(g, T) / hours(g, T)); for (const g of GROUP) y(B, g.name, (passed(g, 0) - passed(g, T)).toFixed(0), benefit(g, T).toFixed(0), hours(g, T), (benefit(g, T) / hours(g, T)).toFixed(1), sK.get(g.name), sR.get(g.name), sV.get(g.name)); const diff = (x, z) => GROUP.filter((g) => x.get(g.name) !== z.get(g.name)).length; console.log(`\nitems -> ru ranking, groups that changed place: ${diff(sK, sR)}/${GROUP.length}; ` + `ru -> ru/hour ranking: ${diff(sR, sV)}/${GROUP.length}; ` + `items -> ru/hour ranking: ${diff(sK, sV)}/${GROUP.length}`); const top = [...GROUP].sort((a, b) => (passed(b, 0) - passed(b, T)) - (passed(a, 0) - passed(a, T))); console.log(`${top[1].name}: item rank ${sK.get(top[1].name)}, ru/hour rank ${sV.get(top[1].name)}`); ``` ``` 5 groups, 252 people; return is counted once gain drops below 1 point 1. retention: gain remaining per week and the return week group people base % eff pt decay h=4 pt h=12 pt return wk field team 120 34 20 0.86 10.9 3.3 20 subscriber team 60 28 18 0.78 6.7 0.9 12 billing 35 22 15 0.72 4.0 0.3 9 technical team 25 16 9 0.62 1.3 0.0 5 management 12 30 21 0.90 13.8 5.9 29 2. retraining every 13 weeks: the same data, three different rankings group saved benefit spent ru/hour rank rank rank items/yr ru/yr hours/yr items ru ru/hr field team 227 9065 720 12.6 2 3 5 subscriber team 314 17288 360 48.0 1 1 2 billing 65 7802 210 37.2 3 4 3 technical team 16 2455 150 16.4 5 5 4 management 48 14457 72 200.8 4 2 1 items -> ru ranking, groups that changed place: 3/5; ru -> ru/hour ranking: 5/5; items -> ru/hour ranking: 4/5 field team: item rank 2, ru/hour rank 5 ``` The first table shows that retention is a property of the group. The return week is 5 for the technical team and 29 for management — more than a fivefold difference. The gain remaining at week four has already fallen to 1.3 points for the technical team while it is still 13.8 points for management; at week twelve those numbers are 0.0 and 5.9. Applying a single retraining calendar to the whole organization means spending the same money at a point where the gain is already gone in one of these two groups and two-thirds of it is still standing in the other. The second table is the real evidence. At the same thirteen-week interval, the same model, the same week — and three different rankings. By item count, subscriber team is first and field team is second; by risk unit, management moves up to second place; by gain per hour, management is first and field team is **fifth**. Going from the item ranking to the risk-unit ranking, three of the five groups change place; going from risk unit to gain per hour, **all five** change place. The field team sums up on its own how these three rankings diverge. Being the most populous group, it saves the most items (227 a year), but it is the group with the lowest weight per item (40 ru) and it swallows half the training hours by itself (720 hours). Management is the reverse: it saves only 48 items a year, but it carries 300 ru per item and, being twelve people, costs 72 hours a year. **The "how many people were trained" metric predicts none of this table's columns.** #### Same Budget, Two Allocations ```js // training/allocation.mjs — the return on retraining interval and the allocation of a fixed hour budget import { GROUP, INTERVALS, sessions, passed, hours, benefit } from './model.mjs'; const CHOICES = [0, ...INTERVALS]; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const total = (f) => GROUP.reduce((a, g) => a + f(g), 0); const A = [-10, 9, 11, 11, 11, 10]; console.log('1. same retraining interval for everyone'); y(A, 'interval', 'sessions', 'saved', 'spent', 'benefit', 'ru/hour'); y(A, 'weeks', 'per year', 'items/yr', 'hours/yr', 'ru/yr', ''); for (const T of INTERVALS) y(A, T, sessions(T), total((g) => passed(g, 0) - passed(g, T)).toFixed(0), total((g) => hours(g, T)), total((g) => benefit(g, T)).toFixed(0), (total((g) => benefit(g, T)) / total((g) => hours(g, T))).toFixed(1)); // Per-group interval choice under a fixed budget (exhaustive search: 6 choices x 5 groups) function bestAllocation(B) { let best = null; const walk = (i, remaining, choice, k) => { if (i === GROUP.length) { if (!best || k > best.k) best = { k, choice: [...choice] }; return; } for (const T of CHOICES) { const s = hours(GROUP[i], T); if (s <= remaining) { choice.push(T); walk(i + 1, remaining - s, choice, k + benefit(GROUP[i], T)); choice.pop(); } } }; walk(0, B, [], 0); return { ...best, hours: GROUP.reduce((a, g, j) => a + hours(g, best.choice[j]), 0) }; } const C = [-9, 12, 11, 12, 11, 26]; console.log('\n2. same budget, two allocations'); y(C, 'budget', 'one interval', 'benefit', 'free', 'benefit', 'free choice'); y(C, 'hours', 'weeks', 'ru/yr', 'hours', 'ru/yr', GROUP.map((g) => g.name.slice(0, 4)).join('/')); for (const B of [756, 1512, 2646]) { const T = INTERVALS.filter((t) => total((g) => hours(g, t)) <= B).sort((a, b) => a - b)[0]; const r = bestAllocation(B); y(C, B, T, total((g) => benefit(g, T)).toFixed(0), r.hours, r.k.toFixed(0), r.choice.join('/')); } ``` ``` 1. same retraining interval for everyone interval sessions saved spent benefit ru/hour weeks per year items/yr hours/yr ru/yr 26 2 364 756 28407 37.6 13 4 671 1512 51066 33.8 8 7 974 2646 73236 27.7 4 13 1333 4914 99659 20.3 2 26 1631 9828 122223 12.4 2. same budget, two allocations budget one interval benefit free benefit free choice hours weeks ru/yr hours ru/yr fiel/subs/bill/tech/mana 756 26 28407 756 44362 0/8/0/0/8 1512 13 51066 1494 64391 0/8/8/8/4 2646 8 73236 2596.5 84941 26/4/4/13/4 ``` The table above shows how many items a given retraining frequency saves. As the interval drops from twenty-six weeks to two weeks, items saved rise from 364 to 1,631 — four and a half times. Hours spent rise from 756 to 9,828: thirteen times. Gain per hour falls from 37.6 ru to 12.4 ru. Tightening the interval always raises the absolute gain and always lowers the efficiency; when the two columns are not written side by side, this trade-off is invisible. The table below holds this lesson's decision. The same 1,512-hour budget yields 51,066 ru when distributed to everyone as a single thirteen-week interval; distributed group by group, and while spending 18 fewer hours, it yields 64,391 ru. The difference is twenty-six percent and requires not a single extra hour. At the tighter budget of seven hundred fifty-six hours, the difference is even larger: 28,407 against 44,362, fifty-six percent. The choice column carries the truly uncomfortable result. Under the first two budgets, the field team — the organization's most populous group, 120 people — **gets no training at all**. The reason lies in the table's own logic: the same hour, in the same budget, cuts up to sixteen times more risk units in management and the subscriber team. The field team enters the list only once the budget rises to 2,646 hours, and even then at the sparsest interval on the list, two sessions a year. The chosen intervals are counterintuitive too. The fastest-forgetting group — the technical team, with a return week of 5 — gets an eight-week interval; the slowest-forgetting group — management, with a return week of 29 — gets a four-week interval, the most frequent one on the list. The rule "retrain more often for whoever forgets fastest" is wrong in this table, because the interval decision comes not from the decay rate but from the product of three numbers: how many people a session costs, how many events the group experiences, and how many risk units an event carries. For a twelve-person group at 300 ru per item, retraining every four weeks costs 234 hours a year; for a hundred-twenty-person group at 40 ru, retraining every twenty-six weeks costs 360 hours. The decay coefficient is only one factor in this decision. A limit of this result must be stated. The model counts only the measured item; the unmeasured effects of leaving a group out of training — the loss of a reporting habit, a group coming to see itself as excluded — are not in this table. The number is enough to decide where the budget should go; it is not enough to decide who gets no training at all. **A ranking that promotes the wrong item starts with treating the unmeasured column as zero.** #### Summary - The return week ranges from 5 to 29 by group; the gain remaining at week four is 1.3 points for the technical team and 13.8 points for management. A single retraining calendar does not see this difference. - At the same thirteen-week interval, three metrics produce three different rankings: going from the item ranking to the risk-unit ranking, 3 of the 5 groups change place; going from risk unit to gain per hour, all 5 change place. - The most populous group saves the most items (227 a year) but ranks last in gain per hour (12.6 ru/hour); the smallest group ranks first (200.8 ru/hour). - As the interval drops from 26 weeks to 2 weeks, items saved rise 4.5 times and hours spent rise 13 times; gain per hour falls from 37.6 to 12.4 ru. - The same 1,512 hours yield 51,066 ru as a single interval and 64,391 ru when distributed per group while spending 18 fewer hours; at a 756-hour budget the difference is fifty-six percent. - Under free allocation, the 120-person field team gets no training at all under the first two budgets. This result says where the budget should go; because the unmeasured column is counted as zero, it does not say who should get no training. #### Next Step All three lessons measured something that keeps flowing: the finding stream, the advisory stream, the decay of behavior. All three shared one thing — the measurement was taken while normal operation continued. The next lesson's subject is the moment normal operation stops, and there is no measurement in hand there, there is a **plan**: which service comes back up in how many minutes, in what order, and who does which step is written down. Being written down is not being verified. What is to be measured is not the quality of the plan but the gap between the plan and reality: how many items fail to hold in the exercise, how many minutes lie between the target time and the measured time, and which item that gap comes from. ### Business Continuity and Disaster Recovery Source: https://academia.sh/en/courses/governance-risk-and-compliance/business-continuity-and-disaster-recovery Twelve recovery items are run through both the plan and the drill: the plan's critical path stays under the declared 300-minute target at 260 minutes, the drill measures 421 minutes and overshoots the target by 121 minutes, and only 161 of the drill report's 293 minutes of delay are reflected in recovery time. When a six-person-day budget is spent by report-delay order the gain is zero; spent by critical-path-gain order it is 40 minutes. The cost of dropping below the target ranges from 32 to 21 person-days depending on the ranking measure. The previous three lessons each measured something that flows continuously, and in all three the measurement was taken while normal operations continued. This lesson's subject is the moment normal operations stop. There is no measurement on hand there, only a **plan**: which service comes back up in how many minutes, in what order, and who does which step is written down. Being written down is not the same as being verified. Backup types, restore mechanics and the definition of recovery objectives were measured in the Information Technology and Systems Fundamentals course; here they are taken as input. This lesson's measure is a single difference: **the gap between plan and reality.** It is written with three numbers — how many items' assumptions fail to hold in the drill, how many minutes separate the target time from the measured time, and which item those minutes come from. The third question is independent of the first two, because one item's delay may not be reflected in recovery time. - **SP19.** The twelve recovery items, the dependency graph, the plan durations and the assumptions that fail to hold in the drill all belong to the fictional regional measurement network and are a **model**. No real recovery is being performed. - **SP20.** An item starts only after all of its dependencies finish, and teams can work in parallel. The recovery team's headcount was not treated as limited; if it were limited, the measured duration would come out longer. - **SP21.** The minutes added by a failed assumption depend on the assumption type and do not vary by item. In reality the same assumption can cost one item ten minutes and another an hour. - **SP22.** The drill actually runs every item. Tabletop exercises never test some assumptions at all; in that case the number of items that fail to hold looks lower than it really is. - **SP23.** The fix cost is expressed in person-days and is **fictional**; fixes do not make each other cheaper. - **SP24.** The data-loss objective was not measured in this lesson; the only thing measured is time. ```js // continuity/model.mjs — MODEL of the recovery plan: fictional regional measurement network's // twelve recovery items, dependency graph, plan duration and the assumptions that fail to hold // in the drill. No real recovery is being performed; backup types and recovery mechanics are // taken as input. export const TARGET_MIN = 300; // recovery target the plan declares (minutes) // Assumption type: minutes added to an item's duration and the fix cost when it fails to hold in the drill export const ASSUMPTION = { 'authorized access': { min: 25, days: 2 }, 'current procedure': { min: 18, days: 3 }, 'readable backup': { min: 40, days: 5 }, 'contact list': { min: 12, days: 1 }, 'backup capacity': { min: 30, days: 8 }, }; // item: name, plan duration (min), dependencies, assumptions that FAIL TO HOLD in the drill export const ITEM = [ { name: 'backup power', plan: 25, deps: [], failed: [] }, { name: 'network backbone', plan: 35, deps: ['backup power'], failed: ['current procedure'] }, { name: 'identity service', plan: 40, deps: ['network backbone'], failed: ['authorized access'] }, { name: 'primary database', plan: 60, deps: ['network backbone'], failed: ['readable backup', 'authorized access'] }, { name: 'intake service', plan: 30, deps: ['identity service', 'primary database'], failed: ['current procedure'] }, { name: 'validation service', plan: 25, deps: ['intake service'], failed: [] }, { name: 'billing', plan: 45, deps: ['primary database', 'validation service'], failed: ['backup capacity'] }, { name: 'work order service', plan: 30, deps: ['identity service', 'primary database'], failed: ['contact list', 'current procedure'] }, { name: 'field application', plan: 20, deps: ['work order service'], failed: ['readable backup', 'contact list'] }, { name: 'subscriber portal', plan: 35, deps: ['identity service', 'billing'], failed: [] }, { name: 'monitoring & logging', plan: 20, deps: ['network backbone'], failed: ['authorized access'] }, { name: 'report generation', plan: 40, deps: ['billing'], failed: ['backup capacity'] }, ]; export const delay = (t) => t.reduce((a, v) => a + ASSUMPTION[v].min, 0); // Latest finish in the dependency graph: duration function item -> minutes export function finish(duration) { const f = new Map(); const compute = (name) => { if (f.has(name)) return f.get(name); const item = ITEM.find((x) => x.name === name); const start = item.deps.reduce((a, d) => Math.max(a, compute(d)), 0); f.set(name, start + duration(item)); return f.get(name); }; for (const item of ITEM) compute(item.name); return f; } export const total = (duration) => Math.max(...finish(duration).values()); export const planDuration = (item) => item.plan; export const measuredDuration = (closed = new Set()) => (item) => item.plan + delay(item.failed.filter((v) => !closed.has(`${item.name}|${v}`))); // Critical path: walk backward from the item whose finish time equals the total export function criticalPath(duration) { const f = finish(duration), last = Math.max(...f.values()); const path = []; let name = [...f.keys()].find((x) => f.get(x) === last); while (name) { path.unshift(name); const item = ITEM.find((x) => x.name === name); name = item.deps.slice().sort((p, q) => f.get(q) - f.get(p))[0]; } return path; } // Slack: minutes an item can slip without entering the critical path export function slack(duration) { const f = finish(duration), last = Math.max(...f.values()); const successors = new Map(ITEM.map((item) => [item.name, []])); for (const item of ITEM) for (const d of item.deps) successors.get(d).push(item.name); const late = new Map(); const compute = (name) => { if (late.has(name)) return late.get(name); const s = successors.get(name); late.set(name, s.length ? Math.min(...s.map((x) => compute(x) - duration(ITEM.find((z) => z.name === x)))) : last); return late.get(name); }; return new Map(ITEM.map((item) => [item.name, compute(item.name) - f.get(item.name)])); } export const CANDIDATE = ITEM.flatMap((item) => item.failed.map((v) => ({ item: item.name, assumption: v, key: `${item.name}|${v}`, min: ASSUMPTION[v].min, days: ASSUMPTION[v].days }))); ``` #### Reported Delay versus Reflected Delay ```js // continuity/drill.mjs — the gap between plan and reality: items that fail to hold, the gap // between target and measured duration, and how much of that delay is reflected in recovery time import { ITEM, TARGET_MIN, CANDIDATE, delay, total, criticalPath, slack, planDuration, measuredDuration } from './model.mjs'; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const planTotal = total(planDuration), measuredTotal = total(measuredDuration()); const pathPlan = criticalPath(planDuration), pathMeasured = new Set(criticalPath(measuredDuration())); const itemSlack = slack(measuredDuration()); console.log(`${ITEM.length} recovery items, declared target ${TARGET_MIN} min`); console.log(`plan critical path ${planTotal} min (${TARGET_MIN - planTotal} min under target), ` + `measured in drill ${measuredTotal} min (${measuredTotal - TARGET_MIN} min over target); difference ${measuredTotal - planTotal} min`); console.log(`items carrying a failed assumption: ` + `${ITEM.filter((item) => item.failed.length).length}/${ITEM.length}, ` + `${CANDIDATE.length} assumptions total, total reported delay ` + `${ITEM.reduce((a, item) => a + delay(item.failed), 0)} min`); const A = [-20, 7, 9, 10, 10, 9, 9]; console.log('\n1. plan, measured and slack per item'); y(A, 'item', 'plan', 'measured', 'delay', 'critical', 'slack', 'impact'); for (const item of ITEM) { const g = delay(item.failed); const solo = total(measuredDuration(new Set(item.failed.map((v) => `${item.name}|${v}`)))); y(A, item.name, item.plan, item.plan + g, g, pathMeasured.has(item.name) ? 'yes' : 'no', itemSlack.get(item.name), measuredTotal - solo); } console.log(`\ncritical path (same in plan and drill): ${pathPlan.join(' -> ')}`); console.log(`delay carried by items outside the critical path: ` + `${ITEM.filter((item) => !pathMeasured.has(item.name)).reduce((a, item) => a + delay(item.failed), 0)} min`); ``` ``` 12 recovery items, declared target 300 min plan critical path 260 min (40 min under target), measured in drill 421 min (121 min over target); difference 161 min items carrying a failed assumption: 9/12, 12 assumptions total, total reported delay 293 min 1. plan, measured and slack per item item plan measured delay critical slack impact backup power 25 25 0 yes 0 0 network backbone 35 53 18 yes 0 18 identity service 40 65 25 no 60 0 primary database 60 125 65 yes 0 60 intake service 30 48 18 yes 0 18 validation service 25 25 0 yes 0 0 billing 45 75 30 yes 0 30 work order service 30 60 30 no 86 0 field application 20 72 52 no 86 0 subscriber portal 35 35 0 no 35 0 monitoring & logging 20 45 25 no 298 0 report generation 40 70 30 yes 0 30 critical path (same in plan and drill): backup power -> network backbone -> primary database -> intake service -> validation service -> billing -> report generation delay carried by items outside the critical path: 132 min ``` The top lines show the plan hitting its target on paper: the critical path is 260 minutes, the declared target is 300 minutes, a forty-minute margin. Running the same graph with real durations in the drill produces 421 minutes — 121 minutes over the target. At least one assumption fails to hold in nine of the twelve items. The real distinction is in the last two columns. The drill report shows 293 minutes of delay, but only 161 minutes of that delay are reflected in recovery time. The remaining 132 minutes sit in items outside the critical path. The monitoring & logging item has 298 minutes of slack; its 25-minute delay does not extend recovery time by even one minute. The field application has a 52-minute delay and 86 minutes of slack; its impact is likewise zero. **A drill report's total delay line does not say how much longer recovery took.** The impact column has its own trap. The five critical-path items' individual impacts are 18, 60, 18, 30 and 30 minutes; they sum to 156. Yet fixing all of them at once yields a gain of 161 minutes. Each individual measurement is bounded by the second-longest path that takes over once that item is fixed; a list that evaluates items one at a time therefore shows a total gain that is **smaller** than it actually is. It is also worth noting that the critical path stayed the same in the plan and in the drill. The delays were large, but not large enough to change the order; the plan's assumption about which item was the bottleneck held, its assumption about durations did not. The two assumptions are tested separately and should be written up separately in the drill report. #### Same Budget, Three Rankings ```js // continuity/fix.mjs — ranks the drill findings by three different measures and runs them // against the same budget import { TARGET_MIN, CANDIDATE, total, planDuration, measuredDuration } from './model.mjs'; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const measuredTotal = total(measuredDuration()); // ties are broken by a fixed item order, so the ranking does not depend on how the names are spelled const TIE_ORDER = ['subscriber portal', 'network backbone', 'intake service', 'validation service', 'billing', 'monitoring & logging', 'work order service', 'identity service', 'report generation', 'field application', 'primary database', 'backup power']; const rank = (name) => TIE_ORDER.indexOf(name); const margin = (a, closed) => total(measuredDuration(closed)) - total(measuredDuration(new Set([...closed, a.key]))); const RULE = { 'report delay': (a) => a.min, 'critical path gain': (a, closed) => margin(a, closed), 'gain/day': (a, closed) => margin(a, closed) / a.days, }; function run(name, budget) { const rule = RULE[name], closed = new Set(); let remaining = budget, n = 0, list = [...CANDIDATE]; while (remaining > 0 && list.length) { list.sort((p, q) => rule(q, closed) - rule(p, closed) || rank(p.item) - rank(q.item)); const a = list.find((x) => x.days <= remaining); if (!a) break; remaining -= a.days; closed.add(a.key); n += 1; list = list.filter((x) => x !== a); } return { n, days: budget - remaining, duration: total(measuredDuration(closed)) }; } console.log(`measured in drill ${measuredTotal} min, plan ${total(planDuration)} min, target ${TARGET_MIN} min; ` + `${CANDIDATE.length} fix candidates, closing all of them costs ${CANDIDATE.reduce((a, x) => a + x.days, 0)} person-days`); const A = [-8, 20, 10, 7, 10, 11]; console.log('\n1. same budget, three different ranking measures'); y(A, 'budget', 'measure', 'fixes', 'days', 'time min', 'gain min'); for (const B of [6, 12, 20]) for (const name of Object.keys(RULE)) { const r = run(name, B); y(A, B, name, r.n, r.days, r.duration, measuredTotal - r.duration); } console.log('\n2. smallest budget needed to drop below the target'); for (const name of Object.keys(RULE)) { let g = 1; while (g <= 43 && run(name, g).duration > TARGET_MIN) g += 1; console.log(` ${name.padEnd(20)} ${g} person-days`); } console.log('\n3. two largest delays in the report'); for (const a of [...CANDIDATE].sort((p, q) => q.min - p.min || rank(p.item) - rank(q.item)).slice(0, 2)) console.log(` ${a.item} / ${a.assumption}: reported ${a.min} min, ${a.days} person-days, ` + `reflected in recovery time ${margin(a, new Set())} min`); ``` ``` measured in drill 421 min, plan 260 min, target 300 min; 12 fix candidates, closing all of them costs 43 person-days 1. same budget, three different ranking measures budget measure fixes days time min gain min 6 report delay 2 6 421 0 6 critical path gain 2 6 381 40 6 gain/day 3 6 378 43 12 report delay 3 12 381 40 12 critical path gain 4 12 338 83 12 gain/day 4 12 338 83 20 report delay 4 20 351 70 20 critical path gain 5 20 308 113 20 gain/day 8 19 320 101 2. smallest budget needed to drop below the target report delay 32 person-days critical path gain 25 person-days gain/day 21 person-days 3. two largest delays in the report field application / readable backup: reported 40 min, 5 person-days, reflected in recovery time 0 min primary database / readable backup: reported 40 min, 5 person-days, reflected in recovery time 40 min ``` The first triplet of rows carries this lesson's harshest result. When a six-person-day budget is spent by report-delay order, two fixes get done, the whole budget is gone, and recovery time **stays** at 421 minutes: zero gain. The same budget spent by critical-path-gain order yields 40 minutes, and by the order that divides gain by days it yields 43 minutes. All three rankings use the same twelve candidates, the same report and the same budget. The reason is written in the third block. The report's two largest delays are both forty minutes and both cost five person-days; the first one's impact on recovery time is **zero**, the second one's is 40 minutes. The number in the report does not give the fix's value; the value only emerges once the item is placed on the dependency graph. In the twenty-person-day row, the two smarter measures swap places. Up to twelve person-days, critical-path gain and gain/day give the same result (338 minutes); at twenty person-days gain/day stays at 320 minutes while critical-path gain drops to 308. **This is the point where the ranking changes**: as the budget grows, the measure that puts cheap items first runs out of budget before it gets around to the expensive item that gains more on its own. At a tight budget the efficiency measure leads; at a wide budget the direct-gain measure leads. The last block ties the decision to a single number. Dropping below the target costs 32 person-days by report-delay order, 25 by critical-path gain, 21 by gain/day order. The findings are the same, the plan is the same, the team is the same — the 11-person-day gap comes from the ranking measure alone. #### Summary - The plan's critical path stays under the declared 300-minute target at 260 minutes; the drill measures 421 minutes and overshoots the target by 121 minutes. At least one assumption fails to hold in nine of the twelve items. - Only 161 of the drill report's 293 minutes of delay are reflected in recovery time; the remaining 132 minutes sit in items that have slack (298 minutes of slack in the monitoring & logging item). - The critical-path items' individual impacts total 156 minutes; fixing all of them at once yields 161 minutes: evaluating them individually shows a smaller gain because of the second path that takes over. - The critical path stayed the same in the plan and the drill: the bottleneck assumption held, the duration assumption did not. The two assumptions are tested separately and written up separately. - When six person-days are spent by report-delay order the gain is zero; by critical-path-gain order it is 40 minutes, by gain/day order 43 minutes. The report's two largest delays are both 40 minutes and 5 person-days; one has an impact of 40 minutes, the other 0. - At twenty person-days the two measures swap places (308 versus 320 minutes); dropping below the target costs 32, 25 and 21 person-days — the 11-person-day gap comes from the ranking measure alone. #### Next Step Four lessons produced four different rankings, and the same pattern repeated in every one: a set of items, a measure, and an order that changes when the measure changes. This lesson's report-delay column was the extreme case — it was a real number, correctly measured, written in the drill report, and it spent a six-person-day budget for zero gain. A column like that is not wrong; it just **carries no decision**. The final lesson separates every program metric through this lens: which metric is tied to a decision and what happens once it crosses a threshold, which one only reaches the report, and how the list gets shorter once the cost of collecting a metric is placed side by side with the number of decisions it triggers. ### Security Metrics Source: https://academia.sh/en/courses/governance-risk-and-compliance/security-metrics Eighteen program metrics split into four groups: eight carry decisions and take 48 percent of the annual collection cost, two exceed their threshold but drop the signal because they have no defined action, five approach the threshold and never cross it, and three never engage their threshold at all. The most expensive metric costs 192 hours a year and produces zero decisions; six of the eight decision-bearing metrics swap places between the decision-count ranking and the hours-per-decision ranking. The previous lesson's reported-delay column is this lesson's starting point. That column was a real number, correctly measured, and it was written into the drill report; even so, it spent a six-person-day budget for zero gain. Its flaw was not that it was wrong — it was that it was never tied to a decision. This lesson sorts an entire program metric set with the same eye. The test is mechanical and has four questions: does the metric have a defined threshold, was the threshold exceeded during the period, is the action to take when it is exceeded defined, and is there a named owner to do that work. If all four hold, the metric has changed behavior; if one is missing, the metric is a report line. The second number to write beside it is the **collection cost**: how many hours per period it takes to produce that metric. - **SP25.** Eighteen metrics, twelve periods of observations, thresholds, and collection hours belong to the fictional regional measurement network and are a **model**. No real measurement runs. - **SP26.** A metric that exceeds its threshold was counted as producing a decision if it had a defined action and a named owner. In reality the defined action is not always carried out; this overstates the decision count. - **SP27.** Approaching the threshold was counted as at least one observation being less than fifteen percent away from the threshold; if that cutoff changes, the boundary between groups changes too. - **SP28.** The collection cost is a fixed number of hours per period; setup and automation cost were not modeled. - **SP29.** The metrics were treated as independent; in reality, one falls while another rises. - **SP30.** The randomness generator is written inline in the lesson and the seed is visible. ```js // metrics/model.mjs — the MODEL of program metrics: eighteen metrics, twelve periods of // observations, threshold, defined action, and collection cost in a fictional regional // measurement network. export const SEED = 20260829, PERIOD = 12; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; // name, base value, periodic amplitude, trend, threshold, threshold direction (+: above is // bad), defined action, named owner, collection hours per period export const METRICS = [ ['open queue', 175, 30, 4, 200, 1, true, true, 3], ['lagging item', 41, 12, 2, 60, 1, true, true, 2], ['average open age', 9, 3, 0.4, 12, 1, true, true, 2], ['patch window', 9, 2.5, 0, 12, 1, true, true, 1], ['unplanned outage', 29, 9, -0.6, 35, 1, true, true, 1], ['test coverage', 43, 8, 1.2, 40, -1, true, true, 4], ['training attendance rate', 68, 8, 0.3, 70, -1, false, true, 6], ['notification rate', 21, 5, 0.8, 19, -1, true, true, 5], ['failed drill item', 9, 2, -0.2, 6, 1, true, true, 8], ['recovery time', 421, 40, -6, 300, 1, true, true, 12], ['control coverage', 76, 5, 1.1, 70, -1, false, true, 9], ['requirement w/o evidence', 14, 4, -0.5, 8, 1, true, true, 7], ['unverified supplier control', 23, 3, 0.1, 40, 1, false, false, 6], ['asset not in inventory', 31, 9, 1.4, 25, 1, true, true, 14], ['risk-list top-ten change', 4, 2, 0, 8, 1, false, false, 5], ['audit finding', 11, 5, -0.4, 10, 1, true, true, 3], ['policy read rate', 94, 4, 0.2, 60, -1, false, false, 16], ['security incident count', 11, 5, 0.1, 12, 1, false, true, 2], ].map(([name, base, amplitude, trend, threshold, direction, action, owner, hours], i) => ({ no: i + 1, name, base, amplitude, trend, threshold, direction, action, owner, hours })); export const observe = (o) => { const r = generator(SEED + o.no * 977); return Array.from({ length: PERIOD }, (_, d) => +(o.base + o.trend * d + (r() - 0.5) * 2 * o.amplitude).toFixed(1)); }; export const exceeds = (o) => observe(o).filter((v) => (v - o.threshold) * o.direction > 0).length; // A crossing produces a decision: the threshold must be exceeded, and a defined action and // named owner must both exist. export const decisions = (o) => (o.action && o.owner ? exceeds(o) : 0); // If the threshold was never approached, the metric is counted as never having been tested // by the threshold at all. export const near = (o) => { const g = observe(o); const dist = g.map((v) => Math.abs(v - o.threshold) / Math.max(1, Math.abs(o.threshold))); return Math.min(...dist) <= 0.15; }; // Four groups: decision-bearing / signaling but not tied to a decision / near threshold / // inactive threshold export const group = (o) => (exceeds(o) > 0 ? (o.action && o.owner ? 'decision-bearing' : 'signal, no decision') : (near(o) ? 'near threshold' : 'inactive threshold')); export const GROUPS = ['decision-bearing', 'signal, no decision', 'near threshold', 'inactive threshold']; export const annualHours = (o) => o.hours * PERIOD; export const TOTAL_HOURS = METRICS.reduce((a, o) => a + annualHours(o), 0); ``` #### Decision-Bearing Metric versus Report Line ```js // metrics/decisions.mjs — sorts metrics by whether they changed behavior or not and // puts the decision count next to the collection cost import { METRICS, GROUPS, PERIOD, TOTAL_HOURS, exceeds, decisions, near, group, annualHours } from './model.mjs'; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd()); const rank = (l, f) => { const s = [...l].sort((a, b) => f(b) - f(a)); return new Map(s.map((o, i) => [o.name, i + 1])); }; console.log(`${METRICS.length} metrics, ${PERIOD} periods, collection cost ${TOTAL_HOURS} hours/yr`); const A = [-20, 8, 10, 9, 11]; console.log('\n1. the four groups of metrics'); y(A, 'group', 'metrics', 'hours/yr', '% hours', 'decis./yr'); for (const g of GROUPS) { const l = METRICS.filter((o) => group(o) === g), s = l.reduce((a, o) => a + annualHours(o), 0); y(A, g, l.length, s, (100 * s / TOTAL_HOURS).toFixed(1), l.reduce((a, o) => a + decisions(o), 0)); } const db = METRICS.filter((o) => decisions(o) > 0); const rD = rank(db, (o) => decisions(o)), rR = rank(db, (o) => decisions(o) / annualHours(o)); const B = [-26, 8, 10, 12, 8, 12]; console.log('\n2. decision-bearing metrics: two rankings'); y(B, 'metric', 'decis.', 'hours/yr', 'decis./hr', 'rank', 'rank'); y(B, '', 'yearly', '', '', 'decis.', 'decis./hr'); for (const o of [...db].sort((a, b) => decisions(b) - decisions(a))) y(B, o.name, decisions(o), annualHours(o), (decisions(o) / annualHours(o)).toFixed(3), rD.get(o.name), rR.get(o.name)); console.log(`metrics that swap rank between the two orderings: ` + `${db.filter((o) => rD.get(o.name) !== rR.get(o.name)).length}/${db.length}`); const priciest = [...METRICS].sort((a, b) => annualHours(b) - annualHours(a))[0]; console.log(`\nmost expensive metric: ${priciest.name}, ${annualHours(priciest)} hours/yr, ` + `threshold exceeded in ${exceeds(priciest)} periods, defined action ${priciest.action ? 'yes' : 'no'}, ` + `owner ${priciest.owner ? 'yes' : 'no'}, near threshold: ${near(priciest) ? 'yes' : 'no'}`); const inact = METRICS.filter((o) => group(o) === 'inactive threshold'); console.log(`removing the ${inact.length} metrics with an inactive threshold ` + `drops ${inact.reduce((a, o) => a + annualHours(o), 0)} hours/yr (` + `${(100 * inact.reduce((a, o) => a + annualHours(o), 0) / TOTAL_HOURS).toFixed(0)} percent of the total), decisions lost: ` + `${inact.reduce((a, o) => a + decisions(o), 0)}`); const sig = METRICS.filter((o) => group(o) === 'signal, no decision'); console.log(`adding a defined action and owner to the ${sig.length} signaling metrics ` + `adds ${sig.reduce((a, o) => a + exceeds(o), 0)} decisions per year; the collection cost stays at ` + `${sig.reduce((a, o) => a + annualHours(o), 0)} hours/yr`); ``` ``` 18 metrics, 12 periods, collection cost 1272 hours/yr 1. the four groups of metrics group metrics hours/yr % hours decis./yr decision-bearing 8 612 48.1 56 signal, no decision 2 96 7.5 0 near threshold 5 240 18.9 0 inactive threshold 3 324 25.5 0 2. decision-bearing metrics: two rankings metric decis. hours/yr decis./hr rank rank yearly decis. decis./hr recovery time 12 144 0.083 1 7 asset not in inventory 12 168 0.071 2 8 failed drill item 10 96 0.104 3 3 requirement w/o evidence 8 84 0.095 4 4 audit finding 6 36 0.167 5 1 open queue 3 36 0.083 6 5 average open age 3 24 0.125 7 2 lagging item 2 24 0.083 8 6 metrics that swap rank between the two orderings: 6/8 most expensive metric: policy read rate, 192 hours/yr, threshold exceeded in 0 periods, defined action no, owner no, near threshold: no removing the 3 metrics with an inactive threshold drops 324 hours/yr (25 percent of the total), decisions lost: 0 adding a defined action and owner to the 2 signaling metrics adds 12 decisions per year; the collection cost stays at 96 hours/yr ``` The first table splits the eighteen metrics into four groups, and the first number worth reading is how the collection cost is distributed. The eight decision-bearing metrics cost 612 hours a year; the remaining ten metrics cost 660 hours and **produce zero decisions**. More than half of the annual measurement effort goes to a set of numbers that never changed behavior at all. The conclusion here is not "drop the metric that produces no decision," because zero has three separate causes. The second group — the two metrics that exceed their threshold but have no defined action — is the worst case: the signal arrived and was dropped. The fix is not in measurement but in definition; the last line counts it: adding a defined action and owner brings twelve more decisions a year, and the collection cost does not change. The cost of a number that rises from zero to twelve is writing two lines into a document. The third group runs the opposite way. The five metrics that approach the threshold never produced an exceedance, but at least one observation of each passed within fifteen percent of the threshold. These are the only evidence that the threshold is **binding** and that the control holds; remove them and what remains is not evidence but optimism. They cost 240 hours a year, and that time was spent not per decision but per assurance. The fourth group is the one that can be dropped: three metrics whose threshold was never exceeded, never approached, and that have no defined action or owner. 324 hours a year, twenty-five percent of the total, zero decisions lost. The most expensive metric is also in this group: 192 hours a year on its own and zero exceedances. **Being expensive does not make a metric important**, because a number nobody has tied to a decision has never been questioned enough to be trimmed. The second table repeats this course's main evidence one last time. Ranked by decision count, recovery time and asset not in inventory come out on top; ranked by hours per decision, the two of them drop to seventh and **eighth** place, while audit finding climbs from fifth to first. Six of the eight metrics swap places: the two metrics that trigger the most decisions are the two most expensive per decision. #### Summary - A metric changes behavior only when it satisfies all four conditions at once: a defined threshold, an exceedance during the period, a defined action, and a named owner. If one is missing, the metric is a report line. - Eight of the eighteen metrics carry decisions (56 decisions a year, 612 hours); the remaining ten metrics cost 660 hours and produce zero decisions — 52 percent of the annual measurement effort. - Adding a defined action and owner to the two metrics that signal but are never tied to a decision adds twelve decisions a year without changing the collection cost. - The five metrics that approach the threshold and never exceed it (240 hours a year) are not dropped: they are the only evidence that the threshold is binding and that the control holds. - The group that can be dropped is the three metrics whose threshold is never even approached: 324 hours a year, 25 percent of the total, zero decisions lost. The most expensive metric (192 hours a year) is in this group. - Six of the eight decision-bearing metrics swap places between the decision-count ranking and the hours-per-decision ranking; the two metrics that trigger the most decisions are the two most expensive per decision. #### Course Wrap-Up These seventeen lessons repeated a single pattern: a set of items was defined, ranked by a measure, the risk each option reduced was written beside the unit it cost, and then the measure was changed and the point where the ranking broke was counted. The last column said the same thing in every lesson — **two different priorities can come out of the same data**; if the measure that produced the ranking is not written down, the list is not a decision, it is a preference with no visible justification. | Lesson | Item count | Ranking measure | Where the ranking breaks | |---|---|---|---| | Risk Assessment Process | 47 risk items | annual risk | 45 items shift once it switches to a ratio | | Qualitative and Quantitative Approaches | 47 items, 1,081 pairs | monetary value / qualitative level | 479 pairs tie, 52 pairs reverse | | Risk Treatment Options | 47 items, 4 options | annual reduction per unit | avoidance never enters even at 1,200 units | | Asset and Inventory Management | 69 items, 30 shadow | annual risk on the full list | 41 recorded items shift, 22 at most | | Third-Party Risk | 62 items, 16 controls | attestation / evidence | 51 items shift, 21 at most | | Security Frameworks | 31 controls, 5 functions | maturity / gain per unit effort | none of the five functions keep the same rank | | Management System Standards | 100 clauses, 3 scopes | absolute cost / cost per unit risk | the order of the three scopes fully reverses | | Audit Control Sets | 28 controls, 25 requirements | control count / evidence burden | three shared members in the top five | | Sector-Specific Compliance Requirements | 16 systems, 20 controls | shielded control / gain per unit day | the action that reduces the most comes last | | Personal Data Protection | 15 systems, 7 right types | missing response / person-hour burden | two fixes rank in reverse across two measures | | Policy, Standard and Procedure | 26 documents, 15 chains | document count / evidence produced | 14 person-days shows fully, adds no evidence | | The Function of Auditor and Compliance Roles | 24 controls, 17 findings | findings caught / days per finding | two ends of the four regulations trade off | | Vulnerability Management Program | 884 findings | score / score-to-effort ratio | two rules swap places at 109 hours/week | | Patch and Change Management | 40 advisories, 6 rollout incidents | exposure + outage cost | the window moves from 1 to 17 days as the cost changes | | Security Awareness Training | 5 groups, 252 people | item / per risk unit / per hour | all five groups shift on the final pass | | Business Continuity and Disaster Recovery | 12 items, 12 candidates | reported delay / critical path gain | two measures swap places at 20 person-days | | Security Metrics | 18 metrics | decision count / hours per decision | six of the eight metrics swap places | This course is also the end of the curriculum. Ten courses built ten separate measurement axes, and each defined its own unit: | Course | Measurement axis | Unit | |---|---|---| | K01 Introduction to Cybersecurity | the path a defense closes | path | | K02 Information Technology and Systems Fundamentals | where the boundary is and what opens if it falls | items needed to cross the boundary | | K03 Network Security | the unverified claim | claim | | K04 Identity, Access and Cryptography | the root of trust and the break point | items the chain must hold | | K05 Endpoint and System Hardening | the foothold | items run automatically | | K06 Attack Techniques and the Threat Landscape | the breakable link in the chain | link and the gain from breaking it early | | K07 Security Tooling and Penetration Testing | the gap between the candidate and the proof | evidenced finding | | K08 Security Operations and Incident Response | the time between detection and containment | minutes and items of evidence | | K09 Cloud and Virtualization Security | items reachable after a single mistake | blast radius | | K10 Governance, Risk and Compliance | risk reduction per unit of effort | rank | Across the ten courses there was one question asked: **what a defense closes and what it stops in return.** The unit changed from course to course — path, item, claim, link, minutes, radius, rank — but the question did not. No course used the word "secure" as an outcome, because what was always measured was **the path that remained**: the foothold that could not be closed, the claim that could not be verified, the link that would not break, the requirement without evidence, the item that could not be kept up with. The close is therefore written as a single rule: **a defense whose cost is not written down is counted as unmeasured.** How many legitimate requests were denied, how many outages it caused, how many hours it took, how many items were managed by hand — if that column is empty, the number left over is not a measurement, it is a claim. What this curriculum leaves behind is not a list of defenses, it is the habit of filling in that column.