--- title: 'The Secure Development Lifecycle' source: 'https://academia.sh/en/courses/secure-lifecycle' language: en updated: '2026-09-26T03:02:39+00:00' license: 'CC BY-SA 4.0' lessons: 11 outcomes: - 'Distributes security activities across stages and calculates the fix cost per stage' - 'Runs a threat modeling session with a data flow diagram and trust boundaries' - 'Writes security requirements in an automatically testable form' --- # The Secure Development Lifecycle The stage in which a vulnerability is born, the stage in which it is caught, and how many times its cost multiplies when it is caught late: the difference between security built into the flow and security standing as a separate gate, the gain of shifting left and the class of item that cannot be seen at design time, turning abuse cases into testable requirements, distributing security competence within the team, the risk-based distribution of limited development capacity, the vulnerability that threat modeling finds at the design stage, counting the flows that cross a trust boundary in a data flow diagram, the ratio of meaningful to meaningless threats produced by systematic threat generation methods, calculating the cheapest path in attack trees, and how much of the model output turns into work items. ## The DevSecOps Approach The measured difference between folding security into the development flow and keeping it as a separate gate: in which stage the same set of items is born, in which stage it is caught, and how many stages late that is, how many times shift left cuts the per-stage fix cost, and the item class for which catching things early does not help at all, the automated testability of security requirements, the requirements that abuse cases surface, spreading security competence across teams, and risk-based allocation of limited development capacity. ### The Difference Between DevSecOps and DevOps Source: https://academia.sh/en/courses/secure-lifecycle/devsecops-and-devops-difference The same 23-item set is run through two flows: a single security gate standing before deployment catches 10 items, never sees 5 of them, and leaves 120 changes waiting in the queue by day 60; once the check is spread across six stages, all 23 items are caught and the average delay drops from 1.94 stages to 0.13 stages. The cost of spreading it out is that 177 of 200 alerts are false positives. The previous curriculum spent ten courses counting the paths that defense closed, and at its close asked which item a scarce budget should go to. All of those measurements shared one silent assumption: **the software was already written.** The path set was extracted from a running system; the check was placed on a running system. This course removes that assumption and takes up the same measurement network's software while it is being built. The unit of measure changes. Instead of a path, a **stage** is counted: in which stage a vulnerability or a wrong decision is born, in which stage it can be caught, and how many stages apart the two are. This first lesson measures the crudest form of that difference — the difference between security being **built into** the flow and standing at the end of the flow as a **separate gate**. Both setups are run on the same item set; three columns come out: how many items are caught, how many stages late they are caught, and what is paid in return. - **DA1.** The network and its software are fictional: the regional measurement network's subscriber portal, meter collection service, billing, and field application are developed by five teams. The change flow and the item set are modeled with `node`; what is measured is this model. - **DA2.** The stage set has six members: requirement, design, code, build, deployment, runtime. An item is born in exactly one stage and cannot be caught before the stage it is born in. - **DA3.** The visibility vector is a modeling decision: it says whether a check placed at that stage can see the item or not. In reality this mapping is worked out by trial; the point of the lesson is not the mapping but the count run on top of it. - **DA4.** A check either sees an item or it does not. Partial visibility and the probability of missing it are not modeled; both would lower the number of items caught without changing the direction of the result. - **DA5.** Gate review happens at the batch level. If a batch contains an item the gate can see, the whole batch is sent back; the other changes in the batch wait along with it. - **DA6.** The alert counts and minute values are given model values. The false-positive count is not given directly; it is found by subtracting the items actually caught from the alert count. #### The Item Set and the Change Flow The model keeps two things separate: the **changes** that pass through the flow, and the **items** that a portion of those changes carry. An item is defined by the stage it is born in and a six-digit visibility vector. ```js // flow.mjs — the change flow and item set of the fictional measurement network's software (model) export const STAGE = ['requirement', 'design', 'code', 'build', 'deployment', 'runtime']; export const TEAM = [['subscriber-portal', 10], ['meter-collection', 8], ['billing', 6], ['field-app', 5], ['shared-library', 6]]; export const DAY = 60; // 12-week measurement window, business days // item: name | stage born in | visibility vector (1/0 per stage: can a check placed // there see this item?). The vectors are a modeling decision, not a measurement. export const ITEM = ` no-data-classification 0 100000 undefined-retention-period 0 100001 unclear-personal-data-scope 0 100010 unauthenticated-internal-call 1 011110 weak-session-model 1 010011 trust-boundary-free-flow 1 010000 missing-role-in-authz-model 1 011001 missing-authorization-check 2 001101 embedded-secret-key 2 001110 unvalidated-input 2 001001 personal-data-in-logs 2 001001 incomplete-error-path 2 001000 weak-randomness 2 001000 unbounded-query-size 2 000001 outdated-dependency 3 000110 unsigned-build-output 3 000110 unnecessary-build-tool 3 000100 default-password 4 000011 overly-broad-network-rule 4 000011 unencrypted-backup 4 000010 open-admin-endpoint 4 000011 race-under-scale 5 000001 third-party-disclosed-flaw 5 000001` .trim().split('\n').map((r) => { const [name, d, g] = r.trim().split(/\s+/); return { name, born: +d, g }; }); export const DAILY = TEAM.reduce((s, [, n]) => s + n, 0) / 5; export const TOTAL_CHANGES = DAILY * DAY; // The change that carries an item is placed at even intervals in the flow (model). export const CARRIER = new Map(ITEM.map((k, i) => [Math.floor((i * TOTAL_CHANGES) / ITEM.length), k])); // First visible stage: the first stage at or after birth where the visibility vector is 1. export const firstSeen = (k, stages) => stages.find((i) => i >= k.born && k.g[i] === '1') ?? null; ``` The vectors carry the lesson's real information. `no-data-classification` can be seen only in the requirement stage; no later check can find it, because there is no faulty line to find, only a missing decision. `unbounded-query-size` is born in the code stage but becomes visible only at runtime: there is nothing wrong in the code, only the absence of a limit that shows up once the data grows. #### The Separate Gate's Queue The first setup puts security at a single point: a review gate standing before deployment. The gate's daily capacity is limited, while the arrival rate is the sum of the teams. ```js // run1.mjs — single gate: changes queue at one point before deployment import { DAY, DAILY, TOTAL_CHANGES, CARRIER, ITEM } from './flow.mjs'; const CAPACITY = 5; // reviews the gate can finish per day (model) function run(b) { const batch = []; for (let i = 0; i < TOTAL_CHANGES; i += b) { const last = Math.min(i + b, TOTAL_CHANGES) - 1, contents = []; for (let c = i; c <= last; c++) if (CARRIER.has(c)) contents.push(CARRIER.get(c)); batch.push({ first: i, n: last - i + 1, ready: Math.floor(last / DAILY), contents, done: null }); } for (let d = 0; d < DAY * 20; d++) { let slot = CAPACITY; for (const p of batch) if (p.done === null && p.ready <= d && slot-- > 0) p.done = d; } // A change's wait: from the day it arrives to the day its batch finishes. let wait = 0; for (const p of batch) for (let c = p.first; c < p.first + p.n; c++) wait += p.done - Math.floor(c / DAILY); const open = batch.filter((p) => p.ready < DAY && p.done >= DAY); return { daily: batch.length / DAY, pending: open.reduce((s, p) => s + p.n, 0), avg: wait / TOTAL_CHANGES, sentBack: batch.filter((p) => p.contents.some((k) => k.g[4] === '1')).reduce((s, p) => s + p.n, 0) }; } const seen = ITEM.filter((k) => k.g[4] === '1').length; console.log(`${DAILY} changes/day, ${DAY} days, ${TOTAL_CHANGES} changes; ` + `gate ${CAPACITY} reviews/day; items the gate can see ${seen}/${ITEM.length}`); console.log('\nbatch reviews/day changes pending at day 60 average wait days changes sent back'); for (const b of [1, 2, 3, 5, 7]) { const r = run(b); console.log(String(b).padEnd(7) + r.daily.toFixed(1).padEnd(14) + String(r.pending).padEnd(30) + r.avg.toFixed(1).padEnd(22) + r.sentBack); } ``` ``` 7 changes/day, 60 days, 420 changes; gate 5 reviews/day; items the gate can see 10/23 batch reviews/day changes pending at day 60 average wait days changes sent back 1 7.0 120 12.0 10 2 3.5 0 0.1 20 3 2.3 0 0.1 30 5 1.4 0 0.3 50 7 1.0 0 0.0 70 ``` The first row gives the gate's capacity problem: seven changes arrive per day, the gate can finish five. Because the remaining two pile up, 120 changes are still in the queue on day sixty, with an average wait of 12 days. This is not a slowdown, it is a **saturated queue**: as long as the gap does not close, the wait time keeps growing as time passes. The teams' response to this is batching — piling up changes and sending them into a single review. The table shows that batching works: once the batch size rises to two, the queue closes and the average wait drops to a tenth of a day. The last column writes down the cost. While the number of items the gate actually sees stays fixed at **ten** in every row, the number of changes sent back because of those ten items rises from 10 to 70. At a batch size of seven, six extra legitimate changes are held up for the sake of a single item. The gate's second limit stands in the very first row: of the twenty-three items, only ten are in a form the gate can look at. The remaining thirteen items pass through the gate without any trouble, because the gate does not look at a requirements decision or a design gap — the only thing in its hands is the build itself, about to be deployed. #### The Same Item Set, a Distributed Check The second setup takes the same item set and spreads the check across six stages. The comparison is fair for exactly this reason: the only thing that changes is **where the check stands**. This is where the naming of the two setups earns its meaning. Development and operations merging into a single flow (DevOps) is not a tool choice, it is a feedback rule: the shorter the loop in which a decision sees its outcome, the more cheaply it gets fixed. The gate setup applies this rule to delivery speed but leaves the security item outside the rule; security's feedback loop, however fast it runs, passes through a single point at the very end of the flow. Building security into the flow (DevSecOps) consists of applying the same rule to the security item as well. What sets it apart is not adding a new kind of check — five of the six checks in the table could just as well be set up in the gate setup. What sets it apart is placing the check into the loop in which the item is born. ```js // run2.mjs — same item set, two flows: check at a single gate and spread across stages import { STAGE, ITEM, firstSeen } from './flow.mjs'; // Per-stage output of a check built into the flow (model): unit run, alerts produced, // developer minutes per alert. const CHECK = [['requirement review', 18, 5, 25], ['design session', 12, 9, 30], ['static code analysis', 420, 96, 6], ['dependency scan', 420, 54, 8], ['configuration audit', 60, 22, 12], ['runtime observation', 60, 14, 20]]; const flows = [['single gate', [4, 5]], ['integrated', [0, 1, 2, 3, 4, 5]]]; console.log('flow caught never seen total delay stages average delay'); const result = new Map(); for (const [name, stages] of flows) { const y = ITEM.map((k) => [k, firstSeen(k, stages)]); const found = y.filter(([, i]) => i !== null); const late = found.reduce((s, [k, i]) => s + i - k.born, 0); result.set(name, y); console.log(name.padEnd(13) + String(found.length).padEnd(11) + String(ITEM.length - found.length).padEnd(16) + String(late).padEnd(22) + (late / found.length).toFixed(2)); } console.log('\nstage born caught in gate flow caught in integrated flow'); for (const [i, a] of STAGE.entries()) console.log(a.padEnd(16) + String(ITEM.filter((k) => k.born === i).length).padEnd(7) + String(result.get('single gate').filter(([, j]) => j === i).length).padEnd(24) + result.get('integrated').filter(([, j]) => j === i).length); console.log('\ncheck unit alert actual false positive developer minutes'); let alertTotal = 0, minuteTotal = 0; for (const [i, [name, unit, alerts, minutes]] of CHECK.entries()) { const actual = result.get('integrated').filter(([, j]) => j === i).length; alertTotal += alerts; minuteTotal += alerts * minutes; console.log(name.padEnd(24) + String(unit).padEnd(7) + String(alerts).padEnd(7) + String(actual).padEnd(8) + String(alerts - actual).padEnd(16) + alerts * minutes); } console.log(`\ntotal ${alertTotal} alerts, ${minuteTotal} minutes (${(minuteTotal / 60).toFixed(1)} hours), ` + `per item ${(minuteTotal / 60 / 23).toFixed(1)} hours`); const gateHours = 300 * 3; console.log(`gate: 300 reviews x 3 hours = ${gateHours} hours, 10 items, ` + `per item ${(gateHours / 10).toFixed(0)} hours`); ``` ``` flow caught never seen total delay stages average delay single gate 18 5 35 1.94 integrated 23 0 3 0.13 stage born caught in gate flow caught in integrated flow requirement 3 0 3 design 4 0 4 code 7 0 6 build 3 0 3 deployment 4 10 4 runtime 2 8 3 check unit alert actual false positive developer minutes requirement review 18 5 3 2 125 design session 12 9 4 5 270 static code analysis 420 96 6 90 576 dependency scan 420 54 3 51 432 configuration audit 60 22 4 18 264 runtime observation 60 14 3 11 280 total 200 alerts, 1947 minutes (32.5 hours), per item 1.4 hours gate: 300 reviews x 3 hours = 900 hours, 10 items, per item 90 hours ``` The first table gives two separate numbers, and they must not be mixed up. In the gate flow, **18** items are caught, but only ten of these are caught at the gate; the remaining eight surface later through a runtime event. Surfacing is not the same as catching, but at least the item becomes known. **5** items, on the other hand, are items that neither setup sees: the classification gap in the requirement, the trust-boundary-free flow in the design, the incomplete error path and the weak randomness in the code, the unnecessary tool in the build. These are not in any record, do not enter any ranking, do not take up any budget. The delay column reduces the difference to a single number. Items caught in the gate flow are found a total of 35 stages late, 1.94 stages per item. In the integrated flow, total delay is 3 stages, and all of it comes from a single item: `unbounded-query-size`, born in the code stage, becomes visible only at runtime. The second table shows this one item directly. Seven items are born in the code stage; the integrated flow catches six of them in the code stage. The seventh belongs to a class no early check can look at, and it stays there no matter how far left you push the check. This course's second lesson counts exactly that class. #### The Cost of Spreading It Out The third table is the bill for the integrated flow. Six checks produce a total of 200 alerts; 23 of these are real items, 177 are false positives. The ratio varies from check to check: the requirement review gets three of its five alerts right, static code analysis gets six of its ninety-six. As frequency rises, the hit rate falls, because code analysis runs on every one of the 420 changes while the requirement review is convened only eighteen times. The total load is 1,947 developer-minutes, that is, 32.5 hours. The time the gate flow spends in the same window is 300 reviews times three hours, that is, 900 hours. The cost paid per item is 1.4 hours in the integrated flow, 90 hours in the gate flow — a sixty-four-fold difference. It should be noted that the two numbers do not measure the same work: the gate looks at every change, the integrated check looks only at the ones that produce an alert. That is also where the difference comes from. The second item of cost does not show up in the table. 177 false positives means interrupting a developer's work 177 times; ninety of these come from a single check. A check's real cost is not the time it runs but how many of the alerts it produces have to be read by a human. When the false-positive rate rises, alerts stop getting read, and the real items that check catches become invisible too — this produces the same outcome as never putting the check in place at all, except the cost keeps being paid. #### Summary - In the fictional measurement network's software, 420 changes flow over 60 business days and carry 23 items; items are born in all six stages, most of all in the code stage (7 items). - A single gate standing before deployment can see only 10 of the 23 items. 7 changes arrive per day, 5 reviews get finished; by day 60, 120 changes are waiting in the queue and the average wait is 12 days. - Batching closes the queue, but its cost is legitimate changes sent back: at batch size 1 it is 10, at batch size 7 it is 70 changes held up because of the same 10 items. - When the check is spread across six stages, all 23 items are caught; items never seen drop from 5 to 0, total delay drops from 35 stages to 3 stages, and delay per item drops from 1.94 to 0.13 stages. - The cost of spreading it out is that 177 of 200 alerts are false positives; 576 of the 1,947 developer-minutes go to the 96 alerts produced by a single check. - Of the 7 items born in the code stage, 6 are caught in the code stage; one becomes visible only at runtime, and an early check does not lower that item's delay. #### Next Step This lesson moved the check and counted that the number of items caught went up while the delay went down. The one thing not measured is what that delay actually costs. A delay of thirty-five stages is a number, but the difference between fixing an item at the design stage and fixing it at runtime is not a stage count, it is the work spent: changing a design decision is a meeting; taking the same decision back out of a deployed system is a data migration, a release plan, and an outage window. The next lesson puts the per-stage fix cost into the model, runs the same item set through two flows with late and early checks, and calculates the overall multiplier. The same lesson also asks a second question: for which items that multiplier does not hold — that is, how large is the class of items for which catching them early gains nothing at all. ### Shift Left Source: https://academia.sh/en/courses/secure-lifecycle/shift-left The fix cost per stage is derived from the work that gets undone, and the same 23-item set is run through two flows: on the 18 items common to both, the late flow costs 318 hours, the early flow 68 hours, a 4.68 multiplier, and once the check load is subtracted from the 250 hours prevented, a net gain of 217.5 hours remains. The multiplier is not a single number — it is 22.5 for an item born in the requirement stage and 13 for one born in deployment; half of the early flow's cost comes from the single item shift left can never touch. The previous lesson moved the check: with the check spread across six stages instead of a single gate, items caught rose from 18 to 23 and total delay dropped from 35 stages to 3 stages. Delay was a number, but a **stage** is not a unit of currency. Fixing an item at the design stage is a meeting's decision; taking the same item back out of a running system is a data migration, a release plan, and an outage window. The two jobs cannot be measured with the same unit. This lesson builds that unit. Moving the check earlier in the flow is called **shift left**, and the phrase by itself carries no decision — it does not answer how far left, for which item, or at what cost. Here the per-stage fix cost enters the model, the same item set runs through two flows with late and early checks, and the overall multiplier is calculated. The second half of the lesson looks the other way: it separates out the class of items for which shifting gains nothing, and counts that class's share of the cost. - **DA7.** The item set is the same as the previous lesson's and does not change; the only thing added is cost. - **DA8.** The fix cost is not a given multiplier, it is derived from the work that gets undone: however many stages have passed since the item was born, the work produced in those stages is redone as well. - **DA9.** The per-stage undo costs and the base fix effort are model values; the unit is fictional developer-hours. In a real organization these numbers are drawn from past fix records. - **DA10.** The comparison is made only on items caught in both flows. The cost of an item the late flow never sees is not zero, it is **unpaid**; it is written down separately. - **DA11.** The running load of the early checks is taken from the previous lesson and subtracted from the net gain as a single item. Setup, training, and maintenance load are not in the model; all of them would lower the net gain further. #### The Per-Stage Fix Cost Cost is not given as a multiplier, it is built. Every stage produces work; if the item is fixed after that stage has passed, the work produced there gets undone too. The total cost is the base fix effort plus the undo cost of every stage passed through. ```js // cost.mjs — the same item set, now with a per-stage fix cost (model) export const STAGE = ['requirement', 'design', 'code', 'build', 'deployment', 'runtime']; // The undo cost of passing a stage (fictional developer-hours): if the item has passed // that stage, fixing it redoes the work produced there as well. export const REDO = [0, 3, 8, 2, 6, 24]; export const BASE = 2; // the item's own fix effort, independent of stage // Cost: base effort plus the undo cost of every stage passed between birth and catch. export const cost = (born, caught) => BASE + REDO.slice(born + 1, caught + 1).reduce((s, x) => s + x, 0); export const ITEM = ` no-data-classification 0 100000 undefined-retention-period 0 100001 unclear-personal-data-scope 0 100010 unauthenticated-internal-call 1 011110 weak-session-model 1 010011 trust-boundary-free-flow 1 010000 missing-role-in-authz-model 1 011001 missing-authorization-check 2 001101 embedded-secret-key 2 001110 unvalidated-input 2 001001 personal-data-in-logs 2 001001 incomplete-error-path 2 001000 weak-randomness 2 001000 unbounded-query-size 2 000001 outdated-dependency 3 000110 unsigned-build-output 3 000110 unnecessary-build-tool 3 000100 default-password 4 000011 overly-broad-network-rule 4 000011 unencrypted-backup 4 000010 open-admin-endpoint 4 000011 race-under-scale 5 000001 third-party-disclosed-flaw 5 000001` .trim().split('\n').map((r) => { const [name, d, g] = r.trim().split(/\s+/); return { name, born: +d, g }; }); export const firstSeen = (k, stages) => stages.find((i) => i >= k.born && k.g[i] === '1') ?? null; ``` ```js // run1.mjs — the multiplier is not one number: it changes with the stage of birth import { STAGE, REDO, cost, ITEM } from './cost.mjs'; console.log('birth stage redo work earliest cost latest cost x items'); for (const [i, a] of STAGE.entries()) { const early = cost(i, i), late = cost(i, STAGE.length - 1); console.log(a.padEnd(16) + String(REDO[i]).padEnd(15) + String(early).padEnd(16) + String(late).padEnd(14) + (late / early).toFixed(1).padEnd(6) + ITEM.filter((k) => k.born === i).length); } console.log('\nsame item, delayed stage by stage (an item born in design)'); console.log('stage caught in cost rise from previous stage'); let previous = null; for (let j = 1; j < STAGE.length; j++) { const b = cost(1, j); console.log(STAGE[j].padEnd(19) + String(b).padEnd(7) + (previous === null ? '-' : `${b - previous} hours, ${(b / previous).toFixed(2)}x`)); previous = b; } ``` ``` birth stage redo work earliest cost latest cost x items requirement 0 2 45 22.5 3 design 3 2 42 21.0 4 code 8 2 34 17.0 7 build 2 2 32 16.0 3 deployment 6 2 26 13.0 4 runtime 24 2 2 1.0 2 same item, delayed stage by stage (an item born in design) stage caught in cost rise from previous stage design 2 - code 10 8 hours, 5.00x build 12 2 hours, 1.20x deployment 18 6 hours, 1.50x runtime 42 24 hours, 2.33x ``` The first table shows why reducing the multiplier column to a single number is wrong. An item born in the requirement stage, when caught at the latest, costs 22.5 times more to fix; for an item born in deployment, the same number is 13. The difference does not come from the item's severity but from **how many stages' worth of work got produced after the point it was born**. The sentence "a vulnerability found late costs multiple times more" is true, but which multiple depends on where the item comes from, and there is a 1.7-fold difference between the two. The second table says something sharper: the rise is not even. When an item born in design slips into the code stage, the cost jumps to five times; moving on from there into the build stage, only to 1.2 times. There are two boundaries where the rise concentrates — the stage where the code is written and the stage where the system becomes live. The build and deployment boundaries between them are cheap. This directly determines where to shift the check to: pulling a check back from deployment to build gains 6 hours, pulling it back from code to design gains 8 hours. The same effort crosses different boundaries, and they are not all equal. #### Two Flows, Overall Multiplier The same item set runs through two flows: a late flow, where the check exists only at deployment and runtime, and an early flow, where it exists in all six stages. ```js // run2.mjs — the same item set in two flows: checked late and checked early import { STAGE, cost, ITEM, firstSeen } from './cost.mjs'; const LATE = [4, 5], EARLY = [0, 1, 2, 3, 4, 5]; const CHECK_LOAD = 32.5; // running load of the early checks (from the previous lesson, hours) const rows = ITEM.map((k) => { const g = firstSeen(k, LATE), e = firstSeen(k, EARLY); return { k, g, e, gb: g === null ? null : cost(k.born, g), eb: cost(k.born, e) }; }); const common = rows.filter((s) => s.g !== null); const earlyOnly = rows.filter((s) => s.g === null); const noGain = common.filter((s) => s.g === s.e); const sum = (l, f) => l.reduce((a, s) => a + f(s), 0); console.log(`${ITEM.length} items; caught in both ${common.length}, ` + `never seen by late flow ${earlyOnly.length}`); const gt = sum(common, (s) => s.gb), et = sum(common, (s) => s.eb); console.log(`late flow ${gt} hours, early flow ${et} hours, x ${(gt / et).toFixed(2)}, ` + `prevented ${gt - et} hours`); console.log(`net gain: ${gt - et} - ${CHECK_LOAD} = ${(gt - et - CHECK_LOAD).toFixed(1)} hours`); console.log(`the ${earlyOnly.length} items the late flow never sees close out at ` + `${sum(earlyOnly, (s) => s.eb)} hours in the early flow; in the late flow their cost is never paid`); console.log('\nwhere the gain concentrates'); console.log('birth stage items late cost early cost prevented share %'); for (const [i, a] of STAGE.entries()) { const l = common.filter((s) => s.k.born === i); if (l.length === 0) { console.log(a.padEnd(16) + '0'); continue; } const g = sum(l, (s) => s.gb), e = sum(l, (s) => s.eb); console.log(a.padEnd(16) + String(l.length).padEnd(7) + String(g).padEnd(11) + String(e).padEnd(13) + String(g - e).padEnd(11) + (((g - e) / (gt - et)) * 100).toFixed(1)); } // Items with no gain fall into two classes: those unreachable because they are born // late, and those born early but visible only late. const lateBorn = noGain.filter((s) => s.k.born >= 4); const lateSeen = noGain.filter((s) => s.k.born < 4); console.log(`\n${noGain.length} items the early check does not help, two classes`); console.log('class items cost in early flow % of early flow cost'); for (const [name, l] of [['born early, seen late', lateSeen], ['born late', lateBorn]]) console.log(name.padEnd(26) + String(l.length).padEnd(7) + String(sum(l, (s) => s.eb)).padEnd(22) + ((sum(l, (s) => s.eb) / et) * 100).toFixed(1)); for (const s of lateSeen) console.log(` ${s.k.name}: born in the ${STAGE[s.k.born]} stage, visible in the ${STAGE[s.e]} ` + `stage, ${s.eb} hours in both flows`); ``` ``` 23 items; caught in both 18, never seen by late flow 5 late flow 318 hours, early flow 68 hours, x 4.68, prevented 250 hours net gain: 250 - 32.5 = 217.5 hours the 5 items the late flow never sees close out at 10 hours in the early flow; in the late flow their cost is never paid where the gain concentrates birth stage items late cost early cost prevented share % requirement 2 66 4 62 24.8 design 3 78 6 72 28.8 code 5 146 42 104 41.6 build 2 16 4 12 4.8 deployment 4 8 8 0 0.0 runtime 2 4 4 0 0.0 7 items the early check does not help, two classes class items cost in early flow % of early flow cost born early, seen late 1 34 50.0 born late 6 12 17.6 unbounded-query-size: born in the code stage, visible in the runtime stage, 34 hours in both flows ``` On the eighteen items common to both, the late flow spends 318 hours, the early flow 68 hours; the overall multiplier is 4.68. This is considerably lower than the 13–22.5 range read off a single item, and it should be lower: the multiplier is weighted by the distribution of when items are born, and a quarter of the set is already born in deployment or runtime and produces no gain at all. A program's real multiplier is always smaller than the multiplier of any single item shift left can reach. Once the running load of the early checks is subtracted from the 250 hours prevented, a net gain of 217.5 hours remains. The form of the number matters as much as the number itself: shift left has become something that can be written not as a principle but as a two-item subtraction. Setup, training, and maintenance loads, which are not in the model, would lower this number further; in a real organization the decision looks at whether the hours prevented also cover these three items. The same subtraction also gives a break-even point. The 250 hours prevented can carry at most 217.5 hours of setup, training, and maintenance on top of the running load; every hour above that threshold turns shifting into a loss. If setting up all six checks does not fit this budget, the right decision is not to set up all of them but to start with the stages whose share of the gain is highest — and that order is written in this table's share column. A silent assumption of the model needs to be said out loud here: every item caught was counted as fixed. In reality, a portion of caught items are deliberately deferred. A deferred item follows the same path as an item never caught, and in the end pays the late flow's cost; the early check has only made the payment **visible**. This is why the 250 hours prevented is not a gain but an **upper bound**: realizing it depends on having the capacity to actually fix the items that were caught. The five items the late flow never sees were left out of the comparison. The early flow closes them out at 10 hours; in the late flow their cost is not zero, it is **unpaid**. This is the most misleading row a fix-cost table can have: an item that is never found looks, in the table, like the cheapest item of all. The distribution of the gain says where to shift to. Of the 250 hours prevented, 41.6% comes from items born in the code stage, 28.8% from design, 24.8% from items born in the requirement stage. Items born in the build stage produce only 4.8% of the total; those born in deployment and runtime produce zero. A check placed on the first three stages carries 95% of the gain. #### The Item Shifting Does Not Help The last table is this lesson's real finding. Seven of the eighteen items gain nothing at all from the early check, and these seven items form two separate classes. The first class has six items and is unreachable by definition: they are born in deployment or runtime. A configuration item or a dependency vulnerability disclosed after the fact cannot be discussed in a design session, because it does not exist yet when that session happens. This class's cost is low — 17.6% of the early flow's cost — because they are caught in the very stage they are born in anyway. The second class is a single item, and it is this lesson's hard finding. `unbounded-query-size` is born in the code stage but becomes visible only at runtime, and it costs 34 hours in both flows. Exactly half of the early flow's total 68-hour cost comes from this one item. No matter how far left you push the check, this number does not drop, because there is nothing to see in the code: there is no wrongly written line, only a limit that was never put in place, and the need for that limit shows up only once the data grows. What this class has in common is that the item's very existence depends on a scale, a data distribution, or an external event. It includes race conditions that surface under load, missing limits that only become meaningful past a certain data volume, and dependency vulnerabilities that become known only through an outside disclosure. For these items, the right decision is not to shift the check but to make **seeing them at runtime cheaper** — measurement, limit checks, and rollback readiness. A program that does not write down the scope of shift left reads this class's cost as a failure of shifting too, and puts more checks in the wrong place. The measurable counterpart of this distinction is: if an item's late catch comes from a missing check, the early flow makes it cheaper; if the item itself only becomes visible late, the early flow's cost stays **exactly the same** as the late flow's. The last column of the table looks for exactly this equality. #### Summary - The fix cost is not given as a multiplier, it is derived from the work that gets undone: base effort plus the undo cost of every stage passed between birth and catch. - The multiplier is not a single number: 22.5 for an item born in the requirement stage, 13 for one born in deployment. The rise is not even either — moving from design to code is a 5x jump, from code to build only 1.2x. - On the 18 items common to both, the late flow spends 318 hours, the early flow 68 hours; the overall multiplier is 4.68, prevented hours 250. After subtracting the check load, net gain is 217.5 hours. - The 5 items the late flow never sees are outside the comparison: their cost is not zero, it is unpaid, and the early flow closes them out at 10 hours. - 95% of the hours prevented come from items born in the first three stages; those born in build produce 4.8%, and those born in deployment and runtime produce 0%. - 7 of the 18 items gain nothing at all from the early check. Six are unreachable because they are born late; the seventh is born in code and becomes visible only at runtime, carrying half of the early flow's cost on its own. #### Next Step This lesson pinned the gain and the limit of shifting to a number, but left the stage carrying the largest share of that gain inside a black box. A quarter of the hours prevented comes from items born in the requirement stage, and the check that catches those items passed by in a single table row: eighteen requirement reviews, five alerts, three real items. How a requirement review catches an item was never opened up. For it to be able to catch one, the requirement has to be written **testably**; a line written as "the system must be secure" produces no item in any review. The next lesson classifies security requirements along the axis of testability: how many requirements are automatically testable, how many can only be checked by eye, how many cannot be tested at all, and how many new requirements enter the list once an abuse case is written. ### Security Requirements Source: https://academia.sh/en/courses/secure-lifecycle/security-requirements Testability is derived from three parts and run on 20 security requirements: 13 are automatically testable, 3 are checked by eye because of a single missing part, and 4 can never move into any class without being rewritten, because of their qualitative predicate. Five abuse cases grow the list by 40 percent, and 7 of the 8 incoming requirements are in denying mode; the 40-hour load of manual checking drops entirely once the missing parts are written. The previous lesson counted that a quarter of the 250 hours prevented comes from items born in the requirement stage, but left the check that catches those items in a single table row: eighteen requirement reviews, five alerts, three real items. How a review catches an item was never opened up. It has exactly one condition for catching one: the requirement has to be written in a form where whether it is met is **testable**. The line "the system must be secure" produces no item in any review, because there is no observation that would show it has been violated. This lesson takes testability out of being an adjective and splits it into three parts, then runs it on a list of twenty requirements. In the second half it asks where the list comes from: the requirements written by reading a feature list and the requirements written by considering an actor pursuing a target the system must **deny** do not produce the same set. - **DA12.** The requirement list is fictional and is modeled with `node`. Each requirement is written with four fields: subject, predicate type, threshold, scope. - **DA13.** Testability is not given, it is **derived**: if the predicate is observable and both the threshold and the scope are written, the requirement is automatically testable. If one of the parts is missing it is checked by eye; if the predicate is qualitative it can never be tested. - **DA14.** The predicate-type and mode fields are a record of how the requirement is **written**, not a judgment of its content's value. A requirement written untestably is not unimportant; the only thing it lacks is that it cannot be checked. - **DA15.** An abuse case is modeled with an actor and a target; no step is given in an executable form. Trying out the scenario outside an organization's own system requires **written authorization and a defined scope**; otherwise it is both illegal and unprofessional. - **DA16.** The run counts and the manual-check minutes are model values; they are consistent with the change and deployment counts from the previous lessons. #### The Three Parts of Testability For a requirement to be automatically testable, three things are needed: an **observable predicate** where the violation would be seen, a **threshold** that says it has been met, and a **scope** that says where it applies. If one of the three is missing, testing is handed to a human. ```js // requirement.mjs — security requirements of the measurement network software (model) // row: name | subject | predicate type | threshold | scope | mode // predicate: event = observable event, state = auditable state, qualitative = qualitative adjective // mode: provides = the work the system does, denies = the work the system refuses const parse = (s) => s.trim().split('\n').map((r) => { const [name, subject, predicate, threshold, scope, mode] = r.trim().split(/\s+/); return { name, subject, predicate, threshold, scope, mode }; }); export const REQUIREMENT = parse(` session-duration-limit session state 30-min subscriber-portal denies failed-login-limit login-endpoint event 5-attempts subscriber-portal denies transport-encryption external-connection state required all-endpoints provides measurement-data-retention measurement-record state 24-months measurement-store provides authorization-check-coverage read-endpoint event every-request billing provides secret-in-source source-file state 0-count all-stores denies personal-data-in-logs log-line state 0-fields all-services denies dependency-age dependency state 180-days all-builds provides admin-endpoint-access admin-endpoint event internal-network meter-collection denies backup-encryption backup-file state required billing-db provides request-rate-limit general-endpoint event 60-requests-min subscriber-portal denies session-termination logout-action event 1-request subscriber-portal provides file-upload-size upload-endpoint state 10-mb field-app denies error-message-content error-response state - all-endpoints denies audit-log-retention authorization-change event every-change - provides password-storage-format subscriber-record state - subscriber-portal provides authentication-strength subscriber-login qualitative - subscriber-portal provides interface-security field-app qualitative - - provides data-integrity measurement-record qualitative - - provides separation-of-duties billing-approval qualitative - billing denies`); // Testability is derived: with an observable predicate + threshold + scope, it is tested automatically. export function classify(g) { if (g.predicate === 'qualitative') return 'untestable'; const missing = ['threshold', 'scope'].filter((a) => g[a] === '-'); return missing.length === 0 ? 'automatic' : 'manual'; } export const missingPart = (g) => ['threshold', 'scope'].filter((a) => g[a] === '-'); // Abuse case: an actor pursues a target the system must deny. // Written as a model; no step is given in an executable form. export const SCENARIO = [ ['subscriber-account-takeover', 'external-actor', ['failed-login-limit', 'session-duration-limit', 'session-termination', 'login-notification']], ['reading-someone-elses-bill', 'authorized-subscriber', ['authorization-check-coverage', 'record-id-guessability']], ['altering-a-measurement-record', 'field-staff', ['audit-log-retention', 'measurement-write-authorization', 'measurement-change-approval']], ['bulk-subscriber-data-pull', 'authorized-subscriber', ['request-rate-limit', 'bulk-download-limit', 'outbound-volume-threshold']], ['device-registration-spoofing', 'external-actor', ['device-registration-verification', 'registration-rate-limit']], ]; // Requirements that come out of the scenarios and are not on the list, written the same way: export const NEW = parse(` login-notification successful-login event every-login subscriber-portal provides record-id-guessability billing-record state not-guessable billing denies measurement-write-authorization measurement-record event role-matched measurement-store denies measurement-change-approval measurement-correction event two-signatures measurement-store denies bulk-download-limit external-export state - subscriber-portal denies outbound-volume-threshold external-export state 50-mb-day all-endpoints denies device-registration-verification device-registration event field-approval field-app denies registration-rate-limit device-registration event 10-registrations-hour - denies`); ``` ```js // run1.mjs — requirements are split by testability; missing parts are counted import { REQUIREMENT, classify, missingPart } from './requirement.mjs'; const count = (l, s) => l.filter((g) => classify(g) === s).length; console.log(`${REQUIREMENT.length} requirements; automatic ${count(REQUIREMENT, 'automatic')}, ` + `manual ${count(REQUIREMENT, 'manual')}, untestable ${count(REQUIREMENT, 'untestable')}`); console.log('\nclass requirements has threshold has scope observable predicate'); for (const s of ['automatic', 'manual', 'untestable']) { const l = REQUIREMENT.filter((g) => classify(g) === s); console.log(s.padEnd(12) + String(l.length).padEnd(14) + String(l.filter((g) => g.threshold !== '-').length).padEnd(15) + String(l.filter((g) => g.scope !== '-').length).padEnd(11) + l.filter((g) => g.predicate !== 'qualitative').length); } console.log('\nrequirements that do not reach automatic, and their missing parts'); for (const g of REQUIREMENT.filter((x) => classify(x) !== 'automatic')) console.log(' ' + g.name.padEnd(25) + classify(g).padEnd(12) + (g.predicate === 'qualitative' ? 'qualitative predicate: cannot change class without a rewrite' : `missing: ${missingPart(g).join(', ')}`)); const singleMissing = REQUIREMENT.filter((g) => classify(g) === 'manual' && missingPart(g).length === 1); console.log(`\n${singleMissing.length} requirements move to automatic by adding a single part; ` + `${count(REQUIREMENT, 'untestable')} requirements need a rewrite`); console.log(`after the fix, automatic ${count(REQUIREMENT, 'automatic') + singleMissing.length}/` + `${REQUIREMENT.length}, remaining ${REQUIREMENT.length - count(REQUIREMENT, 'automatic') - singleMissing.length}`); ``` ``` 20 requirements; automatic 13, manual 3, untestable 4 class requirements has threshold has scope observable predicate automatic 13 13 13 13 manual 3 1 2 3 untestable 4 0 2 0 requirements that do not reach automatic, and their missing parts error-message-content manual missing: threshold audit-log-retention manual missing: scope password-storage-format manual missing: threshold authentication-strength untestable qualitative predicate: cannot change class without a rewrite interface-security untestable qualitative predicate: cannot change class without a rewrite data-integrity untestable qualitative predicate: cannot change class without a rewrite separation-of-duties untestable qualitative predicate: cannot change class without a rewrite 3 requirements move to automatic by adding a single part; 4 requirements need a rewrite after the fix, automatic 16/20, remaining 4 ``` Thirteen of the twenty requirements are automatically testable, three are checked by eye, four can never be tested. The real information is in what separates the second row from the third. The three requirements checked by eye have **observable predicates**; what is missing is a single field. `error-message-content` says what an error response must not contain but writes no threshold; `audit-log-retention` asks for a log entry of every authorization change but does not say which services it applies to. Both move to automatic by adding one line. The four untestable ones are a different case. `data-integrity` or `interface-security` are not requirements with a missing field; their predicates do not correspond to an observation. Writing a threshold for them would not help either, because it is not clear what would be measured. What they need to change class is not completion but a **rewrite**: each one has to be translated into an event or a state whose violation can be seen, and this translation usually splits a single requirement into several. The decision counterpart of the distinction is clear. For three requirements the job is filling in a field; for four requirements the job is a design discussion. After the fix the list splits into 16 automatic and 4 untestable requirements — and these four requirements produce no alert in the code stage no matter how much checking is set up. A warning is needed here: testability is **not correctness**. Having written thirty minutes as the session duration limit makes it automatically testable, but it does not show that thirty minutes is the right number; a wrong threshold is tested just as flawlessly. This classification measures whether a requirement is fit for checking, not whether it is well-aimed. Mixing the two up produces a common flaw: the threshold gets written so that it is testable, and where the number came from is never discussed. #### What an Abuse Case Adds The twenty requirements above are written by reading the feature list: they are derived from the work the system **must do**. An **abuse case** starts from the opposite direction — an actor pursues a target the system must deny, and the requirements needed to deny that target are written down. ```js // run2.mjs — how much and in what form abuse cases grow the list import { REQUIREMENT, NEW, SCENARIO, classify } from './requirement.mjs'; const existing = new Set(REQUIREMENT.map((g) => g.name)); console.log('scenario actor requirements on list new'); let total = 0, inList = 0; for (const [name, actor, produced] of SCENARIO) { const v = produced.filter((x) => existing.has(x)).length; total += produced.length; inList += v; console.log(name.padEnd(31) + actor.padEnd(23) + String(produced.length).padEnd(14) + String(v).padEnd(9) + (produced.length - v)); } console.log(`${SCENARIO.length} scenarios, ${total} requirement references, ${inList} on the list, ` + `${total - inList} new; the list grows ${(((total - inList) / REQUIREMENT.length) * 100).toFixed(0)}%`); const count = (l, s) => l.filter((g) => classify(g) === s).length; const modeCount = (l, k) => l.filter((g) => g.mode === k).length; console.log('\nset requirements automatic manual untestable denies denies %'); for (const [name, l] of [['from feature list', REQUIREMENT], ['from scenarios', NEW]]) console.log(name.padEnd(19) + String(l.length).padEnd(14) + String(count(l, 'automatic')).padEnd(11) + String(count(l, 'manual')).padEnd(8) + String(count(l, 'untestable')).padEnd(12) + String(modeCount(l, 'denies')).padEnd(8) + ((modeCount(l, 'denies') / l.length) * 100).toFixed(0)); // Testability class determines which stage a requirement is tested in, and its cost. const DEPLOY_RUNS = 60, CHANGE_RUNS = 420, REVIEW_MIN = 8; console.log('\nclass tested in stage run count human minutes'); const all = [...REQUIREMENT, ...NEW]; for (const [s, stage, runs, min] of [['automatic', 'code', CHANGE_RUNS, 0], ['manual', 'deployment', DEPLOY_RUNS, REVIEW_MIN], ['untestable', 'runtime', 0, 0]]) console.log(s.padEnd(12) + stage.padEnd(17) + String(count(all, s) * runs).padEnd(14) + count(all, s) * runs * min); const manualMinutes = count(all, 'manual') * DEPLOY_RUNS * REVIEW_MIN; console.log(`\n${all.length} requirements: ${count(all, 'automatic')} tested in the code stage, ` + `${count(all, 'manual')} in the deployment stage, ${count(all, 'untestable')} never tested`); console.log(`manual checking's yearly total is ${manualMinutes} minutes (${(manualMinutes / 60).toFixed(1)} hours); ` + `if the missing parts are written, all ${(manualMinutes / 60).toFixed(1)} hours drop`); ``` ``` scenario actor requirements on list new subscriber-account-takeover external-actor 4 3 1 reading-someone-elses-bill authorized-subscriber 2 1 1 altering-a-measurement-record field-staff 3 1 2 bulk-subscriber-data-pull authorized-subscriber 3 1 2 device-registration-spoofing external-actor 2 0 2 5 scenarios, 14 requirement references, 6 on the list, 8 new; the list grows 40% set requirements automatic manual untestable denies denies % from feature list 20 13 3 4 9 45 from scenarios 8 6 2 0 7 88 class tested in stage run count human minutes automatic code 7980 0 manual deployment 300 2400 untestable runtime 0 0 28 requirements: 19 tested in the code stage, 5 in the deployment stage, 4 never tested manual checking's yearly total is 2400 minutes (40.0 hours); if the missing parts are written, all 40.0 hours drop ``` Five abuse cases touch fourteen requirements; six of them are already on the list, eight are new. The list grows by forty percent, and where the growth concentrates matters: **both** of the two requirements the last scenario produces are new. Device registration is a feature on the feature list, and that feature's requirements are written; being able to register a device **under someone else's name** is not a feature, so it cannot be derived from any feature-list line. A terminology distinction is needed here. A **use case** writes out, step by step, a use the system supports and wants that use to produce the right outcome. An abuse case uses the same form, but its owner sits on the system's opposing side, and the outcome it wants is the outcome the system must **not** give. The second table turns the difference into a number: 45% of the twenty requirements from the feature list are in denying mode, versus 88% of the eight requirements from the scenarios. If the share of denying mode in a requirement list is low, what is missing is not security attention, it is **source**. Six of the eight requirements coming from the scenarios are written directly as automatically testable. This is not because the writer was more careful, it is the **shape of the problem**: a target that must be denied already demands an observation, a threshold, and a scope. The sentence "device registration requires field approval" carries both the event where the violation would be seen and the threshold; "the interface must be secure" carries neither. There is something the table does not measure, and that is this lesson's limit: **where the five scenarios came from.** The scenarios are on the list because someone thought of them. The table has no answer to how many more requirements a sixth scenario would add, because the scenario set itself was not produced by a systematic method. What is measured is the gain **per scenario**, not the share of targets covered; when the two get conflated, a team that has written five scenarios considers its job done. #### The Stage Counterpart of Testability The last table ties back into this course's measurement axis. Nineteen of the twenty-eight requirements are automatically testable, so they are tested in the **code stage** and produce 7,980 runs a year; the human cost of these runs is zero. Five requirements can only be checked by eye, so the point of testing shifts to the **deployment stage**: 300 checks a year, 2,400 minutes, that is 40 hours. The two numbers should be read side by side. An automatically tested requirement is tested at the first opportunity after the stage it is born in, and on every change; its violation is caught in the code stage. A requirement checked by eye is only looked at while a deployment is being prepared, and its violation shows up in the previous lesson's table as an item caught three stages late. Testability looks like a property of how something is written, but its consequence is directly the **detection stage**. The four untestable requirements sit in the third row and produce no runs at all. They are not ignored by the system; their violations are only reported by an **incident**, not a check. In the previous lesson's language: these four requirements are the requirement-stage source of the item class no check ever sees. A requirement list's security value is measured not by how many lines it contains, but by how many of those lines correspond to a run count; in this list, that ratio is nineteen out of twenty-eight. #### Summary - Automatic testability is derived from three parts: an observable predicate, a threshold, a scope. If one of the three is missing the requirement is checked by eye; if the predicate is qualitative it can never be tested. - Of the 20 requirements, 13 are automatic, 3 are checked by eye, 4 are untestable. The 3 checked by eye move to automatic by adding a single field; the 4 untestable ones cannot change class without a rewrite. - 5 abuse cases touch 14 requirements, 6 of which are already on the list, 8 of which are new; the list grows by 40 percent, and the scenario that is entirely new is the one with no counterpart feature on the feature list. - 45% of the requirements from the feature list are in denying mode, versus 88% of those from the scenarios; a low share of denying mode reflects the list's source, not the writer's attention. - 19 of the 28 requirements are tested in the code stage 7,980 times a year at zero human cost; 5 are tested in the deployment stage, consuming 40 hours across 300 checks, and 4 are never tested. - When the missing parts are written, the entire 40-hour load of manual checking drops; the four untestable requirements, meanwhile, keep producing no alert in any check. #### Next Step This lesson measured how a requirement should be written and produced two jobs: writing the missing field into three requirements, rethinking four requirements. Both are jobs someone has to do, and who that someone is was never asked. The person who will write the missing threshold, the person who will run the abuse case, and the person who will notice an unobservable predicate in a design session are not the same person, and they are not present on all five teams. The previous two lessons spread the checks across stages, this lesson turned requirements into a testable form — but both assumed that the **competence** to do the work is ready and waiting on every team. The next lesson measures that assumption: how many teams there are, how many of them have someone with a security eye, how many teams a single person's capacity can cover, and how many reviews go through without a security eye when capacity falls short. ### Security Champions Source: https://academia.sh/en/courses/secure-lifecycle/security-champions The distribution of security competence is measured over five teams' 35 weekly reviews: the central team alone covers 16 reviews and 19 go through without a security eye, and coverage saturates with four champions. A single champion covers an average of 0.86 teams, and even with a champion on every team, 6 reviews a week are left to the center; when a fifth champion is added, catching items that require expertise drops from 2 to 1, because the center no longer sees those reviews. The previous three lessons, in turn, spread checks across stages, computed the cost per stage, and turned requirements into a testable form. All three called on a person but never gave that person a name: the one who notices an unobservable predicate in a design session, the one who separates the six real items out of static analysis's ninety warnings, the one who runs the abuse case. This person was assumed to be ready and waiting on every team. This lesson measures that assumption. A **security champion** is a developer who works inside their own team and sets aside part of their time for security review; not an alternative to a central security team, but an extension of its capacity. There are three things to measure: how many reviews have a security eye, how many are waiting, and how many teams a champion's capacity covers. - **DA17.** The teams, developer counts, and weekly review counts are fictional; the total review load is consistent with the previous lessons' change flow. - **DA18.** A champion only looks at **their own team's** reviews; context knowledge is a resource that cannot be transferred. A central specialist can look at any team, and their capacity sits in a shared pool. - **DA19.** Item difficulty has three levels: 1 is seen with a checklist, 2 with a trained eye, 3 with expertise. A champion catches at most level 2; level 3 is the central specialist's job. - **DA20.** An item is only caught in a review that has a security eye. An item that passes through an uncovered review falls, in the previous lessons' language, into the late flow. - **DA21.** The escalation rate is the share of cases where a champion still forwards an item they cannot see to the center, and it is a model value. When it is zero, the champion model's depth ceiling becomes visible. #### Covered and Uncovered Reviews The model ranks teams by review load and separates two sources of capacity: the time champions spend within their own teams, and the central team's shared pool. ```js // team.mjs — five teams, review load, and the distribution of security competence (model) // row: team | developer | weekly review export const TEAM = ` subscriber-portal 6 10 meter-collection 5 8 billing 4 6 field-app 4 5 shared-library 3 6` .trim().split('\n').map((r) => { const [name, g, i] = r.trim().split(/\s+/); return { name, developer: +g, review: +i }; }); export const CHAMPION_SHARE = 0.2; // the share a champion sets aside from their own time export const CHAMPION_REVIEWS = 6; // the number of reviews they can handle per week with that share export const CENTRAL_PEOPLE = 2, CENTRAL_REVIEWS = 8; // central security team // Item difficulty: 1 is seen with a checklist, 2 with a trained eye, 3 with expertise. // The distribution is modeled on the previous lessons' 23-item set. export const DIFFICULTY = [[1, 11], [2, 7], [3, 5]]; export const CHAMPION_DEPTH = 2; // the highest difficulty a champion can catch export const TOTAL_REVIEWS = TEAM.reduce((s, e) => s + e.review, 0); export const TOTAL_DEVELOPERS = TEAM.reduce((s, e) => s + e.developer, 0); ``` ```js // run1.mjs — as the champion count rises: reviews with a security eye and reviews left waiting import { TEAM, CHAMPION_SHARE, CHAMPION_REVIEWS, CENTRAL_PEOPLE, CENTRAL_REVIEWS, TOTAL_REVIEWS, TOTAL_DEVELOPERS } from './team.mjs'; const central = CENTRAL_PEOPLE * CENTRAL_REVIEWS; // A champion only looks at their own team's reviews; the central pool can look at any team. const sorted = [...TEAM].sort((a, b) => b.review - a.review); console.log(`${TEAM.length} teams, ${TOTAL_DEVELOPERS} developers, ` + `${TOTAL_REVIEWS} reviews/week; central ${central} reviews/week`); console.log('\nchampion by champions central used central idle covered waiting person-equiv'); for (let s = 0; s <= TEAM.length; s++) { const coverage = sorted.slice(0, s).reduce((a, e) => a + Math.min(e.review, CHAMPION_REVIEWS), 0); const remaining = TOTAL_REVIEWS - coverage, used = Math.min(remaining, central); const personEquiv = (s * CHAMPION_SHARE + CENTRAL_PEOPLE).toFixed(1); console.log(String(s).padEnd(10) + String(coverage).padEnd(18) + String(used).padEnd(19) + String(central - used).padEnd(14) + String(coverage + used).padEnd(10) + String(remaining - used).padEnd(10) + personEquiv); } console.log('\ndoes a single champion cover their own team'); console.log('team review champion capacity gap % left to central'); for (const e of sorted) console.log(e.name.padEnd(18) + String(e.review).padEnd(10) + String(CHAMPION_REVIEWS).padEnd(21) + String(Math.max(0, e.review - CHAMPION_REVIEWS)).padEnd(6) + ((Math.max(0, e.review - CHAMPION_REVIEWS) / e.review) * 100).toFixed(0)); const gap = sorted.reduce((a, e) => a + Math.max(0, e.review - CHAMPION_REVIEWS), 0); console.log(`even with a champion on every team, ${gap} reviews/week are left to the center; ` + `a single champion covers an average of ${(CHAMPION_REVIEWS / (TOTAL_REVIEWS / TEAM.length)).toFixed(2)} teams`); ``` ``` 5 teams, 22 developers, 35 reviews/week; central 16 reviews/week champion by champions central used central idle covered waiting person-equiv 0 0 16 0 16 19 2.0 1 6 16 0 22 13 2.2 2 12 16 0 28 7 2.4 3 18 16 0 34 1 2.6 4 24 11 5 35 0 2.8 5 29 6 10 35 0 3.0 does a single champion cover their own team team review champion capacity gap % left to central subscriber-portal 10 6 4 40 meter-collection 8 6 2 25 billing 6 6 0 0 shared-library 6 6 0 0 field-app 5 6 0 0 even with a champion on every team, 6 reviews/week are left to the center; a single champion covers an average of 0.86 teams ``` Row zero gives the starting condition: the two-person central team can handle 16 reviews a week, and 19 reviews go through without a security eye. The covered share is 46 percent, and that number has nothing to do with the central team's diligence — 16 units of capacity have been allocated to 35 reviews. As champions are added, the covered count rises from 16 to 22, to 28, to 34, and **saturates at the fourth champion**. The fifth champion does not change the covered count at all; it only raises the center's idle capacity from 5 to 10. The person-equivalent column gives the cost: with four champions, total security competence adds up to 2.8 full-time people, and 0.8 of that comes out of the twenty-two existing developers. Getting the same coverage by growing the central team would require enough specialists to handle 35 reviews; the champion model is a way to saturate coverage without hiring anyone new. #### How Many Teams Does One Champion Cover The second table disproves a common design assumption with a number. The sentence "one champion per team" implies that coverage is complete; the numbers do not say that. A champion's weekly capacity is six reviews, the average team load is seven reviews — meaning a single champion covers an average of **0.86 teams**. The distribution is sharper than the average. The subscriber portal produces ten reviews a week; the champion handles six of them, four are left to the center, and that is 40 percent of that team's reviews. For the meter collection service the share is 25 percent. The three smaller teams have a zero gap, and the field app's champion has one review's worth of idle capacity, but that capacity cannot be moved to another team — **context does not travel**. The result is this: even with a champion on every team, six reviews a week are left to the center. The champion count saturates coverage **at the team boundary**, not organization-wide; a central capacity to cover the remaining load is needed in every case. And if capacity is not distributed according to team load, the team that produces the most reviews carries the biggest gap — that is the column the table is sorted by. #### Depth Does Not Grow With Coverage A covered review count is not the same thing as a caught item count. Who sits in a review determines which difficulty of item gets seen. ```js // run2.mjs — what happens to depth as coverage grows: whose eyes do difficulty-3 items pass through import { TEAM, CHAMPION_REVIEWS, CENTRAL_PEOPLE, CENTRAL_REVIEWS, DIFFICULTY, CHAMPION_DEPTH, TOTAL_REVIEWS } from './team.mjs'; const central = CENTRAL_PEOPLE * CENTRAL_REVIEWS; const sorted = [...TEAM].sort((a, b) => b.review - a.review); const shallow = DIFFICULTY.filter(([z]) => z <= CHAMPION_DEPTH).reduce((a, [, n]) => a + n, 0); const deep = DIFFICULTY.filter(([z]) => z > CHAMPION_DEPTH).reduce((a, [, n]) => a + n, 0); const ESCALATION = 0.6; // the share of deep items the champion flags as "not sure" and forwards to the center console.log(`${shallow + deep} items: ${shallow} visible to a champion, ${deep} requiring expertise`); console.log('\nchampion by champions by central shallow found deep found + escalation total'); for (let s = 0; s <= TEAM.length; s++) { const coverage = sorted.slice(0, s).reduce((a, e) => a + Math.min(e.review, CHAMPION_REVIEWS), 0); const centralCoverage = Math.min(TOTAL_REVIEWS - coverage, central); const shallowCaught = Math.round((shallow * (coverage + centralCoverage)) / TOTAL_REVIEWS); const deepCaught = Math.round((deep * centralCoverage) / TOTAL_REVIEWS); const withEscalation = Math.round((deep * (centralCoverage + ESCALATION * coverage)) / TOTAL_REVIEWS); console.log(String(s).padEnd(10) + String(coverage).padEnd(18) + String(centralCoverage).padEnd(16) + String(shallowCaught).padEnd(15) + String(deepCaught).padEnd(17) + String(withEscalation).padEnd(14) + (shallowCaught + withEscalation)); } console.log(`\nescalation's extra load on the center at 5 champions is ` + `${(deep * ESCALATION * 29 / TOTAL_REVIEWS).toFixed(1)} items; ` + `the center's idle capacity is ${central - 6} reviews/week`); ``` ``` 23 items: 18 visible to a champion, 5 requiring expertise champion by champions by central shallow found deep found + escalation total 0 0 16 8 2 2 10 1 6 16 11 2 3 14 2 12 16 14 2 3 17 3 18 16 17 2 4 21 4 24 11 18 2 4 22 5 29 6 18 1 3 21 escalation's extra load on the center at 5 champions is 2.5 items; the center's idle capacity is 10 reviews/week ``` The shallow-item column behaves as expected: it rises from eight to eighteen and completes at the fourth champion. The deep-item column does not behave. The central team alone catches two deep items, and that number **does not change at all** until the champion count reaches four; at the fifth champion it **drops to one**. The reason for the drop sits in the neighboring column. When champions cover 29 reviews, only 6 reviews are left to the center; as the number of reviews the center looks at falls, so does the odds of it seeing an item that requires expertise. The champion is sitting in those reviews but is not at the level to see a deep item. **The decision that grew coverage shrank depth.** The total column gives this in a single number: the best result is 22 items at four champions, 21 items at five. The last column measures the countermeasure. If there is an **escalation path** through which a champion can still forward an item they cannot see to the center, the deep-item count at five champions rises from 1 to 3. The cost of this is the 2.5-item extra load it brings to the center, and the center's 10 reviews/week of idle capacity covers it. Whether the champion model works depends not on distributing competence but on **being able to report insufficiency**: a champion who can say "I'm not sure" gets more items caught than a champion trying to be sure. A limitation has to be stated. The model holds the champion's depth ceiling fixed; in reality the ceiling rises with training, and that is outside this measurement. What is measured is how coverage behaves at a given ceiling; training's contribution is a separate decision with its own cost. #### What Happens to an Uncovered Review What happens to an uncovered review is not a question of capacity, it is a question of **rule**. If the security review is mandatory, an uncovered change waits; if it is not mandatory, it goes through and the item it carries slips past. ```js // run3.mjs — an uncovered review can be tied to two different rules: blocking and non-blocking import { TEAM, CHAMPION_REVIEWS, CENTRAL_PEOPLE, CENTRAL_REVIEWS, DIFFICULTY, CHAMPION_DEPTH, TOTAL_REVIEWS } from './team.mjs'; const WEEKS = 12, central = CENTRAL_PEOPLE * CENTRAL_REVIEWS; const sorted = [...TEAM].sort((a, b) => b.review - a.review); const shallow = DIFFICULTY.filter(([z]) => z <= CHAMPION_DEPTH).reduce((a, [, n]) => a + n, 0); const deep = DIFFICULTY.filter(([z]) => z > CHAMPION_DEPTH).reduce((a, [, n]) => a + n, 0); console.log(' gap blocking rule non-blocking rule'); console.log('champions /week changes waiting slip waiting items slipped'); for (let s = 0; s <= TEAM.length; s++) { const coverage = sorted.slice(0, s).reduce((a, e) => a + Math.min(e.review, CHAMPION_REVIEWS), 0); const centralCoverage = Math.min(TOTAL_REVIEWS - coverage, central), gap = TOTAL_REVIEWS - coverage - centralCoverage; const caught = Math.round((shallow * (coverage + centralCoverage)) / TOTAL_REVIEWS) + Math.round((deep * (centralCoverage + 0.6 * coverage)) / TOTAL_REVIEWS); console.log(String(s).padEnd(11) + String(gap).padEnd(8) + String(gap * WEEKS).padEnd(21) + '0'.padEnd(7) + '0'.padEnd(10) + (shallow + deep - caught)); } console.log('\nunder the blocking rule no item slips through, and up to 228 changes wait over 12 weeks;'); console.log('under the non-blocking rule no change waits, and up to 13 items go through unseen.'); ``` ``` gap blocking rule non-blocking rule champions /week changes waiting slip waiting items slipped 0 19 228 0 0 13 1 13 156 0 0 9 2 7 84 0 0 6 3 1 12 0 0 2 4 0 0 0 0 1 5 0 0 0 0 2 under the blocking rule no item slips through, and up to 228 changes wait over 12 weeks; under the non-blocking rule no change waits, and up to 13 items go through unseen. ``` The same gap count produces two completely different outcomes. With competence not distributed at all, the blocking rule holds back **228 changes** over twelve weeks and slips no item through; the non-blocking rule holds back no change at all, and **13 items** pass through without a security eye. This is the same trade-off as the first lesson's gate table, but this time the reason for the hold is not the gate's capacity, it is **the absence of competence to review**. The direction of the decision depends on the champion count. At zero and one champion, the blocking rule cannot actually be run — the 228 and 156 accumulated changes stop the teams, the rule turns into an exception list within a few weeks, and it effectively degrades into the non-blocking rule. After the third champion, the gap falls to one review a week; it holds back 12 changes over twelve weeks, and the blocking rule becomes workable. **A blocking rule cannot be written before competence is distributed**; if it is written anyway, it will not be enforced, and an unenforced rule leaves even the reviews it was supposed to cover uncovered. The last row shows the previous table's finding once more. At the fifth champion the gap is zero, the waiting-changes count is zero, and everything looks covered; and yet the slipped-item count still rises from 1 at four champions to **2**. A measurement that only looks at the coverage column never sees this increase. #### Summary - 5 teams produce 35 reviews a week; the two-person central team covers 16 of them, and 19 reviews go through without a security eye. - As champions are added, covered reviews rise from 16 to 34 and saturate at 35 with the fourth champion; the fifth champion does not change coverage, it only raises the center's idle capacity from 5 to 10. - A champion's capacity is 6 reviews, the average team load is 7 reviews: a single champion covers 0.86 teams. Even with a champion on every team, 6 reviews a week are left to the center, and 4 of those come from a single team. - A four-champion arrangement costs 2.8 person-equivalents, and 0.8 of that comes out of the existing 22 developers; coverage saturates without hiring anyone new. For uncovered reviews, the blocking rule holds back 228 changes over 12 weeks and slips no item through, while the non-blocking rule holds back nothing and slips 13 items through; the blocking rule only becomes workable after the third champion. - Catching the 5 items that require expertise does not rise with the champion count: it stays flat at 2 up to four champions, and drops to 1 at the fifth, because the center no longer sees those reviews. - An escalation path raises the deep-item count at five champions from 1 to 3; its cost is a 2.5-item extra load on the center, and the center's 10 reviews of idle capacity cover it. #### Next Step This lesson measured where to place competence and found that a four-champion arrangement catches 22 of the 23 items. Being caught does not close an item. Every previous lesson assumed that a found item gets fixed; even shift left's 250-hour gain stood on top of that assumption, and it was noted there that a ceiling exists. The same twenty-two developers are the ones who will fix a found item, and their time is shared with feature work. The next lesson distributes limited development capacity across the item set: how many items a sprint's capacity takes, how many items close per unit of capacity, how many items are left out when the ranking criterion changes, and how much the set of items left out changes when the criterion changes. ### Risk-Based Prioritization Source: https://academia.sh/en/courses/secure-lifecycle/risk-based-prioritization The 23-item stack carries 68 developer-days of effort and 667 hours of deferred cost; 22-day sprint capacity is given to four different sorting criteria. The born-first ordering closes 6 items while the low-effort ordering closes 13 items with the same budget, and prevented hours per unit rise from 10.95 to 17.10. Only 2 items in the stack are inside under every criterion, 5 are outside under every criterion; the fate of the remaining 16 items is decided by the criterion choice. The previous four lessons measured catching: checks were distributed across the stages, cost per stage was computed, requirements were converted into a testable form, and competence was spread across teams. All four stopped at the same place — they assumed the found item gets **fixed**. Shift left's 250-hour gain sat on top of that assumption, and it was written there in plain terms that it was an upper bound. This lesson removes that assumption. The same twenty-two developers will do the fixing, and their time is shared with feature work. Once the stack is larger than capacity, the question becomes "which item first," and that question calls for a sorting criterion. The lesson runs four criteria under the same budget, compares closed items per unit of capacity, and finally separates the set that is left out no matter which criterion is chosen. One point needs separating from the start: the ordering here is not a **risk** ordering. Risk ordering works with an event that has not happened and carries a probability term. The items in this stack have already been caught; a found item has no probability of existing, only a cost of being deferred. None of the criteria contain a probability term. - **DA22.** The item set is the same as in the previous lessons. The two added fields are fix effort and deferred cost. - **DA23.** Fix effort is in **fictional developer-days** and is a model value. Deferred cost is derived: if the item is not closed now, it carries forward to runtime and pays the undo difference in between. - **DA24.** Sprint capacity is the total developer-days the teams set aside for security work; an item is either closed completely or not opened at all, there is no partial closing. - **DA25.** The budget is spent in the criterion's order, and an item that does not fit is skipped — the order does not stop there. This rule works in favor of small-effort items, and its effect shows up in the table. - **DA26.** None of the criteria carry a probability or impact term. The orderings are built only from the birth stage, deferred cost, and effort. #### Capacity and Deferred Cost The model carries two new numbers: how many developer-days closing an item takes, and how many hours carrying it forward to runtime adds if it is not closed. The second one is not given directly; it is derived from the previous lesson's undo cost. ```js // capacity.mjs — the same 23 items, now with fix effort and the multiplied cost of delay (model) export const REDO = [0, 3, 8, 2, 6, 24], BASE = 2; export const cost = (born, caught) => BASE + REDO.slice(born + 1, caught + 1).reduce((s, x) => s + x, 0); // row: name | birth stage | visibility | fix effort (fictional developer-days) export const ITEM = ` no-data-classification 0 100000 5 undefined-retention-period 0 100001 3 unclear-personal-data-scope 0 100010 4 unauthenticated-internal-call 1 011110 6 weak-session-model 1 010011 4 trust-boundary-free-flow 1 010000 7 missing-role-in-authz-model 1 011001 5 missing-authorization-check 2 001101 2 embedded-secret-key 2 001110 3 unvalidated-input 2 001001 2 personal-data-in-logs 2 001001 1 incomplete-error-path 2 001000 1 weak-randomness 2 001000 2 unbounded-query-size 2 000001 4 outdated-dependency 3 000110 1 unsigned-build-output 3 000110 3 unnecessary-build-tool 3 000100 1 default-password 4 000011 1 overly-broad-network-rule 4 000011 2 unencrypted-backup 4 000010 2 open-admin-endpoint 4 000011 1 race-under-scale 5 000001 6 third-party-disclosed-flaw 5 000001 2` .trim().split('\n').map((r) => { const [name, d, g, e] = r.trim().split(/\s+/); const born = +d, caught = [...g].findIndex((c, i) => i >= born && c === '1'); // Cost of delay: if the item is not closed now, it carries forward to runtime. return { name, born, caught, effort: +e, prevented: cost(born, 5) - cost(born, caught) }; }); export const SPRINT = 22; // developer-days per sprint set aside for security work export const TOTAL_EFFORT = ITEM.reduce((s, k) => s + k.effort, 0); export const TOTAL_PREVENTED = ITEM.reduce((s, k) => s + k.prevented, 0); // Sort criteria: none contains a probability term, the item has already been caught. export const CRITERIA = { 'born-first': (a, b) => a.born - b.born || a.effort - b.effort, 'high-multiple': (a, b) => b.prevented - a.prevented || a.effort - b.effort, 'low-effort': (a, b) => a.effort - b.effort || b.prevented - a.prevented, 'per-unit': (a, b) => b.prevented / b.effort - a.prevented / a.effort, }; export function distribute(name, budget) { const taken = []; let spent = 0; for (const k of [...ITEM].sort(CRITERIA[name])) if (spent + k.effort <= budget) { taken.push(k); spent += k.effort; } return { taken, spent, closed: taken.length, prevented: taken.reduce((s, k) => s + k.prevented, 0), outside: ITEM.filter((k) => !taken.includes(k)) }; } ``` ```js // run1.mjs — the same sprint capacity is given to four sorting criteria import { ITEM, SPRINT, TOTAL_EFFORT, TOTAL_PREVENTED, CRITERIA, distribute } from './capacity.mjs'; console.log(`${ITEM.length} items, ${TOTAL_EFFORT} developer-days effort, ` + `${TOTAL_PREVENTED} hours deferred cost; sprint capacity ${SPRINT} days ` + `(stack ${(TOTAL_EFFORT / SPRINT).toFixed(1)} sprints)`); console.log('\ncriterion closed spent prevented hrs items per day hours per day outside'); for (const name of Object.keys(CRITERIA)) { const r = distribute(name, SPRINT); console.log(name.padEnd(15) + String(r.closed).padEnd(9) + String(r.spent).padEnd(10) + String(r.prevented).padEnd(15) + (r.closed / r.spent).toFixed(2).padEnd(18) + (r.prevented / r.spent).toFixed(2).padEnd(17) + r.outside.length); } console.log('\nclosed items as capacity grows (per criterion)'); console.log('capacity ' + Object.keys(CRITERIA).map((a) => a.padEnd(14)).join('')); for (const b of [11, 22, 33, 44, 68]) console.log(String(b).padEnd(10) + Object.keys(CRITERIA).map((a) => { const r = distribute(a, b); return `${r.closed} items`.padEnd(14); }).join('')); ``` ``` 23 items, 68 developer-days effort, 667 hours deferred cost; sprint capacity 22 days (stack 3.1 sprints) criterion closed spent prevented hrs items per day hours per day outside born-first 6 22 241 0.27 10.95 17 high-multiple 6 22 241 0.27 10.95 17 low-effort 13 21 359 0.62 17.10 10 per-unit 13 22 391 0.59 17.77 10 closed items as capacity grows (per criterion) capacity born-first high-multiple low-effort per-unit 11 3 items 3 items 8 items 8 items 22 6 items 6 items 13 items 13 items 33 10 items 10 items 16 items 16 items 44 14 items 14 items 19 items 18 items 68 23 items 23 items 23 items 23 items ``` The first row gives the scale: 23 items call for 68 developer-days, the sprint capacity is 22 days, so the stack takes 3.1 sprints. This does not mean capacity is insufficient; it means the ordering matters **across three sprints**. The first two rows being identical is not a coincidence, and it needs to be read correctly. Deferred cost is derived from the birth stage alone; that is why the "biggest multiplier" ordering is a renamed version of the born-first ordering and **carries no independent information**. A program that sorts the same measure under two different names believes it is choosing between two options, when it has only one. The independent information sits in effort, and that is what produces the difference. With the same 22 days, born-first closes **6 items**, low-effort closes **13 items**; prevented hours per unit rise from 10.95 to 17.10. A difference of nearly double is obtained by changing a single column, and it requires no extra capacity. The lower table shows that this holds at every budget, but it reverses somewhere: at 44 days the low-effort ordering closes 19 items, the per-unit ordering closes 18. The ratio ordering takes a high-ratio but high-effort item early, and that item later takes the place of two small items. **The ordering best for per-unit is not the ordering best for item count**; which one is wanted has to be stated in advance. At sixty-eight days all four criteria arrive at the same place — ordering is only a decision while capacity is scarce. #### Cost Carried Across Three Sprints A single-sprint view is incomplete, because the stack closes out over three sprints. The real measure is how many sprints the deferred cost stays open: at the end of each sprint, the hours carried by the still-open items are summed. ```js // run3.mjs — deferred cost carried across three sprints: what does ordering speed up import { ITEM, SPRINT, TOTAL_PREVENTED, CRITERIA } from './capacity.mjs'; console.log('criterion sprint 1 open sprint 2 open sprint 3 open carried over 3 sprints'); for (const name of Object.keys(CRITERIA)) { const remaining = [...ITEM].sort(CRITERIA[name]); const trace = []; let carried = 0; for (let s = 0; s < 3; s++) { let spent = 0; for (const k of [...remaining]) if (spent + k.effort <= SPRINT) { spent += k.effort; remaining.splice(remaining.indexOf(k), 1); } const open = remaining.reduce((a, k) => a + k.prevented, 0); trace.push(open); carried += open; } console.log(name.padEnd(15) + trace.map((x) => String(x).padEnd(16)).join('') + carried); } console.log(`\nstack total ${TOTAL_PREVENTED} hours; the carried figure measures how many sprints ` + 'the deferred cost stays open for'); ``` ``` criterion sprint 1 open sprint 2 open sprint 3 open carried over 3 sprints born-first 426 158 0 584 high-multiple 426 158 0 584 low-effort 308 163 40 511 per-unit 276 80 0 356 stack total 667 hours; the carried figure measures how many sprints the deferred cost stays open for ``` Three criteria close the stack out in three sprints, one does not. The low-effort ordering was the criterion that closed the most items in the first sprint, but at the end of the third sprint it still carries a **40-hour** gap: because it puts small items first, the high-effort items keep failing to fit into each sprint's remaining budget and get pushed to the end. An ordering's success in the first sprint does not say whether it finishes the stack. The carried column reduces the difference between criteria to a single number. The born-first ordering carries **584 hour-sprints** of open cost across three sprints, the per-unit ordering **356**. The difference between them is 39 percent, and it is obtained without spending any extra capacity; only the order of the same work has changed. The cost remaining after the first sprint points the same way: 426 hours against 276 hours. #### Whose Fate the Ordering Decides The closed-item count gives the criterion's value, but the real question is this: when the criterion changes, **who** is left out. Two criteria can leave out the same number of items, and they can leave out entirely different items. ```js // run2.mjs — how much does the left-out set change when the ordering changes import { ITEM, SPRINT, CRITERIA, distribute } from './capacity.mjs'; const names = Object.keys(CRITERIA); const outside = new Map(names.map((a) => [a, new Set(distribute(a, SPRINT).outside)])); console.log('shared items in the left-out set between criterion pairs (22-day sprint)'); console.log(' ' + names.map((a) => a.padEnd(14)).join('')); for (const a of names) console.log(a.padEnd(15) + names.map((b) => { const shared = [...outside.get(a)].filter((k) => outside.get(b).has(k)).length; return String(shared).padEnd(14); }).join('')); const alwaysOut = ITEM.filter((k) => names.every((a) => outside.get(a).has(k))); const neverOut = ITEM.filter((k) => names.every((a) => !outside.get(a).has(k))); const switching = ITEM.length - alwaysOut.length - neverOut.length; console.log(`\n${ITEM.length} items: inside in every criterion ${neverOut.length}, outside in ` + `every criterion ${alwaysOut.length}, switching by criterion ${switching}`); console.log(`the deferred cost of items always left out is ` + `${alwaysOut.reduce((s, k) => s + k.prevented, 0)} hours, fix effort ` + `${alwaysOut.reduce((s, k) => s + k.effort, 0)} days`); console.log('\nitems left out under every criterion'); console.log('item effort prevented hrs hrs/day'); for (const k of [...alwaysOut].sort((a, b) => b.prevented / b.effort - a.prevented / a.effort)) console.log(k.name.padEnd(31) + String(k.effort).padEnd(8) + String(k.prevented).padEnd(15) + (k.prevented / k.effort).toFixed(1)); ``` ``` shared items in the left-out set between criterion pairs (22-day sprint) born-first high-multiple low-effort per-unit born-first 17 17 6 6 high-multiple 17 17 6 6 low-effort 6 6 10 9 per-unit 6 6 9 10 23 items: inside in every criterion 2, outside in every criterion 5, switching by criterion 16 the deferred cost of items always left out is 110 hours, fix effort 26 days items left out under every criterion item effort prevented hrs hrs/day unsigned-build-output 3 30 10.0 unauthenticated-internal-call 6 40 6.7 trust-boundary-free-flow 7 40 5.7 unbounded-query-size 4 0 0.0 race-under-scale 6 0 0.0 ``` The intersection table carries this lesson's hardest number. Of the 17 items the born-first ordering leaves out and the 10 items the low-effort ordering leaves out, **only 6 are shared**. Two programs look at the same stack, spend the same budget, and defer largely different items. The bottom line sums this up in one sentence: of the 23 items, **2** are inside under every criterion, **5** are outside under every criterion, **16** switch sides depending on the criterion. The fate of 70 percent of the stack is decided not by the items' own properties but by the choice of sorting criterion. A security program's most decisive decision is therefore not deciding which item matters, but **writing down which column it is sorted by**; when it is not written down, the decision still gets made, just somewhere nobody sees it. #### Left Out Under Every Criterion The last list shows the residue that is independent of the criterion choice: five items, 110 hours of deferred cost, 26 days of fix effort. These five items are outside for two separate reasons, and each requires a separate decision. Three of them have high effort. Fixing the trust-boundary-free data flow takes seven days, fixing the unauthenticated internal call takes six days; both prevent forty hours each, but their per-hour ratio sits in the middle of the list, and they cannot get into a 22-day budget under any ordering. The right response to these is not to force the ordering, it is **to split the effort**: if a single seven-day item can be broken into two three-day steps, it gets onto the list; if it cannot be split, it needs a separate budget. A stack measured with a single sprint's capacity structurally keeps out high-effort items that cannot be split. The remaining two are different, and their prevented hours are zero. The race under scale and the unbounded query size are the class set apart in the second lesson: born early, visible only at runtime. These do not fall to the bottom of the ordering, they **never enter it**, because closing them early prevents nothing. The right decision for these two items is not prioritization; it is building a way to see them cheaply at runtime, and that work is outside this budget. Not keeping zero-value rows separate in a stack list corrupts the ordering itself. #### Summary - The 23-item stack carries 68 developer-days of effort and 667 hours of deferred cost; with 22-day sprint capacity the stack takes 3.1 sprints, and the ordering produces a decision across all three sprints. - The born-first ordering and the biggest-multiplier ordering give the same result, because deferred cost is derived from the birth stage alone; the two names are a single criterion. - With the same 22 days, the born-first ordering closes 6 items, the low-effort ordering closes 13; prevented hours per unit rise from 10.95 to 17.10, and this requires no extra capacity. - The ordering best for per-unit is not the ordering best for item count: at 44 days the low-effort ordering closes 19 items, the ratio ordering 18. The cost carried across three sprints is 584 hour-sprints for the born-first ordering, 356 for the per-unit ordering, and the low-effort ordering, despite being best in the first sprint, leaves a 40-hour gap at the end of the third. - Of the 23 items, 2 are inside under every criterion, 5 are outside under every criterion, 16 switch sides depending on the criterion; the criterion choice decides the fate of 70 percent of the stack. - The 5 items left out under every criterion carry 110 hours: three have high effort and cannot get in unless split, two have zero prevented hours and should never enter the ordering at all. #### Next Step This topic turned security from a single stage into a decision with a different price at every stage. The gate's queue was counted, the per-stage fix cost was derived from the work that gets undone, requirements were split into testability classes, competence's coverage and depth were measured, and finally limited capacity was distributed under four separate criteria. All of these measurements had a single input: **that we already have a caught set of items.** Twenty-three items were given from the start, and where they came from was never asked. The stages that carried the largest share of the gain were design and requirements — more than half of the prevented hours came from there. But which item gets found in a design session is too decisive to be left to the eye of whoever finds it. The third lesson noted that it was not measured where the five abuse cases came from; the fourth lesson noted that the difficulty level a champion could see stood as a ceiling. Both point to the same gap: **finding which item is born at the design stage has a systematic method of its own, and that method was never explained.** The next topic, **Threat Modeling**, builds that method: the system's data flow is written as a graph, the flows that cross a trust boundary are counted, threats are generated by applying a systematic classification to the graph's elements, and how many of the generated threats are meaningful, how many turn into a countermeasure, is measured. ## Threat Modeling Measuring which item can be found at the design stage and how many hours finding it costs: the duration of a threat modeling session and its hours per item, counting the flows that cross a trust boundary in a data flow diagram, the flows that stay invisible when no boundary is drawn, the raw threat count that systematic threat generation produces and the share of it that is meaningful, elimination time, the coverage of scenario-based modeling and its intersection with the systematic method, the cost of the cheapest path in an attack tree and how many times closing a node multiplies the root cost, how much of the model's output converts into work items, and the written acceptance of residual risk. ### The Purpose of Threat Modeling Source: https://academia.sh/en/courses/secure-lifecycle/purpose-of-threat-modeling A threat modeling session's own measure: in an 18-item inventory, the session takes 40 developer hours, finds 10 items, saves 866 hours against 4.0 hours per item, produces a cost difference ranging from 4x to 30x when the same item is caught late, brings total fix hours down from 1,872 to 1,006, and leaves the eight items the session never touches carrying 48.4% of the total hours. The previous topic distributed security work across stages, wrote requirements in testable form, and asked how to divide limited development capacity. The common input to all of those calculations was a ready-made item list: which gap existed was already known, and only the order was under discussion. At the design stage, though, no such list appears on its own. **Threat modeling** is the method for producing that list at the design stage. This lesson does not measure the method's steps but its **purpose and price**: how many hours a session takes, how many items it finds, how many hours it spends per item, and how many times more expensive the items it finds would have been if caught late. The topic works on the same fictional regional measurement network's software throughout. #### Where the Session Stops The inventory below is a **model**: there is no real product, no real team measurement, and no real record. Items are described by only three columns — where they originate, where they can be seen, and how many developer hours their fix takes. No exploit step is written. **TM1:** fix work is measured in developer hours; the base hour count is the item's cost at the design stage, and it grows by a multiplier in later stages. **TM2:** the stage an item originates in and the stage it **becomes visible** in are separate things; an item born in code may only become visible at runtime. **TM3:** the session runs at the design stage and finds only items whose visible stage is no later than design. **TM4:** the session's cost is the product of participant count and its duration. ```js // threat/item.mjs — stage set and item inventory (model): derived from the development records // of a fictional regional measurement network's software; not a real measurement. No exploit // step is present; an item is described only by where it originates, where it becomes visible, // and how many developer hours its fix takes. export const STAGE = ["requirement", "design", "code", "build", "deploy", "runtime"]; // TM1: fix work is measured in developer hours. `base` is the hour count for fixing the item at // the design stage; it grows by the multiplier below in later stages. export const MULTIPLIER = { requirement: 1, design: 1, code: 4, build: 6, deploy: 12, "runtime": 30 }; // TM2: rows are written as "name origin visible default base". `origin` is the stage the item is // produced in, `visible` is the earliest stage it can be seen in, `default` is the stage it is // actually caught in when there is no threat modeling session. export const ITEM = ` trust-boundary-not-drawn design design deploy 6 meter-identity-not-verified design design runtime 8 billing-record-modifiable design design deploy 10 subscriber-data-flows-unprotected design design deploy 4 authorization-decision-on-client design design code 7 audit-log-missing requirement design runtime 5 field-device-carries-secret design design deploy 9 reporting-endpoint-unauthenticated design design code 3 measurement-backup-unprotected design design runtime 6 session-duration-undefined requirement design code 2 input-length-unbounded code code build 2 query-concatenated-manually code code build 4 timing-difference-leaks code code runtime 5 concurrent-request-race-condition code runtime runtime 12 dependency-version-unpinned build build deploy 3 leftover-tool-in-image build build deploy 2 default-credential-exposed deploy deploy runtime 4 credential-in-log-entry code runtime runtime 6` .trim().split("\n").map((r) => { const [name, origin, visible, def, base] = r.trim().split(/\s+/); return { name, origin, visible, default: def, base: +base }; }); export const order = (a) => STAGE.indexOf(a); export const cost = (k, stage) => k.base * MULTIPLIER[stage]; // TM3: the threat modeling session runs at the design stage and finds only the items whose // `visible` stage is no later than design; it never touches the rest. export const isFound = (k) => order(k.visible) <= order("design"); // TM4: the session's cost is participant count x session count x session duration. export const SESSION = { participants: 5, count: 4, hours: 2 }; export const SESSION_HOURS = SESSION.participants * SESSION.count * SESSION.hours; ``` ```js // threat/session.mjs — how many hours the session takes, how many items it finds, hours per // item, and how many times more expensive the same item is when caught late. import { STAGE, MULTIPLIER, ITEM, SESSION, SESSION_HOURS, order, cost, isFound } from "./item.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const sum = (ks, f) => ks.reduce((s, k) => s + f(k), 0); const found = ITEM.filter(isFound), missed = ITEM.filter((k) => !isFound(k)); console.log(`${ITEM.length} items, ${STAGE.length} stages; multipliers ` + `${STAGE.map((a) => `${a} x${MULTIPLIER[a]}`).join(", ")}`); const A = [-36, -13, -9, -9, 7, 11, 12, 8]; console.log("\n1. where the item originates, where it can be seen, where it is caught"); write(A, "item", "origin", "visible", "default", "delay", "at design", "at default", "factor"); for (const k of ITEM) { const t = cost(k, "design"), v = cost(k, k.default); write(A, k.name, k.origin, k.visible, k.default, order(k.default) - order(k.origin), t, v, (v / t).toFixed(1)); } const B = [-15, 7, 8, 15, 8, 9]; console.log("\n2. item class by visible stage"); write(B, "visible stage", "items", "item %", "default hours", "hour %", "session"); for (const a of STAGE) { const g = ITEM.filter((k) => k.visible === a); if (!g.length) continue; const s = sum(g, (k) => cost(k, k.default)); write(B, a, g.length, ((g.length / ITEM.length) * 100).toFixed(1), s, ((s / sum(ITEM, (k) => cost(k, k.default))) * 100).toFixed(1), order(a) <= order("design") ? "finds" : "misses"); } const WITHOUT_SESSION = sum(ITEM, (k) => cost(k, k.default)); const WITH_SESSION = sum(found, (k) => cost(k, "design")) + sum(missed, (k) => cost(k, k.default)) + SESSION_HOURS; const C = [-40, 14, 22]; console.log("\n3. the session's own measure"); for (const [name, x, y] of [ ["session count x participants x hours", `${SESSION.count}x${SESSION.participants}x${SESSION.hours}`, SESSION_HOURS + " hours"], ["items found", found.length, (SESSION_HOURS / found.length).toFixed(1) + " hours/item"], ["items untouched", missed.length, ((missed.length / ITEM.length) * 100).toFixed(1) + "%"], ["total fix without session", WITHOUT_SESSION + " hours", "-"], ["total fix with session", WITH_SESSION + " hours", (WITHOUT_SESSION / WITH_SESSION).toFixed(2) + "x"], ["hours the session saves", WITHOUT_SESSION - WITH_SESSION + " hours", ((WITHOUT_SESSION - WITH_SESSION) / SESSION_HOURS).toFixed(1) + "x session hours"], ]) write(C, name, x, y); const D = [-19, 7, 19, 18, 8]; console.log("\n4. items the session touches and does not, separately"); write(D, "group", "items", "hours w/o session", "hours w/ session", "factor"); for (const [name, ks, hours] of [ ["found by session", found, sum(found, (k) => cost(k, "design"))], ["missed by session", missed, sum(missed, (k) => cost(k, k.default))], ]) { const o = sum(ks, (k) => cost(k, k.default)); write(D, name, ks.length, o, hours, (o / hours).toFixed(1)); } const worst = [...ITEM].sort((x, y) => cost(y, y.default) - cost(x, x.default))[0]; console.log(`the single most expensive item is ${worst.name}: ${cost(worst, worst.default)} hours, ` + `the session ${isFound(worst) ? "finds" : "misses"} it; the ${missed.length} items the session ` + `misses carry ${((sum(missed, (k) => cost(k, k.default)) / WITHOUT_SESSION) * 100).toFixed(1)}% ` + `of the total fix hours`); ``` ``` 18 items, 6 stages; multipliers requirement x1, design x1, code x4, build x6, deploy x12, runtime x30 1. where the item originates, where it can be seen, where it is caught item origin visible default delay at design at default factor trust-boundary-not-drawn design design deploy 3 6 72 12.0 meter-identity-not-verified design design runtime 4 8 240 30.0 billing-record-modifiable design design deploy 3 10 120 12.0 subscriber-data-flows-unprotected design design deploy 3 4 48 12.0 authorization-decision-on-client design design code 1 7 28 4.0 audit-log-missing requirement design runtime 5 5 150 30.0 field-device-carries-secret design design deploy 3 9 108 12.0 reporting-endpoint-unauthenticated design design code 1 3 12 4.0 measurement-backup-unprotected design design runtime 4 6 180 30.0 session-duration-undefined requirement design code 2 2 8 4.0 input-length-unbounded code code build 1 2 12 6.0 query-concatenated-manually code code build 1 4 24 6.0 timing-difference-leaks code code runtime 3 5 150 30.0 concurrent-request-race-condition code runtime runtime 3 12 360 30.0 dependency-version-unpinned build build deploy 1 3 36 12.0 leftover-tool-in-image build build deploy 1 2 24 12.0 default-credential-exposed deploy deploy runtime 1 4 120 30.0 credential-in-log-entry code runtime runtime 3 6 180 30.0 2. item class by visible stage visible stage items item % default hours hour % session design 10 55.6 966 51.6 finds code 3 16.7 186 9.9 misses build 2 11.1 60 3.2 misses deploy 1 5.6 120 6.4 misses runtime 2 11.1 540 28.8 misses 3. the session's own measure session count x participants x hours 4x5x2 40 hours items found 10 4.0 hours/item items untouched 8 44.4% total fix without session 1872 hours - total fix with session 1006 hours 1.86x hours the session saves 866 hours 21.6x session hours 4. items the session touches and does not, separately group items hours w/o session hours w/ session factor found by session 10 966 60 16.1 missed by session 8 906 906 1.0 the single most expensive item is concurrent-request-race-condition: 360 hours, the session misses it; the 8 items the session misses carry 48.4% of the total fix hours ``` The numbers are in the **measurement** class; their inputs are TM1–TM4. #### The Forty-Hour Session The third table answers this lesson's main question. Four sessions, five participants, two hours per session: a total of **40 developer hours**. The session finds ten items, meaning **4.0 hours per item**. This number describes threat modeling not as a method but as a **work item**; for a team that needs approval, this is the only form that can be argued over. The payoff sits in the same row. Without the session, the total fix work for the eighteen items would be **1,872 hours**; with the session it drops to 1,006 hours. The difference is 866 hours, which is **21.6 times** the 40 hours spent. The size of this ratio comes from a single fact: the items the session finds were items that would otherwise have been caught late, and the late-stage multiplier was high. Looked at over the total, though, the gain is far more modest: 1,872 hours drops to 1,006 hours, which is **1.86 times**. The same measurement gives two different magnitudes, and both are correct. For the items the session touches, the ratio is 16.1; for the whole inventory, it is 1.86. Which number gets written down depends on whether what the session covers is stated explicitly. #### The Same Item, Three Stages Later The last column of the first table answers a single question for every item: when the same item is caught at its default stage, how many times its design-stage cost does it end up costing. The values spread **between 4.0 and 30.0**, and this range is the lesson's most important finding — the sentence "catching it late is expensive" does not carry one single factor; it carries a distribution that depends on how far the item can travel unseen. It is enough to compare the two extremes. `authorization-decision-on-client` is caught in code: 28 hours instead of 7, a factor of 4.0. `meter-identity-not-verified` travels all the way to runtime: **240 hours** instead of 8, a factor of 30.0. The two items' base hours are close to each other; what produces the thirtyfold difference between them is not the item's own size but **how many stages it advances through unseen**. The `delay` column counts this directly. `audit-log-missing` originates at the requirement stage and is caught at runtime: **five stages** late. The same item's base is only 5 hours; a five-stage delay raises it to 150 hours. By contrast, `leftover-tool-in-image` originates at the build stage and is caught at deploy — one stage late, 24 hours instead of 2. `concurrent-request-race-condition`, the most expensive item in the inventory, reaches 360 hours with a three-stage delay and alone carries roughly a fifth of the total without the session. #### The Coverage Ceiling Read on its own, the first table's `origin` column gives the session's upper bound. In the inventory, 2 items originate at the requirement stage, 8 at design, 5 at code, 2 at build, 1 at deploy. When the design stage ends, the number of items **not yet born** is eight. An item cannot be seen before it is born; the `visible` column is never earlier than the `origin` column in any row. This puts a structural ceiling on the session's coverage: **no item born after design can be found in a design-stage session** — that item does not yet exist while the session is running. The eight items being left out comes from sequence, not from any weakness in the session. Below the ceiling, the session's own success is measured. Of the ten items born at the requirement or design stage, all ten show `design` in the `visible` column; in this inventory, the session reaches **the entire ceiling**. In a real session this ratio is lower, and the two numbers must be written separately: how many items could structurally be found, and how many of those were actually found. When the two are conflated, a team reporting 100% coverage has never measured 44.4% of the inventory at all. Two items show this distinction directly. `concurrent-request-race-condition` and `credential-in-log-entry` originate at the code stage but only become visible at runtime; both have a three-stage delay. These fall within the scope of neither the design session nor a code-reading review — they are not visible even at the stage where they are born. #### The Eight Items That Cannot Be Seen at Design The fourth table states this lesson's limit. The ten items the session touches drop from 966 hours to 60 hours — a factor of 16.1. The eight items the session **misses** drop from 906 hours to 906 hours — a factor of **1.0**. No matter how well the session is run, it does nothing for these eight items. The second table separates out where those eight stand. Three items can only be seen in code, two in build, one in deploy, two only at runtime. As a share of item count that is **44.4%** of the inventory; as a share of fix hours it is **48.4%**. So even the most thorough session run at the design stage leaves behind a share approaching half of the total work. This class is not random. A race condition, a timing difference, and a credential landing in a log entry do not appear in the design as a box or an arrow; they only surface in the behavior of code that has been written or a system that is running. The design document does not **carry** these items, so no session that reads the design can find them. The two items visible only at runtime alone carry 540 hours — roughly two-thirds of the 866 hours the session saves. The conclusion that follows is plain: a threat modeling session does not replace the reviews that come after it. The claim "we catch everything at design," made without stating the session's coverage limit, is a claim that, once measured, leaves out half of the inventory. #### Summary - A threat modeling session is a measurable work item: four sessions, five participants, two hours each — **40 developer hours**, 10 items, **4.0 hours per item**. - The session saves 866 hours, which is 21.6 times the 40 hours spent. The same measurement gives 1.86 times over the whole inventory and 16.1 times over just the items it touches. - The same item's late-catch factor spreads between 4.0 and 30.0; what determines the factor is not the item's size but how many stages it advances through unseen. - A 5-hour item delayed by five stages rises to 150 hours; a 2-hour item delayed by one stage rises to 24 hours — the delay count matters more than the base hours. - Eight items in the inventory cannot be seen at design, and the session never touches them — 44.4% of the item count, **48.4%** of the fix hours. For these items, the session's factor is 1.0. #### Next Step This lesson stated the session's price and its coverage limit, but never opened up **what happens inside** the session. How the ten items were found remained a black box: "the design was read and items came out." Reading the design is not a method; a design document contains hundreds of sentences, and which sentence gives rise to a security item is not obvious. The method's first step is to turn the design into a structure that **can be counted over**: which component takes which data from whom and gives it to whom, and which of those exchanges cross from one trust level to another. The next lesson builds that structure and takes the topic's core measurement: the **number of flows that cross a trust boundary** in a data flow diagram, the items each boundary-crossing flow carries, and how many flows stay invisible when no boundary is drawn. ### Data Flow Diagrams Source: https://academia.sh/en/courses/secure-lifecycle/data-flow-diagrams The topic's core measurement: in a graph of 17 nodes, 26 flows, and six trust zones, 23 flows cross a trust boundary, and these crossings produce a total of 58 items at 2.52 per flow, with 58.6% of the items coming from flows that go from a lower to a higher level; a coarse outer/inner drawing sees only 7 of the 23 crossings and only 15 of the 58 items, and the 16 crossings that stay invisible carry 43 items. The previous lesson stated the session's price and its coverage ceiling, but left what happens inside the session as a black box. Reading the design text is not a method: which of hundreds of sentences gives rise to a security item is not obvious, and two people extract two different lists from the same text. The method's first step is to turn the design into a structure that can be counted over. That structure is the **data flow diagram**: which component takes which data from whom and gives it to whom. This lesson builds the graph and takes the topic's core measurement — the number of flows that cross a trust boundary, the items each crossing produces, and how many flows stay invisible when no boundary is drawn. #### A Graph Is Not a Diagram A diagram is a drawing; what gets counted is not the drawing but the graph behind it. In the terms drawn from the Architectural Decisions and Documentation course, a diagram is a **view**, and the graph is the structure the view is derived from. The same graph can be written as a box-and-arrow drawing or, as below, row by row; the measurement gives the same number either way. The graph below is a **model**: there is no real system, no real hostname, no real service endpoint. It works with the fictional regional measurement network's software. **TM5:** every node sits in a **trust zone**, and the zone has a numeric trust level. **TM6:** a node is one of three types — external entity, process, data store. **TM7:** every flow carries a data class. **TM8:** a flow whose two ends sit in different zones crosses the **trust boundary**; every boundary-crossing flow produces items under the four rules below, and a flow that does not cross produces no items. ```js // threat/graph.mjs — the fictional regional measurement network's software's data flow graph // (model): there is no real system, no real hostname, no real service endpoint. The graph only // says "who takes what from whom"; it carries no attack step. // TM5: every node sits in a trust zone, and the zone has a trust level. export const ZONE = { internet: 0, field: 1, edge: 2, internal: 3, management: 4, data: 5 }; // TM6: a node is written as "name type zone"; type is external entity (D), process (S), or store (P). export const NODE = Object.fromEntries(` subscriber D internet | oversight-body D internet | meter-device D field | field-team D field subscriber-portal S edge | collection-endpoint S edge | field-app S field validator S internal | billing S internal | work-order S internal | report-generator S internal | identity-service S internal management-console S management | measurement-store P data | billing-store P data identity-store P data | audit-log P data` .trim().split(/\s*[|\n]\s*/).map((r) => { const [name, type, zone] = r.trim().split(/\s+/); return [name, { name, type, zone, level: ZONE[zone] }]; })); // TM7: a flow is written as "source target class"; class is the data class carried. export const FLOW = ` subscriber subscriber-portal identity | subscriber-portal identity-service identity identity-service identity-store identity | subscriber-portal billing billing billing billing-store billing | billing subscriber-portal billing meter-device collection-endpoint measurement | collection-endpoint validator measurement validator measurement-store measurement | validator billing measurement field-team field-app identity | field-app work-order order work-order field-app order | work-order measurement-store measurement field-app collection-endpoint measurement | management-console validator setting management-console collection-endpoint setting | management-console identity-service setting report-generator measurement-store measurement | report-generator billing-store billing oversight-body report-generator identity | report-generator oversight-body billing billing audit-log record | identity-service audit-log record work-order audit-log record | billing work-order order` .trim().split(/\s*[|\n]\s*/).map((r) => { const [source, target, cls] = r.trim().split(/\s+/); return { source, target, class: cls, name: `${source}->${target}` }; }); // TM8: a flow crosses the trust boundary if its two ends' zones differ. Every boundary-crossing // flow produces items under the four rules below; a flow that does not cross produces no items. export function items(a, zoneOf = (n) => NODE[n].zone, levelOf = (n) => NODE[n].level) { if (zoneOf(a.source) === zoneOf(a.target)) return []; const k = []; if (levelOf(a.source) < levelOf(a.target)) k.push("input-validation", "source-identity"); if (levelOf(a.source) > levelOf(a.target)) k.push("output-filtering"); if (a.class === "identity" || a.class === "billing") k.push("transport-protection"); if (NODE[a.target].type === "P") k.push("authorization"); return k; } ``` ```js // threat/boundary.mjs — how many flows cross the trust boundary, the items each crossing // carries, and how many flows stay invisible when boundary resolution drops. import { ZONE, NODE, FLOW, items } from "./graph.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const crossing = FLOW.filter((a) => NODE[a.source].zone !== NODE[a.target].zone); console.log(`${Object.keys(NODE).length} nodes, ${FLOW.length} flows, ` + `${Object.keys(ZONE).length} trust zones; boundary-crossing flows ${crossing.length}, ` + `non-crossing ${FLOW.length - crossing.length}`); const A = [-12, 7, 8, 9, 7, 11, 10]; console.log("\n1. zones and their contents"); write(A, "zone", "level", "entity", "process", "store", "out flows", "crossing"); for (const [z, l] of Object.entries(ZONE)) { const ns = Object.values(NODE).filter((x) => x.zone === z); const out = FLOW.filter((a) => NODE[a.source].zone === z); write(A, z, l, ns.filter((x) => x.type === "D").length, ns.filter((x) => x.type === "S").length, ns.filter((x) => x.type === "P").length, out.length, out.filter((a) => NODE[a.target].zone !== z).length); } const B = [-42, -14, -8, -67]; console.log("\n2. every boundary-crossing flow and the items it produces"); write(B, "flow", "class", "level", "items"); for (const a of crossing) write(B, a.name, a.class, `${NODE[a.source].level}->${NODE[a.target].level}`, items(a).join(" ")); const ALL = crossing.flatMap((a) => items(a)); console.log(`the ${crossing.length} boundary-crossing flows produce a total of ${ALL.length} ` + `items, ${(ALL.length / crossing.length).toFixed(2)} per flow`); const C = [-22, 7, 8, 8]; console.log("\n3. items by type"); write(C, "item type", "count", "item %", "flow %"); for (const t of [...new Set(ALL)].sort()) { const n = ALL.filter((x) => x === t).length; write(C, t, n, ((n / ALL.length) * 100).toFixed(1), ((n / FLOW.length) * 100).toFixed(1)); } // Resolution: the same graph is measured with three separate boundary drawings. The coarse // drawing recognizes only the outer/inner split; the boundary-less drawing counts every node // in a single zone. const COARSE = { internet: "outer", field: "outer", edge: "inner", internal: "inner", management: "inner", data: "inner" }; const drawing = { "no boundary drawn": { z: () => "single", l: () => 0 }, "coarse: outer/inner": { z: (x) => COARSE[NODE[x].zone], l: (x) => (COARSE[NODE[x].zone] === "outer" ? 0 : 1) }, "detailed: 6 zones": { z: (x) => NODE[x].zone, l: (x) => NODE[x].level }, }; const D = [-21, 15, 11, 7, 8]; console.log("\n4. what stays invisible when boundary resolution drops"); write(D, "drawing", "visible flows", "visible %", "items", "item %"); const full = { flows: crossing.length, items: ALL.length }; for (const [name, c] of Object.entries(drawing)) { const v = FLOW.filter((a) => c.z(a.source) !== c.z(a.target)); const k = v.flatMap((a) => items(a, c.z, c.l)); write(D, name, v.length, ((v.length / full.flows) * 100).toFixed(1), k.length, ((k.length / full.items) * 100).toFixed(1)); } const coarse = drawing["coarse: outer/inner"]; const coarseV = FLOW.filter((a) => coarse.z(a.source) !== coarse.z(a.target)); const missed = crossing.filter((a) => !coarseV.includes(a)); console.log(`the coarse drawing misses ${missed.length} boundary crossings carrying ` + `${missed.flatMap((a) => items(a)).length} items; ${missed.filter((a) => NODE[a.target].type === "P").length} of these enter a data store`); ``` ``` 17 nodes, 26 flows, 6 trust zones; boundary-crossing flows 23, non-crossing 3 1. zones and their contents zone level entity process store out flows crossing internet 0 2 0 0 2 2 field 1 2 1 0 4 3 edge 2 0 2 0 3 3 internal 3 0 5 0 14 12 management 4 0 1 0 3 3 data 5 0 0 4 0 0 2. every boundary-crossing flow and the items it produces flow class level items subscriber->subscriber-portal identity 0->2 input-validation source-identity transport-protection subscriber-portal->identity-service identity 2->3 input-validation source-identity transport-protection identity-service->identity-store identity 3->5 input-validation source-identity transport-protection authorization subscriber-portal->billing billing 2->3 input-validation source-identity transport-protection billing->billing-store billing 3->5 input-validation source-identity transport-protection authorization billing->subscriber-portal billing 3->2 output-filtering transport-protection meter-device->collection-endpoint measurement 1->2 input-validation source-identity collection-endpoint->validator measurement 2->3 input-validation source-identity validator->measurement-store measurement 3->5 input-validation source-identity authorization field-app->work-order order 1->3 input-validation source-identity work-order->field-app order 3->1 output-filtering work-order->measurement-store measurement 3->5 input-validation source-identity authorization field-app->collection-endpoint measurement 1->2 input-validation source-identity management-console->validator setting 4->3 output-filtering management-console->collection-endpoint setting 4->2 output-filtering management-console->identity-service setting 4->3 output-filtering report-generator->measurement-store measurement 3->5 input-validation source-identity authorization report-generator->billing-store billing 3->5 input-validation source-identity transport-protection authorization oversight-body->report-generator identity 0->3 input-validation source-identity transport-protection report-generator->oversight-body billing 3->0 output-filtering transport-protection billing->audit-log record 3->5 input-validation source-identity authorization identity-service->audit-log record 3->5 input-validation source-identity authorization work-order->audit-log record 3->5 input-validation source-identity authorization the 23 boundary-crossing flows produce a total of 58 items, 2.52 per flow 3. items by type item type count item % flow % authorization 9 15.5 34.6 input-validation 17 29.3 65.4 output-filtering 6 10.3 23.1 source-identity 17 29.3 65.4 transport-protection 9 15.5 34.6 4. what stays invisible when boundary resolution drops drawing visible flows visible % items item % no boundary drawn 0 0.0 0 0.0 coarse: outer/inner 7 30.4 15 25.9 detailed: 6 zones 23 100.0 58 100.0 the coarse drawing misses 16 boundary crossings carrying 43 items; 9 of these enter a data store ``` The numbers are in the **measurement** class; their inputs are TM5–TM8. #### Twenty-Three Crossings Seventeen nodes, twenty-six flows, six trust zones. None of these three numbers is threat modeling's input; the number that is an input is **23** — the number of flows that cross a trust boundary. Only three of the twenty-six flows stay within the same zone: the field team talking to its own app, the validator giving billing a measurement, and billing producing a work order. This ratio comes from the graph's shape. Zones exist to increase isolation, but the work of the system has to travel across them: a subscriber's request starts in the internet zone and ends in the data zone, meaning a single transaction crosses three separate boundaries. The first table's last column states this zone by zone. Of the fourteen flows originating in the `internal` zone, **twelve** leave it; no flow leaves the `data` zone, because stores are only written-to and read-from endpoints. This is the number that is meaningful on its own for security. The Attack Surface lesson measured the surface as a path count; at the design stage there are no paths yet, only flows. At this stage the surface's countable equivalent is the **number of flows that cross a trust boundary**, and that number depends on where the boundaries are drawn. #### The Item a Crossing Produces The second table turns every boundary crossing into an item list. Twenty-three crossings produce 58 items; 2.52 per crossing. The rule set has four entries, and none of them relies on intuition: a flow whose trust level goes from lower to higher requires input validation and source identity, a flow going the opposite direction requires output filtering, a flow carrying identity- or billing-class data requires transport protection, a flow entering a data store requires authorization. The third table gives the distribution. `input-validation` and `source-identity` lead with seventeen items each; together they carry **58.6%** of the 58 items. Both have a single source: seventeen of the twenty-three crossings go from a lower trust level to a higher one. `output-filtering` is the smallest group with six items; `transport-protection` and `authorization` sit in the middle with nine each. The busiest rows produce four items at once: `identity-service->identity-store`, `billing->billing-store`, and `report-generator->billing-store`. All three go from lower to higher, carry a data class that needs protection, and write to a store. The plainest row is `work-order->field-app`: a single item, `output-filtering`. The item count is not determined by how important the flow is but by **how many rules it triggers at once**; this distinction is what makes the method systematic. The same systematicity also produces the rule's blind spot. The management console sits at the highest level short of the fifth trust level, and all three of its outgoing flows go downward; so all three produce only `output-filtering`, and none produces `input-validation`. This is consistent from the rule set's point of view — data coming from a more trusted source is not validated — but what the three flows carry is `setting`-class data, and changing the validator's setting affects the entire measurement chain. **Trust level is a ranking, not a certificate of immunity.** The item set that systematic generation produces cannot go beyond the accuracy of the trust-level assignments; if an assignment is wrong, the method stays silent, and that silence never shows up as a zero in any table. #### When No Boundary Is Drawn The fourth table is this lesson's most striking result. The same twenty-six flows are measured at three separate drawing resolutions, and the result gives three separate numbers. If no boundary is drawn at all, every node is counted in a single zone: **zero crossings, zero items**. The system has not changed; only its measure has been erased. This is the most common reason a threat modeling session ends with zero items — the graph was drawn, but no boundary was placed on it. The coarse drawing — only the outer/inner split — sees **7 crossings**, meaning 30.4% of the real crossings, and produces 15 items, meaning 25.9% of them. The detailed drawing gives 23 crossings and 58 items. The difference is **16 crossings and 43 items**; nine of these items come from flows entering a data store. Where the loss happens is clear. The coarse drawing counts the internal network as a single trusted box, so flows like `validator->measurement-store`, `billing->billing-store`, or `management-console->identity-service` never cross a boundary at all. These are the system's highest-value flows: the places where identity and billing data get written to stores. **The items that disappear as boundary resolution drops are not chosen at random; what stays inside the internal zone is what disappears, and what sits inside the internal zone is the store itself.** The rule that follows is plain: a threat modeling session's first output is not the threat list — it is the **zone list**. The number of zones and the number of items that can be found move in the same direction, and when zones are drawn wrong, every step that follows runs on an incomplete set. #### Summary - At the design stage, the surface's countable equivalent is the number of flows that cross a trust boundary: in a graph of 17 nodes and 26 flows, 23 flows cross a boundary, and only 3 stay within the same zone. - The 23 boundary-crossing flows produce 58 items under four rules; 2.52 per crossing. The item count depends not on how important the flow is but on how many rules it triggers at once. - 58.6% of the items come from the 17 flows that go from a lower trust level to a higher one; output filtering is the smallest group with only 6 items. - When no boundary is drawn at all, the measurement gives zero crossings, zero items; the system has not changed, only its measure has been erased. - A coarse outer/inner drawing sees 7 of the 23 crossings and 15 of the 58 items; the 16 crossings that stay invisible carry 43 items, and 9 of these are flows entering a data store. #### Next Step This lesson produced a countable surface: 23 crossings and 58 items. But 58 items are not a threat list — the items are control types; they say "an identity check is required here," not "this specific thing could happen here." The difference between the two is the method's second step. Looking at a graph element and asking "what could go wrong here" depends on the person and on their attention that day. What makes this independent of the person is applying a **fixed category list per element type**: asking the same questions of every flow, every process, and every store. The next lesson runs this systematic generation and separates two numbers: how many threats the method produces, and how many of those are actually valid for this system. Generation is cheap; **elimination** is expensive, and it is measured in minutes. ### Threat Classification Methods Source: https://academia.sh/en/courses/secure-lifecycle/threat-classification-methods Systematic threat generation's two numbers: applying six categories to 43 graph elements produces 156 raw threats, 108 of which (69.2%) are valid for this system and 48 of which turn out invalid, elimination spends 72 minutes for no return, the total session load comes to 666 minutes, and generation limited to exposed elements alone finds 92.6% of the valid threats 63.5 minutes cheaper. The previous lesson derived 58 items from 23 boundary crossings. Those 58 items were not a threat list: each one stated a control type — "a source identity check is required here," "authorization is required here." It did not say what could go wrong. The step that closes this gap is looking at a graph element and asking "what could happen here." This question depends on the person and on their attention that day; two people produce two different lists. What makes it independent of the person is applying a **fixed category list to every element type**. This lesson runs that systematic generation and separates two numbers: how many threats the method produces, and how many of those are actually valid for this system. #### A Fixed Question List per Element Type The widely used method is a **classification method** that generates threats systematically for every data flow diagram element. It has six categories, and the categories are known by what they do: **identity spoofing**, **tampering**, **repudiation**, **information disclosure**, **denial of service**, and **privilege escalation**. The method has a single rule: every element is asked all of the categories applicable to its type — none is skipped, none is added on top. The inventory below is a **model**; it is the same structure as the graph from the previous lesson and carries no attack step. Element properties are not written by hand — they are derived from the graph. **TM9:** every element's five properties are calculated from the graph — exposure, carrying a sensitive class, being in the work flow, having multiple roles, and producing a transaction. **TM10:** generation is the product of an element's type and the categories applicable to that type. **TM11:** a generated threat is **invalid** for this system if the element's property does not satisfy the relevant condition. **TM12:** every raw threat spends a fixed elimination time, and every threat found valid spends an additional writing time. ```js // threat/element.mjs — inventory of graph elements (model): the same structure as the graph in // 02, this time as an element list. There is no real system; no attack step is carried. const ZONE = { internet: 0, field: 1, edge: 2, internal: 3, management: 4, data: 5 }; const NODE = Object.fromEntries(` subscriber D internet | oversight-body D internet | meter-device D field | field-team D field subscriber-portal S edge | collection-endpoint S edge | field-app S field validator S internal | billing S internal | work-order S internal | report-generator S internal | identity-service S internal management-console S management | measurement-store P data | billing-store P data identity-store P data | audit-log P data` .trim().split(/\s*[|\n]\s*/).map((r) => { const [name, type, zone] = r.trim().split(/\s+/); return [name, { name, type, zone, level: ZONE[zone] }]; })); const FLOW = ` subscriber subscriber-portal identity | subscriber-portal identity-service identity identity-service identity-store identity | subscriber-portal billing billing billing billing-store billing | billing subscriber-portal billing meter-device collection-endpoint measurement | collection-endpoint validator measurement validator measurement-store measurement | validator billing measurement field-team field-app identity | field-app work-order order work-order field-app order | work-order measurement-store measurement field-app collection-endpoint measurement | management-console validator setting management-console collection-endpoint setting | management-console identity-service setting report-generator measurement-store measurement | report-generator billing-store billing oversight-body report-generator identity | report-generator oversight-body billing billing audit-log record | identity-service audit-log record work-order audit-log record | billing work-order order` .trim().split(/\s*[|\n]\s*/).map((r) => { const [source, target, cls] = r.trim().split(/\s+/); return { source, target, class: cls, name: `${source}->${target}` }; }); // TM9: every element's property is derived from the graph; not written by hand. const incoming = (d) => FLOW.filter((a) => a.target === d); const outgoing = (d) => FLOW.filter((a) => a.source === d); const neighbors = (d) => [...incoming(d), ...outgoing(d)]; const isSensitiveClass = (s) => s === "identity" || s === "billing"; const isCriticalClass = (s) => s === "measurement" || s === "billing" || s === "order"; export const ELEMENT = [ ...Object.values(NODE).map((d) => ({ name: d.name, type: d.type, exposed: d.type === "D" ? outgoing(d.name).length > 0 : incoming(d.name).some((a) => NODE[a.source].level < d.level), sensitive: neighbors(d.name).some((a) => isSensitiveClass(a.class)), critical: neighbors(d.name).some((a) => isCriticalClass(a.class)), multiRole: new Set(incoming(d.name).map((a) => NODE[a.source].zone)).size >= 2, transacts: d.type === "P" || (d.type === "D" ? outgoing(d.name).length > 0 : outgoing(d.name).some((a) => NODE[a.target].type === "P")), })), ...FLOW.map((a) => ({ name: a.name, type: "A", exposed: NODE[a.source].zone !== NODE[a.target].zone, sensitive: isSensitiveClass(a.class), critical: isCriticalClass(a.class), multiRole: false, transacts: false, })), ]; // TM10: systematic generation applies to every element type the categories applicable to that type. export const CATEGORY = ["identity-spoofing", "tampering", "repudiation", "information-disclosure", "denial-of-service", "privilege-escalation"]; export const APPLIES = { D: ["identity-spoofing", "repudiation"], S: CATEGORY, P: ["tampering", "repudiation", "information-disclosure", "denial-of-service"], A: ["tampering", "information-disclosure", "denial-of-service"], }; // TM11: a generated threat is invalid if the element's property does not satisfy the condition below. export const VALID_IF = { "identity-spoofing": (o) => o.exposed, "tampering": (o) => o.type !== "A" || o.exposed, "repudiation": (o) => o.transacts, "information-disclosure": (o) => o.sensitive, "denial-of-service": (o) => o.critical, "privilege-escalation": (o) => o.multiRole, }; // TM12: every generated threat spends a fixed elimination time, and every threat found valid // spends an additional writing time (fictional minutes). export const ELIMINATION_MIN = 1.5, WRITE_MIN = 4; ``` ```js // threat/generation.mjs — how many threats systematic generation gives, how many are invalid // for this system, how many minutes elimination takes. import { ELEMENT, CATEGORY, APPLIES, VALID_IF, ELIMINATION_MIN, WRITE_MIN } from "./element.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const generate = (els) => els.flatMap((o) => APPLIES[o.type].map((k) => ({ element: o, category: k, valid: VALID_IF[k](o) }))); const RAW = generate(ELEMENT), VALID = RAW.filter((t) => t.valid); const TYPE = { D: "external entity", S: "process", P: "data store", A: "data flow" }; console.log(`${ELEMENT.length} elements, ${CATEGORY.length} categories; raw generation ${RAW.length} threats, ` + `valid ${VALID.length} (${((VALID.length / RAW.length) * 100).toFixed(1)}%), ` + `invalid ${RAW.length - VALID.length}`); const A = [-17, 10, 12, 5, 7, 9, 9]; console.log("\n1. generation per element type"); write(A, "element type", "elements", "categories", "raw", "valid", "invalid", "valid %"); for (const t of ["D", "S", "P", "A"]) { const o = ELEMENT.filter((x) => x.type === t), h = generate(o), g = h.filter((x) => x.valid); write(A, TYPE[t], o.length, APPLIES[t].length, h.length, g.length, h.length - g.length, ((g.length / h.length) * 100).toFixed(1)); } const B = [-23, 5, 7, 9, 9, 2, -32]; console.log("\n2. generation and elimination per category"); write(B, "category", "raw", "valid", "invalid", "valid %", "", "validity condition"); const CONDITION = { "identity-spoofing": "exposed", "tampering": "if a flow, crosses a boundary", "repudiation": "transacts", "information-disclosure": "carries a sensitive class", "denial-of-service": "in the work flow", "privilege-escalation": "multi-role process" }; for (const k of CATEGORY) { const h = RAW.filter((t) => t.category === k), g = h.filter((t) => t.valid); write(B, k, h.length, g.length, h.length - g.length, ((g.length / h.length) * 100).toFixed(1), "", CONDITION[k]); } const C = [-23, 10, 5, 7, 11, 11]; console.log("\n3. three generation sets: all, exposed only, processes only"); write(C, "set", "elements", "raw", "valid", "total min", "valid/min"); const SET = { "all elements": ELEMENT, "exposed elements only": ELEMENT.filter((o) => o.exposed), "processes only": ELEMENT.filter((o) => o.type === "S") }; for (const [name, els] of Object.entries(SET)) { const h = generate(els), g = h.filter((x) => x.valid); const min = h.length * ELIMINATION_MIN + g.length * WRITE_MIN; write(C, name, els.length, h.length, g.length, min.toFixed(1), (g.length / min).toFixed(3)); } const exposed = generate(ELEMENT.filter((o) => o.exposed)).filter((x) => x.valid); console.log(`generation limited to exposed elements finds ` + `${((exposed.length / VALID.length) * 100).toFixed(1)}% of the valid threats; the ` + `${VALID.length - exposed.length} threats it misses arise only in internal elements`); const CODE = Object.fromEntries(CATEGORY.map((k) => [k, k.slice(0, 2).toUpperCase()])); const D = [-30, -11, 5, 7, 2, -19]; console.log("\n4. the three elements producing the most and fewest valid threats (" + CATEGORY.map((k) => `${CODE[k]}=${k}`).join(", ") + ")"); write(D, "element", "type", "raw", "valid", "", "valid categories"); const ranked = ELEMENT.map((o) => ({ o, h: APPLIES[o.type], g: APPLIES[o.type].filter((k) => VALID_IF[k](o)) })).sort((x, y) => y.g.length - x.g.length); for (const s of [...ranked.slice(0, 3), ...ranked.slice(-3)]) write(D, s.o.name, TYPE[s.o.type], s.h.length, s.g.length, "", s.g.map((k) => CODE[k]).join(" ") || "-"); for (const t of ["S", "A"]) { const g = VALID.filter((x) => x.element.type === t).length; console.log(`${TYPE[t]} elements: ` + `${((ELEMENT.filter((o) => o.type === t).length / ELEMENT.length) * 100).toFixed(1)}% of the elements, ` + `${((g / VALID.length) * 100).toFixed(1)}% of the valid threats`); } console.log(`eliminating the ${RAW.length - VALID.length} invalid threats spends ` + `${((RAW.length - VALID.length) * ELIMINATION_MIN).toFixed(1)} minutes and returns nothing`); ``` ``` 43 elements, 6 categories; raw generation 156 threats, valid 108 (69.2%), invalid 48 1. generation per element type element type elements categories raw valid invalid valid % external entity 4 2 8 8 0 100.0 process 9 6 54 40 14 74.1 data store 4 4 16 12 4 75.0 data flow 26 3 78 48 30 61.5 2. generation and elimination per category category raw valid invalid valid % validity condition identity-spoofing 13 11 2 84.6 exposed tampering 39 36 3 92.3 if a flow, crosses a boundary repudiation 17 13 4 76.5 transacts information-disclosure 39 17 22 43.6 carries a sensitive class denial-of-service 39 24 15 61.5 in the work flow privilege-escalation 9 7 2 77.8 multi-role process 3. three generation sets: all, exposed only, processes only set elements raw valid total min valid/min all elements 43 156 108 666.0 0.162 exposed elements only 38 135 100 602.5 0.166 processes only 9 54 40 241.0 0.166 generation limited to exposed elements finds 92.6% of the valid threats; the 8 threats it misses arise only in internal elements 4. the three elements producing the most and fewest valid threats (ID=identity-spoofing, TA=tampering, RE=repudiation, IN=information-disclosure, DE=denial-of-service, PR=privilege-escalation) element type raw valid valid categories billing process 6 6 ID TA RE IN DE PR subscriber-portal process 6 5 ID TA IN DE PR validator process 6 5 ID TA RE DE PR identity-service->audit-log data flow 3 1 TA work-order->audit-log data flow 3 1 TA billing->work-order data flow 3 1 DE process elements: 20.9% of the elements, 37.0% of the valid threats data flow elements: 60.5% of the elements, 44.4% of the valid threats eliminating the 48 invalid threats spends 72.0 minutes and returns nothing ``` The numbers are in the **measurement** class; their inputs are TM9–TM12. #### One Hundred Fifty-Six Raw Threats When the applicable ones of the six categories are asked of forty-three elements, **156 raw threats** come out. Generation is cheap and automatic: nobody thinks about "what could happen here" — the table just gets filled in. This is where the method's entire value comes from — no element is skipped, and the list does not change from person to person. The first table shows where the generation comes from. The twenty-six data flow elements each take three categories, producing **78 threats** — half of the raw generation. The nine process elements each take six categories and produce 54 threats. The four external entity elements take only two categories: an external entity is not asked about tampering or privilege escalation, because the element sits outside the system and holds no authority inside it. The second table breaks the raw count down category by category. Tampering, information disclosure, and denial of service are the busiest three with 39 threats each; because all three are applicable to processes, stores, and flows alike, they produce a count close to the total element count. Privilege escalation produces only nine threats, because it is asked only of processes. #### One Hundred Eight Valid The second number comes out after elimination. Of the 156 raw threats, **108** are valid for this system (69.2%), and **48 are invalid**. Invalidity is not arbitrary: every category has a validity condition, and the condition looks at a property of the element that is derived from the graph. The split varies widely from category to category. **92.3%** of tampering is valid; its condition only asks flows to cross a boundary, and 23 of the 26 flows do. Information disclosure, on the other hand, is only **43.6%** valid: the condition is that the element carries a sensitive class — identity or billing — and the category falls flat on the 22 elements that carry a measurement, setting, or record class. Denial of service sits in the middle (61.5%): interrupting an element that is not in the work flow does not stop the work. The fourth table shows the same split at the element level. `billing` produces a valid threat in all six of the six categories — it is exposed, carries a sensitive class, sits in the work flow, has multiple roles, and writes to a store. At the other end, flows like `work-order->audit-log` are valid in only one of three categories. The concentration is striking: the nine process elements, only 20.9% of the elements, carry **37.0%** of the valid threats; the 26 data flow elements, 60.5% of the elements, carry 44.4% of the threats. #### Elimination's Minute Generation's cheapness is what makes elimination expensive. Every raw threat has to be assessed — the invalid ones too. In the model, eliminating a threat takes 1.5 fictional minutes, and writing up a threat found valid takes an additional 4 minutes. The total is **666 minutes** — 11.1 hours. Of this, 234 minutes go to elimination and 432 minutes to writing. **72 minutes of elimination are spent on the 48 invalid threats and return nothing** — reading and discarding rows that will never turn into a countermeasure. A threat modeling session's duration should therefore be read together with the number of threats **eliminated**, not the number generated. The third table compares three separate generation sets. All elements: 156 raw, 108 valid, 666 minutes, 0.162 valid threats per minute. Exposed elements only: 135 raw, 100 valid, 602.5 minutes, 0.166 per minute. Processes only: 54 raw, 40 valid, 241 minutes, 0.166 again. The three sets' per-minute efficiency is nearly identical, and this runs against expectation. Prioritizing elements that cross a boundary saves 63.5 minutes and loses 8 threats — a saving of less than a tenth of the total, because 88% of the elements are already exposed. Looking only at processes, on the other hand, saves 425 minutes but leaves out **63%** of the valid threats. The conclusion that follows is this: in this method, a shortcut does not work by raising efficiency — it works by **shrinking coverage**. For a team that wants to cut the time down, the only honest path is to write down which set of elements is being left out. #### Summary - Systematic generation asks 43 graph elements the categories applicable to their type and produces 156 raw threats; generation is cheap, does not vary by person, and skips no element. - 108 of the raw threats (69.2%) are valid for this system, and 48 are invalid; the validity condition looks at a property of the element derived from the graph. - Validity ranges from 43.6% to 92.3% depending on category; information disclosure falls flat on 22 elements, while tampering almost never does. - The session's duration is 666 minutes; 72 minutes go to eliminating invalid threats and return nothing. Duration is read together with the threats eliminated, not the threats generated. - The three generation sets' per-minute efficiency is nearly identical (0.162–0.166); a shortcut saves from coverage, not efficiency — looking only at processes misses 63% of the valid threats. #### Next Step Systematic generation starts from the element: the same questions are asked of every box, every arrow, and every store. This approach's strength and its limit sit in the same place — it treats the element **on its own**. Every one of the 108 valid threats is the intersection of one element and one category; none of them arises from two elements being used together. But real damage usually does not arise from a single element — it arises from several elements being used one after another: information obtained in one place becomes valid somewhere else. The way to find items of this kind is to start not from the element but from the **goal** — what the attacker wants to obtain and by what path they get there. The next lesson runs scenario-based modeling over the same graph and measures it on the coverage axis: how many threats the scenarios find, how many of the systematic method's finds they miss, and what the intersection of the two is. ### Risk-Focused Modeling Source: https://academia.sh/en/courses/secure-lifecycle/risk-focused-modeling Scenario-based modeling's coverage measurement: five scenarios producing 51 threats, 39 of which intersect with the systematic set and 12 of which are chain threats not defined on any single element, scenarios missing 63.9% of the systematic set, 24 of the missed ones falling in the denial-of-service category, the scenario method running at 0.234 threats per minute — 1.44 times the speed of systematic generation — and the union of the two methods coming to 120 threats. Systematic generation starts from the element: the same questions are asked of every box, every arrow, and every store. Every one of the 108 valid threats is **the intersection of one element and one category**; none of them arises from two elements being used together. But real damage usually does not arise from a single element — it arises from several elements being used one after another. The way to find items of this kind is to start not from the element but from the **goal**: what the attacking side wants to obtain and by what path they get there. This lesson runs scenario-based modeling over the same graph and asks a single question — how many threats do scenarios find, how many of the systematic method's finds do they miss, and what is the intersection of the two. #### A Scenario Is a Node Sequence A scenario is a path built from the attacker's point of view; it shares its root with the **abuse case** from the Security Requirements lesson, but there it was used to write a requirement — here it is used to generate a threat. The scenarios below are a **model** and only write down **which elements are used**; no step is described by how it is done. Trying a scenario like this on a real system, **without written authorization and a defined scope, is both illegal and unprofessional.** **TM13:** a scenario is a node sequence running from an attacker profile to a target. **TM14:** the elements a scenario touches are the nodes in the sequence and the flows between them. **TM15:** a scenario produces two kinds of threat — an **element threat**, which overlaps with the scenario's category on a touched element, and a **chain threat**, which arises from two consecutive nodes being used together; the latter has no counterpart in systematic generation because it is not defined on a single element. **TM16:** the cost of building a scenario is a fixed time per scenario plus a time per touched element. ```js // threat/scenario.mjs — the same graph, this time with scenarios built backward from the // target (model). A scenario only writes down which elements are used; how a step is carried // out is not written. There is no real system, no real endpoint, and no working step. const ZONE = { internet: 0, field: 1, edge: 2, internal: 3, management: 4, data: 5 }; export const NODE = Object.fromEntries(` subscriber D internet | oversight-body D internet | meter-device D field | field-team D field subscriber-portal S edge | collection-endpoint S edge | field-app S field validator S internal | billing S internal | work-order S internal | report-generator S internal | identity-service S internal management-console S management | measurement-store P data | billing-store P data identity-store P data | audit-log P data` .trim().split(/\s*[|\n]\s*/).map((r) => { const [name, type, zone] = r.trim().split(/\s+/); return [name, { name, type, zone, level: ZONE[zone] }]; })); export const FLOW = ` subscriber subscriber-portal identity | subscriber-portal identity-service identity identity-service identity-store identity | subscriber-portal billing billing billing billing-store billing | billing subscriber-portal billing meter-device collection-endpoint measurement | collection-endpoint validator measurement validator measurement-store measurement | validator billing measurement field-team field-app identity | field-app work-order order work-order field-app order | work-order measurement-store measurement field-app collection-endpoint measurement | management-console validator setting management-console collection-endpoint setting | management-console identity-service setting report-generator measurement-store measurement | report-generator billing-store billing oversight-body report-generator identity | report-generator oversight-body billing billing audit-log record | identity-service audit-log record work-order audit-log record | billing work-order order` .trim().split(/\s*[|\n]\s*/).map((r) => { const [source, target, cls] = r.trim().split(/\s+/); return { source, target, class: cls, name: `${source}->${target}` }; }); // The previous lesson's systematic generation is re-derived here with the same rules. const incoming = (d) => FLOW.filter((a) => a.target === d); const outgoing = (d) => FLOW.filter((a) => a.source === d); const neighbors = (d) => [...incoming(d), ...outgoing(d)]; const isSensitiveClass = (s) => s === "identity" || s === "billing"; const isCriticalClass = (s) => s === "measurement" || s === "billing" || s === "order"; export const ELEMENT = Object.fromEntries([ ...Object.values(NODE).map((d) => [d.name, { name: d.name, type: d.type, exposed: d.type === "D" ? outgoing(d.name).length > 0 : incoming(d.name).some((a) => NODE[a.source].level < d.level), sensitive: neighbors(d.name).some((a) => isSensitiveClass(a.class)), critical: neighbors(d.name).some((a) => isCriticalClass(a.class)), multiRole: new Set(incoming(d.name).map((a) => NODE[a.source].zone)).size >= 2, transacts: d.type === "P" || (d.type === "D" ? outgoing(d.name).length > 0 : outgoing(d.name).some((a) => NODE[a.target].type === "P")) }]), ...FLOW.map((a) => [a.name, { name: a.name, type: "A", exposed: NODE[a.source].zone !== NODE[a.target].zone, sensitive: isSensitiveClass(a.class), critical: isCriticalClass(a.class), multiRole: false, transacts: false }]), ]); export const CATEGORY = ["identity-spoofing", "tampering", "repudiation", "information-disclosure", "denial-of-service", "privilege-escalation"]; const APPLIES = { D: ["identity-spoofing", "repudiation"], S: CATEGORY, P: ["tampering", "repudiation", "information-disclosure", "denial-of-service"], A: ["tampering", "information-disclosure", "denial-of-service"] }; const VALID_IF = { "identity-spoofing": (o) => o.exposed, "tampering": (o) => o.type !== "A" || o.exposed, "repudiation": (o) => o.transacts, "information-disclosure": (o) => o.sensitive, "denial-of-service": (o) => o.critical, "privilege-escalation": (o) => o.multiRole }; export const isValid = (o, k) => APPLIES[o.type].includes(k) && VALID_IF[k](o); export const SYSTEMATIC = new Set(Object.values(ELEMENT).flatMap((o) => CATEGORY.filter((k) => isValid(o, k)).map((k) => `${o.name}|${k}`))); // TM13: a scenario is a node sequence running from an attacker profile to a target. The // sequence only says which element is used; how the step is carried out is not written. export const SCENARIO = [ { name: "subscriber-account-takeover", profile: "external", target: "identity-store", path: ["subscriber", "subscriber-portal", "identity-service", "identity-store"], categories: ["identity-spoofing", "information-disclosure"] }, { name: "billing-amount-tampering", profile: "subscriber", target: "billing-store", path: ["subscriber", "subscriber-portal", "billing", "billing-store"], categories: ["tampering", "privilege-escalation"] }, { name: "false-measurement-record", profile: "field", target: "measurement-store", path: ["meter-device", "collection-endpoint", "validator", "measurement-store"], categories: ["identity-spoofing", "tampering"] }, { name: "audit-trail-corruption", profile: "insider", target: "audit-log", path: ["management-console", "identity-service", "audit-log"], categories: ["tampering", "repudiation"] }, { name: "bulk-billing-data-extraction", profile: "external-organization", target: "billing-store", path: ["oversight-body", "report-generator", "billing-store"], categories: ["identity-spoofing", "information-disclosure"] }, ]; // TM14: the elements a scenario touches are the nodes in the sequence and the flows between them. export function touched(s) { const ls = [...s.path]; for (let i = 0; i + 1 < s.path.length; i++) ls.push(`${s.path[i]}->${s.path[i + 1]}`); return ls; } // TM15: a scenario produces two kinds of threat. An element threat overlaps with the scenario's // category on a touched element and is valid there. A chain threat arises from two consecutive // nodes being used together; it has no counterpart in systematic generation because it is not // defined on a single element. export const elementThreats = (s) => touched(s) .flatMap((a) => s.categories.filter((k) => isValid(ELEMENT[a], k)).map((k) => `${a}|${k}`)); export const chainThreats = (s) => s.path.slice(0, -1) .map((d, i) => `${d}=>${s.path[i + 1]}|chain`); // TM16: the cost of building a scenario is a fixed time per scenario plus a time per touched element. export const SCENARIO_MIN = 25, ELEMENT_MIN = 3; // Systematic generation's duration is derived from the same inputs too. export const RAW_COUNT = Object.values(ELEMENT).reduce((a, o) => a + APPLIES[o.type].length, 0); export const ELIMINATION_MIN = 1.5, WRITE_MIN = 4; ``` ```js // threat/coverage.mjs — how many threats scenario-based generation finds, how many of // systematic generation's finds it misses, what the intersection of the two is. import { ELEMENT, CATEGORY, SCENARIO, SYSTEMATIC, touched, elementThreats, chainThreats, SCENARIO_MIN, ELEMENT_MIN, RAW_COUNT, ELIMINATION_MIN, WRITE_MIN } from "./scenario.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const ELEMENT_T = new Set(SCENARIO.flatMap(elementThreats)); const CHAIN = new Set(SCENARIO.flatMap(chainThreats)); const SCENARIO_SET = new Set([...ELEMENT_T, ...CHAIN]); const INTERSECTION = [...ELEMENT_T].filter((t) => SYSTEMATIC.has(t)); const SYSTEMATIC_ONLY = [...SYSTEMATIC].filter((t) => !ELEMENT_T.has(t)); const SCENARIO_MINUTES = SCENARIO.length * SCENARIO_MIN + SCENARIO.reduce((a, s) => a + touched(s).length, 0) * ELEMENT_MIN; console.log(`${SCENARIO.length} scenarios; scenario set ${SCENARIO_SET.size} threats ` + `(${ELEMENT_T.size} element threats + ${CHAIN.size} chain threats), systematic set ` + `${SYSTEMATIC.size}; intersection ${INTERSECTION.length}`); const A = [-30, -23, -19, 7, 9, 12, 10, 9]; console.log("\n1. scenarios and what they produce"); write(A, "scenario", "profile", "target", "steps", "touched", "element t.", "chain t.", "minutes"); for (const s of SCENARIO) { const d = touched(s); write(A, s.name, s.profile, s.target, s.path.length, d.length, elementThreats(s).length, chainThreats(s).length, SCENARIO_MIN + d.length * ELEMENT_MIN); } const B = [-17, 9, 17, 22]; console.log("\n2. the two methods' coverage"); write(B, "set", "threats", "% of systematic", "ratio to systematic"); for (const [name, n, p] of [ ["systematic set", SYSTEMATIC.size, "-"], ["scenario set", SCENARIO_SET.size, "-"], ["intersection", INTERSECTION.length, (INTERSECTION.length / SYSTEMATIC.size * 100).toFixed(1)], ["scenario only", SCENARIO_SET.size - INTERSECTION.length, "-"], ["systematic only", SYSTEMATIC_ONLY.length, (SYSTEMATIC_ONLY.length / SYSTEMATIC.size * 100).toFixed(1)], ["union", SYSTEMATIC.size + SCENARIO_SET.size - INTERSECTION.length, "-"], ]) write(B, name, n, p === "-" ? "-" : p + " %", (n / SYSTEMATIC.size).toFixed(2) + " x"); const C = [-24, 9, 9, 13, 8]; console.log("\n3. threats found per minute"); write(C, "method", "threats", "minutes", "threats/min", "missed"); const SYSTEMATIC_MIN = RAW_COUNT * ELIMINATION_MIN + SYSTEMATIC.size * WRITE_MIN; for (const [name, n, min, missed] of [ ["systematic generation", SYSTEMATIC.size, SYSTEMATIC_MIN, CHAIN.size], ["scenario-based", SCENARIO_SET.size, SCENARIO_MINUTES, SYSTEMATIC_ONLY.length], ["both together", SYSTEMATIC.size + SCENARIO_SET.size - INTERSECTION.length, SYSTEMATIC_MIN + SCENARIO_MINUTES, 0], ]) write(C, name, n, min.toFixed(1), (n / min).toFixed(3), missed); const D = [-17, 12, 14, 8, 10]; console.log("\n4. where the threats scenarios miss stand"); write(D, "element type", "systematic", "intersection", "missed", "missed %"); const TYPE = { D: "external entity", S: "process", P: "data store", A: "data flow" }; for (const t of ["D", "S", "P", "A"]) { const s = [...SYSTEMATIC].filter((x) => ELEMENT[x.split("|")[0]].type === t); const k = s.filter((x) => !ELEMENT_T.has(x)); write(D, TYPE[t], s.length, s.length - k.length, k.length, ((k.length / s.length) * 100).toFixed(1)); } const cat = CATEGORY.map((k) => ({ k, n: SYSTEMATIC_ONLY.filter((x) => x.endsWith("|" + k)).length })) .sort((a, b) => b.n - a.n); console.log(`category distribution of the missed ${SYSTEMATIC_ONLY.length} threats: ` + cat.map((x) => `${x.k} ${x.n}`).join(", ")); console.log(`none of the ${CHAIN.size} chain threats appear in systematic generation; ` + `they are not defined on a single element`); ``` ``` 5 scenarios; scenario set 51 threats (39 element threats + 12 chain threats), systematic set 108; intersection 39 1. scenarios and what they produce scenario profile target steps touched element t. chain t. minutes subscriber-account-takeover external identity-store 4 7 9 3 46 billing-amount-tampering subscriber billing-store 4 7 8 3 46 false-measurement-record field measurement-store 4 7 9 3 46 audit-trail-corruption insider audit-log 3 5 7 2 40 bulk-billing-data-extraction external-organization billing-store 3 5 6 2 40 2. the two methods' coverage set threats % of systematic ratio to systematic systematic set 108 - 1.00 x scenario set 51 - 0.47 x intersection 39 36.1 % 0.36 x scenario only 12 - 0.11 x systematic only 69 63.9 % 0.64 x union 120 - 1.11 x 3. threats found per minute method threats minutes threats/min missed systematic generation 108 666.0 0.162 12 scenario-based 51 218.0 0.234 69 both together 120 884.0 0.136 0 4. where the threats scenarios miss stand element type systematic intersection missed missed % external entity 8 3 5 62.5 process 40 17 23 57.5 data store 12 6 6 50.0 data flow 48 13 35 72.9 category distribution of the missed 69 threats: denial-of-service 24, tampering 19, repudiation 11, information-disclosure 7, privilege-escalation 5, identity-spoofing 3 none of the 12 chain threats appear in systematic generation; they are not defined on a single element ``` The numbers are in the **measurement** class; their inputs are TM13–TM16. #### Five Scenarios, Fifty-One Threats The first table gives the scenarios' own measure. Three of the five scenarios are four steps, two are three steps; the number of elements touched ranges from seven to five. The total comes to **51 threats**, and the composition of this number is the real finding: **39 element threats and 12 chain threats**. Scenario durations, ranging between forty-six and forty minutes, add up to 218 minutes. Compared with systematic generation's 666 minutes, this is roughly a third. Scenarios touch fewer elements because the path to the target does not traverse the whole graph: the five scenarios together use only a portion of the 43 elements. #### Intersection, Difference, and Union The second table is this lesson's main measurement. The systematic set is 108, the scenario set is 51 threats. The intersection is **39** — that is, all of the element threats the scenarios find are already in the systematic set. This is an expected result: when a scenario touches an element, it produces the category that is valid on that element, and the systematic method has already produced every valid category on every element. **All 12 of the threats found only in the scenarios are chain threats.** These arise from two nodes being used together, and because they are not defined on a single element, they fall into no row of systematic generation. Given the method's structure, they cannot fall in: as long as the question is "what could happen at this element," the question "what is obtained at this element becoming valid at that one" is never asked. **The 69 threats found only in the systematic set**, on the other hand, lie outside the set the scenarios see and make up **63.9%** of the systematic set. The five scenarios touch 26 of the 43 elements, but at each element they only produce the category the scenario itself carries; the 17 untouched elements are left entirely unexamined. The union is 120 threats — 1.11 times the systematic set. The two methods do not substitute for each other; the only reason the union is larger than the systematic set is the 12 chain threats. #### What It Misses The fourth table separates out where the misses stand. The miss rate is highest in data flow elements: **35** of 48 threats are missed (72.9%). It is lowest in data stores (50.0%), because all five of the five scenarios end at a store and stores are the scenarios' target. The category distribution is sharper. Of the 69 missed threats, **24 are denial of service**, 19 are tampering, 11 are repudiation; identity spoofing, with only three threats, is the least-missed category. The reason lies in the very nature of building a scenario: a scenario describes **reaching** a target, while denial of service does not aim to reach the target — it aims to stop the service. Scenarios built from the attacker's point of view are after access; a threat class that does not want access has no natural place on a scenario list. This is the scenario method's measurable blind spot. The sentence "look through the attacker's eyes" sounds inclusive, but once measured, it leaves an entire category almost completely out. #### Threats Found Per Minute The third table compares the two methods in the same unit. Systematic generation finds 108 threats in 666 minutes: **0.162** per minute. Scenario-based generation finds 51 threats in 218 minutes: **0.234** per minute — 1.44 times faster. This ratio is misleading read on its own. The reason the scenario method looks efficient is that it carries no elimination load: because a scenario is built backward from a target, every threat it produces already sits on a path, and no row comes up to be discarded. Systematic generation, by contrast, spends 72 minutes on 48 invalid threats. What creates the efficiency is not the method's intelligence — it is **the smallness of the search space**. The last column writes down the price of that. Used alone, systematic generation misses 12 chain threats; used alone, scenario-based generation misses **69** threats. The two together give 120 threats in 884 minutes — 0.136 per minute, slower than either method alone. Completing coverage lowers per-minute efficiency, and this is a decision the measurement itself hands down: **the fast method is the incomplete one.** #### Summary - A scenario is a node sequence running from an attacker profile to a target; five scenarios touch a portion of the 43 elements and produce **51 threats** in 218 minutes. - 39 of the 51 threats are element threats, **12 are chain threats**; chain threats arise from two nodes being used together and have no counterpart in systematic generation. - The intersection is 39; all of the element threats scenarios find sit inside the systematic set. Scenarios miss **63.9%** of the systematic set (69 threats). - The category distribution of the missed threats is uneven: 24 are denial of service, only 3 are identity spoofing. Because a scenario is after access, it leaves out the threat class that does not want access. - The scenario method runs at 0.234 threats per minute, 1.44 times the speed of systematic generation, but this speed comes from the smallness of the search space; the two together give 120 threats in 884 minutes. #### Next Step Chain threats were this lesson's most notable finding: twelve items, defined on no single element, visible only on a path. But this lesson only **counted** paths — it did not rank them. All five of the five scenarios stand at equal weight on the list. In reality, paths are not equal. A target can be reached by more than one path, and some of these paths are cheaper than others; what defense cares about is not the most expensive path but **the cheapest one**. Counting this requires writing the scenario not as a flat sequence but as a structure branching backward from the target: a root goal, sub-goals, and leaf steps whose costs are written together with their preconditions. The next lesson builds this structure, calculates the cheapest path's cost, and asks a single question: by what factor does closing off one node raise the tree's cost. ### Attack Trees Source: https://academia.sh/en/courses/secure-lifecycle/attack-trees Cost measurement of goal-backward decomposition: in a tree of 12 leaves and three sub-goals, the root cost is 9 fictional units and the cheapest path is two steps, closing 10 of the 12 leaves never changes the root cost at all, the single most efficient closure yields 1.22x for 2 countermeasure units, a four-countermeasure-unit OR node yields 1.89x, the greedy order closes the root at 28 units, and the same result is reached at 16 units by picking one OR node per branch. The previous lesson's most notable finding was chain threats: twelve items, defined on no single element, visible only on a path. But that lesson only counted paths; all five of the five scenarios stood at equal weight on the list. In reality, paths are not equal. A target can be reached by more than one path, and some of these are cheaper than others; what defense cares about is not the most expensive path but **the cheapest one**. Counting this requires writing the scenario not as a flat sequence but as a structure branching backward from the target. The **attack tree** is this structure: a root goal, sub-goals, and leaf steps whose cost is written together with their precondition. #### Root, Sub-Goal, Leaf The tree below is a **model**. Leaf nodes are described only by their **precondition and fictional cost**; nowhere is it written how a step is carried out — the tree's value on the defense side lies not in the step itself but in the **combining structure** of the steps. Running such an analysis on a real system, **without written authorization and a defined scope, is both illegal and unprofessional**. The cost and countermeasure numbers are fictional units, not currency. **TM17:** every node is either OR (one of its children is enough), AND (all are required), or a leaf; a leaf carries a precondition, a cost, and a **countermeasure price**. **TM18:** an OR node's cost is the smallest of its children's, an AND node's cost is their sum. **TM19:** a closed leaf's cost is infinite. **TM20:** the cheapest path is the leaf set obtained by picking the cheapest child at every OR node. ```js // threat/tree.mjs — attack tree (model): root goal, sub-goals, and leaf steps. // Leaf steps are written ONLY with their precondition and fictional cost; nowhere is it // described how a step is carried out. The cost and countermeasure units are fictional, not currency. // TM17: a node is either OR (one of its children is enough), AND (all are required), or a leaf. // A leaf carries "name precondition cost countermeasure": the countermeasure is the fictional // price of closing that leaf. export const TREE = { name: "altering-the-billing-record", type: "OR", children: [ { name: "writing-to-the-store-directly", type: "AND", children: [ { name: "access-to-the-data-zone", type: "OR", children: [ { name: "access-from-the-management-network", precondition: "internal-network-access", cost: 8, countermeasure: 4 }, { name: "access-via-the-backup-channel", precondition: "backup-endpoint-open", cost: 12, countermeasure: 3 }] }, { name: "store-write-authority", type: "OR", children: [ { name: "obtaining-a-service-identity", precondition: "identity-store-readable", cost: 15, countermeasure: 6 }, { name: "changing-a-role-assignment", precondition: "management-authority", cost: 20, countermeasure: 5 }] }] }, { name: "via-billing", type: "AND", children: [ { name: "a-valid-subscriber-session", type: "OR", children: [ { name: "obtaining-subscriber-credentials", precondition: "portal-externally-exposed", cost: 6, countermeasure: 7 }, { name: "session-hijacking", precondition: "token-visible", cost: 9, countermeasure: 5 }] }, { name: "bypassing-the-authorization-check", type: "OR", children: [ { name: "changing-the-object-id", precondition: "authorization-decision-on-client", cost: 3, countermeasure: 2 }, { name: "changing-the-role-claim", precondition: "role-from-client", cost: 5, countermeasure: 2 }] }] }, { name: "altering-the-validator-output", type: "AND", children: [ { name: "entering-the-measurement-flow", type: "OR", children: [ { name: "meter-identity-spoofing", precondition: "meter-identity-unverified", cost: 10, countermeasure: 9 }, { name: "access-to-the-collection-endpoint", precondition: "collection-endpoint-externally-exposed", cost: 14, countermeasure: 4 }] }, { name: "bypassing-the-validation-rule", type: "OR", children: [ { name: "changing-a-setting", precondition: "management-flow-unverified", cost: 11, countermeasure: 3 }, { name: "exceeding-the-threshold-value", precondition: "threshold-fixed", cost: 7, countermeasure: 2 }] }] }] }; export const LEAVES = (d = TREE) => d.children ? d.children.flatMap((c) => LEAVES(c)) : [d]; // TM18: an OR node's cost is the smallest of its children's, an AND node's cost is their sum. // TM19: a closed leaf's cost is infinite; every path running through a closed leaf is closed. export function cost(d = TREE, closed = []) { if (!d.children) return closed.includes(d.name) ? Infinity : d.cost; const c = d.children.map((x) => cost(x, closed)); return d.type === "OR" ? Math.min(...c) : c.reduce((a, b) => a + b, 0); } // TM20: the cheapest path is the leaf set obtained by picking the cheapest child at every OR node. export function cheapestPath(d = TREE, closed = []) { if (!d.children) return cost(d, closed) === Infinity ? [] : [d.name]; if (d.type === "OR") { const best = d.children.reduce((a, b) => cost(a, closed) <= cost(b, closed) ? a : b); return cost(best, closed) === Infinity ? [] : cheapestPath(best, closed); } return d.children.flatMap((c) => cheapestPath(c, closed)); } ``` ```js // threat/closing.mjs — the cheapest path's cost and by what factor closing a node // makes the root more expensive. import { TREE, LEAVES, cost, cheapestPath } from "./tree.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const format = (x) => (x === Infinity ? "closed" : String(x)); const ratio = (x) => (x === Infinity ? "closed" : (x / BASE).toFixed(2)); const leaves = LEAVES(); const BASE = cost(); console.log(`${leaves.length} leaves, ${TREE.children.length} sub-goals; root cost ` + `${BASE} units, cheapest path ${cheapestPath().join(" + ")}`); const A = [-32, 6, 6, 8, 15]; console.log("\n1. sub-goals and their costs"); write(A, "sub-goal", "type", "cost", "leaves", "cheapest path"); for (const c of TREE.children) write(A, c.name, c.type, cost(c), LEAVES(c).length, cheapestPath(c).length + " steps"); const B = [-37, -41, 6, 16, 10, 7, 22]; console.log("\n2. closing each leaf one at a time"); write(B, "leaf", "precondition", "cost", "countermeasure", "new root", "ratio", "ratio/countermeasure"); const measure = leaves.map((y) => ({ y, root: cost(TREE, [y.name]) })) .sort((a, b) => b.root - a.root || a.y.countermeasure - b.y.countermeasure); for (const { y, root } of measure) write(B, y.name, y.precondition, y.cost, y.countermeasure, root, ratio(root), ((root - BASE) / y.countermeasure).toFixed(2)); const C = [-35, 16, 7, 10, 12, 13]; console.log("\n3. greedy closing: the leaf that raises the root most each round"); write(C, "closed leaf", "countermeasure", "spent", "new root", "step ratio", "total ratio"); const closed = []; let spent = 0; while (cost(TREE, closed) !== Infinity) { const candidate = leaves.filter((y) => !closed.includes(y.name)) .map((y) => ({ y, root: cost(TREE, [...closed, y.name]) })) .sort((a, b) => b.root - a.root || a.y.countermeasure - b.y.countermeasure)[0]; const previous = cost(TREE, closed); closed.push(candidate.y.name); spent += candidate.y.countermeasure; write(C, candidate.y.name, candidate.y.countermeasure, spent, format(candidate.root), candidate.root === Infinity ? "closed" : (candidate.root / previous).toFixed(2), ratio(candidate.root)); } console.log(`closing the root completely calls for ${closed.length} leaves and ${spent} ` + `countermeasure units; the remaining ${leaves.length - closed.length} leaves sit on no open path`); // Closing every leaf of an OR node closes that node and eliminates the AND branch containing it. const orNodes = (d = TREE) => (d.children ? [...(d.type === "OR" && d !== TREE ? [d] : []), ...d.children.flatMap((c) => orNodes(c))] : []); const D = [-36, 8, 16, 10, 7, 10]; console.log("\n4. closing an OR node in full"); write(D, "OR node", "leaves", "countermeasure", "new root", "ratio", "per unit"); for (const d of orNodes()) { const ls = LEAVES(d), o = ls.reduce((a, y) => a + y.countermeasure, 0); const root = cost(TREE, ls.map((y) => y.name)); write(D, d.name, ls.length, o, format(root), ratio(root), root === Infinity ? "-" : ((root - BASE) / o).toFixed(2)); } const path = leaves.filter((y) => cheapestPath().includes(y.name)); console.log(`comparison: closing the ${path.length} leaves of the cheapest path spends ` + `${path.reduce((a, y) => a + y.countermeasure, 0)} countermeasure units and raises the root ` + `to ${cost(TREE, path.map((y) => y.name))} units`); // The cheapest way to close the root: pick one OR node from each AND branch, try every // combination, and take the one with the smallest total countermeasure. const branchOptions = TREE.children.map((branch) => orNodes(branch) .map((d) => ({ name: d.name, leaves: LEAVES(d) }))); let cheapestClosure = null; const walk = (i, choice) => { if (i === branchOptions.length) { const ls = choice.flatMap((s) => s.leaves), o = ls.reduce((a, y) => a + y.countermeasure, 0); if (cost(TREE, ls.map((y) => y.name)) === Infinity && (!cheapestClosure || o < cheapestClosure.o)) cheapestClosure = { choice, o, n: ls.length }; return; } for (const s of branchOptions[i]) walk(i + 1, [...choice, s]); }; walk(0, []); console.log(`\nthe cheapest full closure is ${cheapestClosure.n} leaves and ${cheapestClosure.o} ` + `countermeasure units (${cheapestClosure.choice.map((s) => s.name).join(", ")}); the greedy ` + `order gives the same result for ${spent} units — ${(spent / cheapestClosure.o).toFixed(2)} ` + `times as expensive`); ``` ``` 12 leaves, 3 sub-goals; root cost 9 units, cheapest path obtaining-subscriber-credentials + changing-the-object-id 1. sub-goals and their costs sub-goal type cost leaves cheapest path writing-to-the-store-directly AND 23 4 2 steps via-billing AND 9 4 2 steps altering-the-validator-output AND 17 4 2 steps 2. closing each leaf one at a time leaf precondition cost countermeasure new root ratio ratio/countermeasure obtaining-subscriber-credentials portal-externally-exposed 6 7 12 1.33 0.43 changing-the-object-id authorization-decision-on-client 3 2 11 1.22 1.00 changing-the-role-claim role-from-client 5 2 9 1.00 0.00 exceeding-the-threshold-value threshold-fixed 7 2 9 1.00 0.00 access-via-the-backup-channel backup-endpoint-open 12 3 9 1.00 0.00 changing-a-setting management-flow-unverified 11 3 9 1.00 0.00 access-from-the-management-network internal-network-access 8 4 9 1.00 0.00 access-to-the-collection-endpoint collection-endpoint-externally-exposed 14 4 9 1.00 0.00 changing-a-role-assignment management-authority 20 5 9 1.00 0.00 session-hijacking token-visible 9 5 9 1.00 0.00 obtaining-a-service-identity identity-store-readable 15 6 9 1.00 0.00 meter-identity-spoofing meter-identity-unverified 10 9 9 1.00 0.00 3. greedy closing: the leaf that raises the root most each round closed leaf countermeasure spent new root step ratio total ratio obtaining-subscriber-credentials 7 7 12 1.33 1.33 session-hijacking 5 12 17 1.42 1.89 exceeding-the-threshold-value 2 14 21 1.24 2.33 changing-a-setting 3 17 23 1.10 2.56 obtaining-a-service-identity 6 23 28 1.22 3.11 changing-a-role-assignment 5 28 closed closed closed closing the root completely calls for 6 leaves and 28 countermeasure units; the remaining 6 leaves sit on no open path 4. closing an OR node in full OR node leaves countermeasure new root ratio per unit access-to-the-data-zone 2 7 9 1.00 0.00 store-write-authority 2 11 9 1.00 0.00 a-valid-subscriber-session 2 12 17 1.89 0.67 bypassing-the-authorization-check 2 4 17 1.89 2.00 entering-the-measurement-flow 2 13 9 1.00 0.00 bypassing-the-validation-rule 2 5 9 1.00 0.00 comparison: closing the 2 leaves of the cheapest path spends 9 countermeasure units and raises the root to 14 units the cheapest full closure is 6 leaves and 16 countermeasure units (access-to-the-data-zone, bypassing-the-authorization-check, bypassing-the-validation-rule); the greedy order gives the same result for 28 units — 1.75 times as expensive ``` The numbers are in the **measurement** class; their inputs are TM17–TM20. #### Nine Units The root cost is **9 units** and the cheapest path consists of two steps. This single number has nothing to do with the twelve leaves' combined cost of 120 units: the tree is not a sum, it is a **minimum choice**. The first table splits this into three branches. Writing directly to the store is 23 units, altering the validator output is 17 units, going via billing is **9 units**. All three branches have four leaves and a two-step cheapest path; what produces the difference is not the number of leaves but the leaves' costs. Because the root is an OR node, it takes the smallest of the three — that is, **the weakest branch decides the tree's cost**. This directly shows a costing mistake commonly made on the defense side. Effort spent making direct writes to the store harder never changes the root cost, because that branch is already the most expensive one and the attacking side will not choose it. **The tree is a structure that writes down which work goes to waste.** #### Ten of the Twelve Leaves Change Nothing The second table closes each leaf one at a time. The result is harsh: **for ten of the twelve leaves the root cost stays at 9**, a ratio of 1.00. Only two leaves change anything — closing `obtaining-subscriber-credentials` raises the root to 12 (1.33x), closing `changing-the-object-id` raises it to 11 (1.22x). Both sit on the cheapest path; the remaining ten do not. The last column gives the return per countermeasure unit and reverses the ranking. `obtaining-subscriber-credentials` costs 7 countermeasure units for a 3-unit increase: 0.43 per unit. `changing-the-object-id` costs 2 countermeasure units for a 2-unit increase: **1.00** per unit. The closure with the smaller ratio is two and a half times more efficient per unit spent. The most expensive countermeasures are the least efficient. `meter-identity-spoofing` demands 9 countermeasure units and changes the root cost not at all; `obtaining-a-service-identity` demands 6 units and again changes nothing. In a threat-modeling output, these two rows would probably look "high priority" — both correspond to a real vulnerability. Measured against the tree, both contribute zero to the root cost, **because both already sit on the most expensive branch**. #### Closing an OR Node The fourth table does the closing at the node level instead of the leaf level. Closing all of an OR node's leaves closes that node and completely eliminates the AND branch containing it. Of six OR nodes, only two change the root cost, and both are in the billing branch: `a-valid-subscriber-session` (12 countermeasure units) and `bypassing-the-authorization-check` (**4 countermeasure units**). Both raise the root to 17 — **1.89x**. For the same result, one costs three times what the other does; the return per unit is 0.67 against **2.00**. The comparison row delivers the real decision. Closing the two leaves of the cheapest path spends 9 countermeasure units and raises the root to 14. Closing the two leaves of the `bypassing-the-authorization-check` node spends **4 units** and raises the root to 17. The choice that spends less pays off more, because the cheapest path's leaves are spread across two separate OR nodes and each one has a sibling leaf ready to replace it. **Closing a leaf shifts the path; closing an OR node eliminates the branch.** #### Closing the Root Completely The third table runs the greedy order: each round closes the leaf that raises the root the most. Six leaves and **28 countermeasure units** later the root closes; the remaining six leaves sit on no open path. Intermediate values run 12, 17, 21, 23, 28, but the step ratios do not decrease evenly: 1.33 · 1.42 · 1.24 · 1.10 · 1.22. One closure's return depends not on the previous one but on the structure of the open paths left remaining. The last line measures this order's inefficiency. A full closure done by picking one OR node from each AND branch also closes six leaves, but spends only **16 countermeasure units**. The greedy order delivers the same result for **1.75 times the price**. The reason is in table 3's first row. The greedy choice starts with `obtaining-subscriber-credentials` because it raises the root the most — but that leaf's countermeasure price is 7, and closing the same branch via `bypassing-the-authorization-check` would have cost 4 units. **Choosing the highest-payoff move each round does not give the choice with the smallest total price**; in a tree, choices' costs affect each other, and item-by-item ranking cannot see that. #### Summary - The number measured in an attack tree is not the leaf count but **the root cost**: in a tree of 12 leaves and three sub-goals, the root is 9 fictional units and the cheapest path is two steps. The root is an OR node, meaning its cost is decided by the weakest branch. - **Closing ten of the twelve leaves one at a time never changes the root cost**; only the two leaves on the cheapest path yield 1.33 and 1.22x. - Return per countermeasure unit reverses the ranking: the closure that yields 1.33x returns 0.43 per unit, the one that yields 1.22x returns 1.00 per unit. - Closing an OR node in full eliminates the branch; a node worth 4 countermeasure units raises the root to 1.89x, while the two leaves of the cheapest path spend 9 units for only 1.56x. - Closing the root completely costs 28 units in greedy order, **16** units by picking one OR node per branch; the highest-payoff move each round is 1.75 times more expensive in total. #### Next Step Across five lessons this topic produced a threat list and a priority order: 23 boundary crossings, 58 items, 108 systematic threats, 12 chain threats, and a handful of closures that actually change the root cost. All these numbers sit **inside the model**. As long as they sit inside the model, nothing changes. A countermeasure only turns into code once someone takes ownership of it, ties it to an acceptance criterion, and writes it into a release; if any one of these is missing, the work item never opens and the item quietly disappears. The next lesson turns the model's output into work items and takes the final measurement: how many threats land on how many separate countermeasures, how many of those are actually planned, how many disappear due to which gap, and how many items of the remaining residual risk are **accepted in writing**. ### Using Model Output Source: https://academia.sh/en/courses/secure-lifecycle/using-model-output Measures the model output's conversion into work items: 120 threats fall into 24 countermeasures, only 11 of which carry all three of owner, acceptance criterion, and release target and turn into work items, closing 73 threats (60.8%); the 13 lost items carry 47 threats and 97 effort-days, 4 of which are accepted in writing and 9 of which remain silent residual risk carrying 33 threats on their own. The previous five lessons produced a threat list and a priority order: 23 boundary crossings, 58 items, 108 systematic threats, 12 chain threats, and a handful of closures that actually change the root cost. All these numbers sit inside the model. As long as they sit inside the model, nothing changes. A countermeasure only turns into code once someone takes ownership of it, ties it to an acceptance criterion, and writes it into a release. This lesson takes the final measurement: how many threats fall into how many separate countermeasures, how many of those are actually planned, how many are lost to which gap, and how many items of the remaining residual risk are accepted in writing. #### One Hundred Twenty Threats, Twenty-Four Countermeasures The list below is a **model**, not a real record. Countermeasures are identified only by the number of threats they cover, their effort-days, and whether three fields are filled in. **TM21:** a single countermeasure closes more than one threat at once; the union set of the previous two lessons was 120 threats and this column's total is verified against it. **TM22:** a countermeasure only turns into a work item if all three fields are filled — **owner**, **acceptance criterion**, and **release target**; if even one is missing, the item is lost. **TM23:** a lost item is on record if the residual risk it carries has written acceptance, otherwise it is silent. ```js // threat/countermeasure.mjs — model output turned into work items (model): the countermeasure // list from the fictional measurement-network software's threat-modeling session. Not a real // record; carries no attack step. // TM21: a single countermeasure closes more than one threat at once; `threats` is that count. // The union set of the previous two lessons was 120 threats and this column's total is // verified against it. // TM22: a countermeasure only turns into a work item if all three fields are filled: `owner`, // acceptance `criterion`, and `release` target. If even one is missing, the item is lost. // TM23: a lost item is on record if it has written `acceptance` of its residual risk, // otherwise it is silent. // The `tree` column is whether that countermeasure changes the root cost in the attack tree. export const COUNTERMEASURE = ` boundary-crossing-input-validation 17 8 Y Y Y - N boundary-crossing-source-identity 17 10 Y Y Y - N authorization-on-store-writes 9 5 Y Y Y - N transport-protection-for-confidential-class 9 4 Y Y Y - N filtering-on-outbound-data 6 6 Y N Y N N moving-the-authorization-decision-to-the-server 5 9 Y Y Y - Y not-taking-the-role-claim-from-the-client 4 3 Y Y Y - Y hiding-the-session-token 4 5 Y Y N Y Y verifying-meter-identity 4 12 Y N N Y N narrowing-the-collection-endpoint 4 6 Y Y N N N signing-the-validator-setting 3 7 N N N N N binding-the-threshold-value-to-the-data 3 4 Y Y Y - N mandatory-audit-log-write 4 6 Y Y N Y N keeping-the-log-immutable 3 8 N N N N N closing-the-backup-endpoint 2 2 Y Y Y - N segmenting-the-management-network 4 11 N Y N N N rotating-the-service-identity 3 7 Y N N N N tying-role-assignment-to-approval 2 5 Y Y Y - N mandatory-identity-on-the-report-endpoint 3 3 Y Y Y - N rate-limiting-on-the-subscriber-portal 2 2 Y Y Y - Y integrity-tag-on-the-measurement-flow 4 9 N Y N N N re-verification-at-chain-steps 4 10 N N N N N not-storing-secrets-in-the-field-app 2 4 Y Y N Y N capacity-protection-on-the-collection-endpoint 2 6 Y N N N N` .trim().split("\n").map((r) => { const [name, threats, effort, owner, criterion, release, accepted, tree] = r.trim().split(/\s+/); return { name, threats: +threats, effort: +effort, owner: owner === "Y", criterion: criterion === "Y", release: release === "Y", accepted: accepted === "Y", tree: tree === "Y" }; }); export const FIELDS = ["owner", "criterion", "release"]; export const isWorkItem = (o) => FIELDS.every((a) => o[a]); export const THREATS = COUNTERMEASURE.reduce((a, o) => a + o.threats, 0); ``` ```js // threat/workitem.mjs — how many countermeasures actually get planned, how many are lost, // how many items of residual risk are accepted in writing. import { COUNTERMEASURE, FIELDS, isWorkItem, THREATS } from "./countermeasure.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const sum = (ls, f) => ls.reduce((a, o) => a + f(o), 0); const delivered = COUNTERMEASURE.filter(isWorkItem), lost = COUNTERMEASURE.filter((o) => !isWorkItem(o)); console.log(`${THREATS} threats fall into ${COUNTERMEASURE.length} countermeasures (` + `${(THREATS / COUNTERMEASURE.length).toFixed(2)} threats per countermeasure); delivered ` + `as work items ${delivered.length}, lost ${lost.length}`); const A = [-11, 8, 9, 17, 16]; console.log("\n1. checking whether the three fields are filled"); write(A, "field", "filled", "missing", "missing threats", "missing effort"); for (const a of FIELDS) { const e = COUNTERMEASURE.filter((o) => !o[a]); write(A, a, COUNTERMEASURE.length - e.length, e.length, sum(e, (o) => o.threats), sum(e, (o) => o.effort)); } const B = [-11, 17, 18, 9, 10, 13]; console.log("\n2. model output turning into work items"); write(B, "set", "countermeasures", "countermeasure %", "threats", "threat %", "effort-days"); for (const [name, ls] of [["delivered", delivered], ["lost", lost], ["total", COUNTERMEASURE]]) write(B, name, ls.length, ((ls.length / COUNTERMEASURE.length) * 100).toFixed(1), sum(ls, (o) => o.threats), ((sum(ls, (o) => o.threats) / THREATS) * 100).toFixed(1), sum(ls, (o) => o.effort)); const C = [-27, 17, 9, 13]; console.log("\n3. which missing field loses which items"); write(C, "missing-field combination", "countermeasures", "threats", "effort-days"); const combo = {}; for (const o of lost) { const e = FIELDS.filter((a) => !o[a]).join("+"); (combo[e] ||= []).push(o); } for (const [e, ls] of Object.entries(combo).sort((x, y) => y[1].length - x[1].length)) write(C, e, ls.length, sum(ls, (o) => o.threats), sum(ls, (o) => o.effort)); const D = [-23, 17, 9, 10, 24]; console.log("\n4. the residual-risk counterpart of lost items"); write(D, "set", "countermeasures", "threats", "threat %", "threats/countermeasure"); for (const [name, ls] of [["accepted in writing", lost.filter((o) => o.accepted)], ["silent residual risk", lost.filter((o) => !o.accepted)]]) write(D, name, ls.length, sum(ls, (o) => o.threats), ((sum(ls, (o) => o.threats) / THREATS) * 100).toFixed(1), (sum(ls, (o) => o.threats) / ls.length).toFixed(2)); const treed = COUNTERMEASURE.filter((o) => o.tree); console.log(`of the ${treed.length} countermeasures that change the root cost in the attack ` + `tree, ${treed.filter(isWorkItem).length} turned into work items and ` + `${treed.filter((o) => !isWorkItem(o)).length} was lost`); const biggest = [...lost].sort((a, b) => b.threats - a.threats)[0]; console.log(`the lost item carrying the most threats is ${biggest.name} (${biggest.threats} ` + `threats, ${biggest.effort} effort-days); the ${lost.length} lost items average ` + `${(sum(lost, (o) => o.effort) / lost.length).toFixed(1)} days of effort, the ` + `${delivered.length} delivered items ${(sum(delivered, (o) => o.effort) / delivered.length).toFixed(1)} days`); ``` ``` 120 threats fall into 24 countermeasures (5.00 threats per countermeasure); delivered as work items 11, lost 13 1. checking whether the three fields are filled field filled missing missing threats missing effort owner 19 5 18 45 criterion 17 7 25 56 release 12 12 41 91 2. model output turning into work items set countermeasures countermeasure % threats threat % effort-days delivered 11 45.8 73 60.8 55 lost 13 54.2 47 39.2 97 total 24 100.0 120 100.0 152 3. which missing field loses which items missing-field combination countermeasures threats effort-days release 4 14 21 criterion+release 3 9 25 owner+criterion+release 3 10 25 owner+release 2 8 20 criterion 1 6 6 4. the residual-risk counterpart of lost items set countermeasures threats threat % threats/countermeasure accepted in writing 4 14 11.7 3.50 silent residual risk 9 33 27.5 3.67 of the 4 countermeasures that change the root cost in the attack tree, 3 turned into work items and 1 was lost the lost item carrying the most threats is filtering-on-outbound-data (6 threats, 6 effort-days); the 13 lost items average 7.5 days of effort, the 11 delivered items 5.0 days ``` The numbers fall in the **measurement** class; their inputs are TM21–TM23. #### Five-to-One Consolidation The first number is the merge ratio. One hundred twenty threats fall into **24 countermeasures** — 5.00 threats per countermeasure. This shows that the threat count alone is not a measure of workload: `120 threats` sounds like a two-month pile, but the corresponding work item count is twenty-four. The merge does not distribute evenly. The first two countermeasures — boundary-crossing input validation and source identity — close **34 threats** on their own, which is 28.3% of the list. The reason for this was already visible in the second lesson: 58.6% of the items came from flows running from a lower trust level to a higher one, and all of those flows wanted the same two controls. The question being answered is not "how many threats are there" but "**how many separate decisions are there**." #### The Item With No Release Target The second table is the lesson's core measurement. Of the twenty-four countermeasures, only **11** carry all three fields and turn into a work item; 13 are lost. By item count, the conversion rate is 45.8%. Looked at by threat count, the table looks better: the 11 delivered items close **73 threats**, or 60.8%. The gap comes from the merge ratio — delivered items carry more threats on average, because the two most crowded countermeasures are also among the delivered ones. The first table states directly which field is the bottleneck. The owner field is filled in 19 of 24 items, the acceptance criterion in 17, **the release target in only 12**. The 12 items missing a release target carry 41 threats and 91 effort-days. Finding an owner is easy; writing a criterion is a little harder; getting an item into a release is the hardest of all, because that is where it runs into other work. The third table reaches the same conclusion at the item level. Of the 13 lost items, **4 are lost solely because the release target is missing** — they have an owner, their criterion is written, the only thing missing is a calendar slot. These four items carry 14 threats and total 21 effort-days. There is exactly one item missing only its acceptance criterion. The three items with all three fields empty are items that were never taken up by anyone during the session. #### Where a Lost Item Goes The last line gives the direction of the loss, and it says the opposite of what you'd expect. The 13 lost items average **7.5 days** of effort, the 11 items delivered as work items average **5.0 days**. In other words, expensive items get lost more often. This is explained by the order in which the three fields get filled. For an item to get a release target, someone has to put it into a sprint; as the effort-days grow, that decision gets harder and the item slips to "the next release." Once a slip happens, the item does not show up as a zero anywhere in the table — it is not in any release. The intersection with the tree measurement shows up in a single line. Of the **4 countermeasures** that actually change the root cost in the attack tree, **3** turned into work items and one was lost. That is a good ratio, but it is a coincidence: those four items' effort-days are 9, 3, 5, and 2 — that is, the cheap end of the list. If those same items had been expensive, the same process would have filtered them out too. **The criterion the process actually selects on is not impact, it is effort.** #### Silent Residual Risk The fourth table makes the course's final distinction. Of the 13 lost items, **4 are accepted in writing**: the countermeasure will not be implemented, the residual risk it carries is known, and it is put on record. These four items carry 14 threats, or 11.7% of the total. The remaining **9 items are silent residual risk**. They have neither turned into a work item nor been written down anywhere as something that will not be done. The **33 threats** they carry are 27.5% of the total set. The number of threats carried per item is nearly identical between the two sets (3.50 and 3.67); so the silent ones are not smaller items, they are just items **no decision has been made about**. The difference between the two is decisive from a measurement standpoint. Residual risk accepted in writing is a decision: it carries an item, an owner, a rationale, and a date; it can be revisited in a later session. Silent residual risk is not a decision; it is a gap no one knows about, and the next threat-modeling session has to reproduce it from scratch. **A threat-modeling session's output is the sum of the items that turn into work items and the items rejected in writing; everything else counts as never produced.** #### Summary - 120 threats fall into 24 countermeasures (5.00 per countermeasure); the two most crowded countermeasures close 34 threats on their own. The measure of workload is not the threat count but **the number of separate decisions**. - Only 11 of the twenty-four countermeasures carry all three of owner, acceptance criterion, and release target; the conversion rate is 45.8% by item count and 60.8% by threat count. - The bottleneck is the release target: it is missing on 12 items and they carry 41 threats and 91 effort-days. Four of the 13 lost items are lost **solely** because the release target is missing. - Lost items average 7.5 days of effort, delivered ones 5.0 days; the process filters **by effort**, not by impact. Three of the four countermeasures that change the root cost were delivered, but that is because they were cheap. - 4 of the 13 lost items are accepted in writing (14 threats), **9 are silent residual risk** and carry 27.5% of the total set; the number of threats they carry per item is the same order of magnitude as the delivered items. #### Course Wrap-Up | Lesson | What stage the item is born in | What stage it is caught in | The cost of catching it | |---|---|---|---| | The Difference Between DevSecOps and DevOps | 23 items across six stages, most in code (7) | 10 at a single gate, 23 with distributed checks; lag 1.94→0.13 stages | 177 of 200 alerts are false positives | | Shift Left | 18 shared items, from requirements to deployment | late flow 318 hours, early flow 68 hours; 4.68x | 32.5 hours of review overhead, net gain 217.5 hours | | Security Requirements | requirements stage, 28 requirements via scenarios | 19 automated in code, 5 by manual review at deployment, 4 never | 40 hours of manual testing across 300 checks | | Security Champions | code stage, 35 reviews per week | 16 centralized, 35 with four champions | 2.8 person-equivalents; deep-dive items from 2 to 1 | | Risk-Based Prioritization | a backlog of 23 items, 68 days of effort | in 22 days, birth order clears 6 items, effort order clears 13 | 5 items excluded under every criterion, 110 hours | | The Purpose of Threat Modeling | 18 items, 10 of them at requirements and design | the session catches 10 items at design, 8 items are 1–3 stages late | 40 developer-hours, 4.0 hours per item | | Data Flow Diagrams | design, 23 of 26 flows cross a boundary | 58 items at design; only 15 in a rough sketch | 16 crossings stay invisible, 43 items never get born | | Threat Classification Methods | design, 43 diagram elements | 108 meaningful out of 156 raw candidates, at design | 666 minutes; 72 eliminated as invalid | | Risk-Focused Modeling | design, 12 chain threats on a path | 51 threats; 63.9% of the systematic set is missed | 218 minutes; 24 denial-of-service items excluded | | Attack Trees | design, 12 leaves, root 9 units | 10 leaves never change the root, 2 give 1.33x and 1.22x | full closure 16 countermeasure units, greedy order 28 | | Using Model Output | design, 120 threats, 24 countermeasures | 11 work items, 73 threats (60.8%) | 13 items lost, 9 silent (33 threats) | The rule holds across all eleven rows: **security is not a stage, it is a decision with a different price at every stage.** The gain from catching things early is countable — 4.68x, 217.5 hours, 866 hours — but the same measurement produced a second number every time: there is a class of item that cannot be seen at design time, and that class is not small. The 7 items shift-left never touches, the 8 items the design session did not find, and the 12 chain threats systematic production cannot see all say the same thing. **A review whose cost is not written down counts as unmeasured; a method whose coverage limit is not written down is no different.** The last lesson marked a threshold. It recorded which threat was born at design, counted which countermeasure was planned, and measured which ones were lost. But **where the 11 planned countermeasures turn into code** was never seen: how many points in the codebase a "boundary-crossing input validation" item corresponds to, how many of those points actually close, and how many stay entirely outside this decision. **M25/K02 Secure Coding** starts with this question — the number of points a defensive pattern closes on its own, against the number of points no one ever looks at.