Using Model Output

The previous five lessons produced a threat list and a priority order: 23 boundary crossings, 58 items, 108 systematic threats, 12 chain threats, and a handful of closures that actually change the root cost. All these numbers sit inside the model.

As long as they sit inside the model, nothing changes. A countermeasure only turns into code once someone takes ownership of it, ties it to an acceptance criterion, and writes it into a release. This lesson takes the final measurement: how many threats fall into how many separate countermeasures, how many of those are actually planned, how many are lost to which gap, and how many items of the remaining residual risk are accepted in writing.

One Hundred Twenty Threats, Twenty-Four Countermeasures

The list below is a model, not a real record. Countermeasures are identified only by the number of threats they cover, their effort-days, and whether three fields are filled in.

TM21: a single countermeasure closes more than one threat at once; the union set of the previous two lessons was 120 threats and this column’s total is verified against it. TM22: a countermeasure only turns into a work item if all three fields are filled — owner, acceptance criterion, and release target; if even one is missing, the item is lost. TM23: a lost item is on record if the residual risk it carries has written acceptance, otherwise it is silent.

// threat/countermeasure.mjs — model output turned into work items (model): the countermeasure
// list from the fictional measurement-network software's threat-modeling session. Not a real
// record; carries no attack step.

// TM21: a single countermeasure closes more than one threat at once; `threats` is that count.
// The union set of the previous two lessons was 120 threats and this column's total is
// verified against it.
// TM22: a countermeasure only turns into a work item if all three fields are filled: `owner`,
// acceptance `criterion`, and `release` target. If even one is missing, the item is lost.
// TM23: a lost item is on record if it has written `acceptance` of its residual risk,
// otherwise it is silent.
// The `tree` column is whether that countermeasure changes the root cost in the attack tree.
export const COUNTERMEASURE = `
  boundary-crossing-input-validation               17  8  Y Y Y - N
  boundary-crossing-source-identity                17 10  Y Y Y - N
  authorization-on-store-writes                     9  5  Y Y Y - N
  transport-protection-for-confidential-class       9  4  Y Y Y - N
  filtering-on-outbound-data                        6  6  Y N Y N N
  moving-the-authorization-decision-to-the-server   5  9  Y Y Y - Y
  not-taking-the-role-claim-from-the-client         4  3  Y Y Y - Y
  hiding-the-session-token                          4  5  Y Y N Y Y
  verifying-meter-identity                          4 12  Y N N Y N
  narrowing-the-collection-endpoint                 4  6  Y Y N N N
  signing-the-validator-setting                     3  7  N N N N N
  binding-the-threshold-value-to-the-data           3  4  Y Y Y - N
  mandatory-audit-log-write                         4  6  Y Y N Y N
  keeping-the-log-immutable                         3  8  N N N N N
  closing-the-backup-endpoint                       2  2  Y Y Y - N
  segmenting-the-management-network                 4 11  N Y N N N
  rotating-the-service-identity                     3  7  Y N N N N
  tying-role-assignment-to-approval                 2  5  Y Y Y - N
  mandatory-identity-on-the-report-endpoint         3  3  Y Y Y - N
  rate-limiting-on-the-subscriber-portal            2  2  Y Y Y - Y
  integrity-tag-on-the-measurement-flow             4  9  N Y N N N
  re-verification-at-chain-steps                    4 10  N N N N N
  not-storing-secrets-in-the-field-app              2  4  Y Y N Y N
  capacity-protection-on-the-collection-endpoint    2  6  Y N N N N`
  .trim().split("\n").map((r) => {
    const [name, threats, effort, owner, criterion, release, accepted, tree] = r.trim().split(/\s+/);
    return { name, threats: +threats, effort: +effort, owner: owner === "Y", criterion: criterion === "Y",
      release: release === "Y", accepted: accepted === "Y", tree: tree === "Y" };
  });

export const FIELDS = ["owner", "criterion", "release"];
export const isWorkItem = (o) => FIELDS.every((a) => o[a]);
export const THREATS = COUNTERMEASURE.reduce((a, o) => a + o.threats, 0);

// threat/workitem.mjs — how many countermeasures actually get planned, how many are lost,
// how many items of residual risk are accepted in writing.
import { COUNTERMEASURE, FIELDS, isWorkItem, THREATS } from "./countermeasure.mjs";

const write = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const sum = (ls, f) => ls.reduce((a, o) => a + f(o), 0);
const delivered = COUNTERMEASURE.filter(isWorkItem), lost = COUNTERMEASURE.filter((o) => !isWorkItem(o));

console.log(`${THREATS} threats fall into ${COUNTERMEASURE.length} countermeasures (` +
  `${(THREATS / COUNTERMEASURE.length).toFixed(2)} threats per countermeasure); delivered ` +
  `as work items ${delivered.length}, lost ${lost.length}`);

const A = [-11, 8, 9, 17, 16];
console.log("\n1. checking whether the three fields are filled");
write(A, "field", "filled", "missing", "missing threats", "missing effort");
for (const a of FIELDS) {
  const e = COUNTERMEASURE.filter((o) => !o[a]);
  write(A, a, COUNTERMEASURE.length - e.length, e.length, sum(e, (o) => o.threats),
    sum(e, (o) => o.effort));
}

const B = [-11, 17, 18, 9, 10, 13];
console.log("\n2. model output turning into work items");
write(B, "set", "countermeasures", "countermeasure %", "threats", "threat %", "effort-days");
for (const [name, ls] of [["delivered", delivered], ["lost", lost], ["total", COUNTERMEASURE]])
  write(B, name, ls.length, ((ls.length / COUNTERMEASURE.length) * 100).toFixed(1),
    sum(ls, (o) => o.threats), ((sum(ls, (o) => o.threats) / THREATS) * 100).toFixed(1),
    sum(ls, (o) => o.effort));

const C = [-27, 17, 9, 13];
console.log("\n3. which missing field loses which items");
write(C, "missing-field combination", "countermeasures", "threats", "effort-days");
const combo = {};
for (const o of lost) {
  const e = FIELDS.filter((a) => !o[a]).join("+");
  (combo[e] ||= []).push(o);
}
for (const [e, ls] of Object.entries(combo).sort((x, y) => y[1].length - x[1].length))
  write(C, e, ls.length, sum(ls, (o) => o.threats), sum(ls, (o) => o.effort));

const D = [-23, 17, 9, 10, 24];
console.log("\n4. the residual-risk counterpart of lost items");
write(D, "set", "countermeasures", "threats", "threat %", "threats/countermeasure");
for (const [name, ls] of [["accepted in writing", lost.filter((o) => o.accepted)],
  ["silent residual risk", lost.filter((o) => !o.accepted)]])
  write(D, name, ls.length, sum(ls, (o) => o.threats),
    ((sum(ls, (o) => o.threats) / THREATS) * 100).toFixed(1),
    (sum(ls, (o) => o.threats) / ls.length).toFixed(2));

const treed = COUNTERMEASURE.filter((o) => o.tree);
console.log(`of the ${treed.length} countermeasures that change the root cost in the attack ` +
  `tree, ${treed.filter(isWorkItem).length} turned into work items and ` +
  `${treed.filter((o) => !isWorkItem(o)).length} was lost`);
const biggest = [...lost].sort((a, b) => b.threats - a.threats)[0];
console.log(`the lost item carrying the most threats is ${biggest.name} (${biggest.threats} ` +
  `threats, ${biggest.effort} effort-days); the ${lost.length} lost items average ` +
  `${(sum(lost, (o) => o.effort) / lost.length).toFixed(1)} days of effort, the ` +
  `${delivered.length} delivered items ${(sum(delivered, (o) => o.effort) / delivered.length).toFixed(1)} days`);

120 threats fall into 24 countermeasures (5.00 threats per countermeasure); delivered as work items 11, lost 13

1. checking whether the three fields are filled
field        filled  missing  missing threats  missing effort
owner            19        5               18              45
criterion        17        7               25              56
release          12       12               41              91

2. model output turning into work items
set          countermeasures  countermeasure %  threats  threat %  effort-days
delivered                 11              45.8       73      60.8           55
lost                      13              54.2       47      39.2           97
total                     24             100.0      120     100.0          152

3. which missing field loses which items
missing-field combination    countermeasures  threats  effort-days
release                                    4       14           21
criterion+release                          3        9           25
owner+criterion+release                    3       10           25
owner+release                              2        8           20
criterion                                  1        6            6

4. the residual-risk counterpart of lost items
set                      countermeasures  threats  threat %  threats/countermeasure
accepted in writing                    4       14      11.7                    3.50
silent residual risk                   9       33      27.5                    3.67
of the 4 countermeasures that change the root cost in the attack tree, 3 turned into work items and 1 was lost
the lost item carrying the most threats is filtering-on-outbound-data (6 threats, 6 effort-days); the 13 lost items average 7.5 days of effort, the 11 delivered items 5.0 days

The numbers fall in the measurement class; their inputs are TM21–TM23.

Five-to-One Consolidation

The first number is the merge ratio. One hundred twenty threats fall into 24 countermeasures — 5.00 threats per countermeasure. This shows that the threat count alone is not a measure of workload: 120 threats sounds like a two-month pile, but the corresponding work item count is twenty-four.

The merge does not distribute evenly. The first two countermeasures — boundary-crossing input validation and source identity — close 34 threats on their own, which is 28.3% of the list. The reason for this was already visible in the second lesson: 58.6% of the items came from flows running from a lower trust level to a higher one, and all of those flows wanted the same two controls. The question being answered is not “how many threats are there” but “how many separate decisions are there.”

The Item With No Release Target

The second table is the lesson’s core measurement. Of the twenty-four countermeasures, only 11 carry all three fields and turn into a work item; 13 are lost. By item count, the conversion rate is 45.8%.

Looked at by threat count, the table looks better: the 11 delivered items close 73 threats, or 60.8%. The gap comes from the merge ratio — delivered items carry more threats on average, because the two most crowded countermeasures are also among the delivered ones.

The first table states directly which field is the bottleneck. The owner field is filled in 19 of 24 items, the acceptance criterion in 17, the release target in only 12. The 12 items missing a release target carry 41 threats and 91 effort-days. Finding an owner is easy; writing a criterion is a little harder; getting an item into a release is the hardest of all, because that is where it runs into other work.

The third table reaches the same conclusion at the item level. Of the 13 lost items, 4 are lost solely because the release target is missing — they have an owner, their criterion is written, the only thing missing is a calendar slot. These four items carry 14 threats and total 21 effort-days. There is exactly one item missing only its acceptance criterion. The three items with all three fields empty are items that were never taken up by anyone during the session.

Where a Lost Item Goes

The last line gives the direction of the loss, and it says the opposite of what you’d expect. The 13 lost items average 7.5 days of effort, the 11 items delivered as work items average 5.0 days. In other words, expensive items get lost more often.

This is explained by the order in which the three fields get filled. For an item to get a release target, someone has to put it into a sprint; as the effort-days grow, that decision gets harder and the item slips to “the next release.” Once a slip happens, the item does not show up as a zero anywhere in the table — it is not in any release.

The intersection with the tree measurement shows up in a single line. Of the 4 countermeasures that actually change the root cost in the attack tree, 3 turned into work items and one was lost. That is a good ratio, but it is a coincidence: those four items’ effort-days are 9, 3, 5, and 2 — that is, the cheap end of the list. If those same items had been expensive, the same process would have filtered them out too. The criterion the process actually selects on is not impact, it is effort.

Silent Residual Risk

The fourth table makes the course’s final distinction. Of the 13 lost items, 4 are accepted in writing: the countermeasure will not be implemented, the residual risk it carries is known, and it is put on record. These four items carry 14 threats, or 11.7% of the total.

The remaining 9 items are silent residual risk. They have neither turned into a work item nor been written down anywhere as something that will not be done. The 33 threats they carry are 27.5% of the total set. The number of threats carried per item is nearly identical between the two sets (3.50 and 3.67); so the silent ones are not smaller items, they are just items no decision has been made about.

The difference between the two is decisive from a measurement standpoint. Residual risk accepted in writing is a decision: it carries an item, an owner, a rationale, and a date; it can be revisited in a later session. Silent residual risk is not a decision; it is a gap no one knows about, and the next threat-modeling session has to reproduce it from scratch. A threat-modeling session’s output is the sum of the items that turn into work items and the items rejected in writing; everything else counts as never produced.

Summary

• 120 threats fall into 24 countermeasures (5.00 per countermeasure); the two most crowded countermeasures close 34 threats on their own. The measure of workload is not the threat count but the number of separate decisions.
• Only 11 of the twenty-four countermeasures carry all three of owner, acceptance criterion, and release target; the conversion rate is 45.8% by item count and 60.8% by threat count.
• The bottleneck is the release target: it is missing on 12 items and they carry 41 threats and 91 effort-days. Four of the 13 lost items are lost solely because the release target is missing.
• Lost items average 7.5 days of effort, delivered ones 5.0 days; the process filters by effort, not by impact. Three of the four countermeasures that change the root cost were delivered, but that is because they were cheap.
• 4 of the 13 lost items are accepted in writing (14 threats), 9 are silent residual risk and carry 27.5% of the total set; the number of threats they carry per item is the same order of magnitude as the delivered items.

Course Wrap-Up

Lesson	What stage the item is born in	What stage it is caught in	The cost of catching it
The Difference Between DevSecOps and DevOps	23 items across six stages, most in code (7)	10 at a single gate, 23 with distributed checks; lag 1.94→0.13 stages	177 of 200 alerts are false positives
Shift Left	18 shared items, from requirements to deployment	late flow 318 hours, early flow 68 hours; 4.68x	32.5 hours of review overhead, net gain 217.5 hours
Security Requirements	requirements stage, 28 requirements via scenarios	19 automated in code, 5 by manual review at deployment, 4 never	40 hours of manual testing across 300 checks
Security Champions	code stage, 35 reviews per week	16 centralized, 35 with four champions	2.8 person-equivalents; deep-dive items from 2 to 1
Risk-Based Prioritization	a backlog of 23 items, 68 days of effort	in 22 days, birth order clears 6 items, effort order clears 13	5 items excluded under every criterion, 110 hours
The Purpose of Threat Modeling	18 items, 10 of them at requirements and design	the session catches 10 items at design, 8 items are 1–3 stages late	40 developer-hours, 4.0 hours per item
Data Flow Diagrams	design, 23 of 26 flows cross a boundary	58 items at design; only 15 in a rough sketch	16 crossings stay invisible, 43 items never get born
Threat Classification Methods	design, 43 diagram elements	108 meaningful out of 156 raw candidates, at design	666 minutes; 72 eliminated as invalid
Risk-Focused Modeling	design, 12 chain threats on a path	51 threats; 63.9% of the systematic set is missed	218 minutes; 24 denial-of-service items excluded
Attack Trees	design, 12 leaves, root 9 units	10 leaves never change the root, 2 give 1.33x and 1.22x	full closure 16 countermeasure units, greedy order 28
Using Model Output	design, 120 threats, 24 countermeasures	11 work items, 73 threats (60.8%)	13 items lost, 9 silent (33 threats)

The rule holds across all eleven rows: security is not a stage, it is a decision with a different price at every stage. The gain from catching things early is countable — 4.68x, 217.5 hours, 866 hours — but the same measurement produced a second number every time: there is a class of item that cannot be seen at design time, and that class is not small. The 7 items shift-left never touches, the 8 items the design session did not find, and the 12 chain threats systematic production cannot see all say the same thing. A review whose cost is not written down counts as unmeasured; a method whose coverage limit is not written down is no different.

The last lesson marked a threshold. It recorded which threat was born at design, counted which countermeasure was planned, and measured which ones were lost. But where the 11 planned countermeasures turn into code was never seen: how many points in the codebase a “boundary-crossing input validation” item corresponds to, how many of those points actually close, and how many stay entirely outside this decision. M25/K02 Secure Coding starts with this question — the number of points a defensive pattern closes on its own, against the number of points no one ever looks at.