The Purpose of Threat Modeling

The previous topic distributed security work across stages, wrote requirements in testable form, and asked how to divide limited development capacity. The common input to all of those calculations was a ready-made item list: which gap existed was already known, and only the order was under discussion. At the design stage, though, no such list appears on its own.

Threat modeling is the method for producing that list at the design stage. This lesson does not measure the method’s steps but its purpose and price: how many hours a session takes, how many items it finds, how many hours it spends per item, and how many times more expensive the items it finds would have been if caught late. The topic works on the same fictional regional measurement network’s software throughout.

Where the Session Stops

The inventory below is a model: there is no real product, no real team measurement, and no real record. Items are described by only three columns — where they originate, where they can be seen, and how many developer hours their fix takes. No exploit step is written.

TM1: fix work is measured in developer hours; the base hour count is the item’s cost at the design stage, and it grows by a multiplier in later stages. TM2: the stage an item originates in and the stage it becomes visible in are separate things; an item born in code may only become visible at runtime. TM3: the session runs at the design stage and finds only items whose visible stage is no later than design. TM4: the session’s cost is the product of participant count and its duration.

// threat/item.mjs — stage set and item inventory (model): derived from the development records
// of a fictional regional measurement network's software; not a real measurement. No exploit
// step is present; an item is described only by where it originates, where it becomes visible,
// and how many developer hours its fix takes.

export const STAGE = ["requirement", "design", "code", "build", "deploy", "runtime"];

// TM1: fix work is measured in developer hours. `base` is the hour count for fixing the item at
// the design stage; it grows by the multiplier below in later stages.
export const MULTIPLIER = { requirement: 1, design: 1, code: 4, build: 6, deploy: 12,
  "runtime": 30 };

// TM2: rows are written as "name origin visible default base". `origin` is the stage the item is
// produced in, `visible` is the earliest stage it can be seen in, `default` is the stage it is
// actually caught in when there is no threat modeling session.
export const ITEM = `
  trust-boundary-not-drawn           design       design         deploy    6
  meter-identity-not-verified        design       design         runtime   8
  billing-record-modifiable          design       design         deploy   10
  subscriber-data-flows-unprotected  design       design         deploy    4
  authorization-decision-on-client   design       design         code      7
  audit-log-missing                  requirement  design         runtime   5
  field-device-carries-secret        design       design         deploy    9
  reporting-endpoint-unauthenticated design       design         code      3
  measurement-backup-unprotected     design       design         runtime   6
  session-duration-undefined         requirement  design         code      2
  input-length-unbounded             code         code           build     2
  query-concatenated-manually        code         code           build     4
  timing-difference-leaks            code         code           runtime   5
  concurrent-request-race-condition  code         runtime        runtime  12
  dependency-version-unpinned        build        build          deploy    3
  leftover-tool-in-image             build        build          deploy    2
  default-credential-exposed         deploy       deploy         runtime   4
  credential-in-log-entry            code         runtime        runtime   6`
  .trim().split("\n").map((r) => {
    const [name, origin, visible, def, base] = r.trim().split(/\s+/);
    return { name, origin, visible, default: def, base: +base };
  });

export const order = (a) => STAGE.indexOf(a);
export const cost = (k, stage) => k.base * MULTIPLIER[stage];

// TM3: the threat modeling session runs at the design stage and finds only the items whose
// `visible` stage is no later than design; it never touches the rest.
export const isFound = (k) => order(k.visible) <= order("design");

// TM4: the session's cost is participant count x session count x session duration.
export const SESSION = { participants: 5, count: 4, hours: 2 };
export const SESSION_HOURS = SESSION.participants * SESSION.count * SESSION.hours;

// threat/session.mjs — how many hours the session takes, how many items it finds, hours per
// item, and how many times more expensive the same item is when caught late.
import { STAGE, MULTIPLIER, ITEM, SESSION, SESSION_HOURS, order, cost, isFound } from "./item.mjs";

const write = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const sum = (ks, f) => ks.reduce((s, k) => s + f(k), 0);
const found = ITEM.filter(isFound), missed = ITEM.filter((k) => !isFound(k));

console.log(`${ITEM.length} items, ${STAGE.length} stages; multipliers ` +
  `${STAGE.map((a) => `${a} x${MULTIPLIER[a]}`).join(", ")}`);

const A = [-36, -13, -9, -9, 7, 11, 12, 8];
console.log("\n1. where the item originates, where it can be seen, where it is caught");
write(A, "item", "origin", "visible", "default", "delay", "at design", "at default", "factor");
for (const k of ITEM) {
  const t = cost(k, "design"), v = cost(k, k.default);
  write(A, k.name, k.origin, k.visible, k.default,
    order(k.default) - order(k.origin), t, v, (v / t).toFixed(1));
}

const B = [-15, 7, 8, 15, 8, 9];
console.log("\n2. item class by visible stage");
write(B, "visible stage", "items", "item %", "default hours", "hour %", "session");
for (const a of STAGE) {
  const g = ITEM.filter((k) => k.visible === a);
  if (!g.length) continue;
  const s = sum(g, (k) => cost(k, k.default));
  write(B, a, g.length, ((g.length / ITEM.length) * 100).toFixed(1), s,
    ((s / sum(ITEM, (k) => cost(k, k.default))) * 100).toFixed(1),
    order(a) <= order("design") ? "finds" : "misses");
}

const WITHOUT_SESSION = sum(ITEM, (k) => cost(k, k.default));
const WITH_SESSION = sum(found, (k) => cost(k, "design")) +
  sum(missed, (k) => cost(k, k.default)) + SESSION_HOURS;
const C = [-40, 14, 22];
console.log("\n3. the session's own measure");
for (const [name, x, y] of [
  ["session count x participants x hours", `${SESSION.count}x${SESSION.participants}x${SESSION.hours}`,
    SESSION_HOURS + " hours"],
  ["items found", found.length, (SESSION_HOURS / found.length).toFixed(1) + " hours/item"],
  ["items untouched", missed.length, ((missed.length / ITEM.length) * 100).toFixed(1) + "%"],
  ["total fix without session", WITHOUT_SESSION + " hours", "-"],
  ["total fix with session", WITH_SESSION + " hours", (WITHOUT_SESSION / WITH_SESSION).toFixed(2) + "x"],
  ["hours the session saves", WITHOUT_SESSION - WITH_SESSION + " hours",
    ((WITHOUT_SESSION - WITH_SESSION) / SESSION_HOURS).toFixed(1) + "x session hours"],
]) write(C, name, x, y);

const D = [-19, 7, 19, 18, 8];
console.log("\n4. items the session touches and does not, separately");
write(D, "group", "items", "hours w/o session", "hours w/ session", "factor");
for (const [name, ks, hours] of [
  ["found by session", found, sum(found, (k) => cost(k, "design"))],
  ["missed by session", missed, sum(missed, (k) => cost(k, k.default))],
]) {
  const o = sum(ks, (k) => cost(k, k.default));
  write(D, name, ks.length, o, hours, (o / hours).toFixed(1));
}
const worst = [...ITEM].sort((x, y) =>
  cost(y, y.default) - cost(x, x.default))[0];
console.log(`the single most expensive item is ${worst.name}: ${cost(worst, worst.default)} hours, ` +
  `the session ${isFound(worst) ? "finds" : "misses"} it; the ${missed.length} items the session ` +
  `misses carry ${((sum(missed, (k) => cost(k, k.default)) / WITHOUT_SESSION) * 100).toFixed(1)}% ` +
  `of the total fix hours`);

18 items, 6 stages; multipliers requirement x1, design x1, code x4, build x6, deploy x12, runtime x30

1. where the item originates, where it can be seen, where it is caught
item                                origin       visible  default    delay  at design  at default  factor
trust-boundary-not-drawn            design       design   deploy         3          6          72    12.0
meter-identity-not-verified         design       design   runtime        4          8         240    30.0
billing-record-modifiable           design       design   deploy         3         10         120    12.0
subscriber-data-flows-unprotected   design       design   deploy         3          4          48    12.0
authorization-decision-on-client    design       design   code           1          7          28     4.0
audit-log-missing                   requirement  design   runtime        5          5         150    30.0
field-device-carries-secret         design       design   deploy         3          9         108    12.0
reporting-endpoint-unauthenticated  design       design   code           1          3          12     4.0
measurement-backup-unprotected      design       design   runtime        4          6         180    30.0
session-duration-undefined          requirement  design   code           2          2           8     4.0
input-length-unbounded              code         code     build          1          2          12     6.0
query-concatenated-manually         code         code     build          1          4          24     6.0
timing-difference-leaks             code         code     runtime        3          5         150    30.0
concurrent-request-race-condition   code         runtime  runtime        3         12         360    30.0
dependency-version-unpinned         build        build    deploy         1          3          36    12.0
leftover-tool-in-image              build        build    deploy         1          2          24    12.0
default-credential-exposed          deploy       deploy   runtime        1          4         120    30.0
credential-in-log-entry             code         runtime  runtime        3          6         180    30.0

2. item class by visible stage
visible stage    items  item %  default hours  hour %  session
design              10    55.6            966    51.6    finds
code                 3    16.7            186     9.9   misses
build                2    11.1             60     3.2   misses
deploy               1     5.6            120     6.4   misses
runtime              2    11.1            540    28.8   misses

3. the session's own measure
session count x participants x hours             4x5x2              40 hours
items found                                         10        4.0 hours/item
items untouched                                      8                 44.4%
total fix without session                   1872 hours                     -
total fix with session                      1006 hours                 1.86x
hours the session saves                      866 hours   21.6x session hours

4. items the session touches and does not, separately
group                items  hours w/o session  hours w/ session  factor
found by session        10                966                60    16.1
missed by session        8                906               906     1.0
the single most expensive item is concurrent-request-race-condition: 360 hours, the session misses it; the 8 items the session misses carry 48.4% of the total fix hours

The numbers are in the measurement class; their inputs are TM1–TM4.

The Forty-Hour Session

The third table answers this lesson’s main question. Four sessions, five participants, two hours per session: a total of 40 developer hours. The session finds ten items, meaning 4.0 hours per item. This number describes threat modeling not as a method but as a work item; for a team that needs approval, this is the only form that can be argued over.

The payoff sits in the same row. Without the session, the total fix work for the eighteen items would be 1,872 hours; with the session it drops to 1,006 hours. The difference is 866 hours, which is 21.6 times the 40 hours spent. The size of this ratio comes from a single fact: the items the session finds were items that would otherwise have been caught late, and the late-stage multiplier was high.

Looked at over the total, though, the gain is far more modest: 1,872 hours drops to 1,006 hours, which is 1.86 times. The same measurement gives two different magnitudes, and both are correct. For the items the session touches, the ratio is 16.1; for the whole inventory, it is 1.86. Which number gets written down depends on whether what the session covers is stated explicitly.

The Same Item, Three Stages Later

The last column of the first table answers a single question for every item: when the same item is caught at its default stage, how many times its design-stage cost does it end up costing. The values spread between 4.0 and 30.0, and this range is the lesson’s most important finding — the sentence “catching it late is expensive” does not carry one single factor; it carries a distribution that depends on how far the item can travel unseen.

It is enough to compare the two extremes. authorization-decision-on-client is caught in code: 28 hours instead of 7, a factor of 4.0. meter-identity-not-verified travels all the way to runtime: 240 hours instead of 8, a factor of 30.0. The two items’ base hours are close to each other; what produces the thirtyfold difference between them is not the item’s own size but how many stages it advances through unseen.

The delay column counts this directly. audit-log-missing originates at the requirement stage and is caught at runtime: five stages late. The same item’s base is only 5 hours; a five-stage delay raises it to 150 hours. By contrast, leftover-tool-in-image originates at the build stage and is caught at deploy — one stage late, 24 hours instead of 2. concurrent-request-race-condition, the most expensive item in the inventory, reaches 360 hours with a three-stage delay and alone carries roughly a fifth of the total without the session.

The Coverage Ceiling

Read on its own, the first table’s origin column gives the session’s upper bound. In the inventory, 2 items originate at the requirement stage, 8 at design, 5 at code, 2 at build, 1 at deploy. When the design stage ends, the number of items not yet born is eight.

An item cannot be seen before it is born; the visible column is never earlier than the origin column in any row. This puts a structural ceiling on the session’s coverage: no item born after design can be found in a design-stage session — that item does not yet exist while the session is running. The eight items being left out comes from sequence, not from any weakness in the session.

Below the ceiling, the session’s own success is measured. Of the ten items born at the requirement or design stage, all ten show design in the visible column; in this inventory, the session reaches the entire ceiling. In a real session this ratio is lower, and the two numbers must be written separately: how many items could structurally be found, and how many of those were actually found. When the two are conflated, a team reporting 100% coverage has never measured 44.4% of the inventory at all.

Two items show this distinction directly. concurrent-request-race-condition and credential-in-log-entry originate at the code stage but only become visible at runtime; both have a three-stage delay. These fall within the scope of neither the design session nor a code-reading review — they are not visible even at the stage where they are born.

The Eight Items That Cannot Be Seen at Design

The fourth table states this lesson’s limit. The ten items the session touches drop from 966 hours to 60 hours — a factor of 16.1. The eight items the session misses drop from 906 hours to 906 hours — a factor of 1.0. No matter how well the session is run, it does nothing for these eight items.

The second table separates out where those eight stand. Three items can only be seen in code, two in build, one in deploy, two only at runtime. As a share of item count that is 44.4% of the inventory; as a share of fix hours it is 48.4%. So even the most thorough session run at the design stage leaves behind a share approaching half of the total work.

This class is not random. A race condition, a timing difference, and a credential landing in a log entry do not appear in the design as a box or an arrow; they only surface in the behavior of code that has been written or a system that is running. The design document does not carry these items, so no session that reads the design can find them. The two items visible only at runtime alone carry 540 hours — roughly two-thirds of the 866 hours the session saves.

The conclusion that follows is plain: a threat modeling session does not replace the reviews that come after it. The claim “we catch everything at design,” made without stating the session’s coverage limit, is a claim that, once measured, leaves out half of the inventory.

Summary

• A threat modeling session is a measurable work item: four sessions, five participants, two hours each — 40 developer hours, 10 items, 4.0 hours per item.
• The session saves 866 hours, which is 21.6 times the 40 hours spent. The same measurement gives 1.86 times over the whole inventory and 16.1 times over just the items it touches.
• The same item’s late-catch factor spreads between 4.0 and 30.0; what determines the factor is not the item’s size but how many stages it advances through unseen.
• A 5-hour item delayed by five stages rises to 150 hours; a 2-hour item delayed by one stage rises to 24 hours — the delay count matters more than the base hours.
• Eight items in the inventory cannot be seen at design, and the session never touches them — 44.4% of the item count, 48.4% of the fix hours. For these items, the session’s factor is 1.0.

Next Step

This lesson stated the session’s price and its coverage limit, but never opened up what happens inside the session. How the ten items were found remained a black box: “the design was read and items came out.”

Reading the design is not a method; a design document contains hundreds of sentences, and which sentence gives rise to a security item is not obvious. The method’s first step is to turn the design into a structure that can be counted over: which component takes which data from whom and gives it to whom, and which of those exchanges cross from one trust level to another. The next lesson builds that structure and takes the topic’s core measurement: the number of flows that cross a trust boundary in a data flow diagram, the items each boundary-crossing flow carries, and how many flows stay invisible when no boundary is drawn.