Risk-Focused Modeling

Systematic generation starts from the element: the same questions are asked of every box, every arrow, and every store. Every one of the 108 valid threats is the intersection of one element and one category; none of them arises from two elements being used together.

But real damage usually does not arise from a single element — it arises from several elements being used one after another. The way to find items of this kind is to start not from the element but from the goal: what the attacking side wants to obtain and by what path they get there. This lesson runs scenario-based modeling over the same graph and asks a single question — how many threats do scenarios find, how many of the systematic method’s finds do they miss, and what is the intersection of the two.

A Scenario Is a Node Sequence

A scenario is a path built from the attacker’s point of view; it shares its root with the abuse case from the Security Requirements lesson, but there it was used to write a requirement — here it is used to generate a threat.

The scenarios below are a model and only write down which elements are used; no step is described by how it is done. Trying a scenario like this on a real system, without written authorization and a defined scope, is both illegal and unprofessional.

TM13: a scenario is a node sequence running from an attacker profile to a target. TM14: the elements a scenario touches are the nodes in the sequence and the flows between them. TM15: a scenario produces two kinds of threat — an element threat, which overlaps with the scenario’s category on a touched element, and a chain threat, which arises from two consecutive nodes being used together; the latter has no counterpart in systematic generation because it is not defined on a single element. TM16: the cost of building a scenario is a fixed time per scenario plus a time per touched element.

// threat/scenario.mjs — the same graph, this time with scenarios built backward from the
// target (model). A scenario only writes down which elements are used; how a step is carried
// out is not written. There is no real system, no real endpoint, and no working step.

const ZONE = { internet: 0, field: 1, edge: 2, internal: 3, management: 4, data: 5 };
export const NODE = Object.fromEntries(`
  subscriber D internet | oversight-body D internet | meter-device D field | field-team D field
  subscriber-portal S edge | collection-endpoint S edge | field-app S field
  validator S internal | billing S internal | work-order S internal | report-generator S internal | identity-service S internal
  management-console S management | measurement-store P data | billing-store P data
  identity-store P data | audit-log P data`
  .trim().split(/\s*[|\n]\s*/).map((r) => {
    const [name, type, zone] = r.trim().split(/\s+/);
    return [name, { name, type, zone, level: ZONE[zone] }];
  }));
export const FLOW = `
  subscriber          subscriber-portal   identity    | subscriber-portal   identity-service   identity
  identity-service    identity-store      identity    | subscriber-portal   billing            billing
  billing             billing-store       billing     | billing             subscriber-portal  billing
  meter-device        collection-endpoint measurement | collection-endpoint validator          measurement
  validator           measurement-store   measurement | validator           billing            measurement
  field-team          field-app           identity    | field-app           work-order         order
  work-order          field-app           order       | work-order          measurement-store  measurement
  field-app           collection-endpoint measurement | management-console  validator          setting
  management-console  collection-endpoint setting     | management-console  identity-service   setting
  report-generator    measurement-store   measurement | report-generator    billing-store      billing
  oversight-body      report-generator    identity    | report-generator    oversight-body     billing
  billing             audit-log           record      | identity-service    audit-log          record
  work-order          audit-log           record      | billing             work-order         order`
  .trim().split(/\s*[|\n]\s*/).map((r) => {
    const [source, target, cls] = r.trim().split(/\s+/);
    return { source, target, class: cls, name: `${source}->${target}` };
  });

// The previous lesson's systematic generation is re-derived here with the same rules.
const incoming = (d) => FLOW.filter((a) => a.target === d);
const outgoing = (d) => FLOW.filter((a) => a.source === d);
const neighbors = (d) => [...incoming(d), ...outgoing(d)];
const isSensitiveClass = (s) => s === "identity" || s === "billing";
const isCriticalClass = (s) => s === "measurement" || s === "billing" || s === "order";
export const ELEMENT = Object.fromEntries([
  ...Object.values(NODE).map((d) => [d.name, { name: d.name, type: d.type,
    exposed: d.type === "D" ? outgoing(d.name).length > 0
      : incoming(d.name).some((a) => NODE[a.source].level < d.level),
    sensitive: neighbors(d.name).some((a) => isSensitiveClass(a.class)),
    critical: neighbors(d.name).some((a) => isCriticalClass(a.class)),
    multiRole: new Set(incoming(d.name).map((a) => NODE[a.source].zone)).size >= 2,
    transacts: d.type === "P" || (d.type === "D" ? outgoing(d.name).length > 0
      : outgoing(d.name).some((a) => NODE[a.target].type === "P")) }]),
  ...FLOW.map((a) => [a.name, { name: a.name, type: "A",
    exposed: NODE[a.source].zone !== NODE[a.target].zone,
    sensitive: isSensitiveClass(a.class), critical: isCriticalClass(a.class), multiRole: false, transacts: false }]),
]);
export const CATEGORY = ["identity-spoofing", "tampering", "repudiation", "information-disclosure",
  "denial-of-service", "privilege-escalation"];
const APPLIES = { D: ["identity-spoofing", "repudiation"], S: CATEGORY,
  P: ["tampering", "repudiation", "information-disclosure", "denial-of-service"],
  A: ["tampering", "information-disclosure", "denial-of-service"] };
const VALID_IF = { "identity-spoofing": (o) => o.exposed,
  "tampering": (o) => o.type !== "A" || o.exposed,
  "repudiation": (o) => o.transacts, "information-disclosure": (o) => o.sensitive,
  "denial-of-service": (o) => o.critical, "privilege-escalation": (o) => o.multiRole };
export const isValid = (o, k) => APPLIES[o.type].includes(k) && VALID_IF[k](o);
export const SYSTEMATIC = new Set(Object.values(ELEMENT).flatMap((o) =>
  CATEGORY.filter((k) => isValid(o, k)).map((k) => `${o.name}|${k}`)));

// TM13: a scenario is a node sequence running from an attacker profile to a target. The
// sequence only says which element is used; how the step is carried out is not written.
export const SCENARIO = [
  { name: "subscriber-account-takeover", profile: "external", target: "identity-store",
    path: ["subscriber", "subscriber-portal", "identity-service", "identity-store"],
    categories: ["identity-spoofing", "information-disclosure"] },
  { name: "billing-amount-tampering", profile: "subscriber", target: "billing-store",
    path: ["subscriber", "subscriber-portal", "billing", "billing-store"],
    categories: ["tampering", "privilege-escalation"] },
  { name: "false-measurement-record", profile: "field", target: "measurement-store",
    path: ["meter-device", "collection-endpoint", "validator", "measurement-store"],
    categories: ["identity-spoofing", "tampering"] },
  { name: "audit-trail-corruption", profile: "insider", target: "audit-log",
    path: ["management-console", "identity-service", "audit-log"],
    categories: ["tampering", "repudiation"] },
  { name: "bulk-billing-data-extraction", profile: "external-organization", target: "billing-store",
    path: ["oversight-body", "report-generator", "billing-store"],
    categories: ["identity-spoofing", "information-disclosure"] },
];

// TM14: the elements a scenario touches are the nodes in the sequence and the flows between them.
export function touched(s) {
  const ls = [...s.path];
  for (let i = 0; i + 1 < s.path.length; i++) ls.push(`${s.path[i]}->${s.path[i + 1]}`);
  return ls;
}

// TM15: a scenario produces two kinds of threat. An element threat overlaps with the scenario's
// category on a touched element and is valid there. A chain threat arises from two consecutive
// nodes being used together; it has no counterpart in systematic generation because it is not
// defined on a single element.
export const elementThreats = (s) => touched(s)
  .flatMap((a) => s.categories.filter((k) => isValid(ELEMENT[a], k)).map((k) => `${a}|${k}`));
export const chainThreats = (s) => s.path.slice(0, -1)
  .map((d, i) => `${d}=>${s.path[i + 1]}|chain`);

// TM16: the cost of building a scenario is a fixed time per scenario plus a time per touched element.
export const SCENARIO_MIN = 25, ELEMENT_MIN = 3;

// Systematic generation's duration is derived from the same inputs too.
export const RAW_COUNT = Object.values(ELEMENT).reduce((a, o) => a + APPLIES[o.type].length, 0);
export const ELIMINATION_MIN = 1.5, WRITE_MIN = 4;

// threat/coverage.mjs — how many threats scenario-based generation finds, how many of
// systematic generation's finds it misses, what the intersection of the two is.
import { ELEMENT, CATEGORY, SCENARIO, SYSTEMATIC, touched, elementThreats, chainThreats,
  SCENARIO_MIN, ELEMENT_MIN, RAW_COUNT, ELIMINATION_MIN, WRITE_MIN } from "./scenario.mjs";

const write = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const ELEMENT_T = new Set(SCENARIO.flatMap(elementThreats));
const CHAIN = new Set(SCENARIO.flatMap(chainThreats));
const SCENARIO_SET = new Set([...ELEMENT_T, ...CHAIN]);
const INTERSECTION = [...ELEMENT_T].filter((t) => SYSTEMATIC.has(t));
const SYSTEMATIC_ONLY = [...SYSTEMATIC].filter((t) => !ELEMENT_T.has(t));
const SCENARIO_MINUTES = SCENARIO.length * SCENARIO_MIN +
  SCENARIO.reduce((a, s) => a + touched(s).length, 0) * ELEMENT_MIN;

console.log(`${SCENARIO.length} scenarios; scenario set ${SCENARIO_SET.size} threats ` +
  `(${ELEMENT_T.size} element threats + ${CHAIN.size} chain threats), systematic set ` +
  `${SYSTEMATIC.size}; intersection ${INTERSECTION.length}`);

const A = [-30, -23, -19, 7, 9, 12, 10, 9];
console.log("\n1. scenarios and what they produce");
write(A, "scenario", "profile", "target", "steps", "touched", "element t.", "chain t.", "minutes");
for (const s of SCENARIO) {
  const d = touched(s);
  write(A, s.name, s.profile, s.target, s.path.length, d.length, elementThreats(s).length,
    chainThreats(s).length, SCENARIO_MIN + d.length * ELEMENT_MIN);
}

const B = [-17, 9, 17, 22];
console.log("\n2. the two methods' coverage");
write(B, "set", "threats", "% of systematic", "ratio to systematic");
for (const [name, n, p] of [
  ["systematic set", SYSTEMATIC.size, "-"],
  ["scenario set", SCENARIO_SET.size, "-"],
  ["intersection", INTERSECTION.length, (INTERSECTION.length / SYSTEMATIC.size * 100).toFixed(1)],
  ["scenario only", SCENARIO_SET.size - INTERSECTION.length, "-"],
  ["systematic only", SYSTEMATIC_ONLY.length, (SYSTEMATIC_ONLY.length / SYSTEMATIC.size * 100).toFixed(1)],
  ["union", SYSTEMATIC.size + SCENARIO_SET.size - INTERSECTION.length, "-"],
]) write(B, name, n, p === "-" ? "-" : p + " %", (n / SYSTEMATIC.size).toFixed(2) + " x");

const C = [-24, 9, 9, 13, 8];
console.log("\n3. threats found per minute");
write(C, "method", "threats", "minutes", "threats/min", "missed");
const SYSTEMATIC_MIN = RAW_COUNT * ELIMINATION_MIN + SYSTEMATIC.size * WRITE_MIN;
for (const [name, n, min, missed] of [
  ["systematic generation", SYSTEMATIC.size, SYSTEMATIC_MIN, CHAIN.size],
  ["scenario-based", SCENARIO_SET.size, SCENARIO_MINUTES, SYSTEMATIC_ONLY.length],
  ["both together", SYSTEMATIC.size + SCENARIO_SET.size - INTERSECTION.length,
    SYSTEMATIC_MIN + SCENARIO_MINUTES, 0],
]) write(C, name, n, min.toFixed(1), (n / min).toFixed(3), missed);

const D = [-17, 12, 14, 8, 10];
console.log("\n4. where the threats scenarios miss stand");
write(D, "element type", "systematic", "intersection", "missed", "missed %");
const TYPE = { D: "external entity", S: "process", P: "data store", A: "data flow" };
for (const t of ["D", "S", "P", "A"]) {
  const s = [...SYSTEMATIC].filter((x) => ELEMENT[x.split("|")[0]].type === t);
  const k = s.filter((x) => !ELEMENT_T.has(x));
  write(D, TYPE[t], s.length, s.length - k.length, k.length,
    ((k.length / s.length) * 100).toFixed(1));
}
const cat = CATEGORY.map((k) => ({ k, n: SYSTEMATIC_ONLY.filter((x) => x.endsWith("|" + k)).length }))
  .sort((a, b) => b.n - a.n);
console.log(`category distribution of the missed ${SYSTEMATIC_ONLY.length} threats: ` +
  cat.map((x) => `${x.k} ${x.n}`).join(", "));
console.log(`none of the ${CHAIN.size} chain threats appear in systematic generation; ` +
  `they are not defined on a single element`);

5 scenarios; scenario set 51 threats (39 element threats + 12 chain threats), systematic set 108; intersection 39

1. scenarios and what they produce
scenario                      profile                target               steps  touched  element t.  chain t.  minutes
subscriber-account-takeover   external               identity-store           4        7           9         3       46
billing-amount-tampering      subscriber             billing-store            4        7           8         3       46
false-measurement-record      field                  measurement-store        4        7           9         3       46
audit-trail-corruption        insider                audit-log                3        5           7         2       40
bulk-billing-data-extraction  external-organization  billing-store            3        5           6         2       40

2. the two methods' coverage
set                threats  % of systematic   ratio to systematic
systematic set         108                -                1.00 x
scenario set            51                -                0.47 x
intersection            39           36.1 %                0.36 x
scenario only           12                -                0.11 x
systematic only         69           63.9 %                0.64 x
union                  120                -                1.11 x

3. threats found per minute
method                    threats  minutes  threats/min  missed
systematic generation         108    666.0        0.162      12
scenario-based                 51    218.0        0.234      69
both together                 120    884.0        0.136       0

4. where the threats scenarios miss stand
element type       systematic  intersection  missed  missed %
external entity             8             3       5      62.5
process                    40            17      23      57.5
data store                 12             6       6      50.0
data flow                  48            13      35      72.9
category distribution of the missed 69 threats: denial-of-service 24, tampering 19, repudiation 11, information-disclosure 7, privilege-escalation 5, identity-spoofing 3
none of the 12 chain threats appear in systematic generation; they are not defined on a single element

The numbers are in the measurement class; their inputs are TM13–TM16.

Five Scenarios, Fifty-One Threats

The first table gives the scenarios’ own measure. Three of the five scenarios are four steps, two are three steps; the number of elements touched ranges from seven to five. The total comes to 51 threats, and the composition of this number is the real finding: 39 element threats and 12 chain threats.

Scenario durations, ranging between forty-six and forty minutes, add up to 218 minutes. Compared with systematic generation’s 666 minutes, this is roughly a third. Scenarios touch fewer elements because the path to the target does not traverse the whole graph: the five scenarios together use only a portion of the 43 elements.

Intersection, Difference, and Union

The second table is this lesson’s main measurement. The systematic set is 108, the scenario set is 51 threats. The intersection is 39 — that is, all of the element threats the scenarios find are already in the systematic set. This is an expected result: when a scenario touches an element, it produces the category that is valid on that element, and the systematic method has already produced every valid category on every element.

All 12 of the threats found only in the scenarios are chain threats. These arise from two nodes being used together, and because they are not defined on a single element, they fall into no row of systematic generation. Given the method’s structure, they cannot fall in: as long as the question is “what could happen at this element,” the question “what is obtained at this element becoming valid at that one” is never asked.

The 69 threats found only in the systematic set, on the other hand, lie outside the set the scenarios see and make up 63.9% of the systematic set. The five scenarios touch 26 of the 43 elements, but at each element they only produce the category the scenario itself carries; the 17 untouched elements are left entirely unexamined.

The union is 120 threats — 1.11 times the systematic set. The two methods do not substitute for each other; the only reason the union is larger than the systematic set is the 12 chain threats.

What It Misses

The fourth table separates out where the misses stand. The miss rate is highest in data flow elements: 35 of 48 threats are missed (72.9%). It is lowest in data stores (50.0%), because all five of the five scenarios end at a store and stores are the scenarios’ target.

The category distribution is sharper. Of the 69 missed threats, 24 are denial of service, 19 are tampering, 11 are repudiation; identity spoofing, with only three threats, is the least-missed category. The reason lies in the very nature of building a scenario: a scenario describes reaching a target, while denial of service does not aim to reach the target — it aims to stop the service. Scenarios built from the attacker’s point of view are after access; a threat class that does not want access has no natural place on a scenario list.

This is the scenario method’s measurable blind spot. The sentence “look through the attacker’s eyes” sounds inclusive, but once measured, it leaves an entire category almost completely out.

Threats Found Per Minute

The third table compares the two methods in the same unit. Systematic generation finds 108 threats in 666 minutes: 0.162 per minute. Scenario-based generation finds 51 threats in 218 minutes: 0.234 per minute — 1.44 times faster.

This ratio is misleading read on its own. The reason the scenario method looks efficient is that it carries no elimination load: because a scenario is built backward from a target, every threat it produces already sits on a path, and no row comes up to be discarded. Systematic generation, by contrast, spends 72 minutes on 48 invalid threats. What creates the efficiency is not the method’s intelligence — it is the smallness of the search space.

The last column writes down the price of that. Used alone, systematic generation misses 12 chain threats; used alone, scenario-based generation misses 69 threats. The two together give 120 threats in 884 minutes — 0.136 per minute, slower than either method alone. Completing coverage lowers per-minute efficiency, and this is a decision the measurement itself hands down: the fast method is the incomplete one.

Summary

• A scenario is a node sequence running from an attacker profile to a target; five scenarios touch a portion of the 43 elements and produce 51 threats in 218 minutes.
• 39 of the 51 threats are element threats, 12 are chain threats; chain threats arise from two nodes being used together and have no counterpart in systematic generation.
• The intersection is 39; all of the element threats scenarios find sit inside the systematic set. Scenarios miss 63.9% of the systematic set (69 threats).
• The category distribution of the missed threats is uneven: 24 are denial of service, only 3 are identity spoofing. Because a scenario is after access, it leaves out the threat class that does not want access.
• The scenario method runs at 0.234 threats per minute, 1.44 times the speed of systematic generation, but this speed comes from the smallness of the search space; the two together give 120 threats in 884 minutes.

Next Step

Chain threats were this lesson’s most notable finding: twelve items, defined on no single element, visible only on a path. But this lesson only counted paths — it did not rank them. All five of the five scenarios stand at equal weight on the list.

In reality, paths are not equal. A target can be reached by more than one path, and some of these paths are cheaper than others; what defense cares about is not the most expensive path but the cheapest one. Counting this requires writing the scenario not as a flat sequence but as a structure branching backward from the target: a root goal, sub-goals, and leaf steps whose costs are written together with their preconditions. The next lesson builds this structure, calculates the cheapest path’s cost, and asks a single question: by what factor does closing off one node raise the tree’s cost.