--- title: 'Security Operations and Incident Response' source: 'https://academia.sh/en/courses/security-operations' language: en updated: '2026-09-26T03:02:56+00:00' license: 'CC BY-SA 4.0' lessons: 22 outcomes: - 'Connects log sources to a central detection pipeline and counts how many steps lie between source and decision' - 'Runs the incident response life cycle and measures the two-way cost of an early and a late decision' - 'Applies forensic examination steps with the order of volatility and the chain of custody' --- # Security Operations and Incident Response Measuring the time between detection and containment: the shift pattern of the operations center and the item lost at the handover point, multi-source log aggregation and the field that cannot be mapped during normalization, the event that correlation rules merge and the one they miss, detection with network flow data, the threshold point where false positives and false negatives cannot be reduced at the same time, the number of techniques detection engineering covers, hypothesis-driven threat hunting, indicator-based and behavior-based intelligence feeds, automating repeated response, the incident response life cycle from preparation to lessons learned, the item difference between early and late containment, notification obligations, evidence integrity and chain of custody, order of volatility, timeline merging, and clock skew. ## Detection Infrastructure Builds the pipeline for the time between detection and containment: shift capacity and the case that fails to merge at the handover point, normalizing three log formats into a single schema and the field that cannot be mapped, the event a correlation rule merges and the one it misses, the question network flow data can answer without payload content, the threshold point where false positives and missed detections cannot be reduced at the same time, the number of techniques two rule versions cover, the item eliminated per hypothesis, the lifespan difference between indicator-based and behavior-based feeds, and automating repeated response. ### Security Operations Center Source: https://academia.sh/en/courses/security-operations/security-operations-center 273 alerts falling on a fictional line are measured against a three-person shift: the day's work is 1,506 minutes, capacity is 1,440 minutes; the plan that splits twenty-four hours evenly leaves 64 alerts never reviewed and pushes the median wait to 222 minutes, while the plan that leans on density leaves eight hours uncovered and brings that down to 13 alerts and 83 minutes. 19 of 26 cases cross the handover point; with an empty handover note, 19 cases never merge and the median case decision is 987 minutes, dropping to zero unmerged cases at a six-item note with the median falling to 273 minutes, while at twelve items the handover eats 72 minutes and pushes never-reviewed alerts to 26. The previous course spent eighteen lessons counting how many log entries each defensive step left behind. That count carried a silent assumption: that the log entry gets read. This course removes that assumption and measures the structure that reads the log entry. The unit of measure changes too — what is counted is no longer a count of log entries, but **time**. The time between an event occurring and it being seen as an alert is called **time to detect**; the time from being seen until it is stopped is called **time to contain**. Every measurement in this course ties back to one of these two. A security operations center is not a room; it is a capacity and a handover arrangement. This lesson derives two numbers from that arrangement: how many alerts land on whom within how many hours, and how many items get lost at shift change. On-call policy — who gets called, at what threshold it escalates — was measured in the Observability and Operations course; what is measured here is not the call, but the capacity of a continuous line. - **DI1.** The regional measurement network is **fictional**: meter collectors, the subscriber portal, and branch endpoints send their logs to a central line. The line produces 273 alerts a day. The numbers and durations are **data**, not measurement. - **DI2.** The center has three people, each on an eight-hour shift. Daily capacity is 3 × 8 × 60 = 1,440 analyst-minutes. - **DI3.** Alerts come in five types. Each type has two numbers: average review minutes and **the minutes between the event occurring and the log entry landing on the line**. - **DI4.** An analyst reviews only one alert at a time and does not take on work they cannot finish before their shift ends. If no one is on duty, the alert waits. - **DI5.** A case is 2-5 alerts belonging to the same fictional actor, spread across 30-240 minutes. This link is **not written in the log**; it lives in the analyst's memory. - **DI6.** A handover note takes two minutes per item and comes out of the receiving analyst's review capacity. #### One Day's Flow and Two Shift Plans ```js // shift.mjs — fictional alert flow and the three-person shift roster. IS A MODEL. // The regional measurement network is fictional; no real organization, address or event name. // The generator is self-written; the seed appears below. export const SEED = 20260401; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = generator(SEED); const randInt = (n) => Math.floor(r() * n); // Alerts per hour (fictional data, not a measurement): busy during business hours. const DENSITY = [4, 3, 3, 2, 2, 3, 6, 11, 18, 22, 24, 21, 19, 23, 25, 22, 17, 12, 9, 7, 6, 5, 5, 4]; // [type, percentage share, average review minutes, minutes between event and log arrival] const TYPES = [["authentication", 34, 3, 1], ["network-flow", 26, 4, 5], ["endpoint", 22, 8, 2], ["data-access", 12, 11, 12], ["control-plane", 6, 6, 9]]; export const ALERT = []; for (const [h, n] of DENSITY.entries()) for (let k = 0; k < n; k++) { let p = randInt(100), t = TYPES[0]; for (const row of TYPES) { if (p < row[1]) { t = row; break; } p -= row[1]; } ALERT.push({ id: ALERT.length, arrival: h * 60 + randInt(60), type: t[0], review: Math.max(1, t[2] + randInt(5) - 2), lag: t[3] }); } ALERT.sort((a, b) => a.arrival - b.arrival); // Shift plan: [name, start hour, end hour]. Three people, eight hours each. export const PLAN = { "even split": [["N1", 0, 8], ["N2", 8, 16], ["N3", 16, 24]], "density-weighted": [["N1", 6, 14], ["N2", 10, 18], ["N3", 14, 22]], }; // Queue simulation: an alert is only reviewed once an on-duty analyst frees up. export function runQueue(shifts, handoverPrep = []) { const free = shifts.map(([, b], i) => b * 60 + (handoverPrep[i] || 0)); const end = shifts.map(([, , s]) => s * 60); const result = []; for (const u of ALERT) { let pick = -1; for (const [i, v] of free.entries()) { const start = Math.max(v, u.arrival); if (start + u.review > end[i]) continue; // no new work once the shift is ending if (pick < 0 || start < Math.max(free[pick], u.arrival)) pick = i; } if (pick < 0) { result.push({ ...u, analyst: null, wait: null }); continue; } const start = Math.max(free[pick], u.arrival); free[pick] = start + u.review; result.push({ ...u, analyst: shifts[pick][0], wait: start - u.arrival, close: free[pick] }); } return result; } const median = (v) => v.slice().sort((a, b) => a - b)[Math.floor(v.length / 2)]; const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); if (import.meta.url === `file://${process.argv[1]}`) { const wide = fmt([18, 12, 16, 13, 14, 13]), narrow = fmt([18, 8, 12, 14]); console.log(`model (seed ${SEED}): ${ALERT.length} alerts, total ` + `${ALERT.reduce((a, u) => a + u.review, 0)} minutes of review work; ` + `three analysts' capacity 3 x 8 x 60 = 1440 minutes`); console.log(narrow(["type", "alerts", "log lag", "review min"])); for (const [name, , , lag] of TYPES) { const l = ALERT.filter((u) => u.type === name); console.log(narrow([name, l.length, `${lag} min`, l.reduce((a, u) => a + u.review, 0)])); } console.log(); console.log(wide(["plan", "uncovered", "never reviewed", "median wait", "longest wait", "over 30 min"])); for (const [name, v] of Object.entries(PLAN)) { const s = runQueue(v), reviewed = s.filter((u) => u.analyst !== null); const covered = new Set(v.flatMap(([, b, e]) => Array.from({ length: e - b }, (_, i) => b + i))); console.log(wide([name, `${24 - covered.size} hours`, s.length - reviewed.length, median(reviewed.map((u) => u.wait)), Math.max(...reviewed.map((u) => u.wait)), reviewed.filter((u) => u.wait > 30).length])); } console.log("\nper-analyst breakdown in the density-weighted plan (fictional day):"); const s2 = runQueue(PLAN["density-weighted"]), narrow2 = fmt([8, 8, 11, 12, 14]); console.log(narrow2(["analyst", "alerts", "review min", "idle min", "longest wait"])); for (const [name, b, e] of PLAN["density-weighted"]) { const l = s2.filter((u) => u.analyst === name), rv = l.reduce((a, u) => a + u.review, 0); console.log(narrow2([name, l.length, rv, (e - b) * 60 - rv, Math.max(...l.map((u) => u.wait))])); } } ``` ``` model (seed 20260401): 273 alerts, total 1506 minutes of review work; three analysts' capacity 3 x 8 x 60 = 1440 minutes type alerts log lag review min authentication 104 1 min 308 network-flow 68 5 min 271 endpoint 53 2 min 439 data-access 39 12 min 430 control-plane 9 9 min 58 plan uncovered never reviewed median wait longest wait over 30 min even split 0 hours 64 222 477 161 density-weighted 8 hours 13 83 363 260 per-analyst breakdown in the density-weighted plan (fictional day): analyst alerts review min idle min longest wait N1 85 480 0 363 N2 86 480 0 101 N3 89 479 1 127 ``` The first line gives the lesson's boundary: the day's work is 1,506 minutes, capacity is 1,440 minutes. Utilization exceeds one hundred percent, and this is a shortfall no shift arrangement can rescue. But **where** the shortfall lands depends on the arrangement, and the two plans show this in opposite directions. The type table gives the first column: how many minutes after the event the log entry lands. An authentication log entry arrives in one minute, a data-access log entry in twelve. This is a delay the line never knows about — by the time the log entry lands on the line, the event already happened twelve minutes ago, and the queue only starts after that. The plan that splits twenty-four hours evenly delivers full **coverage**: zero uncovered hours. In exchange, 64 alerts are never reviewed, and the median wait is 222 minutes. The reason is plain: the night shift takes three alerts an hour and sits idle, while the day shift meets twenty-five alerts an hour and grows the queue. **Coverage is not a measure of capacity.** Having someone on duty does not mean that person can handle that hour's load. The plan that leans on density leaves eight hours completely uncovered — no one is there between 22:00 and 06:00. In exchange, never-reviewed alerts drop from 64 to 13, and the median wait drops from 222 to 83 minutes. Two columns move in opposite directions: alerts exceeding thirty minutes **rise** from 161 to 260. The reason lies in how the number is defined — in the even-split plan, the 64 never-reviewed alerts do not enter any wait column at all. A lost item improves the average. The last table says all three analysts are fully occupied: idle minutes of 0, 0, and 1. The load looks evenly distributed, but the longest-wait column writes out what that costs. An alert N1 picked up waited 363 minutes in the queue; a log entry that lands at night sits in front of no one until six in the morning. Time to detect breaks into three parts in this model, and all three can be read separately in the table: how many minutes after the event the log entry lands, how many minutes it waits in the queue, and how many minutes the review takes. #### The Case That Fails to Merge at the Handover Point Queue time alone is an incomplete measure, because it assumes the decision is made per alert. The real decision is usually made by **reading multiple alerts together**, and in this model what forms that link is the analyst's memory. Shift change is exactly the moment that memory resets. ```js // handover.mjs — the cost of the case left unmerged at shift handover and of the handover note. IS A MODEL. import { ALERT, PLAN, runQueue } from "./shift.mjs"; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r2 = generator(760413); // Case: 2-5 alerts belonging to the same fictional actor, spread across 30-240 minutes. The // analyst forms the link; it is not written in the log. Seed alerts are chosen by the seed. const taken = new Set(); export const CASE = []; for (const u of ALERT) { if (taken.has(u.id) || r2() > 0.16 || CASE.length === 26) continue; const group = [u.id]; taken.add(u.id); for (const v of ALERT) if (!taken.has(v.id) && group.length < 2 + Math.floor(r2() * 4) && v.arrival > u.arrival + 30 && v.arrival < u.arrival + 240) { group.push(v.id); taken.add(v.id); } CASE.push(group); } const V = PLAN["density-weighted"]; const END = V.map(([, , e]) => e * 60); function day(limit) { const s = runQueue(V, [2 * limit, 2 * limit, 2 * limit]); // two minutes of handover per item const d = new Map(s.map((u) => [u.id, u])); const solo = [], handover = []; for (const g of CASE) { const items = g.map((i) => d.get(i)); const seen = items.filter((u) => u.analyst !== null); const who = [...new Set(seen.map((u) => u.analyst))]; if (who.length === 1 && seen.length === items.length) solo.push({ g, decision: Math.max(...items.map((u) => u.close)) }); else handover.push({ g, boundaryIndex: who.length ? V.findIndex(([a]) => a === who[0]) : 2, first: Math.min(...items.map((u) => u.arrival)), lastClose: Math.max(...seen.map((u) => u.close), 0) }); } let carried = 0, unmerged = 0, strayAlerts = 0; for (const i of [0, 1, 2]) { const list = handover.filter((v) => v.boundaryIndex === i).sort((a, b) => a.first - b.first); for (const [j, v] of list.entries()) if (j < limit) { carried++; v.decision = Math.max(v.lastClose, END[i]) + 3; } else { unmerged++; strayAlerts += v.g.length; v.decision = 1800; } // 06:00 the next day } const span = [...solo, ...handover] .map((v) => v.decision - Math.min(...v.g.map((i) => d.get(i).arrival - d.get(i).lag))) .sort((a, b) => a - b); return { solo: solo.length, handover: handover.length, carried, unmerged, strayAlerts, handoverMin: 6 * limit, neverReviewed: s.filter((u) => u.analyst === null).length, median: span[Math.floor(span.length / 2)] }; } const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([12, 12, 10, 12, 13, 13, 16, 12]); console.log(`${CASE.length} cases exist, containing ${CASE.reduce((a, g) => a + g.length, 0)} ` + `alerts; what forms the link is the analyst's memory, not written in the log`); console.log(`cases closed within a single shift: ${day(0).solo}, ` + `cases crossing the handover point: ${day(0).handover}`); console.log(); console.log(yz(["note limit", "spans shift", "carried", "unmerged", "stray alerts", "handover min", "never reviewed", "median case"])); for (const limit of [0, 2, 4, 6, 12]) { const g = day(limit); console.log(yz([`${limit} cases`, g.handover, g.carried, g.unmerged, g.strayAlerts, g.handoverMin, g.neverReviewed, `${g.median} min`])); } ``` ``` 26 cases exist, containing 128 alerts; what forms the link is the analyst's memory, not written in the log cases closed within a single shift: 7, cases crossing the handover point: 19 note limit spans shift carried unmerged stray alerts handover min never reviewed median case 0 cases 19 0 19 94 0 13 987 min 2 cases 18 6 12 59 12 15 528 min 4 cases 18 12 6 29 24 18 436 min 6 cases 18 18 0 0 36 18 273 min 12 cases 20 20 0 0 72 26 285 min ``` Only seven of twenty-six cases close within a single shift; **nineteen cross the handover point**. This ratio is not a consequence of the model but of the arithmetic of durations: a case spans at most four hours, a shift is eight hours, and once queue waiting is added, it becomes routine for a case to fall to two analysts. With an empty handover note, 19 cases never merge. This does not mean those cases' alerts go **unreviewed** — all 94 alerts get opened and closed one by one. What is lost is not the alert, but the link between alerts, and without that link, each alert looks unimportant on its own. The median case decision rises to 987 minutes, or sixteen hours. Raising the note limit to six items zeroes out unmerged cases and brings the median down to 273 minutes. The cost is written in the table: 36 analyst-minutes go to the handover meeting, and never-reviewed alerts rise from 13 to 18. At twelve items, though, the curve turns back — unmerged cases are already at zero, the handover eats 72 minutes, never-reviewed alerts jump to 26, and the median **climbs back up** from 273 to 285 minutes. Capacity spent shortening the decision time lengthens it again past a certain point. The two-way cost reads together in this table. The cost of a late decision is the case staying open for sixteen hours; how many more items get reached during that time is not written into this model, and will be counted in later lessons. The cost of an early decision is capacity: every minute set aside for handover means one alert that never gets opened at all. #### Summary - The fictional line's daily work is 1,506 minutes; the three-person shift's capacity is 1,440 minutes. Once utilization exceeds one hundred percent, no arrangement closes the gap — it only decides where it lands. - The plan that splits twenty-four hours evenly zeroes out uncovered hours but never gets 64 alerts reviewed and pushes the median wait to 222 minutes; coverage is not a measure of capacity. - The plan that leans on density leaves eight hours uncovered, bringing never-reviewed alerts down to 13 and the median wait down to 83 minutes; alerts exceeding thirty minutes rising from 161 to 260 is because the lost item improves the average. - 19 of 26 cases cross the handover point; with an empty handover note, none of them merge, all 94 alerts close one by one, and the median case decision stays at 987 minutes. - A six-item handover note zeroes out unmerged cases and brings the median down to 273 minutes; at twelve items, the handover eats 72 minutes, never-reviewed alerts rise to 26, and the median climbs back up to 285 minutes. #### Next Step What the handover note did was put into writing a link that lived in an analyst's memory. Building that link into the log from the start — that is, letting the machine see that two alerts belong to the same actor — was assumed in this lesson's model. In reality, it cannot be assumed: the meter collector's log and the subscriber portal's log write the same person with different field names, different time formats, and different levels of detail. The next lesson fits three separately formatted fictional logs into a single schema and establishes the course's unit of measure: how many fields map, how many fields fit no counterpart at all, and what question an unmappable field leaves unanswered. ### Log Aggregation and Normalization Source: https://academia.sh/en/courses/security-operations/log-collection-and-normalization Three separately formatted fictional logs are normalized into a single schema, and the fields that do not map are counted: 15 of 24 source fields fit an eight-field schema, 9 remain unmapped; four schema fields are empty in the network-flow log, and none of the 404 records has an actor. The sources write the same event 1, 5, and 8 minutes later. Two lookup tables fill in the actor field: 115 of 139 control-plane records convert to an account, 24 do not convert; in a five-minute window, 190 of 404 flow records get an actor assigned, 212 stay empty, and at sixty minutes the assignment rises to 208 while ambiguous records jump from 2 to 39. Of 26 events that leave a trace in two sources, 21 get collected; in a one-minute window, none do. The previous lesson's handover note put into writing a link that lived in the analyst's memory. That link being present in the log from the start — that is, the machine being able to see that two alerts belong to the same actor — was assumed in the previous lesson. This lesson removes that assumption and establishes the course's unit of measure. The measure is this: which of a record's fields fit the shared schema, which do not, and **what goes in place of the field that does not fit**. The source inventory was counted in the Endpoint and System Hardening course; here the inventory is an input. What is counted is not the source's existence, but **its linkability to the others**. - **DI7.** The regional measurement network has three log sources: a cloud control-plane record, endpoint process-creation logs, and network flow data. All three are **fictional**; the addresses come from non-routable fictional blocks. - **DI8.** The target schema has eight fields: time, source, host, address, actor, action, target, result. A field a source cannot fill stays `null` and **gets counted**. - **DI9.** A source field is in one of three states: it **maps** to the schema, it stays **leftover** because the schema has no counterpart for it, or a schema field stays **empty** because no source field can fill it. - **DI10.** A record forms at the moment the source writes it, not the moment the event happens. The model gives this gap as data, per source: 1 minute at the endpoint, 5 at network flow, 8 at the control plane. - **DI11.** Thirty fictional events leave a trace in more than one source. This link is **knowledge the model holds**; it is written into no log line. - **DI12.** The actor identity takes three forms across three sources: a portal ID, a domain account, and address alone. The lookup tables live in the asset inventory, not in the log. #### Three Formats, One Schema ```js // source.mjs — three separately formatted fictional logs are produced and normalized into one schema. // IS A MODEL. Addresses come from RFC1918 fictional blocks; no real organization or host. export const SEED = 20260402; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = generator(SEED); const randInt = (n) => Math.floor(r() * n); const pick = (d) => d[randInt(d.length)]; export const HOST = Array.from({ length: 60 }, (_, i) => `uc-${101 + i}`); export const ACCOUNT = Array.from({ length: 40 }, (_, i) => `sh-${201 + i}`); export const ADDRESS = HOST.map((_, i) => `10.40.${1 + (i % 3)}.${10 + i}`); // Portal ID: 28 accounts have a match; six of them (service identities) have none at all. const PORTAL = [...ACCOUNT.slice(0, 28).map((_, i) => `ab-${4401 + i}`), ...Array.from({ length: 6 }, (_, i) => `ab-${4429 + i}`)]; const ACTION = ["object.read", "object.write", "permission.change", "key.use"]; const PATH = ["/opt/measurement/collector", "/usr/bin/archive", "/opt/measurement/verifier"]; // Target schema: eight fields. Each source fills part of it. export const SCHEMA = ["t", "source", "host", "address", "actor", "action", "target", "result"]; // [source, format, source fields, fields mapped to schema, arrival lag min, resolution] export const SOURCE = [["control-plane", "json", 8, 6, 8, "1 ms"], ["endpoint", "key=value", 8, 5, 1, "1 s"], ["network-flow", "csv", 8, 4, 5, "1 s"]]; const DAY = Date.UTC(2026, 3, 2); const iso = (ms) => new Date(DAY + ms).toISOString(); export const cp = [], ep = [], nf = []; // three raw logs, three separate formats const tagged = { cp: [], ep: [], nf: [] }; // the model's ground truth; NOT WRITTEN to the log line // Shared event: the same work item leaves a trace in more than one source. The model knows this, // the line does not. export const SHARED = []; for (let k = 0; k < 30; k++) { const i = randInt(HOST.length), h = ACCOUNT[randInt(28)]; SHARED.push({ t: 3600e3 + randInt(50000e3), host: HOST[i], address: ADDRESS[i], account: h, portal: PORTAL[ACCOUNT.indexOf(h)], ep: r() < 0.8, nf: r() < 0.9, cp: r() < 0.7 }); } const write = (o, e) => { if (o.cp) { cp.push({ ts: iso(o.t + 480e3 + randInt(9000)), actor: { id: o.portal, type: "user" }, op: pick(ACTION), target: `bucket/measurement/${randInt(900)}`, result: r() < 0.9 ? "allowed" : "denied", srcIp: o.address, requestId: `q${randInt(1e5)}` }); tagged.cp.push(e); } if (o.ep) { ep.push(`time=${Math.floor((DAY + o.t + 60e3 + randInt(9000)) / 1000)} ` + `host=${o.host} event=process-creation user=${o.account} pid=${1000 + randInt(9000)} ` + `parent=${400 + randInt(600)} path=${pick(PATH)} ` + `hash=${randInt(1e9).toString(16).padStart(8, "0")}`); tagged.ep.push(e); } if (o.nf) { nf.push([Math.floor((DAY + o.t + 300e3 + randInt(9000)) / 1000), 1 + randInt(60), o.address, `10.90.2.${10 + randInt(40)}`, pick([443, 8443, 5671]), "outbound", 400 + randInt(9e5), 5 + randInt(900)].join(",")); tagged.nf.push(e); } }; SHARED.forEach((o, i) => write(o, i)); // Background traffic: records unrelated to any shared event. for (let k = 0; k < 480; k++) { const i = randInt(HOST.length); write({ t: randInt(86400e3), host: HOST[i], address: ADDRESS[i], account: pick(ACCOUNT), portal: pick(PORTAL), ep: r() < 0.45, nf: r() < 0.75, cp: r() < 0.25 }, -1); } // Normalization: one mapper per source. A field that cannot be filled stays null. export const normal = { "control-plane": (s) => { const o = JSON.parse(s); return { t: Date.parse(o.ts), source: "control-plane", host: null, address: o.srcIp, actor: o.actor.id, action: o.op, target: o.target, result: o.result }; }, "endpoint": (s) => { const a = Object.fromEntries(s.split(" ").map((p) => p.split("="))); return { t: Number(a.time) * 1000, source: "endpoint", host: a.host, address: null, actor: a.user, action: a.event, target: a.path, result: null }; }, "network-flow": (s) => { const [ts, , src, dst, port] = s.split(","); return { t: Number(ts) * 1000, source: "network-flow", host: null, address: src, actor: null, action: null, target: `${dst}:${port}`, result: null }; }, }; export const RAW = { "control-plane": cp.map((o) => JSON.stringify(o)), "endpoint": ep, "network-flow": nf }; const TAGGED = { "control-plane": tagged.cp, "endpoint": tagged.ep, "network-flow": tagged.nf }; const all = Object.entries(RAW).flatMap(([k, l]) => l.map((s, i) => [normal[k](s), TAGGED[k][i]])).sort((a, b) => a[0].t - b[0].t); export const EVENT = all.map((x) => x[0]); export const LABEL = all.map((x) => x[1]); // shared-event index, or -1; the line NEVER SEES this if (import.meta.url === `file://${process.argv[1]}`) { const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([18, 15, 8, 7, 14, 12, 11, 13, 14]); console.log(`model (seed ${SEED}): ${EVENT.length} records, ${SCHEMA.length}-field schema`); for (const k of ["control-plane", "endpoint", "network-flow"]) console.log(RAW[k][0]); console.log(); console.log(yz(["source", "format", "records", "fields", "mapped", "leftover", "empty", "time res.", "write lag"])); for (const [name, format, fields, mapped, lag, res] of SOURCE) { const sample = normal[name](RAW[name][0]); const filled = SCHEMA.filter((a) => a !== "source" && sample[a] !== null).length; console.log(yz([name, format, RAW[name].length, fields, mapped, fields - mapped, SCHEMA.length - 1 - filled, res, `${lag} min`])); } const t = SOURCE.reduce((a, k) => a + k[2], 0), e = SOURCE.reduce((a, k) => a + k[3], 0); console.log(`\ntotal ${t} source fields: ${e} map to the schema, ` + `${t - e} remain leftover; the actor field is absent from every network-flow record`); } ``` ``` model (seed 20260402): 787 records, 8-field schema {"ts":"2026-04-02T10:26:21.519Z","actor":{"id":"ab-4401","type":"user"},"op":"permission.change","target":"bucket/measurement/888","result":"allowed","srcIp":"10.40.1.22","requestId":"q96644"} time=1775125160 host=uc-113 event=process-creation user=sh-201 pid=4052 parent=405 path=/usr/bin/archive hash=3718b893 1775125395,7,10.40.1.22,10.90.2.18,5671,outbound,120991,443 source format records fields mapped leftover empty time res. write lag control-plane json 139 8 6 2 1 1 ms 8 min endpoint key=value 244 8 5 3 2 1 s 1 min network-flow csv 404 8 4 4 4 1 s 5 min total 24 source fields: 15 map to the schema, 9 remain leftover; the actor field is absent from every network-flow record ``` Three sample lines show three separate worlds. The control-plane record writes a nested object and keeps the actor at the `actor.id` path. The endpoint record writes plain `key=value` pairs and calls the actor `user`. The network-flow record, meanwhile, is a headerless comma-separated line: **the field name is not in the record**, only its position is. The only thing that knows the first column is time is the parser. 15 of 24 source fields fit the schema, 9 remain leftover. A leftover field is not a total loss — the raw record is kept — but it **cannot be queried** through the shared schema; since the process ID or byte count finds no place in the schema, no correlation rule can use it as a criterion. The real loss is in the `empty` column. Four schema fields are empty in the network-flow record: host, actor, action, and result. The heaviest of these is actor, because every correlation operation in this course will run along the actor axis. **None of the 404 flow records say who did it.** In the endpoint record, the result field is empty: it writes that a process ran, not whether it was permitted. The last two columns separate two different problems with time. Resolution means two sources round events to the second; two records in the same second lose their order. Write lag is more decisive: the same event lands in the record one minute later at the endpoint, five at network flow, eight at the control plane. The line sees the three records **spread across seven minutes** and cannot tell whether that spread is real or comes from transmission. #### Filling the Field That Does not Map The actor field cannot be left empty, because if it stays empty the three sources never link to each other. There are two ways to fill it, and both rely on a table outside the log: a static mapping that converts the portal ID to an account, and a time-sensitive lookup that ties the address to the host and the host to the session at that moment. ```js // merge.mjs — the field that does not map is filled with lookup tables, then records are collected. import { EVENT, LABEL, SHARED, HOST, ADDRESS, ACCOUNT } from "./source.mjs"; // Lookup tables are model data: absent from the log, they come from the asset inventory. const addrToHost = new Map(ADDRESS.map((a, i) => [a, HOST[i]])); const portalToAccount = new Map(ACCOUNT.slice(0, 28).map((h, i) => [`ab-${4401 + i}`, h])); const endpointEvents = EVENT.filter((o) => o.source === "endpoint"); // The network-flow record has no actor: an attempt is made to tie the address to the host, and // the host to the account in an endpoint record within a +/- W-minute window. function assignActor(window) { const resolved = new Array(EVENT.length).fill(null); let assigned = 0, ambiguous = 0, empty = 0, portalConverted = 0; for (const [i, o] of EVENT.entries()) { if (o.source === "endpoint") { resolved[i] = o.actor; continue; } if (o.source === "control-plane") { resolved[i] = portalToAccount.get(o.actor) || null; if (resolved[i]) portalConverted++; continue; } const host = addrToHost.get(o.address); const candidates = [...new Set(endpointEvents.filter((u) => u.host === host && Math.abs(u.t - o.t) <= window * 60e3).map((u) => u.actor))]; if (candidates.length === 1) { resolved[i] = candidates[0]; assigned++; } else if (candidates.length > 1) ambiguous++; else empty++; } return { resolved, assigned, ambiguous, empty, portalConverted }; } // Records with the actor field filled are clustered along the account axis, by a // W-minute gap rule. function cluster(resolved, window) { const groups = new Map(); for (const [i, a] of resolved.entries()) if (a) (groups.get(a) || groups.set(a, []).get(a)).push(i); const clusters = []; for (const [, l] of groups) { let s = [l[0]]; for (const i of l.slice(1)) { if (EVENT[i].t - EVENT[s[s.length - 1]].t <= window * 60e3) s.push(i); else { clusters.push(s); s = [i]; } } clusters.push(s); } const multiSource = clusters.filter((s) => new Set(s.map((i) => EVENT[i].source)).size >= 2); const collected = new Set(), mixed = multiSource.filter((s) => { const labels = new Set(s.map((i) => LABEL[i]).filter((x) => x >= 0)); for (const x of labels) if (s.filter((i) => LABEL[i] === x).length >= 2) collected.add(x); return labels.size >= 2; }); const threeSource = [...collected].filter((x) => multiSource.some((s) => new Set(s.filter((i) => LABEL[i] === x) .map((i) => EVENT[i].source)).size === 3)).length; return { clusters: multiSource.length, collected: collected.size, threeSource, mixed: mixed.length }; } const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([12, 11, 11, 9, 9, 15, 13, 11]); const nf = EVENT.filter((o) => o.source === "network-flow").length; console.log(`${EVENT.length} normalized records; the actor field is empty in ${nf} network-flow ` + `records, and in ${EVENT.filter((o) => o.source === "control-plane").length} control-plane ` + `records the actor is written as a portal ID`); const twoPlus = SHARED.filter((o) => [o.ep, o.nf, o.cp].filter(Boolean).length >= 2).length; console.log(`shared events the model knows about: ${SHARED.length}, of which ${twoPlus} leave a ` + `trace in at least two sources (the line does not see this)`); console.log(); console.log(yz(["window", "assigned", "ambiguous", "empty", "clusters", "collected", "three-source", "mixed"])); for (const w of [1, 5, 15, 60]) { const a = assignActor(w), k = cluster(a.resolved, w); console.log(yz([`${w} min`, a.assigned, a.ambiguous, a.empty, k.clusters, `${k.collected}/${twoPlus}`, k.threeSource, k.mixed])); } const a5 = assignActor(5); console.log(`\ncontrol-plane records where the portal ID converts to an account: ` + `${a5.portalConverted}; unconverted ${EVENT.filter((o) => o.source === "control-plane").length - a5.portalConverted}`); console.log(`records with the actor field filled: ${a5.resolved.filter(Boolean).length}/${EVENT.length}`); ``` ``` 787 normalized records; the actor field is empty in 404 network-flow records, and in 139 control-plane records the actor is written as a portal ID shared events the model knows about: 30, of which 26 leave a trace in at least two sources (the line does not see this) window assigned ambiguous empty clusters collected three-source mixed 1 min 3 0 401 3 0/26 0 0 5 min 190 2 212 184 21/26 13 1 15 min 195 7 202 179 21/26 13 1 60 min 208 39 157 145 20/26 12 1 control-plane records where the portal ID converts to an account: 115; unconverted 24 records with the actor field filled: 549/787 ``` The static mapping is the easy one, and even it is not complete: 115 of 139 control-plane records convert to an account, **24 cannot convert**. The six portal IDs with no match are service identities and are not tied to a person in the inventory. These 24 records show the actor field as filled in the schema, but the value inside it never matches the other two sources — a filled field is more misleading than an empty one. The time-sensitive lookup, by contrast, depends directly on the window, and the table shows the window's two-sided effect. In a one-minute window, an actor can be assigned to only 3 of the 404 flow records; since the sources' write lags are 1 and 5 minutes, there is already a four-minute gap between a flow record and an endpoint record. Widening to five minutes makes assignment jump to 190. At sixty minutes, assignment gains only 18 more records, while **ambiguous records climb from 2 to 39**: as the window widens, more than one account's session becomes visible at the same host, and it becomes impossible to say whose flow it is. The last two columns show what the measure amounts to. Of the 26 fictional events that leave a trace in at least two sources, 21 are collected in a five-minute window, 13 of them with all three sources. In a one-minute window, **none** are collected — that zero comes not from missing data, but from the sources writing at different moments. All five of the uncollected events are missing **an endpoint record**; the remaining network-flow and control-plane records are three minutes apart, so the window is not the problem. The obstacle is this: the network-flow record's actor field is empty, and the lookup that fills it comes from the endpoint record. The count of mixed clusters stays at one: in this model, the cost of widening the window is not false merging, but ambiguity. In a dense environment, that column grows. #### Summary - 15 of the three sources' 24 fields map to the eight-field schema, 9 remain leftover; a leftover field is kept but cannot be queried through the shared schema, so it cannot serve as a criterion for any rule. - Four schema fields are empty in the network-flow record, and the heaviest is actor: none of the 404 records say who did it. In the endpoint record, the result field is empty. - The same event lands in the record 1 minute later at the endpoint, 5 at network flow, 8 at the control plane; the line sees the three records spread across seven minutes and cannot tell whether that spread is real or comes from transmission. - The static mapping that converts a portal ID to an account converts 115 of 139 records, and cannot convert 24; for service identities with no match, the actor field looks filled but matches nothing. - The window works both ways: 3 assignments and zero collected events at one minute, 190 assignments and 21/26 at five minutes, 208 assignments at sixty minutes but ambiguous records climbing from 2 to 39. #### Next Step In this lesson, clustering was done with a single criterion: the same actor and a time window. This criterion produced 184 clusters, and only 21 of them correspond to an event the model knows about; the rest is the same account's ordinary work over the course of a day. A security information and event management system does not cluster this way; it writes a condition sequence — which field from which source, in what order, after what count. The next lesson runs a correlation rule over the stream normalized in this lesson and produces two numbers: how many events the rule merges into a single alert, and **what it never sees at all** outside the ones it merges. ### Security Information and Event Management Source: https://academia.sh/en/courses/security-operations/security-information-and-event-management A three-condition correlation rule runs against a fictional 3,083-record event stream: the rule produces 10 alerts, and these alerts merge 161 raw events, 16.1 events per alert. Three of five fictional campaigns are caught; one stays below the threshold, one never appears because its third step exists only in network flow. The funnel narrows from 3,083 records to 12 candidates, from 12 candidates to 10 alerts, from 10 alerts to 3 decisions, and takes 43 minutes from the first record landing. In the threshold, window and lag sweep, no setting brings in all five of the five campaigns; the best row is 4/5 and 1 false positive. The previous lesson clustered records with a single criterion: the same actor and a time window. That criterion produced 184 clusters, and only 21 of them corresponded to an event the model actually knew about. A security information and event management system does not cluster this way; it writes a **condition sequence** — which field from which source, in which order, after which count. This lesson runs such a rule over a normalized stream and produces two numbers: how many raw events the rule merges into a single alert, and what it never sees outside what it merges. The stream below is a **model**, and events enter only **as records**; nowhere does it write how an attack step is carried out. **Repeated password attempts against a set of accounts, without written authorization and a defined scope, is both illegal and unprofessional**; what is measured here is the record the attempt leaves behind. - **DI13.** The stream covers one day: 120 personnel accounts, 6 service identities and four event types. A service identity cannot change permissions, so that record **never exists** for it. - **DI14.** Five fictional campaigns are given as record sequences. The label is the model's ground truth and is **never written to the stream**; the rule cannot see it. - **DI15.** Correlation rule R1 has three conditions: at least `E` failed records at the same actor within `P` minutes, then a success, then a permission change within `G` minutes. - **DI16.** An alert's **evidence** is the set of raw events the rule held on to while producing it. The merge measure is the size of this set. - **DI17.** The chain's fixed lags are model data: transmission and normalization take 3 minutes, the alert queue 14 minutes, analyst review 9 minutes per alert. #### A Correlation Rule's State Machine ```js // flow.mjs — fictional event stream and a correlation rule's state machine. IS A MODEL. // Events enter as RECORDS; not the attack step but the record the step leaves behind is modeled. export const SEED = 20260403; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = generator(SEED); const randInt = (n) => Math.floor(r() * n); const ACCOUNT = Array.from({ length: 120 }, (_, i) => `sh-${301 + i}`); const SERVICE = Array.from({ length: 6 }, (_, i) => `sv-${11 + i}`); const raw = []; // {minute, actor, type, label} — label is the model's ground truth const add = (t, a, type, e) => raw.push({ t: Math.round(t), actor: a, type, label: e }); const times = (n, f) => { for (let k = 0; k < n; k++) f(); }; // Background: routine daily work. A service identity cannot change permissions, so that record // never exists for it. for (const h of ACCOUNT) { times(2 + randInt(6), () => add(randInt(1440), h, "identity.success", "routine")); times(randInt(9), () => add(randInt(1440), h, "identity.failure", "routine")); times(randInt(4), () => add(randInt(1440), h, "permission.change", "routine")); times(4 + randInt(14), () => add(randInt(1440), h, "flow.outbound", "routine")); } for (const h of SERVICE) { times(6 + randInt(10), () => add(randInt(1440), h, "identity.success", "routine")); times(randInt(20), () => add(randInt(1440), h, "identity.failure", "routine")); times(20 + randInt(40), () => add(randInt(1440), h, "flow.outbound", "routine")); } // Dense password reset: a legitimate burst that crosses the threshold, followed by a login and // permission work. for (const h of ACCOUNT.slice(0, 9)) { let t = 420 + randInt(600); times(6 + randInt(5), () => add(t += 1 + r() * 3, h, "identity.failure", "reset")); add(t += 12, h, "identity.success", "reset"); if (r() < 0.55) add(t + 5 + randInt(30), h, "permission.change", "reset"); } // Five fictional campaigns: [name, actor, failure-record count, third-step type, lag min]. // Only the RECORD sequence is given; the method is not described. const CAMPAIGN = [["k1", ACCOUNT[8], 40, "permission.change", 25], ["k2", ACCOUNT[21], 22, "permission.change", 48], ["k3", ACCOUNT[55], 61, "permission.change", 9], ["k4", SERVICE[2], 33, "flow.outbound", 20], // third step only in network flow ["k5", ACCOUNT[96], 4, "permission.change", 35]]; // failure count below threshold for (const [name, h, n, third, lag] of CAMPAIGN) { let t = 60 + randInt(1200); times(n, () => add(t += 0.4 + r() * 1.2, h, "identity.failure", name)); add(t += 3, h, "identity.success", name); add(t + lag, h, third, name); } raw.sort((a, b) => a.t - b.t); export const EVENT = raw.map(({ t, actor, type }) => ({ t, actor, type })); export const LABEL = raw.map((o) => o.label); // the pipeline never SEES this export const NAME = CAMPAIGN.map(([a]) => a); // Correlation rule R1 (state machine): at the same actor, >=E failed records within P // minutes, then a success, then a permission change within G minutes. export function R1({ threshold = 5, window = 30, lag = 60 } = {}) { const state = new Map(), alerts = [], candidates = new Set(), passed = new Set(); for (const [i, o] of EVENT.entries()) { const d = state.get(o.actor) || { failed: [], open: null, evidence: [] }; state.set(o.actor, d); if (o.type === "identity.failure") { d.failed = d.failed.filter((x) => EVENT[x].t > o.t - window).concat(i); if (d.failed.length >= threshold) candidates.add(o.actor); } else if (o.type === "identity.success" && d.failed.length >= threshold) { d.open = o.t; d.evidence = [...d.failed, i]; passed.add(o.actor); } else if (o.type === "permission.change" && d.open !== null && o.t - d.open <= lag) { alerts.push({ actor: o.actor, t: o.t, evidence: [...d.evidence, i] }); d.open = null; d.failed = []; } } return { alerts, candidates: candidates.size, passed: passed.size }; } export const catches = (alerts, name) => alerts.find((x) => x.evidence.some((i) => LABEL[i] === name)); export const falsePositives = (alerts) => alerts.filter((x) => !x.evidence.some((i) => NAME.includes(LABEL[i]))); if (import.meta.url === `file://${process.argv[1]}`) { const s = R1(), merged = s.alerts.reduce((a, u) => a + u.evidence.length, 0); console.log(`model (seed ${SEED}): ${EVENT.length} events, ${ACCOUNT.length} personnel and ` + `${SERVICE.length} service accounts, ${CAMPAIGN.length} fictional campaigns (as records)`); console.log(`R1 (threshold 5 / window 30 min / lag 60 min): ${s.alerts.length} alerts; ` + `these alerts merge ${merged} raw events, per alert ` + `${(merged / s.alerts.length).toFixed(1)}`); const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([11, 9, 18, 11, 15, 32]); console.log(); console.log(yz(["campaign", "records", "third step", "caught", "merged events", "reason not seen"])); for (const [name, , n, third] of CAMPAIGN) { const u = catches(s.alerts, name); console.log(yz([name, n + 2, third, u ? "yes" : "no", u ? u.evidence.length : 0, u ? "-" : third === "flow.outbound" ? "third step only in network flow" : "failure count below threshold"])); } console.log(`\ncandidate actors ${s.candidates}, passed second condition ${s.passed}, alerts ` + `${s.alerts.length}; ${falsePositives(s.alerts).length} of these contain no campaign ` + `record`); } ``` ``` model (seed 20260403): 3083 events, 120 personnel and 6 service accounts, 5 fictional campaigns (as records) R1 (threshold 5 / window 30 min / lag 60 min): 10 alerts; these alerts merge 161 raw events, per alert 16.1 campaign records third step caught merged events reason not seen k1 42 permission.change yes 32 - k2 24 permission.change yes 24 - k3 63 permission.change yes 32 - k4 35 flow.outbound no 0 third step only in network flow k5 6 permission.change no 0 failure count below threshold candidate actors 12, passed second condition 12, alerts 10; 7 of these contain no campaign record ``` The merge ratio is the rule's most visible work: 10 alerts gather 161 raw events, 16.1 records per alert. The analyst faces not 161 lines but 10 items, each with an evidence set underneath it. This is the number that replaces the previous lesson's 184 clusters, and it is the rule's **reason for existing**. The third column shows a detail: k3 has 63 records, but only 32 events merge into the alert. The thirty-minute window drops older failures, because the counter is a sliding window. The evidence set is not the whole event, but **the part the rule saw**; the rest is searched by hand during review. The bottom two rows are the lesson's real result. k5 stays below the threshold: four failed records do not cross the threshold of five, and the rule never wakes up. k4 is invisible for a different reason — its third step is not a permission change but an outbound network connection, and the rule's third condition never asks about that type. A permission-change record **never occurs** for a service identity; the rule cannot fire for that actor, by definition. These two rows connect directly to the previous lesson's measure. The rule can only ask about fields that exist in the schema; k4's third record fits the shared schema, but the rule's condition sequence has no place for that type. The gap does not come from the source not being collected — it comes from **which field the rule asks about** — and this distinction is decisive in counting detection coverage. The last line gives the cost: 7 of the 10 alerts contain no campaign record at all. Most of these are the dense password-reset cluster — a legitimate burst of failures that crosses the threshold, followed by a login, followed by routine permission work. The rule's condition sequence cannot tell this sequence apart from a campaign, because both leave the same three records in the same order. #### From Record to Decision: Funnel and Minutes ```js // funnel.mjs — the chain from record to decision: how many candidates are eliminated and how // many minutes are added at each step. import { EVENT, R1, NAME, catches, falsePositives } from "./flow.mjs"; // The chain's fixed lags are model DATA, not a measurement (minutes). const TRANSMISSION = 3, QUEUE = 14, REVIEW = 9; const s = R1(), fp = falsePositives(s.alerts); // The average time the rule has to wait for the third step. const wait = s.alerts.reduce((a, u) => a + (u.t - EVENT[u.evidence[u.evidence.length - 2]].t), 0) / s.alerts.length; const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yh = fmt([48, 12, 11, 13]); console.log(`${EVENT.length} raw events, ${s.alerts.length} alerts, ` + `${s.alerts.length - fp.length} of these contain a campaign record`); console.log(); console.log(yh(["step", "remaining", "eliminated", "added min"])); const step = [["raw records", EVENT.length, 0], ["transmission and normalization", EVENT.length, TRANSMISSION], ["condition 1: 5 failures / 30 min", s.candidates, 0], ["condition 2: then a success", s.passed, 0], ["condition 3: a permission change within 60 min", s.alerts.length, Math.round(wait)], ["alert queue", s.alerts.length, QUEUE], ["analyst review", s.alerts.length - fp.length, REVIEW]]; let previous = EVENT.length, total = 0; for (const [name, remaining, mins] of step) { total += mins; console.log(yh([name, remaining, previous - remaining, mins])); previous = remaining; } console.log(`total: ${EVENT.length} records to ${s.alerts.length - fp.length} decisions, ` + `${total} minutes after the first record lands`); const yz = fmt([10, 10, 10, 8, 8, 12, 13, 13]); console.log(); console.log(yz(["threshold", "window", "lag", "cand.", "alerts", "caught", "false pos.", "review min"])); for (const [threshold, window, lag] of [[5, 30, 60], [3, 30, 60], [8, 30, 60], [5, 5, 60], [3, 5, 60], [5, 30, 15], [5, 30, 240]]) { const k = R1({ threshold, window, lag }); const y = NAME.filter((a) => catches(k.alerts, a)).length; const n = falsePositives(k.alerts).length; console.log(yz([threshold, `${window} min`, `${lag} min`, k.candidates, k.alerts.length, `${y}/${NAME.length}`, n, n * REVIEW])); } ``` ``` 3083 raw events, 10 alerts, 3 of these contain a campaign record step remaining eliminated added min raw records 3083 0 0 transmission and normalization 3083 0 3 condition 1: 5 failures / 30 min 12 3071 0 condition 2: then a success 12 0 0 condition 3: a permission change within 60 min 10 2 17 alert queue 10 0 14 analyst review 3 7 9 total: 3083 records to 3 decisions, 43 minutes after the first record lands threshold window lag cand. alerts caught false pos. review min 5 30 min 60 min 12 10 3/5 7 63 3 30 min 60 min 18 11 4/5 7 63 8 30 min 60 min 10 8 3/5 5 45 5 5 min 60 min 4 2 2/5 0 0 3 5 min 60 min 12 5 4/5 1 9 5 30 min 15 min 12 6 1/5 5 45 5 30 min 240 min 12 11 3/5 8 72 ``` The funnel's first row does all the eliminating in a single step: the first condition narrows 3,083 events down to 12 candidates. The second condition **eliminates nothing** — all twelve actors that crossed the threshold eventually log in. What keeps the rule narrow is not, as one might think, the three conditions coming one after another — it is the first condition; the second condition only marks the start of the chain. The minutes column shows a less obvious cost. The 17 minutes the third condition adds is not a processing delay — the rule **has to wait**, because whether the third record arrives is only known once the lag window closes. The total of 43 minutes is this rule's time to detect, and 17 of it comes from the rule's own definition, 14 from the queue, 9 from review. Transmission and normalization account for only 3 minutes; that is the most any effort to speed up the pipeline could gain. The bottom table draws the parameters' boundary. Pulling the threshold down to three brings in k5, and caught becomes 4/5, but the candidate count rises from 12 to 18. Narrowing the window to five minutes zeroes out the false positive and drops the review load from 63 minutes to zero — in exchange, k2 disappears, because its records are spread across the thirty-minute window. The best row is the combination of threshold 3 and window 5: 4/5 and a single false positive. No row gives 5/5. k4 stays invisible under every setting, because what is missing is not the threshold or the window — it is the **source**: the third step's record exists only in network flow data, and R1 never asks about that type. A parameter sweep cannot step **outside a rule's own definition**. #### Summary - The three-condition correlation rule narrows 3,083 raw events down to 10 alerts, and these alerts merge 161 events; 16.1 records per alert reach the analyst as a single item. - The evidence set is not the whole event: only 32 of k3's 63 records enter the alert, because the counter is a sliding window and drops the older ones. - Three of the five campaigns are caught; k5 stays below the threshold, and k4 never appears — by the rule's own definition — because it leaves its third step only in network flow. - 7 of the 10 alerts contain no campaign record; the dense password-reset cluster leaves the same three records in the same order as a campaign, so the condition sequence cannot tell them apart. - 43 minutes pass from record to decision, and of that, 17 comes from the rule waiting for its third step, 14 from the queue, 9 from review; transmission and normalization account for 3 minutes. #### Next Step k4's third step existed in no identity or management record; it existed only as an outbound connection, in network flow data. That source is structurally different from the other two: a flow record does not write the connection's **content**, only who talked to whom, for how long, and how many bytes. The next lesson turns this constraint into a measurement — which question can be answered without content, which cannot, and how the pattern readable from metadata in an encrypted connection separates out a number of items along an axis the other sources never see at all. ### Network Flow Data Source: https://academia.sh/en/courses/security-operations/network-flow-data Four metadata-only metrics run against 2,979 fictional flow records: regular interval flags 7 pairs and 6 of them are meter transfer, exit ratio flags 23 and 22 of them are nightly backup, new destination flags 7 and 6 of them are a new work item, long session flags 6 and 5 of them are an admin session. All four of the fictional items get flagged, but 39 of the 43 total pairs are legitimate. Of ten questions asked against the same record set, 6 are answered from metadata alone; what was sent, which account did it, whether the request was accepted, and which process started it stay unanswered. If everything flagged were cut, 630 flows and 8.8 GB of legitimate work would stop. The previous lesson's k4 item appeared in no identity or management record; its third step existed only as an outbound connection, in network flow data. That source is **structurally** different from the other two. A flow record does not write the connection's content; it writes who talked to whom, on which port, for how long, and how many bytes. This lesson turns that constraint into a measurement. The question is not "is flow data useful" but **which question can be answered without content** — and, in return, how many legitimate connections get flagged. The record set below is a **model**; the external addresses are drawn from blocks reserved for documentation and do not correspond to a real endpoint. - **DI18.** A flow record has seven fields: time, host, destination, port, duration, bytes sent, bytes received. **There is no content field**, and this is not a configuration gap — it is the source's definition. - **DI19.** The fictional network has four legitimate traffic patterns: personnel browsing, meter collectors' fifteen-minute transfers, nightly backup, and long admin sessions. - **DI20.** Four fictional items are given only **as records**: a regularly spaced connection sequence, a high-exit-ratio transfer, a destination absent from the baseline, and a single long session. - **DI21.** A metric operates on a `(host, destination, port)` triple and looks only at metadata. The label is the model's ground truth and is not written to the record. - **DI22.** The baseline is the set of `(destination, port)` pairs seen on previous days, and it is given as model data. #### Four Metadata-Only Metrics ```js // flowdata.mjs — fictional network flow records and four metadata-only metrics. THIS IS A MODEL. // External addresses come from blocks reserved for documentation; there is no real endpoint or organization. export const SEED = 20260404; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = generator(SEED); const randInt = (n) => Math.floor(r() * n); const ENDPOINT = Array.from({ length: 40 }, (_, i) => `uc-${101 + i}`); const COLLECTOR = Array.from({ length: 6 }, (_, i) => `tp-${11 + i}`); const INTERNAL = Array.from({ length: 8 }, (_, i) => `10.90.1.${20 + i}`); const EXTERNAL = Array.from({ length: 30 }, (_, i) => `198.51.100.${10 + i}`); // Flow record: the connection has no CONTENT. Only who, with whom, how much, how long. const raw = []; const add = (t, host, dest, port, duration, sent, received, label) => raw.push({ t: Math.round(t), host, dest, port, duration, sent, received, label }); // Personnel browsing: bytes received are much larger than bytes sent, destination varies. for (const u of ENDPOINT) for (let k = 0, n = 30 + randInt(60); k < n; k++) add(randInt(1440), u, EXTERNAL[randInt(30)], 443, 1 + randInt(90), 800 + randInt(9000), 20000 + randInt(900000), "browsing"); // Meter collectors: a small and REGULAR transfer to an internal destination every fifteen minutes. for (const [j, tp] of COLLECTOR.entries()) for (let t = j * 2; t < 1440; t += 15) add(t, tp, INTERNAL[j % 8], 5671, 2 + randInt(3), 40000 + randInt(4000), 900 + randInt(400), "measurement-transfer"); // Nightly backup: large sent, small received, once a day. for (const u of ENDPOINT.slice(0, 22)) add(120 + randInt(120), u, INTERNAL[6], 8443, 40 + randInt(40), 3e8 + randInt(2e8), 5000 + randInt(4000), "backup"); // Long admin sessions. for (const u of ENDPOINT.slice(0, 5)) add(540 + randInt(120), u, INTERNAL[7], 22, 150 + randInt(180), 90000 + randInt(50000), 300000 + randInt(200000), "admin-session"); // Four fictional items. Only the RECORD is given; the method is not described. const ITEM = ["d1-regular-interval", "d2-exit-ratio", "d3-new-destination", "d4-long-session"]; for (let t = 30, k = 0; k < 96; k++, t += 15) // d1 add(t + (r() < 0.5 ? 0 : 1), ENDPOINT[7], "203.0.113.24", 443, 1 + randInt(2), 1200 + randInt(400), 600 + randInt(300), ITEM[0]); for (let k = 0, t = 800; k < 12; k++) // d2 add(t += 4 + randInt(9), ENDPOINT[19], "203.0.113.31", 443, 20 + randInt(40), 4e7 + randInt(2e7), 3000 + randInt(2000), ITEM[1]); for (let k = 0; k < 3; k++) // d3 add(1010 + k * 7, ENDPOINT[31], "203.0.113.9", 9443, 3 + randInt(6), 9000 + randInt(4000), 12000 + randInt(9000), ITEM[2]); add(300, ENDPOINT[24], INTERNAL[3], 8443, 415, 2200000, 6100000, ITEM[3]); // d4 // New work item: three legitimate external destinations seen for the first time during the day. for (const [j, u] of ENDPOINT.slice(30, 36).entries()) for (let k = 0, n = 2 + randInt(5); k < n; k++) add(600 + randInt(500), u, `192.0.2.${40 + (j % 3)}`, 443, 2 + randInt(30), 3000 + randInt(9000), 40000 + randInt(300000), "new-work-item"); raw.sort((a, b) => a.t - b.t); export const FLOW = raw.map(({ label, ...a }, i) => ({ ...a, i })); export const LABEL = raw.map((a) => a.label); // ground truth of the model; metrics Do not see it // Baseline: (destination, port) pairs seen on previous days (model data). export const BASELINE = new Set([...INTERNAL.flatMap((h) => [`${h}|5671`, `${h}|8443`, `${h}|22`]), ...EXTERNAL.map((h) => `${h}|443`), "203.0.113.24|443", "203.0.113.31|443"]); // Four metrics: they look only at metadata, not at content. const pairs = () => { const m = new Map(); for (const a of FLOW) { const k = `${a.host}|${a.dest}|${a.port}`; (m.get(k) || m.set(k, []).get(k)).push(a); } return m; }; export const METRIC = { "regular interval": () => [...pairs()].filter(([, l]) => { if (l.length < 12) return false; const d = l.slice(1).map((a, i) => a.t - l[i].t); const o = d.reduce((x, y) => x + y, 0) / d.length; return o > 0 && Math.sqrt(d.reduce((x, y) => x + (y - o) ** 2, 0) / d.length) / o < 0.15; }), "exit ratio": () => [...pairs()].filter(([, l]) => { const s = l.reduce((x, a) => x + a.sent, 0), rc = l.reduce((x, b) => x + b.received, 0); return s > 5e6 && s / rc > 20; }), "new destination": () => [...pairs()].filter(([k]) => { const [, h, p] = k.split("|"); return !BASELINE.has(`${h}|${p}`); }), "long session": () => [...pairs()].filter(([, l]) => l.some((a) => a.duration > 120)), }; if (import.meta.url === `file://${process.argv[1]}`) { const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const line = fmt([16, 12, 10, 13, 34]); console.log(`model (seed ${SEED}): ${FLOW.length} flow records, ` + `${new Set(FLOW.map((a) => a.host)).size} hosts; no content field in the record`); console.log(`example: ${JSON.stringify(FLOW[LABEL.indexOf(ITEM[0])])}`); console.log(); console.log(line(["metric", "flagged", "item", "false pos.", "source of false positives"])); for (const [name, f] of Object.entries(METRIC)) { const l = f(); const lab = ([, x]) => LABEL[x[0].i]; const correct = l.filter((c) => ITEM.includes(lab(c))); const fp = l.filter((c) => !ITEM.includes(lab(c))); console.log(line([name, l.length, correct.map((c) => lab(c).slice(0, 2)).join(",") || "-", fp.length, [...new Set(fp.map(lab))].join(", ") || "-"])); } const all = new Set(Object.values(METRIC).flatMap((f) => f().map(([, x]) => LABEL[x[0].i]) .filter((e) => ITEM.includes(e)))); const total = Object.values(METRIC).reduce((a, f) => a + f().length, 0); console.log(`\n${all.size} of the four fictional items are flagged by at least one metric; ` + `the four metrics together flag ${total} pairs`); } ``` ``` model (seed 20260404): 2979 flow records, 46 hosts; no content field in the record example: {"t":31,"host":"uc-108","dest":"203.0.113.24","port":443,"duration":1,"sent":1402,"received":676,"i":70} metric flagged item false pos. source of false positives regular interval 7 d1 6 measurement-transfer exit ratio 23 d2 22 backup new destination 7 d3 6 new-work-item long session 6 d4 5 admin-session 4 of the four fictional items are flagged by at least one metric; the four metrics together flag 43 pairs ``` The example record shows the lesson's limit: there are seven fields and none of them says what was carried. Against that, the four metrics flag all four of the fictional items. A pattern can be seen without seeing the content; what is measured is not the content itself but the connection's **shape**. The shape's discriminative power, though, is low, and the table's last two columns record that. The regular-interval metric flags seven pairs, six of which are the meter collectors' fifteen-minute transfer. That same regularity — a fixed interval, small and equal-sized packets — is this network's **normal work**; the fictional item has exactly the shape of normal work. The exit-ratio metric is even worse: twenty-two of twenty-three pairs are nightly backup. The new-destination metric flags six legitimate items, because all three new work connections are also seen for the first time during the day. One detail matters here: d1's destination **is** in the baseline, so the new-destination metric never sees it at all. No single metric catches all four items at once; coverage comes from the union of the four metrics, and that union flags 43 pairs. #### The Question Answered Without Content ```js // question.mjs — which questions a flow record can and cannot answer, then the cost of cutting it off. import { FLOW, LABEL, BASELINE, METRIC } from "./flowdata.mjs"; const pair = FLOW.filter((a) => LABEL[a.i] === "d1-regular-interval"); const bytes = (n) => (n > 1e9 ? `${(n / 1e9).toFixed(1)} GB` : `${(n / 1e6).toFixed(1)} MB`); const median = (v) => v.slice().sort((a, b) => a - b)[Math.floor(v.length / 2)]; // Each question is a function: it returns the answer if it can produce one from the flow record, null if it cannot. const QUESTION = [ ["which two endpoints talked", (l) => `${l[0].host} -> ${l[0].dest}:${l[0].port}`], ["how many times did it connect", (l) => `${l.length} times`], ["at what intervals", (l) => `${median(l.slice(1).map((x, i) => x.t - l[i].t))} min`], ["how much data went out", (l) => bytes(l.reduce((a, x) => a + x.sent, 0))], ["how long did it last", (l) => `at most ${Math.max(...l.map((x) => x.duration))} min`], ["was the destination seen before", (l) => (BASELINE.has(`${l[0].dest}|${l[0].port}`) ? "yes" : "no")], ["what was sent", () => null], ["which account did it", () => null], ["was the request accepted", () => null], ["which process started it", () => null], ]; const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const line = fmt([34, 16, 28]); console.log(`d1 item: ${pair.length} flow records. The same record set is asked ten questions.`); console.log(); console.log(line(["question", "answerable", "answer"])); for (const [q, f] of QUESTION) { const y = f(pair); console.log(line([q, y === null ? "no" : "yes", y === null ? "-" : y])); } const answerable = QUESTION.filter(([, f]) => f(pair) !== null).length; console.log(`\n${answerable} of the ten questions are answered from metadata alone, ` + `${QUESTION.length - answerable} need another source`); // Early decision: legitimate work cut if a connection is severed based only on the flow metric. const SOURCE = { "measurement-transfer": ["meter collection stops", 0], "backup": ["nightly backup", 0], "new-work-item": ["new work connection", 0], "admin-session": ["admin access", 0] }; const flagged = new Map(); for (const [, f] of Object.entries(METRIC)) for (const [k, l] of f()) if (!flagged.has(k)) flagged.set(k, l); let legitPairs = 0, legitRecords = 0, legitBytes = 0; for (const [, l] of flagged) { const e = LABEL[l[0].i]; if (!SOURCE[e]) continue; legitPairs++; legitRecords += l.length; legitBytes += l.reduce((a, x) => a + x.sent + x.received, 0); SOURCE[e][1]++; } const line2 = fmt([22, 10, 30]); console.log(`\n${flagged.size} distinct pairs are flagged, ${legitPairs} of them are legitimate; ` + `cutting them stops ${legitRecords} flows and ${bytes(legitBytes)}`); console.log(line2(["work type cut", "pairs", "work stopped"])); for (const [e, [ne, n]] of Object.entries(SOURCE)) console.log(line2([e, n, ne])); ``` ``` d1 item: 96 flow records. The same record set is asked ten questions. question answerable answer which two endpoints talked yes uc-108 -> 203.0.113.24:443 how many times did it connect yes 96 times at what intervals yes 15 min how much data went out yes 0.1 MB how long did it last yes at most 2 min was the destination seen before yes yes what was sent no - which account did it no - was the request accepted no - which process started it no - 6 of the ten questions are answered from metadata alone, 4 need another source 43 distinct pairs are flagged, 39 of them are legitimate; cutting them stops 630 flows and 8.8 GB work type cut pairs work stopped measurement-transfer 6 meter collection stops backup 22 nightly backup new-work-item 6 new work connection admin-session 5 admin access ``` The question table turns network flow data's limit into a number: six of ten questions get answered, four stay unanswered. The answered ones alone are not enough to make a **decision**. `uc-108` connected to the same destination 96 times every fifteen minutes and sent a total of 0.1 megabytes; that description fits both a recurring health check and something else. The fields that would tell them apart — which account, which process, whether the request was accepted — **are not** in the flow record. Each of the four unanswered questions names a specific source. The question of which account did it requires the address mapping from the second lesson, and that mapping could answer 190 of 404 records in a five-minute window. The question of which process started it wants the endpoint record; whether the request was accepted wants the control-plane record. Network flow data is not a **decision source**, it is a **routing source**: it says which host and which time window to look at. The last table writes the cost of that early decision. Of the 43 flagged pairs, 39 are legitimate, and cutting all of them stops 630 flows and 8.8 gigabytes of work. That is not a flat number: six of the cut pairs are meter-collection transfers, and when that transfer stops, measurement data never reaches the center at all. Twenty-two pairs are nightly backup; going a day without a backup is a loss that directly affects the recovery decision later in the course. #### Summary - A flow record has seven fields and no content field; despite that, four metrics that look only at shape flag all four fictional items. - Discriminative power is low: regular interval flags meter transfer in 6 of 7 pairs, exit ratio flags nightly backup in 22 of 23 pairs, new destination flags a new work connection in 6 of 7 pairs. - No single metric catches all four items at once; because d1's destination is in the baseline, the new-destination metric never sees it, and coverage comes from the union of the four metrics. - 6 of ten questions are answered from metadata alone; the four questions of what was sent, which account did it, whether the request was accepted, and which process started it need another source. - Of the 43 flagged pairs, 39 are legitimate; cutting all of them stops 630 flows and 8.8 gigabytes of work, six pairs of which are meter-collection transfers. #### Next Step This lesson's four metrics ran with fixed thresholds, and the result was that 39 of 43 flags turned out legitimate. Tightening the thresholds is the first fix that comes to mind: making the coefficient of variation 0.05 instead of 0.15, requiring an exit ratio of two hundred instead of twenty. The next lesson runs that as a sweep and plots two curves on the same axis — as the threshold tightens, false positives drop, but missed detections rise. What is measured is not which threshold is right but the point where the two curves **do not cross**: in the band where the fictional items' scores overlap with legitimate work's scores, no threshold can reduce both at once, and how many items that band contains can be counted. ### False Positives and False Negatives Source: https://academia.sh/en/courses/security-operations/false-positive-and-false-negative 20 of 900 candidate items are fictional, and the score distributions from four attributes overlap in the 37–88 band: that band holds 157 legit and 17 fictional items, and no threshold can separate the two at once. At threshold 40, false positive is 124 and missed is 1; at threshold 80, false positive is 1 and missed is 11; even at the threshold 70 that minimizes total error, 4 false positives and 4 missed remain. A daily budget of 144 analyst-minutes nails the threshold to 72: 16 alerts, 7 missed items, and 42 items reached. A fifth attribute narrows the band to 49–75 and to 15 legit and 7 items, pulling the budget threshold down to 66, but a zero-zero threshold is still nowhere to be found. The previous lesson's four metrics ran with fixed thresholds, and the result was that 39 of 43 flags turned out legitimate. Tightening the thresholds is the first fix that comes to mind: making the coefficient of variation 0.05 instead of 0.15, requiring an exit ratio of two hundred instead of twenty. This lesson runs that fix as a **sweep** and draws both curves on the same axis. The result is not whether a single threshold is right. What gets counted is the band where the two distributions overlap: for the items in that band, no threshold can reduce both false positive and missed at once. This is the measurement the rest of the course builds on. The score distribution below is a **model**; the scores come from fictional attributes, not a real measurement. - **DI23.** The candidate set is 900 items: 880 legitimate work, 20 fictional items. The label is the model's ground truth and does not enter the scoring. - **DI24.** The score comes out as a weighted sum of four attributes: regularity, exit ratio, destination novelty and off-hours. The weights are model data. - **DI25.** Both distributions have a **tail**: some legit items score high, some fictional items score low. This is not a flaw in the model — it is the lesson's subject. - **DI26.** The overlap band is the range between the lowest fictional item score and the highest legit score. Items outside the band are classified correctly at every threshold. - **DI27.** The cost is two-sided and both sides are counted with model data: 9 minutes of review per alert, 6 extra items a missed item reaches by day's end, and 144 daily analyst-minutes allotted to this rule. #### Score Distribution and Threshold Sweep ```js // threshold.mjs — candidate items are scored, the threshold is swept and the overlap band is counted. THIS IS A MODEL. export const SEED = 20260405; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = generator(SEED); // Four attributes, each between 0 and 1. Weights are model data. export const WEIGHT = { regularity: 0.30, exitRatio: 0.30, destNovelty: 0.25, offHours: 0.15 }; const low = () => r() ** 3; // legit item: mostly low, has a tail const high = () => 1 - r() ** 3; // fictional item: mostly high, has a tail export const computeScore = (o) => Math.round(100 * Object.entries(WEIGHT).reduce((a, [k, w]) => a + w * o[k], 0)); export const CANDIDATE = []; for (let k = 0; k < 880; k++) CANDIDATE.push({ regularity: low(), exitRatio: low(), destNovelty: low(), offHours: low(), label: "legit" }); for (let k = 0; k < 20; k++) CANDIDATE.push({ regularity: high(), exitRatio: high(), destNovelty: high(), offHours: high(), label: "item" }); for (const o of CANDIDATE) o.score = computeScore(o); CANDIDATE.sort((a, b) => a.score - b.score); export const legit = CANDIDATE.filter((o) => o.label === "legit"); export const item = CANDIDATE.filter((o) => o.label === "item"); export const sweep = (threshold) => ({ threshold, falsePositive: legit.filter((o) => o.score >= threshold).length, missed: item.filter((o) => o.score < threshold).length, alert: CANDIDATE.filter((o) => o.score >= threshold).length, }); if (import.meta.url === `file://${process.argv[1]}`) { const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([9, 9, 15, 13, 13, 12]); console.log(`model (seed ${SEED}): ${CANDIDATE.length} candidate items, ${item.length} of them ` + `fictional; score comes out as a weighted sum of four attributes`); console.log(`legit score range ${legit[0].score}-${legit[legit.length - 1].score}, ` + `item score range ${item[0].score}-${item[item.length - 1].score}`); const yd = fmt([16, 10, 14]); console.log(); console.log(yd(["score range", "legit", "item"])); for (let a = 0; a < 100; a += 20) { const ic = (l) => l.filter((o) => o.score >= a && o.score < a + 20).length; console.log(yd([`${a}-${a + 19}`, ic(legit), ic(item)])); } console.log(); console.log(yz(["threshold", "alert", "false positive", "missed", "total error", "precision"])); for (const e of [10, 20, 30, 40, 50, 60, 70, 80]) { const t = sweep(e); console.log(yz([e, t.alert, t.falsePositive, t.missed, t.falsePositive + t.missed, t.alert ? `${Math.round(100 * (t.alert - t.falsePositive) / t.alert)}%` : "-"])); } // Overlap band: between the lowest fictional item score and the highest legit score. const lo = item[0].score, hi = legit[legit.length - 1].score; const bandLegit = legit.filter((o) => o.score >= lo && o.score <= hi).length; const bandItem = item.filter((o) => o.score >= lo && o.score <= hi).length; console.log(`\noverlap band ${lo}-${hi}: this band holds ${bandLegit} legit and ${bandItem} ` + `fictional items; no threshold can separate the two at once`); const best = [...Array(101).keys()].map(sweep) .reduce((a, b) => (b.falsePositive + b.missed < a.falsePositive + a.missed ? b : a)); console.log(`threshold minimizing total error ${best.threshold}: false positive ` + `${best.falsePositive}, missed ${best.missed}; no threshold gives zero-zero`); } ``` ``` model (seed 20260405): 900 candidate items, 20 of them fictional; score comes out as a weighted sum of four attributes legit score range 0-88, item score range 37-97 score range legit item 0-19 357 0 20-39 399 1 40-59 108 2 60-79 15 8 80-99 1 9 threshold alert false positive missed total error precision 10 766 746 0 746 3% 20 543 523 0 523 4% 30 302 282 0 282 7% 40 143 124 1 125 13% 50 74 56 2 58 24% 60 33 16 3 19 52% 70 20 4 4 8 80% 80 10 1 11 12 90% overlap band 37-88: this band holds 157 legit and 17 fictional items; no threshold can separate the two at once threshold minimizing total error 70: false positive 4, missed 4; no threshold gives zero-zero ``` The distribution table gives the lesson's whole subject in five rows. 756 of the legit items are below 40, and 19 of the fictional items are above 40; that means the score does useful work. But the third and fourth rows show the two distributions sharing the same bins: in the 40–59 range, 108 legit items sit alongside 2 fictional items, and in the 60–79 range, 15 legit items sit alongside 8 fictional items. In these rows, the score produces no separation. The threshold sweep writes the same fact in the language of decisions, and there is no flat region in between. At threshold 40, 124 false positives and 1 missed; at threshold 80, 1 false positive and 11 missed. Lowering the threshold reduces missed and grows the review load; raising it lightens the review and grows missed. **The transition in between is continuous**; there is no point that drives both numbers to zero at once. The reason is written in the second line: legit scores fall between 0 and 88, fictional item scores fall between 37 and 97. These two ranges **overlap** between 37–88, and that band holds 157 legit items and 17 fictional items. Wherever a threshold lands inside the band, it misses the fictional items below it and flags the legit items above it. The 726 items outside the band are classified correctly at every threshold; the measured uncertainty is only as large as the band. The precision column corrects another misconception. For threshold 10, precision comes out to 3 percent: 98 percent of alerts are legitimate work. At threshold 80, precision rises to 90 percent, but missed jumps to 11. Precision alone is not a quality measure, because it is undefined for **a rule that produces no alerts at all**, and any rule that produces few alerts shows high precision. #### Budget Threshold and Narrowing the Band The threshold that minimizes total error is 70, and there 4 false positives and 4 missed remain. But this choice is not made in practice, because threshold 70 produces 20 alerts and every alert demands analyst time. Capacity, not accuracy, determines the operating point. ```js // point.mjs — the threshold's two-sided cost in minutes, then the band with a fifth attribute. import { CANDIDATE, sweep } from "./threshold.mjs"; // Model data (not a measurement): review minutes, the number of extra items a missed item // reaches by day's end, and the daily analyst-minutes allotted to this rule. const REVIEW = 9, EXTRA_ITEM = 6, BUDGET = 144; const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([9, 8, 13, 12, 15, 13]); console.log(`daily budget ${BUDGET} analyst-minutes, review per alert ${REVIEW} minutes; ` + `each missed item reaches ${EXTRA_ITEM} extra items by day's end (model data)`); console.log(); console.log(yz(["threshold", "alert", "review min", "missed", "item reached", "budget"])); for (const e of [40, 50, 60, 70, 80, 90]) { const t = sweep(e); console.log(yz([e, t.alert, t.alert * REVIEW, t.missed, t.missed * EXTRA_ITEM, t.alert * REVIEW <= BUDGET ? "fits" : "does not fit"])); } const fitting = [...Array(101).keys()].map(sweep).filter((t) => t.alert * REVIEW <= BUDGET); const pick = fitting[0]; console.log(`\nlowest threshold that fits the budget ${pick.threshold}: ${pick.alert} alerts, ` + `${pick.missed} missed items, ${pick.missed * EXTRA_ITEM} items reached`); // Fifth attribute: whether a matching record exists at the identity source. Discriminating but not perfect. const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r2 = generator(880517); const W2 = { regularity: 0.21, exitRatio: 0.21, destNovelty: 0.18, offHours: 0.10, noIdentityMatch: 0.30 }; const CANDIDATE2 = CANDIDATE.map((o) => ({ ...o, noIdentityMatch: (o.label === "item" ? r2() < 0.8 : r2() < 0.03) ? 1 : 0 })); for (const o of CANDIDATE2) o.score = Math.round(100 * Object.entries(W2).reduce((a, [k, w]) => a + w * o[k], 0)); const band = (list) => { const m = list.filter((o) => o.label === "legit"), k = list.filter((o) => o.label === "item"); const lo = Math.min(...k.map((o) => o.score)), hi = Math.max(...m.map((o) => o.score)); const count = (l) => l.filter((o) => o.score >= lo && o.score <= hi).length; // Operating point: the LOWEST threshold that fits the budget. const best = [...Array(101).keys()].map((e) => ({ e, n: list.filter((o) => o.score >= e).length, fp: m.filter((o) => o.score >= e).length, mi: k.filter((o) => o.score < e).length })) .find((x) => x.n * REVIEW <= BUDGET); return { lo, hi, m: count(m), k: count(k), best }; }; const yb = fmt([18, 9, 16, 16, 17, 13]); console.log(); console.log(yb(["scoring", "band", "legit in band", "item in band", "budget threshold", "fp / missed"])); for (const [name, list] of [["four attributes", CANDIDATE], ["five attributes", CANDIDATE2]]) { const b = band(list); console.log(yb([name, `${b.lo}-${b.hi}`, b.m, b.k, b.best.e, `${b.best.fp} / ${b.best.mi}`])); } console.log(`\nthe fifth attribute lowers the item count in the band but does not close it; ` + `no threshold gives zero false positive and zero missed in either scoring`); ``` ``` daily budget 144 analyst-minutes, review per alert 9 minutes; each missed item reaches 6 extra items by day's end (model data) threshold alert review min missed item reached budget 40 143 1287 1 6 does not fit 50 74 666 2 12 does not fit 60 33 297 3 18 does not fit 70 20 180 4 24 does not fit 80 10 90 11 66 fits 90 3 27 17 102 fits lowest threshold that fits the budget 72: 16 alerts, 7 missed items, 42 items reached scoring band legit in band item in band budget threshold fp / missed four attributes 37-88 157 17 72 3 / 7 five attributes 49-75 15 7 66 1 / 5 the fifth attribute lowers the item count in the band but does not close it; no threshold gives zero false positive and zero missed in either scoring ``` The budget column takes the threshold choice out of our hands. The threshold 70 that minimizes total error demands 180 minutes of review and does not fit the 144-minute budget. The lowest threshold that fits is 72; there, 16 alerts are reviewed, 7 fictional items get missed, and per the model data, those 7 items reach 42 extra items by day's end. **What chooses the threshold is not accuracy — it is analyst-minutes.** The two-sided cost has different units, and that is the real difficulty that makes comparison hard. The left side is minutes, paid directly: 1,287 minutes at threshold 40, more than a nine-person day. The right side is items, paid with a delay: 17 missed items at threshold 90, 102 items reached. Adding the two on the same axis is only possible once someone writes down how many minutes an item is worth, and that number is an organizational decision, not a technical one. The last table shows the one real way out: narrowing the band. The fifth attribute — the item having no matching record at the identity source — brings the band from 37–88 down to 49–75, the legit count in the band from 157 to 15, and the fictional item count in the band from 17 to 7. The budget threshold drops from 72 to 66, and at that point false positive falls from 3 to 1, missed falls from 7 to 5. **Both numbers improve at the same time**, because what changes is not the threshold — it is the score itself. The fifth attribute has a cost too, and that cost was measured in the second lesson. To say "no matching record at the identity source," the mapping has to be attempted first; in a five-minute window, that mapping resolved part of the records and left the rest ambiguous. For an item that stays ambiguous, this attribute cannot be filled in. So the improvement that narrows the band rests on a **limitation of the pipeline measured earlier**, and as that limitation grows, the improvement shrinks. Yet the band does not close. The fifth attribute is present in 80 percent of fictional items and also present in 3 percent of legit items; the remaining overlap is a direct result of these two proportions. No threshold gives zero false positive and zero missed in either scoring, and adding more attributes narrows the band, it does not close it. #### Summary - 900 candidate items' scores overlap in the 37–88 band; that band holds 157 legit and 17 fictional items and no threshold can separate the two at once. The 726 items outside the band are classified correctly at every threshold. - At threshold 40, 124 false positives and 1 missed; at threshold 80, 1 false positive and 11 missed; even at the threshold 70 that minimizes total error, 4 and 4 remain. - Precision alone is not a quality measure: any rule that produces few alerts shows high precision — at threshold 80, precision is 90 percent while missed is 11. - Capacity, not accuracy, picks the operating point: the lowest threshold that fits 144 analyst-minutes is 72, where 7 fictional items get missed, reaching 42 extra items. - The fifth attribute narrows the band to 49–75, the legit count to 15 and the item count to 7; at the budget threshold, false positive falls from 3 to 1 and missed falls from 7 to 5, but a zero-zero threshold still does not exist. #### Next Step What narrowed the band was not a threshold adjustment but a new attribute — that is, a change to the scoring itself. This is not tuning a rule, it is **rewriting** it, and that is a routine part of the job. The next lesson treats that work as a cycle: it runs two versions of a rule set against the same technique list and compares the number of techniques each covers. What is measured there is not how many rules were written, but how many cells of the tactic-technique matrix are covered by at least one rule, how many are never covered, and in how many of the cells marked as covered the rule could never actually fire — because that cell's data source is somewhere the pipeline never collects from. ### Detection Engineering Source: https://academia.sh/en/courses/security-operations/detection-engineering Two rule versions are compared on a fictional classification matrix with eight tactic columns and 26 technique rows: version 1's 6 rules cover 11 techniques but only 8 are firable, 15 techniques are never covered; version 2's 11 rules cover 24 techniques, 19 firable, never-covered drops to 2. Over 51 fictional samples, version 1 catches 10, version 2 catches 30; once the reason for a miss is separated out, no-rule drops from 32 to 3 while no-source rises from 4 to 11. An extra 220 minutes of writing work comes to 11 minutes per sample, daily false positive drops from 30 to 27, but review load rises from 360 minutes to 513 minutes. In the previous lesson, what narrowed the band was not a threshold adjustment but a new attribute — that is, a change to the scoring itself. This is not tuning a rule, it is **rewriting** it, and that is a routine part of the job. Detection engineering is the name for that job: write a rule, run it, measure what it misses, fix it. The cycle's measure cannot be how many rules were written, because the rule count gives neither coverage nor accuracy. This lesson builds the measure on a classification matrix. The matrix is treated by its **structure**: columns are tactics, rows are techniques; the techniques and their traces here are **fictional** and do not correspond to the content of a real classification effort. - **DI28.** The matrix has 8 tactic columns and 26 technique rows. Each technique row is labeled with which source its trace lands on. - **DI29.** A technique is in one of three states: its trace lands on a **collected** source, its trace lands on an uncollected source, or it **leaves no trace in any source**. In the last two cases, a rule can be written but it can never fire. - **DI30.** A rule covers one or more techniques. Every rule has three numbers: write-and-tune minutes, sensitivity, and daily false positive. All three are model data. - **DI31.** A **covered** technique is one that at least one rule targets. **Firable coverage** is the subset of that whose source is collected. The gap between the two is coverage that stays on paper. - **DI32.** The sample set consists of fictional examples of the techniques and is **identical** for both versions; the comparison's validity depends on this. #### Coverage in the Matrix and Two Versions ```js // coverage.mjs — two rule-set versions' coverage over a tactic-technique matrix. THIS IS A MODEL. // The matrix is treated by its STRUCTURE: columns are tactics, rows are techniques. The techniques are fictional. export const TACTIC = [["initial access", 3], ["execution", 4], ["persistence", 4], ["privilege escalation", 3], ["credential access", 4], ["discovery", 3], ["lateral movement", 3], ["exfiltration", 2]]; // Where each technique's trace lands: I identity, E endpoint, F network flow, // M management plane, S UNcollected server application log, - no trace in any source. const TRACE = "IES EFEM E-EF IMS I-EM FFE S-I FF".replace(/ /g, "").split(""); export const TECHNIQUE = []; for (const [tactic, n] of TACTIC) for (let k = 0; k < n; k++) { const i = TECHNIQUE.length; TECHNIQUE.push({ id: `t${String(i + 1).padStart(2, "0")}`, tactic, trace: TRACE[i] }); } // Sources collected in the pipeline (model data). Source S exists but is not collected. export const COLLECTED = new Set(["I", "E", "M", "F"]); // Rule: [name, covered technique indexes, write+tune minutes, sensitivity, daily false positive] export const VERSION1 = [["s1", [0, 2], 90, 0.8, 3], ["s2", [6, 8], 120, 0.7, 5], ["s3", [12, 16], 150, 0.6, 4], ["s4", [20, 23], 90, 0.75, 2], ["s5", [1, 13], 180, 0.5, 9], ["s6", [4], 240, 0.4, 7]]; export const VERSION2 = [["s1", [0, 2], 30, 0.85, 2], ["s2", [6, 8], 40, 0.8, 2], ["s3", [12, 16], 40, 0.7, 2], ["s4", [20, 23], 20, 0.8, 1], ["s5", [1, 13, 17], 60, 0.6, 3], ["s7", [3, 9], 150, 0.7, 3], ["s8", [5, 10, 14], 210, 0.65, 4], ["s9", [18, 21], 180, 0.55, 3], ["s10", [7, 11], 120, 0.5, 2], ["s11", [24, 25], 90, 0.6, 2], ["s12", [15, 19], 150, 0.45, 3]]; export function coverage(rules) { const covered = new Set(rules.flatMap(([, l]) => l)); const count = (f) => [...covered].filter((i) => f(TECHNIQUE[i].trace)).length; return { covered, firable: new Set([...covered].filter((i) => COLLECTED.has(TECHNIQUE[i].trace))), uncollected: count((z) => z === "S"), traceless: count((z) => z === "-"), none: TECHNIQUE.length - covered.size, minutes: rules.reduce((a, k) => a + k[2], 0), fp: rules.reduce((a, k) => a + k[4], 0) }; } if (import.meta.url === `file://${process.argv[1]}`) { const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yt = fmt([26, 9, 10, 12, 12]); const k1 = coverage(VERSION1), k2 = coverage(VERSION2); console.log(`matrix: ${TACTIC.length} tactic columns, ${TECHNIQUE.length} technique rows; ` + `${TECHNIQUE.filter((t) => t.trace === "S").length} techniques' source is not collected, ` + `${TECHNIQUE.filter((t) => t.trace === "-").length} techniques have no trace in any source`); console.log(); console.log(yt(["tactic", "technique", "version 1", "version 2", "no trace"])); for (const [tactic] of TACTIC) { const l = TECHNIQUE.filter((t) => t.tactic === tactic); const count = (s) => l.filter((t) => s.has(TECHNIQUE.indexOf(t))).length; console.log(yt([tactic, l.length, count(k1.covered), count(k2.covered), l.filter((t) => !COLLECTED.has(t.trace)).length])); } const yk = fmt([10, 8, 12, 15, 14, 10, 16, 11]); console.log(); console.log(yk(["version", "rules", "covered", "firable", "no source", "no trace", "never covered", "write min"])); for (const [name, k, l] of [["version 1", k1, VERSION1], ["version 2", k2, VERSION2]]) console.log(yk([name, l.length, `${k.covered.size}/${TECHNIQUE.length}`, `${k.firable.size}/${TECHNIQUE.length}`, k.uncollected, k.traceless, k.none, k.minutes])); } ``` ``` matrix: 8 tactic columns, 26 technique rows; 3 techniques' source is not collected, 3 techniques have no trace in any source tactic technique version 1 version 2 no trace initial access 3 3 3 1 execution 4 2 3 0 persistence 4 1 4 1 privilege escalation 3 2 3 1 credential access 4 1 4 1 discovery 3 1 3 0 lateral movement 3 1 2 2 exfiltration 2 0 2 0 version rules covered firable no source no trace never covered write min version 1 6 11/26 8/26 2 1 15 870 version 2 11 24/26 19/26 3 2 2 1090 ``` The tactic table gives the first finding: version 1's coverage **is not evenly distributed**. All three initial-access rows are covered; none of the exfiltration rows are. This is detection writing's natural tendency — the easy rule gets written on the source whose fields are most distinguishing, and that source is usually near the start of the chain. The result shows up not in a coverage map but in **missed items**. The exfiltration row staying empty does not come from that tactic being seen as unimportant — it comes from its trace sitting in network flow data, and from the difficulty, measured in the fourth lesson, of writing a distinguishing rule on that source. The gap between the two columns in the lower table is the lesson's central concept. Version 1 covers eleven techniques but only eight are firable: three techniques have a rule and it never triggers. Two of those have a source, but the pipeline does not collect it; one technique leaves no trace in any source. The sentence "we cover eleven of twenty-six techniques" is a **false sentence** when this distinction goes unwritten. Version 2 raises coverage to 24, firable coverage to 19, and brings never-covered down from 15 to 2. Of the two remaining techniques, one cannot be covered because it leaves no trace in any source; the other could have been covered, but a rule from version 1 got **removed** in the cycle. Writing work rose from 870 minutes to 1,090 minutes; that number includes tuning old rules alongside writing new ones, and the first four rules' minutes in the second version are noticeably lower, because tuning is cheaper than writing. #### Running the Versions Against the Same Sample Set ```js // cycle.mjs — two rule versions are run against a fictional sample set; the reason for each miss is separated. import { TECHNIQUE, COLLECTED, VERSION1, VERSION2, coverage } from "./coverage.mjs"; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const REVIEW = 9; // Fictional sample set: 0-4 samples per technique. The sample count is DATA. const SAMPLE = []; { const r = generator(20260406); for (const [i] of TECHNIQUE.entries()) for (let k = 0, n = Math.floor(r() * 5); k < n; k++) SAMPLE.push({ technique: i, z: r() }); } function run(rules) { const k = coverage(rules); const result = { caught: 0, noRule: 0, noSource: 0, sensitivity: 0 }; const firing = new Set(), share = new Map(rules.map(([name]) => [name, 0])); for (const o of SAMPLE) { const covering = rules.filter(([, l]) => l.includes(o.technique)); if (covering.length === 0) { result.noRule++; continue; } if (!COLLECTED.has(TECHNIQUE[o.technique].trace)) { result.noSource++; continue; } const top = Math.max(...covering.map(([, , , d]) => d)); if (o.z < top) { result.caught++; covering.forEach(([name]) => { firing.add(name); share.set(name, share.get(name) + 1); }); } else result.sensitivity++; } const dead = rules.filter(([name]) => !firing.has(name)).map(([name, l]) => { const ns = l.filter((i) => !COLLECTED.has(TECHNIQUE[i].trace)).length; const smp = SAMPLE.filter((o) => l.includes(o.technique)).length; return `${name} (${l.length} technique, ${ns} with no source, ${smp} samples)`; }); return { ...result, fp: k.fp, minutes: k.minutes, dead, share, review: (result.caught + k.fp) * REVIEW }; } const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([10, 8, 11, 11, 12, 13, 13, 13]); console.log(`fictional sample set: ${SAMPLE.length} samples, spread over ${TECHNIQUE.length} techniques`); console.log(); console.log(yz(["version", "sample", "caught", "no rule", "no source", "sensitivity", "false pos.", "review min"])); const [a, b] = [VERSION1, VERSION2].map(run); for (const [name, s] of [["version 1", a], ["version 2", b]]) console.log(yz([name, SAMPLE.length, s.caught, s.noRule, s.noSource, s.sensitivity, s.fp, s.review])); console.log(`\nrule that never fires — version 1: ${a.dead.join(", ") || "none"}; ` + `version 2: ${b.dead.join(", ") || "none"}`); console.log(`version 2's extra ${b.minutes - a.minutes} minutes of writing work ` + `catches ${b.caught - a.caught} extra samples: ` + `${Math.round((b.minutes - a.minutes) / (b.caught - a.caught))} minutes per sample`); console.log(`daily false positive ${a.fp} -> ${b.fp}, daily review ` + `${a.review} -> ${b.review} minutes`); const yp = fmt([8, 9, 13, 12, 12, 11, 17]); console.log("\nversion 2 by rule:"); console.log(yp(["rule", "technique", "no source", "caught", "daily fp", "write min", "min per sample"])); for (const [name, l, dk, , fp] of VERSION2) { const n = b.share.get(name); console.log(yp([name, l.length, l.filter((i) => !COLLECTED.has(TECHNIQUE[i].trace)).length, n, fp, dk, n ? Math.round(dk / n) : "-"])); } ``` ``` fictional sample set: 51 samples, spread over 26 techniques version sample caught no rule no source sensitivity false pos. review min version 1 51 10 32 4 5 30 360 version 2 51 30 3 11 7 27 513 rule that never fires — version 1: s6 (1 technique, 0 with no source, 1 samples); version 2: s9 (2 technique, 1 with no source, 3 samples) version 2's extra 220 minutes of writing work catches 20 extra samples: 11 minutes per sample daily false positive 30 -> 27, daily review 360 -> 513 minutes version 2 by rule: rule technique no source caught daily fp write min min per sample s1 2 1 1 2 30 30 s2 2 1 4 2 40 10 s3 2 0 3 2 40 13 s4 2 0 3 1 20 7 s5 3 1 4 3 60 15 s7 2 0 5 3 150 30 s8 3 0 1 4 210 210 s9 2 1 0 3 180 - s10 2 0 3 2 120 40 s11 2 0 5 2 90 18 s12 2 1 1 3 150 150 ``` The three reasons for a miss sit in separate columns, and it is exactly this separation that shows what the cycle is doing. In version 1, 32 of 41 misses are from no rule; in version 2, that number drops to 3. Against that, no-source **rises** from 4 to 11. This rise is not a regression: as rules get written, the reason for a miss shifts from a missing rule to a missing source. Once that second column runs dry, the remaining number cannot be reduced by writing rules; it needs a collection decision. The sensitivity column holds the third reason: there is a rule, there is a source, but the sample did not cross the rule's threshold. This number rises from 5 to 7, because version 2 covers more techniques and the new rules have low sensitivity. Growing a rule set changes the **distribution** of misses; it does not shrink every reason at once. All three columns correspond to a separate piece of work: the first calls for writing a rule, the second for a collection decision, the third for the threshold-and-attribute work from the previous lesson. Rolling the same number up into a single "missed" column leaves it unclear which work needs to be done. The dead-rule row is maintenance's hidden line item. In version 1, s6 never fired, and that rule had cost 240 minutes of writing work; in version 2, s9 never fired, and one of the two techniques it covers already has an uncollected source. A dead rule shows up as **covered** in a coverage table, never shows up in an alert list, and gets reviewed all over again at every version upgrade. The cycle's bill is this: an extra 220 minutes of writing work catches 20 extra samples, 11 minutes per sample. Daily false positive drops from 30 to 27 — it falls even though the rule count nearly doubled, because version 1's noisiest rule got removed. But daily review load rises from 360 minutes to 513 minutes: a correct detection has a review cost too, and a good rule set does not lighten the shift — it **fills** it. The final table, broken down by rule, opens up what the average hides. The minutes-per-sample column ranges from 7 to 210: s4 catches three samples with twenty minutes of tuning, s8 catches one sample with two hundred ten minutes of writing work. Both are single rows in the coverage table and look identical there. Taking coverage as the measure is not enough, then; **how many samples a covered cell brings in** is a separate number. In the same table, the `no source` column is greater than zero for five rules. These rules cover two techniques but can only fire on one; they contribute two to the coverage count and one to the firable count. The most extreme example is s9: two techniques, one on an uncollected source, zero catches all day, 180 minutes of writing work. Whether a rule is good or bad can only be said after it is run, and running it is this cycle's real step. #### Summary - Covered technique and firable coverage are counted separately: version 1 covers eleven techniques but only eight are firable, three have a rule and it never triggers. - Coverage is not evenly distributed across tactics; version 1 covers all three initial-access rows while covering none of the exfiltration rows. - Version 2 raises coverage to 24, firable coverage to 19, and brings never-covered down from 15 to 2; one of the remaining two cannot be covered because it leaves no trace in any source. - As rules get written, the reason for a miss shifts: no-rule drops from 32 to 3 while no-source rises from 4 to 11, and that number cannot be reduced by writing rules. - An extra 220 minutes of writing work comes to 11 minutes per sample; daily false positive drops from 30 to 27 while review load rises from 360 minutes to 513 minutes. - Rule-by-rule efficiency ranges from 7 to 210 minutes, and this gap does not show up in the coverage table; s9 covers two techniques but never fires, and its 180 minutes of writing work go unrewarded. #### Next Step After version 2, 21 of 51 samples are still missed, and 11 of those sit where no rule can ever reach: techniques whose source is not collected or that leave no trace at all. Writing rules does not reduce that number, because a rule can only ask the field it can query. The next lesson takes a different route: it starts not from a rule but from a **hypothesis**. A hypothesis asks the data set a question, and the answer is not an alert — it is a count of eliminated items. The measure there is how many items get eliminated per hypothesis, how many hypotheses return zero results, and how many of the eliminated items turn into a new rule. ### Threat Hunting Source: https://academia.sh/en/courses/security-operations/threat-hunting Six hypotheses are run against a fictional telemetry set of 12,910 records, and the items eliminated per hypothesis are counted: h1 eliminates 12,906 records in three steps and leaves 4 items, h4 leaves 6 items in two steps, h3 leaves 283 items and becomes impossible to review; h2, h5, and h6 find no fictional item at all but write 370 items to the baseline. A total of 1,691 minutes of search finds two fictional items, 846 minutes per item; writing a rule cost 11 minutes per sample. Two of the six hypotheses convert into a rule because they stay under the daily ten-alert limit, and four exceed the limit or return empty. After the previous lesson's second version, 21 of 51 samples were still missed, and 11 of those sat where no rule could ever reach. Writing rules does not reduce that number, because a rule can only ask the field it can query and only search for the form it was written to match. Threat hunting starts from a different place: not from a rule, but from a **hypothesis**. A hypothesis does not produce an alert; it asks a question, and the answer is a count of eliminated items. This lesson runs six hypotheses against a fictional telemetry set and produces three numbers: how many items get eliminated per hypothesis, how many hypotheses return zero results, and how many of the eliminated ones can convert into a rule. In the set below, two fictional items are embedded only **as records**; nowhere does it write out how an attack step was carried out. **Running this kind of search on a system outside your own environment, without written authorization and a defined scope, is both illegal and unprofessional.** **DI33.** The telemetry set covers one day: 12,910 records, 192 hosts, 90 personnel, and 8 service identities. A record carries a process-execution event and its associated outbound transfer. **DI34.** The baseline is each path's usual parent process. In three percent of records, the parent process falls outside that match; this is not a flaw in the model, it is the counterpart of noise in a real environment. **DI35.** A hypothesis consists of sequential filters. Each filter narrows the set, and **how many items it eliminates is counted separately**. **DI36.** The two fictional items are embedded only as records, and their labels never enter the filters. Both are behaviors that the previous lesson's rule set does not cover. **DI37.** Cost has two components: search minutes per hypothesis and 2 review minutes per remaining item. A hypothesis can convert into a rule if its remaining item count stays under the daily ten-alert limit. #### Six Hypotheses, Six Funnels ```js // hunt.mjs — fictional telemetry set and the items six hypotheses eliminate step by step. THIS IS A MODEL. export const SEED = 20260407; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = generator(SEED); const randInt = (n) => Math.floor(r() * n); const pick = (d) => d[randInt(d.length)]; const HOST = [...Array.from({ length: 180 }, (_, i) => `uc-${101 + i}`), ...Array.from({ length: 12 }, (_, i) => `tp-${11 + i}`)]; const ACCOUNT = Array.from({ length: 90 }, (_, i) => `sh-${201 + i}`); const SERVICE = Array.from({ length: 8 }, (_, i) => `sv-${11 + i}`); // Baseline: each path's usual parent process (model data). const PAIR = [["/usr/bin/measurement-client", "/sbin/service-manager"], ["/usr/bin/archive", "/usr/bin/scheduler"], ["/opt/measurement/collector", "/opt/measurement/verifier"], ["/usr/bin/report", "/usr/bin/scheduler"], ["/opt/portal/client", "/usr/bin/session"], ["/usr/bin/backup", "/usr/bin/scheduler"], ["/usr/bin/shell", "/usr/bin/session"]]; export const BASELINE = new Set(PAIR.map(([y, u]) => `${u}|${y}`)); const COLLECTOR_PATH = ["/opt/measurement/collector", "/usr/bin/archive", "/usr/bin/backup"]; // Record: a process-execution event with its associated outbound transfer (fictional). export const RECORD = []; const add = (o, label) => RECORD.push({ ...o, label }); for (let k = 0; k < 12000; k++) { const host = pick(HOST); const path = host.startsWith("tp-") ? pick(COLLECTOR_PATH) : pick(PAIR)[0]; // Three percent of records: the parent process matches unusually: real-world noise. const parent = r() < 0.03 ? pick(PAIR)[1] : PAIR.find(([y]) => y === path)[1]; add({ host, account: r() < 0.06 ? pick(SERVICE) : pick(ACCOUNT), path, parent, hour: 7 + randInt(12), signed: r() < 0.97, external: r() < 0.22, bytes: 1000 + randInt(200000) }, "routine"); } // Night runs: scheduled legit jobs. for (let k = 0; k < 900; k++) { const path = pick(["/usr/bin/backup", "/usr/bin/archive"]); add({ host: pick(HOST), account: pick(SERVICE), path, parent: "/usr/bin/scheduler", hour: r() < 0.5 ? randInt(6) : 20 + randInt(4), signed: true, external: r() < 0.1, bytes: 2e6 + randInt(4e8) }, "night-run"); } // Two fictional items, given only as RECORD entries. for (let k = 0; k < 4; k++) add({ host: "uc-137", account: "sh-244", path: "/usr/bin/shell", parent: "/usr/bin/session", hour: 2 + k, signed: false, external: true, bytes: 40000 + randInt(20000) }, "a1"); for (let k = 0; k < 6; k++) add({ host: `tp-${11 + k}`, account: "sv-13", path: "/usr/bin/report", parent: "/opt/measurement/verifier", hour: 13, signed: true, external: false, bytes: 8000 + randInt(4000) }, "a2"); // Hypothesis: [name, steps (label + filter), search-and-review minutes] const manyHosts = (l) => { const m = new Map(); for (const o of l) { const a = `${o.parent}|${o.path}`; (m.get(a) || m.set(a, new Set()).get(a)).add(o.host); } return l.filter((o) => m.get(`${o.parent}|${o.path}`).size >= 5); }; export const HYPOTHESIS = [ ["h1 unsigned binary, off-hours, external target", [ ["unsigned", (l) => l.filter((o) => !o.signed)], ["off-hours", (l) => l.filter((o) => o.hour < 7 || o.hour >= 19)], ["external target", (l) => l.filter((o) => o.external)]], 75], ["h2 shell with a service identity", [ ["service identity", (l) => l.filter((o) => o.account.startsWith("sv-"))], ["shell path", (l) => l.filter((o) => o.path.endsWith("shell"))]], 40], ["h3 off-baseline parent-path, 5+ hosts", [ ["off-baseline pair", (l) => l.filter((o) => !BASELINE.has(`${o.parent}|${o.path}`))], ["5+ hosts", manyHosts]], 90], ["h4 unexpected path on a collector", [ ["collector host", (l) => l.filter((o) => o.host.startsWith("tp-"))], ["off-baseline path", (l) => l.filter((o) => !COLLECTOR_PATH.includes(o.path))]], 60], ["h5 large night-time outbound transfer", [ ["off-hours", (l) => l.filter((o) => o.hour < 7 || o.hour >= 19)], ["over 10 MB", (l) => l.filter((o) => o.bytes > 1e7)], ["external target", (l) => l.filter((o) => o.external)]], 55], ["h6 unsigned binary from the scheduler", [ ["unsigned", (l) => l.filter((o) => !o.signed)], ["scheduler parent", (l) => l.filter((o) => o.parent === "/usr/bin/scheduler")]], 45], ]; export function run([, steps]) { let l = RECORD, funnel = [RECORD.length]; for (const [, f] of steps) { l = f(l); funnel.push(l.length); } return { remaining: l, funnel, item: [...new Set(l.map((o) => o.label))].filter((e) => e.startsWith("a")) }; } if (import.meta.url === `file://${process.argv[1]}`) { const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([48, 8, 12, 10, 10, 12]); console.log(`model (seed ${SEED}): ${RECORD.length} records, ` + `${new Set(RECORD.map((o) => o.host)).size} hosts; two fictional items embedded as records`); console.log(); console.log(yz(["hypothesis", "step", "eliminated", "remaining", "minute", "item"])); for (const h of HYPOTHESIS) { const s = run(h); console.log(yz([h[0], h[1].length, RECORD.length - s.remaining.length, s.remaining.length, h[2], s.item.join(",") || "-"])); } const h1 = HYPOTHESIS[0], s1 = run(h1); console.log(`\n${h1[0]} — step by step funnel:`); const yh = fmt([22, 10, 11]); console.log(yh(["step", "remaining", "eliminated"])); console.log(yh(["start", s1.funnel[0], "-"])); for (const [i, [name]] of h1[1].entries()) console.log(yh([name, s1.funnel[i + 1], s1.funnel[i] - s1.funnel[i + 1]])); } ``` ``` model (seed 20260407): 12910 records, 192 hosts; two fictional items embedded as records hypothesis step eliminated remaining minute item h1 unsigned binary, off-hours, external target 3 12906 4 75 a1 h2 shell with a service identity 2 12800 110 40 - h3 off-baseline parent-path, 5+ hosts 2 12627 283 90 a2 h4 unexpected path on a collector 2 12904 6 60 a2 h5 large night-time outbound transfer 3 12816 94 55 - h6 unsigned binary from the scheduler 2 12744 166 45 - h1 unsigned binary, off-hours, external target — step by step funnel: step remaining eliminated start 12910 - unsigned 378 12532 off-hours 4 374 external target 4 0 ``` The funnel table shows hunting's real work. h1's first filter eliminates 12,532 records and leaves 378; the second filter eliminates 374 more. **The third filter eliminates nothing** — all four remaining records were already going to an external target. This says the hypothesis's third condition is unnecessary on this data set and can be dropped when it is converted into a rule. Running a hypothesis produces not just the items found but also **which condition actually did the work**. Two of the six hypotheses give a usable result: h1 leaves four items, h4 leaves six, and both can be reviewed by hand. h3 finds the same fictional item, but buried inside 283 items; a hypothesis being correct does not make it useful, because if there is no time to review the rest, a finding counts as not found. h3 and h4 find the same fictional item by different routes, and the only difference between them is what the first filter relies on: h4 knows that collector machines have a narrow list of paths, h3 does not. Three hypotheses find no fictional item at all. h2 searches for "shell execution with a service identity" and finds 110 records; this means the hypothesis was **falsified** — that behavior is routine in this fictional environment. h6's 166 records are unsigned binaries running from the scheduler, and they say the same thing. The number of eliminated items alone does not give a hypothesis's quality. h6 and h1 use the same first filter and leave the same 378 records; where they part ways is the second filter. h1 asks about the time and leaves 4 items, h6 asks about the parent process and leaves 166. Both hypotheses look at the same data at the same cost; what makes the difference is **how rare the second question is in the environment**, and that cannot be known while writing the hypothesis — it can only be measured after running it. #### Cost, the Value of a Zero Result, and Converting to a Rule ```js // cost.mjs — cost per hypothesis, the value of a zero result, and the measure for converting to a rule. import { RECORD, HYPOTHESIS, run } from "./hunt.mjs"; // Model data: review minutes per remaining item and a rule's daily alert limit. const ITEM_MIN = 2, RULE_LIMIT = 10; const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([48, 11, 10, 11, 13, 8, 15]); const s = HYPOTHESIS.map((h) => ({ h, ...run(h) })); console.log(yz(["hypothesis", "eliminated", "remaining", "search min", "review min", "item", "rule fit"])); let totalMin = 0, found = new Set(); for (const x of s) { const rev = x.remaining.length * ITEM_MIN; totalMin += x.h[2] + rev; x.item.forEach((k) => found.add(k)); console.log(yz([x.h[0], RECORD.length - x.remaining.length, x.remaining.length, x.h[2], rev, x.item.join(",") || "-", x.remaining.length <= RULE_LIMIT ? "yes" : "no"])); } const empty = s.filter((x) => x.item.length === 0); console.log(`\ntotal ${totalMin} minutes of hunting, ${found.size} fictional items found: ` + `${Math.round(totalMin / found.size)} minutes per item`); console.log(`${empty.length} hypotheses found no item at all, but ` + `${empty.reduce((a, x) => a + x.remaining.length, 0)} items get written to the baseline; ` + `these items will not be eliminated by the same hypothesis again`); // Converting to a rule: a hypothesis that stays under the daily alert limit becomes a rule. const candidate = s.filter((x) => x.remaining.length <= RULE_LIMIT); const yk = fmt([48, 14, 19, 13]); console.log(); console.log(yk(["rule-candidate hypothesis", "daily alert", "monthly review min", "item type"])); for (const x of candidate) console.log(yk([x.h[0], x.remaining.length, x.remaining.length * ITEM_MIN * 30, x.item.join(",")])); console.log(`\n${s.length} hypotheses, ${candidate.length} convert into a rule; ` + `${s.length - candidate.length} exceed the daily alert limit or return empty`); console.log(`in hunting, ${Math.round(totalMin / found.size)} minutes per item, ` + `in rule writing, 11 minutes per sample; hunting finds the expensive one`); ``` ``` hypothesis eliminated remaining search min review min item rule fit h1 unsigned binary, off-hours, external target 12906 4 75 8 a1 yes h2 shell with a service identity 12800 110 40 220 - no h3 off-baseline parent-path, 5+ hosts 12627 283 90 566 a2 no h4 unexpected path on a collector 12904 6 60 12 a2 yes h5 large night-time outbound transfer 12816 94 55 188 - no h6 unsigned binary from the scheduler 12744 166 45 332 - no total 1691 minutes of hunting, 2 fictional items found: 846 minutes per item 3 hypotheses found no item at all, but 370 items get written to the baseline; these items will not be eliminated by the same hypothesis again rule-candidate hypothesis daily alert monthly review min item type h1 unsigned binary, off-hours, external target 4 240 a1 h4 unexpected path on a collector 6 360 a2 6 hypotheses, 2 convert into a rule; 4 exceed the daily alert limit or return empty in hunting, 846 minutes per item, in rule writing, 11 minutes per sample; hunting finds the expensive one ``` The cost table's real information is in the review column. h3's search cost is 90 minutes, but reviewing the 283 items it leaves takes 566 minutes — the hypothesis's total bill is 656 minutes, and 86 percent of that comes from the filter's failure to narrow enough. The way to make a hypothesis cheaper is not searching faster, it is **asking a narrower question**. A total of 1,691 minutes finds two fictional items: 846 minutes per item. In the previous lesson, writing a rule cost 11 minutes per sample. The two numbers cannot be compared directly, and here is the direction the comparison points: a rule catches, cheaply, what is **already been seen**; hunting finds, expensively, what was never written in a form any rule could ask about. Both fictional items are behaviors that the previous lesson's rule set does not cover. The three hypotheses that return zero results do not count as wasted. They write 370 items to the baseline: a service identity running a shell, an unsigned binary coming from the scheduler, and a large night-time transfer are all **routine** in this environment, and they do not need to be eliminated by the same hypothesis again. Hunting's output is not findings — it is a map of the environment, and the map grows with every run. The rule-candidate table closes the cycle. h1 would generate 4 alerts a day, h4 would generate 6; both stay under the ten-alert limit, and their monthly review loads are 240 and 360 minutes. The remaining four hypotheses cannot convert into a rule: h3 would generate 283 alerts a day, and the others would become a rule that contains no items at all. Whether a hypothesis converts into a rule depends not on its correctness but on its **volume**. #### Summary - A hypothesis produces not an alert but a count of eliminated items: h1 eliminates 12,906 records in three steps and leaves 4 items, h4 leaves 6 items in two steps, and both can be reviewed by hand. - h1's third filter eliminates nothing; running it produces, along with the items found, which condition did no work, and that condition can be dropped when it is converted into a rule. - h3 finds the correct fictional item but leaves it buried inside 283 items; if there is no time to review the rest, a finding counts as not found. - The three hypotheses that return zero results write 370 items to the baseline; this proves those behaviors are routine, and they do not need to be eliminated by the same hypothesis again. - A total of 1,691 minutes of search finds two fictional items, 846 minutes per item; writing a rule cost 11 minutes per sample, and a rule only catches what has already been seen. - Two of the six hypotheses convert into a rule because they stay under the daily ten-alert limit; the measure for converting is not correctness, it is volume. #### Next Step All six of this lesson's hypotheses came from the environment's own data: the baseline, the usual parent process, the usual hour. None of them used information about "what has been seen elsewhere." That information comes from outside, and it comes in two separate forms: individual indicators — an address, a hash, a name — and behavior definitions. The two look like the same thing, but their **lifespans** differ. The next lesson measures that difference: how many days an indicator stays valid, how many months a behavior definition does; how the match rate changes as the feed grows, and where the storage cost comes from. ### Threat Intelligence Source: https://academia.sh/en/courses/security-operations/threat-intelligence A fictional feed of 900 indicators and 12 behavior definitions is compared on the lifespan axis over a 90-day window: the median life of an address indicator is 5 days, a file hash 2 days, a name 15 days; a behavior definition's median life is 358 days. Of the 900 indicators added by the ninetieth day, only 72 are valid, 828 are dead. Of 60 fictional contacts, 5 match the indicator list, 28 have an indicator not yet on the list that day, and 27 have an indicator whose life has expired. Raising retention from 3 days to 90 days moves caught contacts from 2 to 5 while raising dead matches from 2 to 190 and review time from 36 minutes to 1,755 minutes. Behavior definitions catch 27 contacts and produce no dead matches; catch per item is 5.6 per mille against 2,250 per mille. The previous lesson's six hypotheses all came from the environment's own data: the baseline, the usual parent process, the usual hour. None of them used information about "what has been seen elsewhere." That information comes from outside, and it comes in two separate forms — individual **indicators** and **behavior definitions**. When the two sit side by side on a list, they look like the same thing; both say "look at this." Where they part ways is **lifespan**. An indicator is a specific value, and it becomes useless once that value changes; a behavior definition describes a method, and it stays valid as long as the method does not change. This lesson counts that difference in days and months. The feed below is a **model**; no provider, source, or campaign name appears in it, and the lifespan values are data. - **DI38.** The feed carries 900 indicators: 420 addresses, 380 file hashes, 100 names. Each indicator has a day it was added and a lifespan. - **DI39.** Lifespan is drawn from an exponential distribution; the averages are 6, 3, and 21 days by type. These numbers are **model data**. - **DI40.** When an address indicator's lifespan expires, the value can be **reassigned** elsewhere. The model sets this probability at 0.55 for addresses, 0.10 for names, and 0 for file hashes. - **DI41.** A behavior definition is 12 items, and its lifespan is measured in months. Reassignment does not apply, because what is described is a method, not a value. - **DI42.** Sixty fictional contact events are each tied to an indicator and a day. Fifteen percent of the contacts have no behavior definition at all. #### An Indicator's Lifespan ```js // feed.mjs — the lifespan of an indicator feed and its matching in a 90-day window. THIS IS A MODEL. // The feed has NO provider, source, or campaign name; the types are fictional. export const SEED = 20260408; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = generator(SEED); const randInt = (n) => Math.floor(r() * n); export const WINDOW_DAYS = 90; // [type, item count, average life in days, probability of reassignment once life expires] const TYPE = [["address", 420, 6, 0.55], ["file hash", 380, 3, 0], ["name", 100, 21, 0.1]]; export const INDICATOR = []; for (const [type, n, life, reassign] of TYPE) for (let k = 0; k < n; k++) INDICATOR.push({ type, added: randInt(WINDOW_DAYS), life: Math.max(1, Math.round(-life * Math.log(1 - r()))), reassigned: r() < reassign }); // Behavior definition: describes a method, not a single value. Its life is measured in months. export const BEHAVIOR = Array.from({ length: 12 }, (_, i) => ({ id: `d${i + 1}`, added: randInt(WINDOW_DAYS), life: 120 + randInt(420), scope: 30 + randInt(60) })); // Fictional contact events: each is tied to one indicator and one day. export const CONTACT = Array.from({ length: 60 }, () => ({ day: randInt(WINDOW_DAYS), indicator: randInt(INDICATOR.length), behavior: r() < 0.85 ? randInt(BEHAVIOR.length) : -1 })); // 15 percent have no behavior definition export const valid = (x, day) => day >= x.added && day <= x.added + x.life; const median = (v) => v.slice().sort((a, b) => a - b)[Math.floor(v.length / 2)]; if (import.meta.url === `file://${process.argv[1]}`) { const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yt = fmt([22, 9, 13, 14, 16]); console.log(`model (seed ${SEED}): ${INDICATOR.length} indicators, ${BEHAVIOR.length} behavior ` + `definitions, ${WINDOW_DAYS}-day window, ${CONTACT.length} fictional contacts`); console.log(); console.log(yt(["type", "item", "median life", "avg. life", "reassigned"])); for (const [type] of TYPE) { const l = INDICATOR.filter((g) => g.type === type); console.log(yt([type, l.length, `${median(l.map((g) => g.life))} day`, `${(l.reduce((a, g) => a + g.life, 0) / l.length).toFixed(1)} day`, l.filter((g) => g.reassigned).length])); } console.log(yt(["behavior definition", BEHAVIOR.length, `${median(BEHAVIOR.map((d) => d.life))} day`, `${(BEHAVIOR.reduce((a, d) => a + d.life, 0) / BEHAVIOR.length).toFixed(1)} day`, 0])); const yl = fmt([16, 10, 16, 10, 13]); console.log(); console.log(yl(["life range", "address", "file hash", "name", "total share"])); for (const [lo, hi] of [[1, 2], [3, 7], [8, 14], [15, 30], [31, 9999]]) { const inRange = (l) => l.filter((g) => g.life >= lo && g.life <= hi).length; const all = inRange(INDICATOR); console.log(yl([hi > 999 ? `${lo}+ day` : `${lo}-${hi} day`, inRange(INDICATOR.filter((g) => g.type === "address")), inRange(INDICATOR.filter((g) => g.type === "file hash")), inRange(INDICATOR.filter((g) => g.type === "name")), `${(100 * all / INDICATOR.length).toFixed(1)}%`])); } const yg = fmt([8, 12, 15, 12, 18]); console.log(); console.log(yg(["day", "added", "valid ind.", "dead ind.", "valid behavior"])); for (const d of [0, 15, 30, 45, 60, 89]) { const va = INDICATOR.filter((x) => valid(x, d)).length; const ad = INDICATOR.filter((x) => x.added <= d).length; console.log(yg([d, ad, va, ad - va, BEHAVIOR.filter((x) => valid(x, d)).length])); } const caught = CONTACT.filter((t) => valid(INDICATOR[t.indicator], t.day)); const early = CONTACT.filter((t) => t.day < INDICATOR[t.indicator].added); console.log(`\n${caught.length} of ${CONTACT.length} contacts match the indicator list; ` + `${early.length} have an indicator not yet on the list that day, ` + `${CONTACT.length - caught.length - early.length} have an indicator whose life has expired`); } ``` ``` model (seed 20260408): 900 indicators, 12 behavior definitions, 90-day window, 60 fictional contacts type item median life avg. life reassigned address 420 5 day 6.6 day 239 file hash 380 2 day 3.0 day 0 name 100 15 day 22.5 day 11 behavior definition 12 358 day 317.4 day 0 life range address file hash name total share 1-2 day 126 220 15 40.1% 3-7 day 156 129 16 33.4% 8-14 day 96 31 18 16.1% 15-30 day 38 0 28 7.3% 31+ day 4 0 23 3.0% day added valid ind. dead ind. valid behavior 0 9 9 0 0 15 142 58 84 3 30 285 77 208 4 45 455 77 378 6 60 622 73 549 8 89 900 72 828 12 5 of 60 contacts match the indicator list; 28 have an indicator not yet on the list that day, 27 have an indicator whose life has expired ``` The first table gives the lifespan gap directly: a file hash's median life is two days, an address's is five, a name's is fifteen. A behavior definition's median life is 358 days — on the same axis, twenty-four to one hundred eighty times longer depending on indicator type. This does not mean a behavior definition is better; it means the two items cannot be managed on the same list with the same procedure. The lifespan distribution table opens up what the average hides. Forty percent of the feed dies within one or two days, and the bulk of that share is file hashes: 220 of 380 hashes live shorter than two days, and none exceed fifteen. Names sit at the opposite end — 51 of 100 names live longer than fifteen days. If all three types are carried on the same list, picking a single retention period for the whole list mismanages at least two of the three types. This shape of the distribution also feeds into an operational decision: since 73 percent of items live shorter than a week, pulling the feed once a day means adding a significant share of items to the list at a moment when they are never valid. Unless the feed's pull frequency is on the same scale as the item's lifespan, the list grows but coverage does not. The second table shows how the list behaves. By the ninetieth day, all 900 indicators have been added, but only 72 are valid; 828 are dead. The list grows linearly, while the valid share flattens out between 72 and 77 after the thirtieth day. An indicator feed's size is not a number about its strength; strength is the count of items still valid on that list on that day. The last line gives the first result: only 5 of 60 contacts match the indicator list. The two causes of missing it split almost evenly — in 28 contacts the indicator is not yet on the list that day, in 27 its life has expired. The first number measures the feed's latency, the second measures the indicator's short lifespan, and they call for two separate fixes: the first wants a faster feed, the second wants a longer-lived measure. #### Retention Period and Comparison with Behavior ```js // life.mjs — retention-window scan, the false positives from dead indicators, and comparison with behavior. import { INDICATOR, BEHAVIOR, CONTACT, WINDOW_DAYS, valid } from "./feed.mjs"; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const REVIEW_MIN = 9, MAINT_ITEM = 0.2, MAINT_BEHAVIOR = 30; // An indicator stays on the list from added..added+S days. // If it stays on the list after its life expires and the value has been reassigned, it matches legit traffic. function scan(S) { const r = generator(4413571); let dead = 0, list = 0; for (const g of INDICATOR) { list += Math.min(S, WINDOW_DAYS - g.added); for (let d = g.added + g.life + 1; d <= Math.min(g.added + S, WINDOW_DAYS - 1); d++) if (g.reassigned && r() < 0.02) dead++; } const caught = CONTACT.filter((t) => { const g = INDICATOR[t.indicator]; return t.day >= g.added && t.day <= g.added + Math.min(S, g.life); }).length; return { list: Math.round(list / WINDOW_DAYS), caught, dead, maint: Math.round(INDICATOR.length * MAINT_ITEM) }; } const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([24, 14, 16, 16, 14, 13]); console.log(yz(["feed / retention", "list size", "caught contact", "dead match", "review min", "maint min"])); for (const S of [3, 7, 14, 30, 90]) { const t = scan(S); console.log(yz([`indicator, ${S} day`, t.list, `${t.caught}/${CONTACT.length}`, t.dead, (t.caught + t.dead) * REVIEW_MIN, t.maint])); } const behaviorCaught = CONTACT.filter((t) => t.behavior >= 0 && valid(BEHAVIOR[t.behavior], t.day)).length; const noDef = CONTACT.filter((t) => t.behavior < 0).length; console.log(yz(["behavior definition", BEHAVIOR.length, `${behaviorCaught}/${CONTACT.length}`, 0, behaviorCaught * REVIEW_MIN, BEHAVIOR.length * MAINT_BEHAVIOR])); const end = [3, 7, 14, 30, 90].map(scan); console.log(`\nraising indicator-list retention from 3 to 90 days moves caught ` + `${end[0].caught} -> ${end[4].caught}, dead match ${end[0].dead} -> ${end[4].dead}`); console.log(`of ${CONTACT.length} contacts, ${noDef} have no behavior definition at all; ` + `${CONTACT.length - behaviorCaught - noDef} have a definition but it was not yet added on the contact day`); const perItem = (n, k) => (n / k * 1000).toFixed(1); console.log(`catch per item (per mille): indicator ` + `${perItem(end[4].caught, INDICATOR.length)}, behavior ` + `${perItem(behaviorCaught, BEHAVIOR.length)}`); ``` ``` feed / retention list size caught contact dead match review min maint min indicator, 3 day 30 2/60 2 36 180 indicator, 7 day 68 4/60 13 153 180 indicator, 14 day 131 4/60 45 441 180 indicator, 30 day 255 5/60 101 954 180 indicator, 90 day 451 5/60 190 1755 180 behavior definition 12 27/60 0 243 360 raising indicator-list retention from 3 to 90 days moves caught 2 -> 5, dead match 2 -> 190 of 60 contacts, 10 have no behavior definition at all; 23 have a definition but it was not yet added on the contact day catch per item (per mille): indicator 5.6, behavior 2250.0 ``` The retention scan repeats the fifth lesson's curve on a different axis. Raising retention from three days to ninety moves caught contacts from 2 to 5; in the same step, dead matches rise from 2 to 190 and review load from 36 minutes to 1,755 minutes. **Three extra catches cost 188 extra false matches**, and almost all of it — 185 of the 190 dead matches — comes from expired address indicators colliding with reassigned values. This line says: an indicator list's most expensive part is the part that has never been cleaned out. Between thirty and ninety days, caught contacts stay flat while dead matches climb from 101 to 190. Extending retention in that range **gains nothing** and nearly doubles the cost. A feed's retention period should be read off the feed's own lifespan distribution; if most items die within a week, keeping the list for thirty days does not pay off. The list-size column says the same thing in a different unit: at three-day retention the average list is 30 items, at ninety days it is 451, and for that fifteen-fold growth, caught contacts only rise from 2 to 5. The behavior row is the measure's real result. Twelve items catch 27 contacts, produce no dead matches, and cost 243 minutes of review — a 900-item indicator list caught 5 contacts and cost 1,755 minutes. Catch per item is 5.6 per mille against 2,250 per mille. Maintenance minutes run higher on the behavior side (360 against 180), because reviewing a behavior definition takes far longer than deleting a value from a list; but that maintenance is **for twelve items, not nine hundred**. This comparison does not mean the two feeds are interchangeable. An indicator, when it matches, is an **exact** match: a value either is or is not on the list, and review finishes in a few minutes. Because a behavior definition is a description, matching it also calls for interpretation, and this lesson's model takes the review time as equal for both — in reality, reviewing a behavior match takes longer. What the measure says is this: **the same maintenance minute covers more contacts on the behavior side**, and the indicator side only pays off when the feed's latency is shorter than the lifespan. The behavior side has its own misses too, and the last line writes them out: 10 of 60 contacts have no behavior definition at all, and 23 have a definition that had not been added yet on the day of contact. The second number is the counterpart of the indicator side's latency, and it is larger here because behavior definitions are written more slowly. A behavior definition is long-lived, but it is **born late**. #### Summary - An indicator's lifespan is measured in days, a behavior definition's in months: median 2-15 days against 358 days; the two cannot be managed on the same list with the same procedure. - Of 900 indicators added by the ninetieth day, only 72 are valid; the list grows linearly while the valid share flattens out after the thirtieth day. - Only 5 of 60 contacts match the indicator list; 28 have an indicator not yet added, 27 have one whose life has expired, and the two causes call for two separate fixes. - Raising retention from 3 to 90 days moves caught contacts from 2 to 5 while raising dead matches from 2 to 190 and review time from 36 minutes to 1,755 minutes; between 30 and 90 days, caught contacts do not rise at all. - Twelve behavior definitions catch 27 contacts and produce no dead matches; catch per item is 5.6 per mille against 2,250 per mille. - A behavior definition is born late: 23 of 60 contacts have a definition that was not yet added on the day of contact, and 10 have no definition at all. #### Next Step Across eight lessons, every step of the pipeline has been counted: sources were merged, rules were written, a threshold was chosen, a hypothesis was run, and information coming from outside was separated on the lifespan axis. Every number that came out of it points to the same place — analyst minutes. In this eighth lesson, ninety-day retention cost 1,755 minutes; in the sixth lesson, the second version cost 513 minutes a day; and in the first lesson, daily capacity was 1,440 minutes. None of these lessons dealt with what happens after an alert turns out to be correct. The next lesson closes the topic and measures automating repeated response with two numbers together: minutes saved and the cost of a wrong automated action. ### Response Automation Source: https://academia.sh/en/courses/security-operations/response-automation Eight recurring response procedures produce 158 items and 1,215 minutes of manual work per day; full automation cuts that to 281 minutes but brings 2.65 wrong automatic actions per day and a 457-minute cost, for a net gain of 477 minutes. Three procedures give a net negative under full automation: account lockout -4, address blocking -72, endpoint isolation -72 minutes. Human-approved mode gives a net gain of 715 minutes with zero wrong actions; selecting the best mode per procedure raises the daily net to 870 minutes, and the 3,120-minute write-up work is repaid in 4 days. When the cut legit work item's minute value is taken as 90, full automation drops to -60 minutes at a 1x wrong rate. Across eight lessons every step of the pipeline was counted, and every resulting number pointed at the same place: analyst minutes. In lesson one, daily capacity was 1,440 minutes; in lesson six, the second rule version cost 513 minutes a day; in lesson eight, ninety-day retention asked for 1,755 minutes. This lesson closes the topic and measures the only way to reclaim minutes: handing repeated work to a machine. The measure is not one-directional. Every automated response takes a wrong action **on its own** when the alert turns out wrong, and that action has two costs: undoing it, and the legitimate work it cuts in the meantime. The procedure inventory below is a **model**; the frequencies, durations, and rates are data, not measurement. - **DI43.** There are eight recurring procedures, producing a total of 158 items per day. Each procedure has a manual duration, an automatic duration, and a one-time write-up duration. - **DI44.** Automation coverage is less than 1: a portion of a procedure's items is always handled manually, because its input is incomplete. - **DI45.** There are three modes. **Manual**: every item by a human. **Approved**: the machine prepares it, a human approves it at 1.5 minutes per item. **Full**: the machine prepares and executes it. - **DI46.** A wrong automatic action occurs only in full mode; the model assumes the human catches **all** of them in approved mode. This is an optimistic assumption and should be measured on a real line. - **DI47.** The cost of a wrong action is two items: rollback minutes and the legit work item cut. The second item's minute value is model data, and it is an organizational, not a technical, decision. #### The Procedure Inventory and What Automation Gains ```js // procedure.mjs — recurring response procedures, automation gain, and payback. THIS IS A MODEL. // Numbers are DATA, not measurement; the procedures are fiction for the regional measurement network. // [name, daily count, manual min, automatic min, write min, automation coverage, wrong action rate, // wrong action rollback min, legit work items cut by wrong action] export const PROCEDURE = [ ["indicator lookup", 62, 4, 0.2, 240, 1.00, 0.000, 0, 0], ["file hash lookup", 38, 5, 0.2, 180, 1.00, 0.000, 0, 0], ["user verification prompt", 21, 9, 2.0, 420, 0.85, 0.030, 12, 1], ["identity session termination", 14, 11, 1.0, 360, 0.80, 0.060, 25, 2], ["account lockout", 9, 12, 1.0, 300, 0.75, 0.080, 40, 3], ["address blocking", 7, 16, 1.5, 480, 0.70, 0.110, 55, 6], ["endpoint isolation", 4, 34, 3.0, 600, 0.60, 0.140, 120, 9], ["daily collection request", 3, 26, 4.0, 540, 0.90, 0.010, 30, 0], ]; // Minute value of a cut legit work item (an organizational decision, model data). export const CUT_ITEM_MIN = 35; export const manualMin = (y) => y[1] * y[2]; export function automate(y, mode) { const [, n, manual, auto, write, coverage, wrongRate, rollback, cut] = y; const autoShare = mode === "manual" ? 0 : coverage; const approval = mode === "approved" ? 1.5 : 0; // human approval costs 1.5 min per item const workMin = n * (autoShare * (auto + approval) + (1 - autoShare) * manual); const wrong = mode === "full" ? n * autoShare * wrongRate : 0; const cost = wrong * (rollback + cut * CUT_ITEM_MIN); return { workMin, wrong, cost, net: manualMin(y) - workMin - cost, write: mode === "manual" ? 0 : write }; } if (import.meta.url === `file://${process.argv[1]}`) { const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const yz = fmt([28, 9, 11, 16, 13, 12, 13]); const totalManual = PROCEDURE.reduce((a, y) => a + manualMin(y), 0); console.log(`${PROCEDURE.length} recurring procedures, ` + `${PROCEDURE.reduce((a, y) => a + y[1], 0)} items/day; manual total ${totalManual} min ` + `(lesson one's daily capacity was 1440 min)`); console.log(); console.log(yz(["procedure", "daily", "manual min", "full auto. min", "wrong action", "daily net", "payback"])); for (const y of PROCEDURE) { const t = automate(y, "full"); console.log(yz([y[0], y[1], manualMin(y), Math.round(t.workMin), t.wrong.toFixed(2), Math.round(t.net), t.net > 0 ? `${Math.ceil(y[4] / t.net)} days` : "never"])); } const t = PROCEDURE.map((y) => automate(y, "full")); console.log(`\nfull automation gives a daily net gain of ` + `${Math.round(t.reduce((a, x) => a + x.net, 0))} min, ` + `${t.reduce((a, x) => a + x.wrong, 0).toFixed(2)} wrong automatic actions/day, ` + `cost ${Math.round(t.reduce((a, x) => a + x.cost, 0))} min`); console.log(`total write-up work ${PROCEDURE.reduce((a, y) => a + y[4], 0)} min`); } ``` ``` 8 recurring procedures, 158 items/day; manual total 1215 min (lesson one's daily capacity was 1440 min) procedure daily manual min full auto. min wrong action daily net payback indicator lookup 62 248 12 0.00 236 2 days file hash lookup 38 190 8 0.00 182 1 days user verification prompt 21 189 64 0.54 100 5 days identity session termination 14 154 42 0.67 48 8 days account lockout 9 108 34 0.54 -4 never address blocking 7 112 41 0.54 -72 never endpoint isolation 4 136 62 0.34 -72 never daily collection request 3 78 19 0.03 59 10 days full automation gives a daily net gain of 477 min, 2.65 wrong automatic actions/day, cost 457 min total write-up work 3120 min ``` The first line ties back to the course's first lesson: the eight procedures produce 1,215 minutes of manual work a day, and that lesson's three-person shift had a total capacity of 1,440 minutes. Recurring response eats 84 percent of that capacity; it is one reason 64 alerts went unlooked-at in lesson one. The table's first two rows are automation's easy side. Indicator and file hash lookup change nothing, they only fetch data; their wrong action rate is zero, and automating them gains 418 minutes a day. Both procedures' write-up work is repaid in one or two days. **Automation that fetches rather than changes has no cost.** The bottom three rows are the lesson's real result. Account lockout, address blocking, and endpoint isolation give a **net negative** under full automation: -4, -72, and -72 minutes respectively. Endpoint isolation alone makes the point — only 4 items a day, 136 minutes manual, 62 minutes automated, a 74-minute gain. But there are 0.34 wrong isolations a day, and each one costs 120 minutes of rollback plus 9 cut legitimate work items. In an organization that counts a cut item at 35 minutes, that is 146 minutes a day. Gain 74, cost 146. Where the wrong action rate comes from was also measured in this topic. In lesson five, at the threshold that fit the budget, 3 of 16 alerts were false positives; if a procedure is tied to those alerts, its wrong action rate is directly the rule's precision. Automation **turns detection precision into action**: a false positive worked manually only costs review minutes, but worked automatically it cuts legitimate work. The same precision is paid in two different units under two modes. The wrong action rate and a procedure's destructiveness rise together in this model, and that is not a coincidence: the more a procedure stops, the more complex its input is. The procedures that look the most tempting to automate — the longest-running ones — are the most expensive to automate. #### Three Modes and the Break-Even Point ```js // mode.mjs — three-mode comparison, per-procedure best mode, and a break-even scan. import { PROCEDURE, CUT_ITEM_MIN, manualMin, automate } from "./procedure.mjs"; const fmt = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const totalManual = PROCEDURE.reduce((a, y) => a + manualMin(y), 0); const modeTotal = (mode) => PROCEDURE.reduce((a, y) => { const t = automate(y, mode); return { workMin: a.workMin + t.workMin, wrong: a.wrong + t.wrong, cost: a.cost + t.cost, net: a.net + t.net, write: a.write + t.write }; }, { workMin: 0, wrong: 0, cost: 0, net: 0, write: 0 }); const yz = fmt([12, 11, 13, 13, 12, 11, 13]); console.log(yz(["mode", "work min", "wrong action", "cost min", "net min", "write min", "payback"])); for (const mode of ["manual", "approved", "full"]) { const t = modeTotal(mode); console.log(yz([mode, Math.round(t.workMin), t.wrong.toFixed(2), Math.round(t.cost), Math.round(t.net), t.write, t.net > 0 ? `${Math.ceil(t.write / t.net)} days` : "-"])); } // Best mode: for each procedure, the mode with the highest net gain. const best = PROCEDURE.map((y) => { const top = ["manual", "approved", "full"] .map((m) => ({ m, ...automate(y, m) })) .reduce((a, b) => (b.net > a.net ? b : a)); return { y, top }; }); const ys = fmt([28, 12, 12, 15, 16]); console.log("\nbest mode per procedure:"); console.log(ys(["procedure", "mode", "net min", "wrong action", "cut items"])); for (const { y, top } of best) console.log(ys([y[0], top.m, Math.round(top.net), top.wrong.toFixed(2), (top.wrong * y[8]).toFixed(2)])); const net = best.reduce((a, s) => a + s.top.net, 0); const write = best.reduce((a, s) => a + s.top.write, 0); console.log(`\nselective-mode daily net ${Math.round(net)} min (manual was ${totalManual} min), ` + `write-up work ${write} min, payback ${Math.ceil(write / net)} days`); // Break-even point: if the wrong action rate and the cut item's minute value change. const yk = fmt([14, 14, 14, 16, 18]); console.log("\nbreak-even point under full automation:"); console.log(yk(["wrong multiplier", "item min 10", "item min 35", "item min 90", "profitable procs"])); for (const c of [0.25, 0.5, 1, 2, 4]) { const calc = (kd) => PROCEDURE.reduce((a, y) => { const [, n, , , , coverage, rate, rollback, cut] = y; const t = automate(y, "full"); const wrong = n * coverage * rate * c; return a + (manualMin(y) - t.workMin - wrong * (rollback + cut * kd)); }, 0); const profitable = PROCEDURE.filter((y) => { const [, n, , , , coverage, rate, rollback, cut] = y; const t = automate(y, "full"); return manualMin(y) - t.workMin - n * coverage * rate * c * (rollback + cut * CUT_ITEM_MIN) > 0; }).length; console.log(yk([`${c}x`, Math.round(calc(10)), Math.round(calc(35)), Math.round(calc(90)), `${profitable}/${PROCEDURE.length}`])); } ``` ``` mode work min wrong action cost min net min write min payback manual 1215 0.00 0 0 0 - approved 500 0.00 0 715 3120 5 days full 281 2.65 457 477 3120 7 days best mode per procedure: procedure mode net min wrong action cut items indicator lookup full 236 0.00 0.00 file hash lookup full 182 0.00 0.00 user verification prompt full 100 0.54 0.54 identity session termination approved 95 0.00 0.00 account lockout approved 64 0.00 0.00 address blocking approved 64 0.00 0.00 endpoint isolation approved 71 0.00 0.00 daily collection request full 59 0.03 0.00 selective-mode daily net 870 min (manual was 1215 min), write-up work 3120 min, payback 4 days break-even point under full automation: wrong multiplier item min 10 item min 35 item min 90 profitable procs 0.25x 881 820 686 8/8 0.5x 827 705 437 7/8 1x 721 477 -60 5/8 2x 508 20 -1054 4/8 4x 81 -894 -3041 4/8 ``` The three-mode table gives an unexpected order: **approved mode gains more than full automation**. Full automation cuts work time to 281 minutes, approved mode leaves it at 500; but full automation carries a 457-minute cost and its net gain stays at 477, while approved mode's is 715. Two hundred nineteen minutes of extra approval work prevents four hundred fifty-seven minutes of wrong-action cost. The right answer is not picking a single mode, it is picking per procedure. The second table does this: the four data-fetching procedures land on full automation, the four that change something land on approved mode. The daily net rises to 870 minutes — 72 percent of the 1,215 manual minutes — and the 3,120-minute write-up work is repaid in **four days**. The cut legitimate work item column is 0.54 a day under this selection; full automation had 2.65 wrong actions a day and far more cut items. The last table shows sensitivity to two unknowns and closes out the course's measure. If the wrong action rate quadruples, full automation is still positive (81 minutes) when a cut item is counted at 10 minutes; it is -894 at 35 minutes, and -3,041 at 90 minutes. Same line, same procedures, same wrong rate — the only thing that flips the result is **how many minutes the cut legitimate work is counted as**. That number is not a security metric, and it cannot be set by the security team. The right-hand column says something else: even if the wrong rate quadruples, four of the eight procedures stay positive. These are the data-fetching procedures, and their wrong action rates are already close to zero. Automation's safe zone is work that **changes nothing**; for every job that changes something, the decision comes down to weighing the minutes gained against the legitimate work cut. #### Summary - Eight recurring procedures produce 158 items and 1,215 minutes of manual work a day; that is 84 percent of lesson one's 1,440-minute daily capacity. - Automation has no cost on data-fetching procedures: indicator and file hash lookup gain 418 minutes a day, and the write-up work is repaid in one or two days. - Three of the change-making procedures give a net negative under full automation: endpoint isolation gains 74 minutes but 0.34 wrong isolations a day cost 146 minutes. - Approved mode gains more than full automation: 715 versus 477 minutes, because 219 minutes of extra approval work prevents 457 minutes of wrong-action cost. - Selecting the mode per procedure raises the daily net to 870 minutes, and the 3,120-minute write-up work is repaid in four days; the cut legitimate work item drops to 0.54 a day — full automation had 2.65 wrong actions a day and far more cut items. - When the cut legitimate work's minute value is taken as 90 instead of 10, full automation drops to -60 minutes at a 1x wrong rate; that number is an organizational, not a technical, decision. #### Next Step This topic built a pipeline and counted every step of it: sources were mapped to a single schema and the unmapped fields were counted, a correlation rule was run and what it missed was written down, a threshold scan measured the band where false positives and misses cannot be reduced at the same time, rule versions were compared, hypotheses were run, feeds were split along a lifespan axis, and recurring response was automated. All of these numbers belonged to detection time — the time **before** the moment an alert turns out **right** — containment time was never measured. What happens after it turns out right was never written down. In this lesson's approved mode, who approves, under what authority endpoint isolation is applied, who decides when nine legitimate work items are cut, and who is informed after the decision — these stand in the model as mere numbers, but in an organization they are a role, an authority boundary, and a sequence of steps. The next topic, **Incident Response**, addresses exactly this: from preparation to identification, containment to eradication and recovery — which step is taken by whom, under what authority, and in what time. ## Incident Response Measuring the time between detection and containment in items: how many minutes preparation items cut from the decision chain, how many assets the incident's verification and scope radius miss, the item gap between early and late containment in the same incident, removing persistence in terms of foothold points, supervised recovery along the axis of reinfection risk, lessons learned along the axis of evidence and scope, and notification obligations measured as duration and stakeholder count. ### Preparation Source: https://academia.sh/en/courses/security-operations/preparation Counting preparation in items: how twelve preparation items cut the six-step chain from alert to decision from 470 minutes to 182 minutes, pulling the containment moment from 656 to 368 and the items reached from 13 to 9, how the asset inventory's absence alone adds 49 minutes, how the communication tree earns 5.00 minutes per person-hour while forensic collection readiness stays at 0.11, and how pre-approved authority earns 103 minutes a year while cutting 233 legitimate work items. The previous topic built the detection pipeline: sources were collected, normalized, correlation rules were written, thresholds were tuned, and repeated response was automated. The question the topic left behind was this: what happens when an alert turns out to be right. The pipeline produced a correct alert; from here on, what gets counted is no longer the quality of the rule but the time between the moment the alert fires and the moment access stops. Throughout this topic the unit of measure is twofold: minutes and items. Time to detect is the output of the detection pipeline and is given here as an input. What gets measured is time to contain — how many minutes pass between the alert and access stopping, and how many items are reached in those minutes. Preparation is this duration's individually countable component: a plan, an authority matrix, or a communication tree is either ready on the day the incident starts or is written during the incident. The latter adds minutes. #### Chain from Alert to Decision The example is the fictional regional measurement network carried through the courses: branch collectors, a validator, billing, a subscriber portal, and work order services; a three-person shift. Both the incident and the records are fictional; there is no real organization, address, or event. **IR1:** The alert fires 186 minutes after the incident starts and is correct; this value is the input coming from the detection pipeline. **IR2:** The access curve consists of fourteen items; each item becomes accessible at a specific minute and that item never closes again. The curve is data, not a measurement; no step sequence belonging to the attack side is written, only which item is accessed at which minute. **IR3:** The chain from alert to containment is six steps: verification, scope, authority, decision, selection, execution. Each step has an unprepared baseline number of minutes. **IR4:** There are twelve preparation items; a ready item shortens the minutes of the step it affects by a multiplier, a missing item does nothing — the step stays at its baseline. **IR5:** Each preparation item's annual maintenance cost is in person-hours. **IR6:** Access stops the moment containment is applied; the number of items already reached by that point is the cost of a late decision. ```js // preparation/model.mjs — incident response preparation model: no real incident record. // The access curve, chain steps, and preparation items are data structures; durations are // fictional data. export const ALERT = 186; // minute the alert fires (output of the detection pipeline) // access curve: [minute, item name] — the item becomes accessible at that minute (data) export const ACCESS = [ [0, "run on collector-03"], [26, "collector-03 local identity store"], [58, "validator-01 service account"], [95, "collector-07"], [134, "collector-11"], [171, "validator-02"], [214, "billing read endpoint"], [263, "subscriber portal copy"], [318, "work order queue"], [372, "field laptop management channel"], [441, "backup store listing"], [505, "billing archive export"], [588, "second-region collector set"], [676, "management console read"]]; export const items = (dk) => ACCESS.filter(([t]) => t <= dk).length; // chain step: name, unprepared baseline minutes export const STEP = [["verification", 68], ["scope", 124], ["authority", 96], ["decision", 82], ["selection", 54], ["execution", 46]]; // preparation item: name, steps it affects, step multiplier, annual maintenance person-hours export const PREP = [ ["response plan", ["selection"], 0.55, 16], ["authority matrix", ["authority"], 0.45, 8], ["communication tree", ["decision"], 0.50, 6], ["asset inventory", ["scope"], 0.55, 24], ["log source map", ["verification"], 0.50, 12], ["backup access channel", ["execution"], 0.75, 10], ["pre-approved containment authority", ["authority"], 0.55, 4], ["identity response playbook", ["selection"], 0.65, 14], ["forensic collection readiness", ["execution"], 0.90, 18], ["external response support", ["decision"], 0.85, 5], ["tabletop exercise", STEP.map(([a]) => a), 0.88, 30], ["restore test", ["execution"], 0.85, 22]]; // chain step minutes and total for a given set of ready items export function chain(ready) { const dk = Object.fromEntries(STEP); for (const [name, steps, mult] of PREP) if (ready.includes(name)) for (const a of steps) dk[a] *= mult; const total = Math.round(Object.values(dk).reduce((s, v) => s + v, 0)); return { dk, total, containment: ALERT + total, items: items(ALERT + total) }; } export const FULL = PREP.map(([a]) => a); export const PARTIAL = ["response plan", "authority matrix", "communication tree", "asset inventory", "log source map", "backup access channel"]; // annual alert set: 6 of 24 verified alerts are real incidents (data) export const ALERTS_PER_YEAR = 24, REAL_PER_YEAR = 6, CUT_PER_HOUR = 26; ``` ```js // preparation/measure.mjs — how many minutes the decision shortens when a preparation item // is ready, how many minutes and how many reached items a missing item adds, what pre-approved // authority's two-sided cost is. import { ALERT, STEP, PREP, FULL, PARTIAL, chain, items, ALERTS_PER_YEAR, REAL_PER_YEAR, CUT_PER_HOUR } from "./model.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); const D = { unprepared: chain([]), partial: chain(PARTIAL), full: chain(FULL) }; console.log(`alert fires at minute ${ALERT}; ${PREP.length} preparation items, ` + `${STEP.length} chain steps`); console.log("\n1. chain from alert to decision (minutes)"); const A = [-24, 14, 12, 10]; write(A, "step", "unprepared", "partial (6)", "full (12)"); for (const [a] of STEP) write(A, a, ...Object.values(D).map((r) => r.dk[a].toFixed(0))); write(A, "total", ...Object.values(D).map((r) => r.total)); write(A, "containment minutes", ...Object.values(D).map((r) => r.containment)); write(A, "items reached", ...Object.values(D).map((r) => `${r.items}/14`)); write(A, "annual person-hours", ...Object.entries({ unprepared: [], partial: PARTIAL, full: FULL }) .map(([, k]) => PREP.filter(([a]) => k.includes(a)) .reduce((s, h) => s + h[3], 0))); console.log("\n2. minutes added to the chain when an item is missing (removed one at a time from the full set)"); const B = [-38, 11, 10, 12, 14, 15]; write(B, "missing item", "total min", "+minutes", "containment", "annual p-hr", "min/p-hr"); const t = D.full; const ranked = PREP.map((h) => [h, chain(FULL.filter((a) => a !== h[0]))]) .sort((x, y) => y[1].total - x[1].total); for (const [h, r] of ranked) write(B, h[0], r.total, "+" + (r.total - t.total), r.containment, h[3], ((r.total - t.total) / h[3]).toFixed(2)); write(B, "(none missing)", t.total, "-", `${t.containment} (${t.items}/14)`, PREP.reduce((s, h) => s + h[3], 0), "-"); console.log("\n3. two sides of pre-approved containment authority (" + `${ALERTS_PER_YEAR} verified alerts a year, ${REAL_PER_YEAR} real incidents)`); const C = [-34, 14, 12, 13]; write(C, "run", "authority min", "annual gain", "annual cut"); for (const [a, k] of [["no pre-approval", FULL.filter((a) => a !== "pre-approved containment authority")], ["pre-approval", FULL], ["pre-approval, no log source map", FULL.filter((a) => a !== "log source map")]]) { const r = chain(k), approved = k.includes("pre-approved containment authority"); const gain = approved ? REAL_PER_YEAR * (chain(k.filter((a) => a !== "pre-approved containment authority")).dk.authority - r.dk.authority) : 0; const cut = approved ? (ALERTS_PER_YEAR - REAL_PER_YEAR) * (r.dk.verification / 60) * CUT_PER_HOUR : 0; write(C, a, r.dk.authority.toFixed(0), gain.toFixed(0) + " min", cut.toFixed(0) + " items"); } console.log("\n4. items reached as containment minute is scanned"); const E = [-16, 9, 9, 9, 9, 9, 9]; const scan = [200, 280, 362, 470, 560, 656]; write(E, "containment min", ...scan); write(E, "items reached", ...scan.map((d) => `${items(d)}/14`)); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band console.log(`\nthree levels and ${PREP.length} marginal runs within the 500 ms band: ` + (ms < 500 ? "yes" : "no")); ``` ``` alert fires at minute 186; 12 preparation items, 6 chain steps 1. chain from alert to decision (minutes) step unprepared partial (6) full (12) verification 68 34 30 scope 124 68 60 authority 96 43 21 decision 82 41 31 selection 54 30 17 execution 46 35 23 total 470 251 182 containment minutes 656 437 368 items reached 13/14 10/14 9/14 annual person-hours 0 76 169 2. minutes added to the chain when an item is missing (removed one at a time from the full set) missing item total min +minutes containment annual p-hr min/p-hr asset inventory 231 +49 417 24 2.04 communication tree 212 +30 398 6 5.00 log source map 212 +30 398 12 2.50 authority matrix 207 +25 393 8 3.13 tabletop exercise 207 +25 393 30 0.83 pre-approved containment authority 199 +17 385 4 4.25 response plan 196 +14 382 16 0.88 identity response playbook 191 +9 377 14 0.64 backup access channel 189 +7 375 10 0.70 external response support 187 +5 373 5 1.00 restore test 186 +4 372 22 0.18 forensic collection readiness 184 +2 370 18 0.11 (none missing) 182 - 368 (9/14) 169 - 3. two sides of pre-approved containment authority (24 verified alerts a year, 6 real incidents) run authority min annual gain annual cut no pre-approval 38 0 min 0 items pre-approval 21 103 min 233 items pre-approval, no log source map 21 103 min 467 items 4. items reached as containment minute is scanned containment min 200 280 362 470 560 656 items reached 6/14 8/14 9/14 11/14 12/14 13/14 three levels and 12 marginal runs within the 500 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Three Preparation Levels The first table converts preparation into minutes. With no item ready, the chain takes 470 minutes: the alert fires at minute 186, containment is applied at minute 656, and by then thirteen of the fourteen items have been reached. With six items it drops to 251 minutes, with twelve items to 182 minutes; containment is pulled to minute 368 and the items reached drop to nine. The step breakdown shows where preparation touches. The two largest baselines are the scope (124 minutes) and authority (96 minutes) steps — that is, the questions "which assets were affected" and "who can cut access." With full preparation, scope drops to 60 and authority to 21 minutes. The verification step drops from 68 to only 30 and remains the chain's third-longest step even at full preparation; verification is not a paperwork task but a record-reading task, and it has a ceiling that preparation can only shorten so far. The difference between the two levels is not evenly distributed. The first six items require 76 person-hours a year and cut 219 minutes from the chain — 2.88 minutes per person-hour. The second six items require 93 more person-hours and cut only 69 minutes; 0.74 minutes per person-hour. Preparation's return is a diminishing curve: the first items touch the chain's longest steps, the later ones split what is left. The fourth table shows the same curve on its own. Pulling containment from minute 656 to 368 saves four items; pulling it from 368 to 280 saves one more. The curve is not flat: the item count rises fast in the early minutes, then thins out. This means preparation's return is not constant either. #### Minutes per Item The second table counts preparation as items. When a single item is removed from the full set, the minutes added to the chain are that item's real value. The asset inventory's absence is the most expensive: it alone adds 49 minutes, because the scope step both has the longest baseline and, without an inventory, there is no starting list at all. The communication tree and the log source map each add thirty minutes; the authority matrix and the tabletop exercise each add twenty-five. What these three items actually are gives the number its meaning. The asset inventory is the list that holds which machine runs which service and which network segment it sits on; if it is missing during an incident, the scope step first has to produce that list itself. The authority matrix writes down which role can carry out which action with which approval: cutting a network segment, locking a service account, or taking a field unit off the network are separate lines. The communication tree wires in advance which information goes to whom through which channel; if this tree is missing during an incident, the decision step is spent hunting for the right person. The last column settles it: minutes per person-hour. The communication tree needs six person-hours of maintenance a year and earns 5.00 minutes per person-hour; pre-approved authority earns 4.25 with four person-hours; the authority matrix earns 3.13. The tabletop exercise is the most expensive item at thirty person-hours and stays at 0.83 — but it does not scale a single step, it scales all six at once, so its reach is broader than the table shows. The two items at the bottom of the list deserve attention. The restore test yields 0.18 and forensic collection readiness 0.11 minutes per person-hour; together they spend forty person-hours and cut only six minutes from the chain. The only thing measured in this lesson is decision minutes. Forensic collection readiness's payoff is not in this table — it is where whether evidence was collected gets counted; the restore test's payoff shows up when how many stages recovery takes is measured. A preparation item measured on a single axis, if cut because it looks cheap on that axis, gets paid for on another axis. #### Two Sides of Preparation The third table shows that preparation is not free. Pre-approved containment authority lets the shift cut access without waiting for senior approval. **IR7:** 24 verified alerts are reviewed a year, six of which are real incidents; with pre-approval, the cut lasts until the alert is ruled out — that is, for the duration of the verification step — and cuts 26 legitimate work items an hour. Without pre-approval the authority step is 38 minutes, with pre-approval 21 minutes; across six real incidents it earns 103 minutes a year. The same authority cuts a total of 233 legitimate work items across the eighteen alerts that turn out not to be real. The third row shows that two preparation items are linked to each other. When the log source map is missing, verification rises from 30 to 60 minutes; the minutes pre-approved authority earns stay the same (103), but the legitimate work it cuts rises to 467 items. The cost of early-cut authority depends on how quickly that cut can be reversed. Early-cut authority granted without fast verification produces a cost twice the minutes it earns. Preparation items are not independent rows; one item's value changes depending on whether another item is ready, and minutes summed one at a time do not show this dependency. #### Summary - The chain from alert to containment is six steps; it takes 470 minutes unprepared, 251 with six items, and 182 with twelve items. The containment moment drops from 656 to 368, and items reached drop from 13 to 9. - The two largest steps are scope (124-minute baseline) and authority (96 minutes); preparation brings these down to 60 and 21 minutes, while verification drops from 68 to only 30 and stays third in the chain. - The cost of a single missing item is counted separately: asset inventory +49, communication tree and log source map +30, authority matrix and tabletop exercise +25 minutes. - Gain per person-hour is 5.00 minutes for the communication tree and 4.25 for pre-approved authority; forensic collection readiness stays at 0.11 — its payoff is not on this axis, it is measured when evidence is counted. - Pre-approved authority earns 103 minutes a year and cuts 233 legitimate work items; with the log source map missing, the same authority's cut rises to 467 items. #### Next Step The chain's first two steps were just single minute values in this lesson: verification 30, scope 60. Both have to answer a single question — is this alert really an incident, and how many assets were affected. The next lesson goes inside these two steps: which asset the alert points to, how many steps out from that asset you need to look, how many of the assets examined have logs, and how many affected assets a narrow scope leaves off the list. ### Identification Source: https://academia.sh/en/courses/security-operations/identification Measuring scope determination in candidates and minutes: how the seven scope criteria bring between 1 and 19 candidates in a thirty-asset inventory, how the scope step ranges between 3 and 73 minutes, how the log indicator's four candidates in 10 minutes leave the same number of items open as radius two's seventeen candidates in 68 minutes, how the widest criterion cuts legitimate work unnecessarily on thirteen assets, and how two reached collectors cannot be verified by any criterion because they transmit only network flow data. The previous lesson split the chain into six steps and counted how many minutes preparation cut from each step. At full preparation, two steps stood out: verification at 30 minutes, scope at 60 minutes. Both answer a single question — is this alert really an incident, and how many assets were affected. The scope step's 60 minutes was a fixed number in the previous lesson; in this lesson that number is derived from the chosen scope criterion. Identification does two jobs: verification and scope. The second is a balancing problem, and both sides of the balance are countable. If scope is kept narrow, the scope step shortens and containment is applied early — but access does not stop on affected assets that never made the list. If scope is kept wide, the scope step lengthens, the curve keeps advancing for every minute containment is delayed, and legitimate work is cut unnecessarily on unaffected assets. #### Asset, Log, and Access Curve **IR8:** The fictional network's inventory is thirty assets: twelve branch collectors, three validators, two billing systems, two portals, one work order service, one backup store, one management console, and eight field units. The neighborhood graph gives which asset talks to which. **IR9:** Each asset transmits specific log sources to the central pipe. Whether an asset was affected can only be verified if it transmits a process, identity, or application log entry; on an asset that transmits only network flow data, the connection is visible but execution or a session cannot be verified. **IR10:** The access curve is eighteen items; each item is reached at one minute on one asset. An asset is affected from the moment its name appears in the curve. **IR11:** The scope criterion determines the candidate set: topological radius, shared service account, log indicator, or a combination of these. The indicator set is read at the moment the verification step ends; an asset that has not dropped into the log by that point is not in the indicator set. **IR12:** The scope step's minutes are derived from the candidate count — a candidate that transmits a verifiable log entry needs 7 minutes of review, one that does not needs 19, and the three-person shift works in parallel. **IR13:** The chain's steps outside scope are at the previous lesson's full-preparation values: verification 30, the remaining four steps 92 minutes total. **IR14:** Containment is applied to every asset in scope; on an affected asset outside scope, access continues; on an unaffected asset in scope, legitimate work is cut unnecessarily. ```js // identification/model.mjs — scope-determination model: no real network or incident record. // The asset inventory, neighborhood graph, log transmission, and access curve are data structures. export const ALERT = 186, FLAGGED = "collector-03"; // asset the alert flags export const VERIFICATION = 30, REMAINING = 92; // chain steps outside scope export const SHIFT = 3; // people on shift export const MIN_LOGGED = 7, MIN_UNCERTAIN = 19; // review minutes per candidate const make = (prefix, n) => Array.from({ length: n }, (_, i) => `${prefix}-${String(i + 1).padStart(2, "0")}`); // asset: name -> { sources it transmits, uses a shared service account? } export const ASSET = {}; for (const [i, a] of make("collector", 12).entries()) ASSET[a] = { sources: i < 6 ? ["process", "network"] : ["network"], account: true }; for (const a of make("validator", 3)) ASSET[a] = { sources: ["process", "identity"], account: true }; for (const a of make("billing", 2)) ASSET[a] = { sources: ["identity", "application"], account: false }; for (const a of make("portal", 2)) ASSET[a] = { sources: ["application"], account: false }; ASSET["workorder-01"] = { sources: ["application"], account: false }; ASSET["store-01"] = { sources: [], account: false }; ASSET["console-01"] = { sources: ["identity"], account: false }; for (const [i, a] of make("field", 8).entries()) ASSET[a] = { sources: i < 5 ? ["process"] : [], account: false }; // neighbor: which asset talks to which (undirected) export const NEIGHBOR = {}; const link = (a, b) => { (NEIGHBOR[a] ??= []).push(b); (NEIGHBOR[b] ??= []).push(a); }; make("collector", 12).forEach((t, i) => link(t, `validator-${String(1 + (i / 4 | 0)).padStart(2, "0")}`)); for (const d of make("validator", 3)) { link(d, "billing-01"); link(d, "billing-02"); link(d, "store-01"); } for (const f of make("billing", 2)) { link(f, "portal-01"); link(f, "portal-02"); } for (const u of make("field", 8)) link(u, "workorder-01"); for (const t of make("collector", 12)) link("console-01", t); // access curve: [minute, asset, item name] — the item is reached at that minute (data) export const ACCESS = [ [0, "collector-03", "execution"], [26, "collector-03", "local identity store"], [58, "validator-01", "service account"], [95, "collector-07", "collector handoff"], [134, "collector-11", "collector handoff"], [171, "validator-02", "second validator"], [214, "billing-01", "read endpoint"], [263, "portal-01", "subscriber copy"], [318, "workorder-01", "work order queue"], [372, "field-01", "management channel"], [441, "store-01", "backup listing"], [505, "billing-01", "archive export"], [588, "collector-09", "second-region set"], [676, "console-01", "console read"], [762, "portal-02", "second portal copy"], [845, "store-01", "backup export"], [951, "billing-02", "second billing endpoint"], [1088, "console-01", "console write authority"]]; export const items = (dk) => ACCESS.filter(([t]) => t <= dk); export const affected = (dk) => [...new Set(items(dk).map(([, v]) => v))]; // assets reached from the flagged asset within radius r export function radius(r) { let set = new Set([FLAGGED]); for (let i = 0; i < r; i++) set = new Set([...set, ...[...set].flatMap((a) => NEIGHBOR[a] ?? [])]); return [...set]; } export const accountSet = () => Object.keys(ASSET).filter((a) => ASSET[a].account); // log indicator: among assets reached by that point, those that dropped an indicator into the log export const indicatorSet = (dk) => affected(dk).filter((a) => ASSET[a].sources.some((k) => k !== "network")); export const verifiable = (a) => ASSET[a].sources.some((k) => k !== "network"); export function scopeMinutes(candidates) { const k = candidates.filter(verifiable).length, b = candidates.length - k; return { candidates: candidates.length, logged: k, uncertain: b, min: Math.ceil((k * MIN_LOGGED + b * MIN_UNCERTAIN) / SHIFT) }; } ``` The measurement file runs seven criteria on the same incident and produces four numbers for each criterion: how many candidates, how many minutes the scope step takes, at which minute containment lands, and how many items are still open at that point. ```js // identification/measure.mjs — how many candidates a scope criterion brings, how many are // verifiable, where the scope minute moves containment, how many items an asset left out of // scope leaves open. import { ALERT, FLAGGED, VERIFICATION, REMAINING, ASSET, ACCESS, items, affected, radius, accountSet, indicatorSet, verifiable, scopeMinutes } from "./model.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); const union = (...k) => [...new Set(k.flat())]; const VERIFY_END = ALERT + VERIFICATION; // the moment verification ends: the indicator set is read here const CRITERIA = [["radius 0", radius(0)], ["radius 1", radius(1)], ["radius 2", radius(2)], ["shared service account", accountSet()], ["log indicator", indicatorSet(VERIFY_END)], ["indicator + account", union(indicatorSet(VERIFY_END), accountSet())], ["radius 2 + account", union(radius(2), accountSet())]]; const ALL = Object.keys(ASSET); console.log(`${ALL.length} assets, ${ALL.filter(verifiable).length} of them transmit verifiable log ` + `entries; the alert flagged the "${FLAGGED}" asset at minute ${ALERT}`); console.log(`verification finished at minute ${VERIFY_END}; assets reached by then ` + `${affected(VERIFY_END).length}: ` + affected(VERIFY_END).join(", ")); const R = CRITERIA.map(([name, set]) => { const k = scopeMinutes(set), containment = ALERT + VERIFICATION + k.min + REMAINING; const aff = affected(containment), covered = aff.filter((a) => set.includes(a)); const entries = items(containment), closed = entries.filter(([, v]) => set.includes(v)); return { name, set, ...k, containment, aff, covered, entries, closed, // verified: in scope, transmits logs, and actually reached verified: set.filter((a) => verifiable(a) && aff.includes(a)).length, ruledOut: set.filter((a) => verifiable(a) && !aff.includes(a)).length, affUncertain: aff.filter((a) => set.includes(a) && !verifiable(a)).length }; }); console.log("\n1. scope criterion, candidate count, and scope-step minutes"); const A = [-25, 7, 9, 10, 11, 13]; write(A, "criterion", "cand.", "logged", "uncertain", "scope min", "contain min"); for (const r of R) write(A, r.name, r.candidates, r.logged, r.uncertain, r.min, r.containment); console.log("\n2. what scope sees and misses (at the moment of containment)"); const B = [-25, 11, 10, 11, 9, 7, 20]; write(B, "criterion", "affected", "covered", "missed", "closed", "open", "cut unnecessarily"); for (const r of R) write(B, r.name, r.aff.length, r.covered.length, r.aff.length - r.covered.length, r.closed.length, r.entries.length - r.closed.length, r.candidates - r.covered.length); console.log("\n3. candidate verification outcome"); const C = [-25, 12, 11, 10, 24]; write(C, "criterion", "verified", "ruled out", "uncertain", "in scope but uncertain"); for (const r of R) write(C, r.name, r.verified, r.ruledOut, r.uncertain, r.affUncertain); console.log("\n4. marginal cost of expanding scope"); const D = [-38, 8, 10, 12, 10, 15]; write(D, "expansion", "+cand.", "+minutes", "+covered", "-open", "min/item"); const find = (name) => R.find((r) => r.name === name); for (const [x, y] of [["radius 0", "radius 1"], ["radius 1", "radius 2"], ["radius 2", "radius 2 + account"], ["log indicator", "indicator + account"], ["log indicator", "radius 2 + account"]]) { const a = find(x), b = find(y); const openA = a.entries.length - a.closed.length, openB = b.entries.length - b.closed.length; write(D, `${x} -> ${y}`, b.candidates - a.candidates, b.min - a.min, b.covered.length - a.covered.length, openA - openB, openA === openB ? "-" : ((b.min - a.min) / (openA - openB)).toFixed(1)); } console.log("\n5. reached but not verifiable by any log"); const widest = find("radius 2 + account"); for (const a of widest.aff.filter((x) => !verifiable(x))) write([-4, -16, -32, -14], "", a, `sources it transmits: ${ASSET[a].sources.join(", ") || "none"}`, widest.set.includes(a) ? "in scope" : "out of scope"); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // raw measurement is not printed, only the band is checked console.log(`\nseven criteria and four scans within the 500 ms band: ${ms < 500 ? "yes" : "no"}`); ``` ``` 30 assets, 20 of them transmit verifiable log entries; the alert flagged the "collector-03" asset at minute 186 verification finished at minute 216; assets reached by then 6: collector-03, validator-01, collector-07, collector-11, validator-02, billing-01 1. scope criterion, candidate count, and scope-step minutes criterion cand. logged uncertain scope min contain min radius 0 1 1 0 3 311 radius 1 3 3 0 7 315 radius 2 17 10 7 68 376 shared service account 15 9 6 59 367 log indicator 4 4 0 10 318 indicator + account 16 10 6 62 370 radius 2 + account 19 12 7 73 381 2. what scope sees and misses (at the moment of containment) criterion affected covered missed closed open cut unnecessarily radius 0 7 1 6 2 6 0 radius 1 7 2 5 3 5 1 radius 2 9 5 4 6 4 12 shared service account 8 5 3 6 3 10 log indicator 8 4 4 5 4 0 indicator + account 8 6 2 7 2 10 radius 2 + account 9 6 3 7 3 13 3. candidate verification outcome criterion verified ruled out uncertain in scope but uncertain radius 0 1 0 0 0 radius 1 2 1 0 0 radius 2 3 7 7 2 shared service account 3 6 6 2 log indicator 4 0 0 0 indicator + account 4 6 6 2 radius 2 + account 4 8 7 2 4. marginal cost of expanding scope expansion +cand. +minutes +covered -open min/item radius 0 -> radius 1 2 4 1 1 4.0 radius 1 -> radius 2 14 61 3 1 61.0 radius 2 -> radius 2 + account 2 5 1 1 5.0 log indicator -> indicator + account 12 52 2 2 26.0 log indicator -> radius 2 + account 15 63 2 1 63.0 5. reached but not verifiable by any log collector-07 sources it transmits: network in scope collector-11 sources it transmits: network in scope seven criteria and four scans within the 500 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Seven Scope Criteria The first table converts candidate count into minutes. Radius zero is a single candidate and the scope step takes 3 minutes; radius one is three candidates at 7 minutes. Radius two jumps to seventeen candidates because the whole collector set enters the list through the validators and the management console — the scope step rises to 68 minutes. The shared service account yields fifteen candidates and 59 minutes. The log indicator is four candidates and **10 minutes**: this criterion takes the assets that had dropped an indicator into the central pipe by the moment verification ends, and none of them is uncertain, because by definition all of them transmit logs. The second table compares the same criteria against the outcome. At radius zero, containment is applied at minute 311; by then seven assets are affected and only one of them is in scope; two of the eight items close, **six stay open**. Radius two waits until minute 376, covers five of nine affected assets, and leaves four items open. The log indicator contains at minute 318, covers four of eight affected assets, and **also leaves four items** open. Radius two's thirteen extra candidates and 58 extra minutes do not change the number of items left open. The best outcome comes from the combination of indicator and account: sixteen candidates, 62 minutes, containment at minute 370, and **two open items**. The combination of radius two and account is wider at nineteen candidates, but the scope step rises to 73 minutes, containment slips to minute 381, and open items rise to three. A wider scope does not close more items; a delayed scope just lets the curve keep advancing. #### Cut Unnecessarily and Unverifiable The second table's last column is the decision's other side. Containment is applied to every asset in scope; on every asset that is in scope but unaffected, legitimate work is cut. Radius zero and the log indicator are zero in this column — every candidate they bring is genuinely affected. Radius two cuts legitimate work unnecessarily on twelve assets, indicator plus account on ten, radius two plus account on **thirteen**. The log indicator's precision shows here: all four of its four candidates are affected. The third table opens up the candidate set. Of radius two plus account's nineteen candidates, four are verified, eight are ruled out, and seven stay **uncertain** — that is, neither verifiable nor ruleable-out, because those assets do not transmit a verifiable log. The last column pulls the most annoying subset out of this: the number of assets that are genuinely affected yet stay uncertain is two. The fifth table names these two assets. `collector-07` and `collector-11` appear in the access curve at minutes 95 and 134; both transmit only network flow data. They are in scope and containment is applied to them too, but that they were affected cannot be verified by any log. On an asset that transmits no logs, the containment decision is either made by the scope criterion or not made at all. #### The Marginal Cost of Expanding The fourth table converts the cost of moving scope from one criterion to another into minutes per item. Going from radius zero to radius one costs two candidates and 4 minutes, and closes one item: 4.0 minutes per item. Going from radius one to radius two costs fourteen candidates and 61 minutes and still closes only one item: **61.0 minutes per item**. Going from radius two to adding account closes one more item for two candidates and 5 minutes. The cheaper of the two transitions that close two items at once is going from indicator to indicator plus account: two items for twelve candidates and 52 minutes, that is, 26.0 minutes per item. Topological radius is an expensive criterion on this network: because the management console is linked to every collector, the second radius step puts half the inventory on the list at once, and most of the assets it brings are unaffected. #### Summary - Twenty of the thirty-asset inventory transmit verifiable logs; the seven scope criteria bring between 1 and 19 candidates, and the scope step ranges between 3 and 73 minutes. - The log indicator pulls containment to minute 318 with four candidates in 10 minutes and leaves four items open; radius two spends 68 minutes on seventeen candidates, contains at minute 376, and also leaves four items open. - The combination of indicator and account gives the fewest open items (two); radius two plus account brings three more candidates and delays by 11 minutes, so it leaves three items open. - The cost of a wide scope is unnecessary cutting: radius two plus account cuts legitimate work on thirteen unaffected assets, while the log indicator cuts none. - Two of the reached collectors transmit only network flow data; they are in scope, but that they were affected cannot be verified by any log — for the same reason, seven of the remaining seventeen candidates also stay uncertain. #### Next Step In this lesson containment was a single action, applied the same way to every asset in scope. In reality containment is not a single action: cutting a network segment is applied within minutes and stops all of a branch's legitimate traffic, while rekeying a service account takes hours but reopens legitimate work. The next lesson runs these two types separately on the same incident: how many legitimate work items short-term containment cuts, how many minutes later long-term containment kicks in, and how many extra items get reached in those minutes. ### Containment Source: https://academia.sh/en/courses/security-operations/containment Calculating the item gap between early and late containment in the same incident: running eight short-term and four long-term actions across seven regimes; early containment stopping at minute 393 with 10 items reached and cutting 1,161 legitimate work items, versus long-term-only containment stopping at minute 765 with 15 items reached and cutting none; the cost of five extra items being 232 legitimate work items per extra item; the same cut rising to 11,687 items with no long-term action; and, in the yearly balance, the early policy preventing 30 extra items against 9,171 legitimate work items. The previous lesson measured scope, and the best criterion produced a decision at minute 370. In that lesson, containment was a single action: it applied to every asset in scope the same way, and access stopped. This lesson removes that simplification. Containment is not a single action; it is a set of actions that kick in at different minutes, close off different assets, and produce different costs. The distinction is between short-term and long-term containment. A short-term action is applied within minutes and, from the moment it is applied, also stops legitimate work on those assets. A long-term action kicks in hours later, does not cut legitimate work, and lifts the short-term cut. What this lesson measures is the item gap between these two types in the same incident. #### Containment Actions **IR15:** The containment set consists of eight short-term and four long-term actions; each action kicks in a set number of minutes after the decision moment, closes off specific assets, and requires a set number of items to be manually rebuilt. **IR16:** Access stops once every asset affected at that moment is closed off; if even a single affected asset remains not closed off, access continues. **IR17:** Every asset class has an hourly legitimate work item count — collector 12, validator 26, billing 18, portal 40, work order 9, field unit 4, backup store and console 2. **IR18:** Short-term containment lasts until the first long-term action covering the same asset kicks in; if no such action exists, it lasts until the end of the 2,880-minute measurement window. **IR19:** The decision moment comes from the previous lesson: minute 370. **IR20:** A long-term action does not cut legitimate work, but it produces items that must be manually rebuilt. **IR21:** Six of the 24 verified alerts per year are real incidents; under the early-cut policy, short-term containment is applied before verification and, on a non-real alert, lasts for the duration of verification. ```js // containment/model.mjs — containment model: not a real incident or network. The access // curve, containment actions and legitimate-work rates are data structures; durations are fictional. export const DECISION = 370, WINDOW = 2880; // previous lesson's decision moment; measurement window (min) // access curve: [minute, asset, item name] (same data as the previous lesson) export const ACCESS = [ [0, "collector-03", "execution"], [26, "collector-03", "local identity store"], [58, "validator-01", "service account"], [95, "collector-07", "collector handoff"], [134, "collector-11", "collector handoff"], [171, "validator-02", "second validator"], [214, "billing-01", "read endpoint"], [263, "portal-01", "subscriber copy"], [318, "workorder-01", "work order queue"], [372, "field-01", "management channel"], [441, "store-01", "backup listing"], [505, "billing-01", "archive export"], [588, "collector-09", "second-region set"], [676, "console-01", "console read"], [762, "portal-02", "second portal copy"], [845, "store-01", "backup export"], [951, "billing-02", "second billing endpoint"], [1088, "console-01", "console write authority"]]; export const items = (min) => ACCESS.filter(([t]) => t <= min); export const affected = (min) => [...new Set(items(min).map(([, v]) => v))]; const makeIds = (prefix, n) => Array.from({ length: n }, (_, i) => `${prefix}-${String(i + 1).padStart(2, "0")}`); export const COLLECTOR = makeIds("collector", 12), VALIDATOR = makeIds("validator", 3); export const ACCOUNT = [...COLLECTOR, ...VALIDATOR]; // those that use the shared service account export const APPLICATION = [...makeIds("billing", 2), ...makeIds("portal", 2), "workorder-01"]; export const SCOPE = ["collector-03", "collector-07", "collector-11", "validator-01", "validator-02", "billing-01"]; // scope verified in the previous lesson // hourly legitimate work items per asset class (data) export const LEGIT = { collector: 12, validator: 26, billing: 18, portal: 40, workorder: 9, field: 4, store: 2, console: 2 }; export const hourly = (set) => set.reduce((s, a) => s + (LEGIT[a.split("-")[0]] ?? 0), 0); // containment action: name, type, minutes after decision to apply, asset closed off, // items to be manually rebuilt export const FIELD = makeIds("field", 8), ALL = [...ACCOUNT, ...APPLICATION, ...FIELD, "store-01", "console-01"]; export const ACTIONS = { accountLock: ["lock the service account", "short", 4, ACCOUNT, 0], consoleClose: ["close the management channel", "short", 9, ["console-01"], 0], segmentCut: ["cut the branch segment", "short", 11, COLLECTOR, 0], storeLock: ["set the backup store to read-only", "short", 13, ["store-01"], 0], applicationLock: ["set the application endpoint to read-only", "short", 16, APPLICATION, 0], scopeCut: ["take the verified scope off the network", "short", 19, SCOPE, 0], fieldIsolate: ["take the field units off the network", "short", 23, FIELD, 0], externalCut: ["cut the central pipe's external connection", "short", 31, ALL, 0], accountRekey: ["rekey the service account", "long", 214, ACCOUNT, 15], segmentRule: ["make the segmentation rule permanent", "long", 286, [...COLLECTOR, "console-01"], 4], applicationPatch: ["patch the application endpoint", "long", 341, [...APPLICATION, "store-01"], 5], endpointRebuild: ["rebuild the endpoint", "long", 395, [...FIELD, ...SCOPE], 6], }; export const actionOf = (k) => { const [name, type, min, assets, manual] = ACTIONS[k]; return { key: k, name, type, min, assets, manual }; }; // access stops once every asset affected at that moment is closed off export function apply(decision, keys) { const sorted = keys.map(actionOf).sort((a, b) => a.min - b.min); const closed = new Set(); for (const y of sorted) { const t = decision + y.min; for (const a of y.assets) closed.add(a); if (affected(t).every((a) => closed.has(a))) return { t, closed, stopped: y }; } return { t: null, closed, stopped: null }; } // short-term containment lasts until the long-term action covering the same assets kicks in export function measure(decision, keys) { const y = keys.map(actionOf), s = apply(decision, keys); const short = y.filter((x) => x.type === "short"), long = y.filter((x) => x.type === "long"); // for each asset, the short-term cut lasts until the first long-term action covering // that asset kicks in; if no such action exists, it lasts until the end of the measurement window const liftAt = (a) => Math.min(WINDOW, ...long.filter((u) => u.assets.includes(a)).map((u) => u.min)); let cut = 0; const cutFrom = new Map(); for (const k of short) for (const a of k.assets) cutFrom.set(a, Math.min(cutFrom.get(a) ?? Infinity, k.min)); for (const [a, start] of cutFrom) cut += (LEGIT[a.split("-")[0]] ?? 0) * Math.max(0, liftAt(a) - start) / 60; const end = s.t ?? decision + WINDOW; return { containment: s.t, items: items(end).length, open: s.t ? affected(s.t).filter((a) => !s.closed.has(a)).length : affected(end).length, cut: Math.round(cut), manual: y.reduce((a, x) => a + x.manual, 0), cutAssets: cutFrom.size, short: short.length, long: long.length }; } ``` The measurement file runs seven containment regimes on the same incident and at the same decision moment, then scans the decision moment to derive the item gap between early and late containment. ```js // containment/measure.mjs — the item gap between early and late containment in the same // incident: how many legitimate work items short-term containment cuts, how many minutes // later long-term containment kicks in, how many extra items are reached in those minutes. import { DECISION, WINDOW, ACCESS, items, ACTIONS, actionOf, hourly, measure } from "./model.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); console.log(`decision moment: minute ${DECISION} (the previous lesson's best scope criterion); ` + `measurement window ${WINDOW} min`); console.log("\n1. containment actions"); const A = [-43, 7, 9, 9, 14, 7]; write(A, "action", "type", "+minutes", "assets", "legit work/hr", "manual"); for (const k of Object.keys(ACTIONS)) { const y = actionOf(k); write(A, y.name, y.type, y.min, y.assets.length, y.type === "short" ? hourly(y.assets) : "-", y.manual || "-"); } const REGIMES = { "narrow short": ["scopeCut"], "account lock": ["accountLock"], "full short": ["scopeCut", "applicationLock", "fieldIsolate"], "wide short": ["accountLock", "consoleClose", "segmentCut", "storeLock", "applicationLock", "fieldIsolate"], "cut external connection": ["externalCut"], "long only": ["accountRekey", "segmentRule", "applicationPatch", "endpointRebuild"], "full short + long": ["scopeCut", "applicationLock", "fieldIsolate", "accountRekey", "segmentRule", "applicationPatch", "endpointRebuild"], }; console.log(`\n2. ${Object.keys(REGIMES).length} containment regimes, same incident and same decision moment`); const B = [-24, 12, 9, 8, 11, 13, 7]; write(B, "regime", "containment", "items", "open", "cut assets", "cut work", "manual"); const R = {}; for (const [name, k] of Object.entries(REGIMES)) { const r = R[name] = measure(DECISION, k); write(B, name, r.containment ?? "not stopped", `${r.items}/${ACCESS.length}`, r.open, r.cutAssets, r.cut, r.manual || "-"); } console.log("\n3. early and late containment in the same incident (decision moment scanned)"); const C = [-9, 11, 8, 11, 8, 12, 12, 16]; write(C, "decision", "early min", "items", "late min", "items", "extra items", "cut work", "work/extra item"); for (const decision of [311, 318, 370, 437, 656]) { const er = measure(decision, REGIMES["full short + long"]), ge = measure(decision, REGIMES["long only"]); const extra = ge.items - er.items; write(C, decision, er.containment, er.items, ge.containment, ge.items, extra, er.cut, extra ? (er.cut / extra).toFixed(0) : "-"); } // yearly alert set: 6 of 24 verified alerts are real incidents (data). Under the early // policy, short-term containment is applied before verification and, on a false alert, lasts VERIFY minutes. const ALERTS_YEAR = 24, REAL = 6, VERIFY = 30; const cutHourly = hourly([...new Set(REGIMES["full short + long"].flatMap((k) => actionOf(k).type === "short" ? actionOf(k).assets : []))]); console.log(`\n4. yearly balance (${ALERTS_YEAR} verified alerts, ${REAL} real incidents; ` + `false-alert cut lasts ${VERIFY} min)`); const D = [-18, 15, 16, 13, 17]; write(D, "policy", "real incident", "false alert", "year cut", "year extra items"); const erY = measure(DECISION, REGIMES["full short + long"]), geY = measure(DECISION, REGIMES["long only"]); const falseCost = Math.round((ALERTS_YEAR - REAL) * cutHourly * VERIFY / 60); write(D, "early cut", REAL * erY.cut, falseCost, REAL * erY.cut + falseCost, 0); write(D, "late cut", 0, 0, 0, REAL * (geY.items - erY.items)); console.log(` early policy's cost per item: ` + `${((REAL * erY.cut + falseCost) / (REAL * (geY.items - erY.items))).toFixed(0)} legitimate work items`); console.log("\n5. extra items late containment allows"); for (const [t, v, name] of items(geY.containment).filter(([t]) => t > erY.containment)) write([-4, -12, -18, -28], "", `minute ${t}`, v, name); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // the raw measurement is not printed, only the band is checked console.log(`\nseven regimes and five decision moments within the 500 ms band: ${ms < 500 ? "yes" : "no"}`); ``` ``` decision moment: minute 370 (the previous lesson's best scope criterion); measurement window 2880 min 1. containment actions action type +minutes assets legit work/hr manual lock the service account short 4 15 222 - close the management channel short 9 1 2 - cut the branch segment short 11 12 144 - set the backup store to read-only short 13 1 2 - set the application endpoint to read-only short 16 5 125 - take the verified scope off the network short 19 6 106 - take the field units off the network short 23 8 32 - cut the central pipe's external connection short 31 30 383 - rekey the service account long 214 15 - 15 make the segmentation rule permanent long 286 13 - 4 patch the application endpoint long 341 6 - 5 rebuild the endpoint long 395 14 - 6 2. 7 containment regimes, same incident and same decision moment regime containment items open cut assets cut work manual narrow short not stopped 18/18 14 6 5054 - account lock not stopped 18/18 14 15 10641 - full short 393 10/18 0 18 11687 - wide short 393 10/18 0 30 18323 - cut external connection 401 10/18 0 30 18186 - long only 765 15/18 0 0 0 30 full short + long 393 10/18 0 18 1161 30 3. early and late containment in the same incident (decision moment scanned) decision early min items late min items extra items cut work work/extra item 311 330 9 706 14 5 1161 232 318 337 9 713 14 5 1161 232 370 393 10 765 15 5 1161 232 437 778 15 832 15 0 1161 - 656 997 17 1051 17 0 1161 - 4. yearly balance (24 verified alerts, 6 real incidents; false-alert cut lasts 30 min) policy real incident false alert year cut year extra items early cut 6966 2205 9171 0 late cut 0 0 0 30 early policy's cost per item: 306 legitimate work items 5. extra items late containment allows minute 441 store-01 backup listing minute 505 billing-01 archive export minute 588 collector-09 second-region set minute 676 console-01 console read minute 762 portal-02 second portal copy seven regimes and five decision moments within the 500 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Eight Regimes, Same Incident The second table runs seven regimes on the same incident and the same decision moment. The first two rows never stop access at all. The narrow short regime cuts six assets, spends 5,054 legitimate work items, and because fourteen assets stay open, all eighteen items are reached; account lock cuts fifteen assets and spends 10,641 items. Both regimes lose on both sides of the decision: legitimate work is cut and access continues. The full short regime stops it at minute 393 with three actions; the items reached are **ten**, and no asset stays open. The wide short regime stops it at the same minute but cuts thirty assets and spends 18,323 items; cutting the external connection stops it at minute 401 and spends 18,186 items. **The cost of three regimes that reach the same result ranges from 11,687 to 18,323.** Long-term actions set the duration of the short-term cut. In the full short regime, with no long-term action, the cut lasts until the end of the measurement window and spends 11,687 items; once four long-term actions are added, it drops to **1,161 items**. Here, the long-term action is the short-term containment's **end date**. #### The Difference Between Early and Late The third table is this lesson's main measurement. Two regimes are compared on the same incident: early (full short plus long) and late (long only). The early regime stops at minute 393 with ten items reached; the late regime stops at minute 765 with **fifteen** items reached. The gap is five items, at a cost of 1,161 legitimate work items — that is, **232 legitimate work items** per extra item. No matter how early the decision is made, this ratio does not change: at minutes 311 and 318 the gap is still five items and the ratio is still 232, because the distance between the short-term and long-term actions is independent of the decision moment. The table's last two rows break this. When the decision shifts to minute 437, the early regime stops at minute 778 instead of 393; with a decision made at minute 656, it climbs to minute 997. The reason is that the access curve moves outside scope: at minute 441 the backup store is reached, and at minute 588 the second-region collector set is reached, and the full short regime's three actions do not cover these assets. Access stops only once the long-term action kicks in. **In these two rows, the extra items early containment gains are zero, but the legitimate work it cuts is still 1,161 items.** The value of short-term containment depends on the decision-moment scope still being valid. The fifth table writes out, by name, the five items that late containment allows. Two are data exports, two are access spreading to new assets. The count is five, but the items are not equal in weight, and this model does not measure that. #### The Yearly Balance The fourth table carries the decision into the year. Under the early-cut policy, 6,966 legitimate work items are cut across six real incidents and 2,205 across eighteen non-real alerts — **9,171 items** a year. Under the late-cut policy, no items are cut, and in exchange **30 extra items** are reached; the early policy's cost per extra item prevented is 306 legitimate work items. These two numbers are not in the same unit, and the model cannot convert one into the other; whoever makes the decision sees the two columns and weighs them by their own measure. What the measurement does is keep the decision from being unnumbered: the difference between "contain early" and "verify first" is, in this incident, 9,171 items against 30 items. #### Summary - The three short regimes that stop access do so at minutes 393 and 401 with ten items reached, but their costs range from 11,687 to 18,323 items; once long-term action is added, the cost of the same cut drops to 1,161 items, at the price of thirty items being manually rebuilt. - Early containment stops at minute 393 with ten items reached, late containment at minute 765 with fifteen; the cost of five extra items is 232 legitimate work items per extra item. - When the decision is delayed to minute 437, the access curve moves outside scope: the extra items early containment gains drop to zero, while the legitimate work it cuts stays at 1,161 items. - In the yearly balance, the early policy cuts 9,171 legitimate work items and prevents 30 extra items; 306 legitimate work items per extra item. #### Next Step Access stopped, but the only thing that stopped is access; when systems return to the network, they return with whatever is on them. The next lesson measures that: how many footholds are on the endpoints, how many of them detection methods find, how many cleanup removes, and how many days later a missed point produces reinfection. ### Eradication Source: https://academia.sh/en/courses/security-operations/eradication Measuring the removal of persistence in terms of footholds: finding 17 of 22 points across nine affected assets in 506 minutes with five discovery methods; three of four configuration hooks going unfound by any method because they were lost on assets taken off the network during containment; across six eradication regimes, the missed-point count dropping from 10 to 0 as total time climbs from 353 minutes to 1,361 minutes; and five missed points producing 8 reinfections in 60 days, with the first on day five. The previous lesson stopped access. Short-term containment was applied at minute 393, long-term actions kicked in within hours, and the cut was lifted. What stopped was access; the items automatically run on the endpoints did not stop. Once the short-term cut lifts and assets return to the network, they return with whatever is on them. Eradication's unit of measure is the **foothold**: an automatically run item. The inventory of point types was measured in the Endpoint Security course; here it is taken as input. Three separate things are counted in this lesson — how many points are **known**, how many are **eradicated**, and how many produce reinfection when **missed**. The three are not the same number; the gap between them is this lesson's subject. No persistence-establishment method is written out; what is counted is the point itself and the discovery method applied to it. #### Point Inventory and Discovery Methods **IR22:** The nine assets affected in the previous lesson have a total of 22 footholds; they are spread across seven types, and each point's daily reactivation probability is a given. **IR23:** There are five discovery methods, and each finds only specific types: baseline comparison, process-creation log scanning, identity store audit, network flow data monitoring, and memory examination. **IR24:** Not every method applies to every asset — baseline comparison works on an asset that has a baseline, process log scanning on an asset that forwards process logs, and **memory examination only on an asset not taken off the network during containment**. **IR25:** A found point can be removed, and removal takes 8 minutes per point; rebuilding an asset takes 95 minutes and removes every point on that asset — including the ones not found. **IR26:** A missed point reactivates each day with its own probability; the scan runs for 60 days with seed 20260214. All dice are rolled from scratch, and the six regimes share the same daily sequence. ```js // eradication/model.mjs — persistence eradication model: no real endpoint, persistence // method or script involved. Footholds, discovery methods and asset properties are data structures. export const SEED = 20260214, DAYS = 60; // reinfection scan export const REBUILD_MIN = 95, REMOVE_MIN = 8; // per asset, per point // affected asset: name -> { has baseline?, forwards process logs?, // taken off network during containment (memory lost) } export const ASSETS = { "collector-03": { baseline: true, process: true, offNetwork: true }, "collector-07": { baseline: true, process: false, offNetwork: true }, "collector-11": { baseline: true, process: false, offNetwork: true }, "validator-01": { baseline: true, process: true, offNetwork: true }, "validator-02": { baseline: true, process: true, offNetwork: true }, "billing-01": { baseline: true, process: false, offNetwork: true }, "portal-01": { baseline: true, process: false, offNetwork: false }, "workorder-01": { baseline: true, process: false, offNetwork: false }, "endpoint-01": { baseline: false, process: true, offNetwork: true }, }; // foothold: [asset, type, daily reactivation probability] // M24/K05 measured the type inventory; here it is taken as input, no method is written out export const POINTS = [ ["collector-03", "service definition", 0.06], ["collector-03", "scheduled task", 0.05], ["collector-03", "configuration hook", 0.04], ["collector-07", "service definition", 0.06], ["collector-07", "session script", 0.03], ["collector-11", "scheduled task", 0.05], ["collector-11", "in-memory run", 0.02], ["validator-01", "service definition", 0.07], ["validator-01", "identity item", 0.09], ["validator-01", "outbound connection", 0.05], ["validator-02", "identity item", 0.09], ["validator-02", "configuration hook", 0.04], ["billing-01", "identity item", 0.08], ["billing-01", "session script", 0.03], ["billing-01", "outbound connection", 0.05], ["portal-01", "configuration hook", 0.04], ["portal-01", "identity item", 0.08], ["workorder-01", "scheduled task", 0.05], ["workorder-01", "in-memory run", 0.02], ["endpoint-01", "session script", 0.03], ["endpoint-01", "service definition", 0.06], ["endpoint-01", "configuration hook", 0.04], ]; // discovery method: name, minutes per asset, types it finds, which assets it applies to export const METHODS = [ ["baseline comparison", 22, ["service definition", "scheduled task", "session script"], (v) => v.baseline], ["process-creation log scan", 14, ["service definition", "scheduled task", "in-memory run"], (v) => v.process], ["identity store audit", 9, ["identity item"], () => true], ["network flow data monitoring", 11, ["outbound connection"], () => true], ["memory examination", 47, ["in-memory run", "configuration hook"], (v) => !v.offNetwork], ]; // index of points found and minutes spent, for a given set of chosen methods export function discover(chosen) { const found = new Set(); let min = 0; for (const [name, minutes, types, applies] of METHODS) { if (!chosen.includes(name)) continue; for (const [a, v] of Object.entries(ASSETS)) { if (!applies(v)) continue; min += minutes; POINTS.forEach(([pa, pt], i) => { if (pa === a && types.includes(pt)) found.add(i); }); } } return { found, min }; } export function rng(seed) { // linear congruential generator let x = seed >>> 0; return () => ((x = (x * 1664525 + 1013904223) >>> 0) / 2 ** 32); } // all dice are rolled from scratch: runs share the same daily sequence const ROLLS = (() => { const z = rng(SEED); return Array.from({ length: DAYS * POINTS.length }, () => z()); })(); // a missed point reactivates daily with its own probability; first activation and total are counted export function reinfection(missed) { let first = null, events = 0; for (let d = 0; d < DAYS; d++) for (const i of missed) if (ROLLS[d * POINTS.length + i] < POINTS[i][2]) { events++; if (first === null) first = d + 1; } return { first, events }; } export function measure(chosen, rebuilt) { const { found, min } = discover(chosen); const rebuiltSet = new Set(rebuilt); const clean = new Set(found); POINTS.forEach(([a], i) => { if (rebuiltSet.has(a)) clean.add(i); }); const missed = POINTS.map((_, i) => i).filter((i) => !clean.has(i)); const removed = [...found].filter((i) => !rebuiltSet.has(POINTS[i][0])).length; return { found: found.size, clean: clean.size, missed, discoveryMin: min, cleanupMin: removed * REMOVE_MIN + rebuiltSet.size * REBUILD_MIN, rebuilt: rebuiltSet.size, ...reinfection(missed) }; } ``` The measurement file first adds the methods one by one and counts how many new points each brings, then runs six eradication regimes against the same inventory. ```js // eradication/measure.mjs — how many footholds there are, how many the discovery method // finds, how many eradication removes, and how many days later a missed point produces reinfection. import { SEED, DAYS, ASSETS, POINTS, METHODS, discover, measure } from "./model.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); const ALL = METHODS.map((y) => y[0]); const assetIds = Object.keys(ASSETS); const TYPES = [...new Set(POINTS.map(([, t]) => t))]; console.log(`${assetIds.length} affected assets, ${POINTS.length} footholds, ` + `${TYPES.length} types; ${METHODS.length} discovery methods; seed ${SEED}`); console.log("\n1. points found as discovery methods are added (cumulative)"); const A = [-32, 10, 8, 12, 15, 16]; write(A, "method", "min/asset", "assets", "new points", "cumul. points", "cumul. minutes"); let chosen = [], prevN = 0; for (const [name, minutes, , applies] of METHODS) { chosen = [...chosen, name]; const k = discover(chosen), fit = assetIds.filter((a) => applies(ASSETS[a])).length; write(A, name, minutes, fit, k.found.size - prevN, `${k.found.size}/${POINTS.length}`, k.min); prevN = k.found.size; } console.log("\n2. point type and number of methods that can find it"); const B = [-24, 8, 9, -40]; write(B, "type", "points", "methods", " missing points / reason"); const all = discover(ALL).found; for (const type of TYPES) { const index = POINTS.map((n, i) => [n, i]).filter(([n]) => n[1] === type); const missing = index.filter(([, i]) => !all.has(i)); write(B, type, index.length, METHODS.filter((y) => y[2].includes(type)).length, missing.length ? ` ${missing.length}: ${missing.map(([n]) => n[0]).join(", ")}` : " -"); } const SCOPE = ["collector-03", "collector-07", "collector-11", "validator-01", "validator-02", "billing-01"]; const REGIMES = { "cheap discovery": [[ALL[0], ALL[2]], []], "excluding memory": [ALL.slice(0, 4), []], "full discovery": [ALL, []], "full discovery + scope rebuild": [ALL, SCOPE], "cheap discovery + scope rebuild": [[ALL[0], ALL[2]], SCOPE], "rebuild all assets": [ALL, assetIds], }; console.log(`\n3. six eradication regimes (${DAYS}-day reinfection scan)`); const C = [-33, 9, 8, 11, 14, 12, 8, 18, 7]; write(C, "regime", "found", "clean", "missed", "discovery min", "cleanup min", "rebuild", "first reinfection", "events"); const R = {}; for (const [name, [s, k]] of Object.entries(REGIMES)) { const r = R[name] = measure(s, k); write(C, name, `${r.found}/${POINTS.length}`, r.clean, r.missed.length, r.discoveryMin, r.cleanupMin, r.rebuilt, r.first ? `day ${r.first}` : "-", r.events); } console.log("\n4. points missed after full discovery"); for (const i of R["full discovery"].missed) write([-4, -16, -24, -18], "", POINTS[i][0], POINTS[i][1], `daily probability ${POINTS[i][2].toFixed(2)}`); console.log("\n5. cost per missed point"); const D = [-33, 11, 12, 21, 16]; write(D, "regime", "total min", "missed", "min/closed point", "events/missed"); const base = R["cheap discovery"]; for (const [name, r] of Object.entries(R)) { const closed = r.clean - base.clean, min = r.discoveryMin + r.cleanupMin - base.discoveryMin - base.cleanupMin; write(D, name, r.discoveryMin + r.cleanupMin, r.missed.length, closed > 0 ? (min / closed).toFixed(0) : "-", r.missed.length ? (r.events / r.missed.length).toFixed(1) : "-"); } const ms = Number(process.hrtime.bigint() - t0) / 1e6; // the raw measurement is not printed, only the band is checked console.log(`\nsix regimes and the ${DAYS}-day scan within the 500 ms band: ${ms < 500 ? "yes" : "no"}`); ``` ``` 9 affected assets, 22 footholds, 7 types; 5 discovery methods; seed 20260214 1. points found as discovery methods are added (cumulative) method min/asset assets new points cumul. points cumul. minutes baseline comparison 22 8 8 8/22 176 process-creation log scan 14 4 1 9/22 232 identity store audit 9 9 4 13/22 313 network flow data monitoring 11 9 2 15/22 412 memory examination 47 2 2 17/22 506 2. point type and number of methods that can find it type points methods missing points / reason service definition 4 2 - scheduled task 3 2 - configuration hook 4 1 3: collector-03, validator-02, endpoint-01 session script 3 1 1: endpoint-01 in-memory run 2 2 1: collector-11 identity item 4 1 - outbound connection 2 1 - 3. six eradication regimes (60-day reinfection scan) regime found clean missed discovery min cleanup min rebuild first reinfection events cheap discovery 12/22 12 10 257 96 0 day 3 19 excluding memory 15/22 15 7 412 120 0 day 5 11 full discovery 17/22 17 5 506 136 0 day 5 8 full discovery + scope rebuild 17/22 20 2 506 610 6 day 5 4 cheap discovery + scope rebuild 12/22 17 5 257 586 6 day 5 9 rebuild all assets 17/22 22 0 506 855 9 - 0 4. points missed after full discovery collector-03 configuration hook daily probability 0.04 collector-11 in-memory run daily probability 0.02 validator-02 configuration hook daily probability 0.04 endpoint-01 session script daily probability 0.03 endpoint-01 configuration hook daily probability 0.04 5. cost per missed point regime total min missed min/closed point events/missed cheap discovery 353 10 - 1.9 excluding memory 532 7 60 1.6 full discovery 642 5 58 1.6 full discovery + scope rebuild 1116 2 95 2.0 cheap discovery + scope rebuild 843 5 98 1.8 rebuild all assets 1361 0 101 - six regimes and the 60-day scan within the 500 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Points Found as Methods Are Added The first table counts discovery cumulatively. Baseline comparison applies to eight assets, takes 176 minutes, and on its own finds **eight** of the 22 points. Process-creation log scanning applies to only four assets — five assets do not forward process logs — and for 56 minutes it brings **one** new point; baseline comparison had already found the rest of what it turns up. Identity store audit adds four new points across nine assets in 81 minutes, and network flow data monitoring adds two new points in 99 minutes. The last row is the most expensive. Memory examination takes 47 minutes per asset and applies to only **two** assets: on the seven assets taken off the network during containment, memory contents were lost. For 94 minutes it brings two new points and raises the total to 17. Five points cannot be found by any method. #### The Five Points That Cannot Be Found The second table breaks the loss down by type. **Three** of the four configuration hooks cannot be found, because only memory examination finds this type, and that method cannot be applied on the `collector-03`, `validator-02` and `endpoint-01` assets. The in-memory run on `collector-11` is lost for the same reason. The session script on `endpoint-01` slips through for a different reason: that asset has no baseline, and only baseline comparison finds this type. The fourth table writes out these five points by name. What they have in common is not a discovery gap but **a side effect of a containment decision**. In the previous lesson, short-term containment took assets off the network and stopped access at minute 393; the same decision made memory examination impossible on seven assets. Here is a third cost of early containment, and it does not show up in the previous lesson's tables. #### Six Regimes and Reinfection The third table compares eradication regimes. Cheap discovery — baseline comparison and identity audit — finds 12 points in 257 minutes, misses 10, and produces **19 reinfections** in 60 days; the first comes on day three. Full discovery finds 17 points in 506 minutes, misses five, and produces eight events; the first comes on day five. The only way to close a missed point is not to wait for it to be found, but to rebuild the asset. When rebuilding the six scope assets is added to full discovery, clean points rise from 17 to **20** — the unfound points on the rebuilt assets go too — missed drops to two and events fall to four. Rebuilding all nine assets brings missed down to zero, and there is no reinfection at all in 60 days. The fifth table gives the minutes for this. Cheap discovery totals 353, full discovery 642, full discovery plus scope rebuild 1,116, rebuilding everything 1,361 minutes. Relative to cheap discovery, the cost per closed point is 58 minutes for full discovery, 95 for scope rebuild, and 101 for full rebuild. The row that stands out is **cheap discovery plus scope rebuild**: it cleans 17 points in 843 minutes — the same count as full discovery, at 201 minutes more — and five points still stay open. Rebuilding does not substitute for discovery; it only holds on the asset that gets rebuilt. #### Summary - Nine affected assets have 22 footholds; five discovery methods find 17 of them in 506 minutes, and five cannot be found by any method. - The methods' marginal contribution is not equal: baseline comparison brings eight points in 176 minutes, process log scanning one point in 56 minutes, and memory examination two points in 94 minutes. - Three of the four configuration hooks and one in-memory run slip through because memory examination cannot be applied on seven assets; those assets were taken off the network during containment. - Five missed points produce eight reinfections in 60 days, the first on day five; when ten points are missed, events rise to 19 and the first moves up to day three. - The only way to bring missed points to zero is to rebuild the asset: nine assets take 1,361 minutes, 101 minutes per closed point. #### Next Step The points are eradicated and the missed count is on record. What comes after this is bringing the service back, and a new question arises there: which point in time does the rebuilt asset return to. Some of the backups were taken after the first foothold moment, and points sit in those backups. The next lesson measures recovery along the axis of reinfection risk: which backup is clean to return to, how many items reverting to a clean backup costs, how many items are manually rebuilt, and at which stage a supervised restoration catches reinfection. ### Recovery Source: https://academia.sh/en/courses/security-operations/recovery Measuring recovery along the axis of reinfection risk: three of seven recovery points being clean, with the newest one nine and a half days back; returning to the clean backup costing 94,715 items of data and 44 manually rebuilt items; rebuilding from scratch giving the same cleanliness for 190 more minutes at a cost of 2,493 lost items; returning to a dirty backup rerunning eradication from scratch and raising total time to 1,347 minutes; and the four-stage supervised restoration bringing the expected points reaching production down from 5.00 to 0.10. The previous lesson counted that five of the 22 footholds could not be found by any discovery method, and showed that the only way to bring the missed count to zero is to rebuild the asset. A rebuilt asset does not come back empty; it is filled from a backup or an image. This lesson measures that choice. Recovery is not measured by how quickly the service comes back up. What is measured is **reinfection risk**: which recovery point was taken before the first foothold, how many items returning to a clean one costs, how many items are manually rebuilt, and across how many stages the service is observed as it returns to production. Backup types, chain structure, and storage accounting were measured in the IT and Systems Fundamentals course; here a backup is treated only as a **point in time**. #### Recovery Points **IR27:** The first foothold was nine days ago; a recovery point is clean only if it was taken before that moment. **IR28:** The day each of the 22 footholds was established is a given; a backup contains every point that was already established at the moment it was taken. **IR29:** Daily transaction volume is 9,970 items — 8,400 meter readings, 1,250 invoice items, 320 work orders; lost items are the volume from the day returned to up to today. **IR30:** Configuration left outside the backup is manually rebuilt: six items baseline, four items per day returned to. **IR31:** When returning to a dirty backup, eradication reruns from scratch — the previous lesson's full discovery regime, 642 minutes — and five points are missed again. **IR32:** Supervised restoration has four stages; each stage has an observation window and a probability of catching a reactivating point. **IR33:** In rebuilding from scratch, the system image is clean and data is taken from the latest backup; rebuilding nine assets from an image takes 430 minutes and 34 manually rebuilt items. ```js // recovery/model.mjs — supervised restoration model: no real backup or system involved. // Recovery points, foothold establishment days and stages are data structures. export const FIRST_FOOTHOLD = 9; // how many days ago the first foothold occurred export const POINT_COUNT = 22, MISSED = 5; // from the previous lesson: total and missed points // day each foothold was established (how many days ago) — 22 points (data) export const ESTABLISHED = [9, 9, 9, 8, 8, 7, 7, 6, 6, 6, 5, 5, 4, 4, 4, 3, 3, 2, 2, 1, 1, 1]; // daily transaction volume: service -> items (data) export const VOLUME = { "meter reading": 8400, "invoice item": 1250, "work order": 320 }; export const DAILY = Object.values(VOLUME).reduce((s, v) => s + v, 0); // recovery point: how many days ago it was taken, restore minutes export const POINTS = [[21, 260], [14, 245], [9.5, 240], [7, 215], [3, 190], [1, 175], [0.25, 165]]; // items to be manually rebuilt: configuration outside the backup, per day (data) export const MANUAL_DAILY = 4, MANUAL_BASE = 6; export const dirtyPoints = (daysAgo) => ESTABLISHED.filter((k) => k >= daysAgo).length; // points already established at that moment are in the backup export const lost = (daysAgo) => Math.round(daysAgo * DAILY); export const manual = (daysAgo) => MANUAL_BASE + Math.round(daysAgo * MANUAL_DAILY); // supervised restoration stage: name, observation minutes, probability of catching a reactivating point export const STAGES = [["opening on an isolated network", 120, 0.55], ["supervised limited traffic", 180, 0.70], ["partial traffic", 240, 0.85], ["full traffic", 0, 0]]; export function staged(count) { // the first `count` stages are applied const subset = STAGES.slice(0, count); const escaped = subset.reduce((p, [, , y]) => p * (1 - y), 1); return { min: subset.reduce((s, [, d]) => s + d, 0), capture: 1 - escaped, escaped }; } // recovery regime: name, recovery point (days ago) or null (rebuild from scratch), // data is taken from the latest backup on a from-scratch rebuild, and the system image is clean export const REGIMES = [ ["return to latest backup", 0.25, false], ["return to 1 day ago", 1, false], ["return to 3 days ago", 3, false], ["return to clean backup", 9.5, false], ["rebuild from scratch + data", 0.25, true], ]; export const SCRATCH_MIN = 430, SCRATCH_MANUAL = 34; // rebuilding nine assets from an image // eradication reruns from scratch when returning to a dirty backup (the previous lesson's full discovery regime) export const CLEANUP_MIN = 642; export function measure([name, day, scratch], stageCount) { const backupPoints = scratch ? 0 : dirtyPoints(day); const remaining = backupPoints > 0 ? MISSED : 0; // remains even if eradication reruns const a = staged(stageCount); const restore = POINTS.find(([g]) => g === day)[1]; return { name, day, backupPoints, remaining, lost: scratch ? lost(0.25) : lost(day), manual: scratch ? SCRATCH_MANUAL : manual(day), min: (scratch ? SCRATCH_MIN : restore) + (backupPoints ? CLEANUP_MIN : 0) + a.min, captured: +(remaining * a.capture).toFixed(1), escaped: +(remaining * a.escaped).toFixed(2), stage: stageCount, observe: a.min, rate: a.capture }; } ``` The measurement file first ranks the seven recovery points, then runs five recovery regimes against the same event and scans the stage count. ```js // recovery/measure.mjs — which backup is clean to return to, how many items are lost, // how many items are manually rebuilt, and where the supervised restoration stages catch reinfection. import { FIRST_FOOTHOLD, ESTABLISHED, VOLUME, DAILY, POINTS, STAGES, REGIMES, CLEANUP_MIN, MISSED, dirtyPoints, lost, manual, staged, measure } from "./model.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); const CLEAN_DAY = POINTS.find(([g]) => dirtyPoints(g) === 0 && g < FIRST_FOOTHOLD + 1)[0]; console.log(`first foothold ${FIRST_FOOTHOLD} days ago; ${ESTABLISHED.length} footholds, ` + `daily transaction volume ${DAILY} items (` + Object.entries(VOLUME).map(([a, v]) => `${a} ${v}`).join(", ") + ")"); console.log("\n1. recovery points"); const A = [-10, 8, 20, 14, 8, 15]; write(A, "days ago", "clean", "footholds in backup", "lost items", "manual", "restore"); for (const [g, dk] of POINTS) write(A, g, dirtyPoints(g) === 0 ? "yes" : "no", dirtyPoints(g), lost(g), manual(g), dk + " min"); console.log(`\n2. five recovery regimes (in a dirty backup, eradication reruns from scratch ` + `for ${CLEANUP_MIN} min, and ${MISSED} points are missed again)`); const B = [-28, 10, 10, 14, 8, 12, 15]; write(B, "regime", "in backup", "remaining", "lost items", "manual", "total min", "escaped points"); const R = REGIMES.map((d) => measure(d, STAGES.length)); for (const r of R) write(B, r.name, r.backupPoints, r.remaining, r.lost, r.manual, r.min, r.escaped.toFixed(2)); console.log("\n3. supervised restoration stages"); const C = [-31, 12, 12, 18, 16]; write(C, "stage", "observe min", "capture", "cumul. capture", "cumul. minutes"); for (const [i, [name, dk, y]] of STAGES.entries()) { const a = staged(i + 1); write(C, name, dk, y.toFixed(2), a.capture.toFixed(4), a.min); } console.log("\n4. expected points reaching production by stage count"); const D = [-8, 12, 18, 16, 14, 14]; write(D, "stages", "observe min", "cumul. capture", "latest backup", "clean backup", "from scratch"); for (let n = 0; n <= STAGES.length; n++) { const a = staged(n); write(D, n, a.min, a.capture.toFixed(4), measure(REGIMES[0], n).escaped.toFixed(2), measure(REGIMES[3], n).escaped.toFixed(2), measure(REGIMES[4], n).escaped.toFixed(2)); } console.log("\n5. difference between the two clean options"); const t = measure(REGIMES[3], STAGES.length), s = measure(REGIMES[4], STAGES.length); const E = [-27, 14, 13, 12]; write(E, "measure", "clean backup", "from scratch", "difference"); write(E, "lost items", t.lost, s.lost, t.lost - s.lost); write(E, "manual items", t.manual, s.manual, t.manual - s.manual); write(E, "total minutes", t.min, s.min, t.min - s.min); write(E, "points reaching production", t.escaped.toFixed(2), s.escaped.toFixed(2), (t.escaped - s.escaped).toFixed(2)); console.log("\n6. loss per service from returning to the clean backup"); const F = [-18, 12, 14, 10]; write(F, "service", "daily", `${CLEAN_DAY}-day`, "share"); for (const [name, v] of Object.entries(VOLUME)) write(F, name, v, Math.round(v * CLEAN_DAY), (100 * v / DAILY).toFixed(1) + "%"); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // the raw measurement is not printed, only the band is checked console.log(`\nfive regimes and the five stage counts within the 500 ms band: ${ms < 500 ? "yes" : "no"}`); ``` ``` first foothold 9 days ago; 22 footholds, daily transaction volume 9970 items (meter reading 8400, invoice item 1250, work order 320) 1. recovery points days ago clean footholds in backup lost items manual restore 21 yes 0 209370 90 260 min 14 yes 0 139580 62 245 min 9.5 yes 0 94715 44 240 min 7 no 7 69790 34 215 min 3 no 17 29910 18 190 min 1 no 22 9970 10 175 min 0.25 no 22 2493 7 165 min 2. five recovery regimes (in a dirty backup, eradication reruns from scratch for 642 min, and 5 points are missed again) regime in backup remaining lost items manual total min escaped points return to latest backup 22 5 2493 7 1347 0.10 return to 1 day ago 22 5 9970 10 1357 0.10 return to 3 days ago 17 5 29910 18 1372 0.10 return to clean backup 0 0 94715 44 780 0.00 rebuild from scratch + data 0 0 2493 34 970 0.00 3. supervised restoration stages stage observe min capture cumul. capture cumul. minutes opening on an isolated network 120 0.55 0.5500 120 supervised limited traffic 180 0.70 0.8650 300 partial traffic 240 0.85 0.9798 540 full traffic 0 0.00 0.9798 540 4. expected points reaching production by stage count stages observe min cumul. capture latest backup clean backup from scratch 0 0 0.0000 5.00 0.00 0.00 1 120 0.5500 2.25 0.00 0.00 2 300 0.8650 0.68 0.00 0.00 3 540 0.9798 0.10 0.00 0.00 4 540 0.9798 0.10 0.00 0.00 5. difference between the two clean options measure clean backup from scratch difference lost items 94715 2493 92222 manual items 44 34 10 total minutes 780 970 -190 points reaching production 0.00 0.00 0.00 6. loss per service from returning to the clean backup service daily 9.5-day share meter reading 8400 79800 84.3% invoice item 1250 11875 12.5% work order 320 3040 3.2% five regimes and the five stage counts within the 500 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Which Backup Is Clean The first table splits the seven recovery points along the axis of cleanliness. Three are clean: 21, 14, and 9.5 days ago. The newest clean point is **9.5 days back**, and returning to it costs 94,715 items of data and 44 manually rebuilt items. Four points are dirty, and the number of footholds they contain rises toward the present: 7 at seven days ago, 17 at three days ago, 22 at one day ago. Lost items and the foothold count in the backup move in opposite directions — there is no single row in this table that is a good option. Restore minutes reflect none of this distinction: the oldest point takes 260 minutes, the newest 165. When recovery time is measured on its own, the latest backup looks like the best option. The decision depends not on restore minutes but on what the backup contains. #### Five Regimes The second table runs all five regimes together. In the three regimes that return to a dirty backup, eradication reruns from scratch and adds 642 minutes; total time falls between 1,347 and 1,372 minutes, and in all three, five points remain missed. Returning to the latest backup loses only 2,493 items of data, but it takes 1,347 minutes and makes eradication redo all of its work. **Recovery can undo the work eradication has done.** Two regimes are clean. Returning to the clean backup is the fastest at 780 minutes and brings back no points at all, but it costs 94,715 items of data and 44 manually rebuilt items. Rebuilding from scratch gives the same cleanliness in 970 minutes; lost items drop to 2,493 and manually rebuilt items to 34. The fifth table writes out the difference item by item: for 190 more minutes, 92,222 items of data and 10 manually rebuilt items are gained. **The 190 minutes paid in recovery is cheaper than nine and a half days of transaction data.** #### Staged Restoration The third table breaks supervised restoration into stages. Opening on an isolated network catches 0.55 with 120 minutes of observation, supervised limited traffic 0.70 with 180 minutes, partial traffic 0.85 with 240 minutes; cumulative capture is 0.5500, 0.8650, and 0.9798 respectively. The fourth stage is full traffic, its observation window is zero, and **it adds nothing** — production itself is not an observation stage. The fourth table converts stage count into the expected points reaching production. In the regime that returns to the latest backup, five points remain; if no stage is applied all five reach production, with one stage this drops to 2.25, with two to 0.68, with three to **0.10**. The cost is 540 minutes of observation. In the clean regimes, the remaining point count is zero, and the stage count changes nothing. Two separate decisions follow from this. If a dirty backup was returned to, staged restoration is mandatory, and 540 minutes of observation brings the expected points reaching production down from five to 0.10 — one in fifty. If the source was clean, the stages gain nothing on this measure — what they gain is on a different axis, verifying that cleanliness was actually achieved, and this model does not measure that. The stages' minutes are large compared with recovery itself. Of the 970 minutes for rebuilding from scratch, 540 are observation; the restore itself takes 430 minutes. If the observation window is shortened, the service opens earlier but the capture rate falls: 0.8650 at 300 minutes with two stages, 0.9798 at 540 minutes with three. The last stage's 240 minutes carries cumulative capture from 0.8650 to 0.9798 — in a dirty regime, that means bringing the expected points reaching production down from 0.68 to 0.10. #### Where the Loss Falls The sixth table breaks the 94,715-item loss from returning to the clean backup down by service. Meter reading holds 84.3 percent of the share at a daily 8,400 items and comes to 79,800 items over 9.5 days; invoice item is 11,875, work order 3,040 items. The loss from the three services does not carry the same weight, and this model counts them as equal. A meter reading is a measurement that can be read again later; an invoice item and a work order are the record of a commitment made to someone outside. The lost-item count carries none of this distinction — whoever makes the decision also has to see which service the number came from. #### Summary - Three of the seven recovery points are clean; the newest clean point is 9.5 days back and costs 94,715 items of data and 44 manually rebuilt items. - In the three regimes that return to a dirty backup, eradication reruns from scratch for 642 minutes, total time rises to 1,347–1,372 minutes, and five points are missed again. - Rebuilding from scratch, for 190 more minutes than the clean backup, gains 92,222 items of data and 10 manually rebuilt items. - Supervised restoration's three observation stages catch a cumulative 0.9798 in 540 minutes; the fourth stage is full traffic and adds nothing. - The expected points reaching production in a dirty regime is 5.00 with no stages, 2.25 with one stage, and 0.10 with three; in clean regimes it is zero regardless of stage count. #### Next Step The service is back and the expected reinfection is down to 0.10. What remains is the incident itself: what happened, when it began, which decision was made at which minute, and which item, done differently next time, would shorten the timeline. The next lesson takes up lessons learned along the axis of evidence and scope: which questions can be answered, which stay unanswered for lack of a record, and how many of the improvement items opened rest directly on a missing record. ### Lessons Learned Source: https://academia.sh/en/courses/security-operations/lessons-learned Measuring the review along the axis of evidence and scope: eleven of sixteen questions being answered with nine record sources, five questions staying unanswered for three distinct reasons, five of twelve improvement items resting directly on a missing record, record-only improvements bringing the answered-question count to 16 for 136 person-hours while shaving 22 minutes off the chain, and preparation and process improvements gaining 55 minutes for 103 person-hours while opening no questions at all. The previous lesson brought the service back and lowered the expected footholds reaching production to 0.10. The incident is closed. What remains is reconstructing the incident itself: what happened, when it began, which decision was made at which minute, and which item, done differently, would have shortened the timeline. This lesson's measure is not the review's output but the review's **evidence base**. Opening, closing, and tracking action items was measured in the DevOps and Observability course; here, the step before that is what counts. A question set is built, the record sources needed to answer each question are written down, and three numbers are drawn out: how many questions are answered, how many stay unanswered **for lack of a record**, and how many of the improvement items opened rest directly on a missing record. #### The Question Set and Record Sources **IR34:** The review takes place twelve days after the incident; the first foothold was nine days ago. **IR35:** There are sixteen questions; each question requires one or two record sources, and which day it needs to look at is known. **IR36:** Each of the nine record sources has three properties — whether it exists, its retention period in days, and whether it persists past containment and recovery. **IR37:** A question is answered only when all the sources it requires are available; if a source does not exist, if its retention period does not cover the day the question needs, or if the source was deleted at containment, the question stays unanswered. **IR38:** There are twelve improvement items; each one either rests directly on a missing record, refreshes a preparation item, or ties in a process step. **IR39:** Each item's annual person-hour cost and the minutes it shaves off the decision chain are given; the minute values come from rerunning the chain. **IR40:** The decision chain's starting value is the full-preparation value from the first lesson: 182 minutes. ```js // lessons/model.mjs — lessons-learned model: no real incident record or review involved. // The question set, record sources and improvement items are data structures. export const REVIEW_DAY = 12, FIRST_FOOTHOLD = 9; // day the review is held; how many days ago the first foothold occurred // record source: name -> { does it exist, retention in days, does it persist past containment and recovery } export const SOURCES = { process: { name: "process-creation record", exists: true, retention: 14, persists: true }, identity: { name: "authentication log", exists: true, retention: 90, persists: true }, network: { name: "network flow data", exists: true, retention: 30, persists: true }, application: { name: "application audit log", exists: true, retention: 7, persists: true }, detection: { name: "detection pipeline log", exists: true, retention: 30, persists: true }, response: { name: "response log", exists: true, retention: 365, persists: true }, file: { name: "file integrity monitoring", exists: false, retention: 0, persists: false }, data: { name: "data access log", exists: false, retention: 0, persists: false }, memory: { name: "memory dump", exists: true, retention: 365, persists: false }, }; // question: text, required sources, how many days ago the question needs to look at export const QUESTIONS = [ ["which asset the first execution occurred on and when", ["process"], 9], ["which identity gained the first access", ["identity"], 9], ["which assets the service account was used on", ["identity", "network"], 9], ["whether collector-07 and -11 were actually affected", ["process"], 0], ["how much data volume was exfiltrated", ["network"], 0], ["which account read the billing archive", ["application"], 9], ["what the last day's application accesses were", ["application"], 3], ["when the persistence points were established", ["file"], 9], ["at what moment the configuration hooks were added", ["memory", "file"], 9], ["why the alert fired at minute 186", ["detection"], 0], ["whether a rule existed that could have fired earlier", ["detection", "process"], 0], ["what data the scope decision was based on", ["response"], 0], ["at what minute containment was applied", ["response"], 0], ["how long the legitimate work that was cut off lasted", ["response", "application"], 0], ["when the missed points were established", ["memory", "file"], 9], ["which subscriber records were read", ["data"], 9], ]; // why a source cannot answer the question export function reason(k, day) { const s = SOURCES[k]; if (!s.exists) return "no source"; if (!s.persists) return "deleted at containment"; if (day > s.retention) return "retention ran out"; return null; } export function answer(question, added = []) { const [, required, day] = question; const missing = required.filter((k) => !added.includes(k) && reason(k, day)); return { answered: missing.length === 0, reasons: missing.map((k) => `${k}: ${reason(k, day)}`) }; } export const answered = (added = []) => QUESTIONS.filter((s) => answer(s, added).answered).length; // improvement item: name, the gap it rests on (source key or preparation item), type, // annual person-hours, minutes it shaves off the decision chain export const IMPROVEMENTS = [ ["turning on file integrity monitoring", "file", "record", 34, 0], ["turning on the data access log", "data", "record", 46, 0], ["extending application retention to 30 days", "application", "record", 12, 0], ["capturing memory before containment", "memory", "record", 18, 0], ["process-creation logging for collector-07..12", "process", "record", 26, 22], ["keeping the asset inventory current", "-", "preparation", 24, 10], ["refreshing the communication tree", "-", "preparation", 6, 6], ["refreshing the authorization matrix", "-", "preparation", 8, 8], ["mapping out record sources", "-", "preparation", 12, 7], ["a quarterly tabletop exercise", "-", "preparation", 30, 9], ["tying containment actions to a procedure", "-", "process", 14, 8], ["turning the scope criterion into a dashboard metric", "-", "process", 9, 7], ]; export const CHAIN_BASE = 182, ALERT = 186; // the previous lessons' full-preparation decision chain // access curve: [minute, asset] (same data as the previous lessons) export const ACCESS = [[0, "collector-03"], [26, "collector-03"], [58, "validator-01"], [95, "collector-07"], [134, "collector-11"], [171, "validator-02"], [214, "billing-01"], [263, "portal-01"], [318, "workorder-01"], [372, "endpoint-01"], [441, "storage-01"], [505, "billing-01"], [588, "collector-09"], [676, "console-01"], [762, "portal-02"], [845, "storage-01"], [951, "billing-02"], [1088, "console-01"]]; export const itemsReached = (min) => ACCESS.filter(([t]) => t <= min).length; // selected improvement set: answered questions, chain minutes, containment moment, items reached export function apply(selected) { const k = IMPROVEMENTS.filter(([name]) => selected.includes(name)); const added = k.filter((x) => x[2] === "record").map((x) => x[1]); const chain = CHAIN_BASE - k.reduce((s, x) => s + x[4], 0); return { items: k.length, answered: answered(added), chain, containment: ALERT + chain, reached: itemsReached(ALERT + chain), hours: k.reduce((s, x) => s + x[3], 0) }; } ``` The measurement file first matches sources to questions, then applies the improvement sets and works out the answered-question count together with the chain minutes. ```js // lessons/measure.mjs — which question stays unanswered for lack of a record, what the reason // for that is, and how many of the improvement items opened rest directly on a missing record. import { REVIEW_DAY, SOURCES, QUESTIONS, IMPROVEMENTS, CHAIN_BASE, reason, answer, answered, apply } from "./model.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); console.log(`review takes place ${REVIEW_DAY} days after the incident; ${QUESTIONS.length} questions, ` + `${Object.keys(SOURCES).length} record sources`); console.log("\n1. record sources"); const A = [-26, 7, 10, 9, 10, 12]; write(A, "source", "exists", "retention", "persists", "required", "blocked"); for (const [k, s] of Object.entries(SOURCES)) { const required = QUESTIONS.filter(([, g]) => g.includes(k)); const blocked = required.filter(([, , day]) => reason(k, day)); write(A, s.name, s.exists ? "yes" : "no", s.exists ? s.retention + " days" : "-", s.persists ? "yes" : "no", required.length, blocked.length); } console.log("\n2. unanswered questions and their reason"); const B = [-4, -50, -48]; for (const s of QUESTIONS) { const y = answer(s); if (!y.answered) write(B, "", s[0], y.reasons.join("; ")); } console.log(` answered ${answered()}/${QUESTIONS.length}`); console.log("\n3. improvement items"); const C = [-52, 12, 18, 12, 11]; write(C, "item", "type", "gap it closes", "annual p-h", "chain min"); for (const [name, gap, type, hours, min] of IMPROVEMENTS) write(C, name, type, gap === "-" ? "-" : SOURCES[gap].name.split(" ")[0], hours, min || "-"); const recordItems = IMPROVEMENTS.filter((x) => x[2] === "record").length; console.log(` items resting directly on a missing record: ${recordItems}/${IMPROVEMENTS.length}`); console.log("\n4. when improvement sets are applied"); const D = [-24, 8, 10, 11, 12, 11, 12]; write(D, "set", "items", "answered", "chain min", "containment", "reached", "annual p-h"); const names = (t) => IMPROVEMENTS.filter((x) => t.includes(x[2])).map((x) => x[0]); const SETS = { "none": [], "record only": names(["record"]), "preparation + process": names(["preparation", "process"]), "all": names(["record", "preparation", "process"]) }; for (const [k, selected] of Object.entries(SETS)) { const r = apply(selected); write(D, k, r.items, `${r.answered}/${QUESTIONS.length}`, r.chain, r.containment, `${r.reached}/18`, r.hours); } console.log("\n5. where the two axes do not convert into each other"); const recordR = apply(SETS["record only"]), prepR = apply(SETS["preparation + process"]); const E = [-23, 12, 17, 15, -22]; write(E, "set", "annual p-h", "questions opened", "minutes gained", " p-h per unit"); write(E, "record only", recordR.hours, recordR.answered - answered(), CHAIN_BASE - recordR.chain, ` ${(recordR.hours / (recordR.answered - answered())).toFixed(1)} / question`); write(E, "preparation + process", prepR.hours, prepR.answered - answered(), CHAIN_BASE - prepR.chain, ` ${(prepR.hours / (CHAIN_BASE - prepR.chain)).toFixed(1)} / minute`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // the raw measurement is not printed, only the band is checked console.log(`\nfour sets and the question scan within the 500 ms band: ${ms < 500 ? "yes" : "no"}`); ``` ``` review takes place 12 days after the incident; 16 questions, 9 record sources 1. record sources source exists retention persists required blocked process-creation record yes 14 days yes 3 0 authentication log yes 90 days yes 2 0 network flow data yes 30 days yes 2 0 application audit log yes 7 days yes 3 1 detection pipeline log yes 30 days yes 2 0 response log yes 365 days yes 3 0 file integrity monitoring no - no 3 3 data access log no - no 1 1 memory dump yes 365 days no 2 2 2. unanswered questions and their reason which account read the billing archive application: retention ran out when the persistence points were established file: no source at what moment the configuration hooks were added memory: deleted at containment; file: no source when the missed points were established memory: deleted at containment; file: no source which subscriber records were read data: no source answered 11/16 3. improvement items item type gap it closes annual p-h chain min turning on file integrity monitoring record file 34 - turning on the data access log record data 46 - extending application retention to 30 days record application 12 - capturing memory before containment record memory 18 - process-creation logging for collector-07..12 record process-creation 26 22 keeping the asset inventory current preparation - 24 10 refreshing the communication tree preparation - 6 6 refreshing the authorization matrix preparation - 8 8 mapping out record sources preparation - 12 7 a quarterly tabletop exercise preparation - 30 9 tying containment actions to a procedure process - 14 8 turning the scope criterion into a dashboard metric process - 9 7 items resting directly on a missing record: 5/12 4. when improvement sets are applied set items answered chain min containment reached annual p-h none 0 11/16 182 368 9/18 0 record only 5 16/16 160 346 9/18 136 preparation + process 7 11/16 127 313 8/18 103 all 12 16/16 105 291 8/18 239 5. where the two axes do not convert into each other set annual p-h questions opened minutes gained p-h per unit record only 136 5 22 27.2 / question preparation + process 103 0 55 1.9 / minute four sets and the question scan within the 500 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Eleven Have an Answer The first table counts the sources. Six sources exist and persist: the process-creation record, the authentication log, network flow data, the application audit log, the detection pipeline log, and the response log. The last column gives how many questions each source **blocks**. File integrity monitoring is required for three questions and blocks all three, because it does not exist at all. The memory dump is required for two questions and blocks both, because it does not persist. The application audit log is required for three questions and blocks only one — that question looks nine days back, while retention is seven days. The second table lists the five unanswered questions along with their reason, and the reason falls into three distinct kinds. **No source:** the questions of when the persistence points were established and which subscriber records were read are unanswered because file integrity monitoring and the data access log were never turned on at all. **Retention ran out:** which account read the billing archive is unanswered because it falls outside the application record's seven-day window — the source existed, the record existed, but it had been deleted. **Deleted at containment:** the questions of when the configuration hooks and the missed points were established are unanswered because the assets were pulled off the network before a memory dump was taken. This third reason comes straight out of this topic. In the third lesson, short-term containment stopped access at minute 393 and gained five items; in the fourth lesson, the same decision made memory examination impossible on seven assets and caused three configuration hooks to be missed. Here the same decision's third counterpart appears: two review questions can never be answered. **The containment decision's cost is paid in three separate lessons, in three separate units.** #### Five Rest on a Record The third table lists the twelve improvement items. Five rest directly on a missing record: turning on file integrity monitoring, turning on the data access log, extending application retention to thirty days, capturing memory before containment, and adding logging to the six collectors that were not forwarding process-creation records. Five refresh preparation items, and two tie in a process step. **Five of the twelve items are the direct consequence of a missing record.** The fourth table applies the sets. Record-only improvements cost five items and 136 person-hours a year; the answered-question count rises from 11 to **16** and the chain drops from 182 to 160 minutes — the only minute gain comes from adding process-creation logging. Preparation and process improvements cost seven items and 103 person-hours; the chain drops to 127 minutes, containment to minute 313, and the items reached fall from nine to eight, but the answered-question count **stays at 11**. All together cost 239 person-hours, give a 105-minute chain, and answer 16/16. #### Two Axes The fifth table shows that the two sets do not convert into each other. Record improvements cost 27.2 person-hours per question and shave only 22 minutes off the chain. Preparation and process improvements cost 1.9 person-hours per minute and open no questions at all. The same budget buys two different things: one buys **stopping the next incident faster**, the other buys **telling this incident's story better**. Ranking the two against each other is not this model's job, but one observation can be made. One of the record improvements — capturing memory before containment — opens two questions for eighteen person-hours and adds no minutes to the chain at all; in exchange, it delays containment, and that delay's cost was measured in the third lesson. If an improvement item carries a cost on another axis beyond the question it opens, the item has been measured incompletely unless that cost is also written into the list. #### Summary - Eleven of the sixteen review questions are answered with nine record sources; five stay unanswered. - The three reasons for being unanswered are counted separately: two questions have no source at all, one question's retention period was not enough, and two questions' sources were deleted at containment. - The containment decision's cost is five items in the third lesson, three missed footholds in the fourth lesson, and two unanswered questions here. - Five of the twelve improvement items rest directly on a missing record; five refresh a preparation item, and two tie in a process step. - Record-only improvements bring the answered-question count to 16/16 for 136 person-hours and shave 22 minutes off the chain; preparation and process improvements gain 55 minutes for 103 person-hours and open no questions at all. #### Next Step The review has drawn out its answers, written down the improvement items, and counted their costs. All of this stayed inside the organization. There is one more face of the incident that looks outward: who needs to be notified, at what threshold, within what time. The next lesson measures that as time and stakeholder count — how many stakeholders there are, at what threshold notification begins, how many corrections a notification made before scope is finalized produces, and how many hours the delay costs. ### Communication and Notification Obligations Source: https://academia.sh/en/courses/security-operations/communication-and-notification-obligations Measuring notification as time and stakeholder count: five of twelve stakeholders being external, the 4,320-minute window of the obligation type that requires notification within a fixed time, the scope estimate dropping from 48,000 to 6,200 without ever settling, early notification over-notifying 14,800 subscribers and producing 46 corrections against staying at 29,967 uninformed subscriber-hours, and late notification exceeding the window by 64,540 minutes across five external stakeholders while producing 1,793,247 subscriber-hours. The previous lesson counted five of sixteen review questions staying unanswered; one of them was which subscriber records were read, and it was unanswered because the data access log was never turned on at all. That gap turns directly into a decision in this lesson, because notification wants exactly this number: how many subscribers to tell what. Notification's unit of measure is **time and stakeholder count**. How many stakeholders there are, at what threshold each one is informed, how many minutes it takes to prepare the notification, and what the delay costs. A portion of the external stakeholders are subject to an obligation type that requires notification within a fixed time; that obligation's window starts from the moment of detection and is fixed. What the measurement will show is this: staying inside the window and reporting the correct number cannot both be achieved at once. #### Stakeholders, Thresholds and the Window **IR41:** There are twelve stakeholders; five external, seven internal. Each stakeholder has a notification threshold, and the threshold corresponds to a moment from the previous lessons: verification at 216, scope at 370, containment at 393, recovery at 1,363, and review at 17,280 minutes. **IR42:** For each stakeholder, preparing and approving the notification takes a set number of minutes; the notification moment is the threshold plus the prep time. **IR43:** The window of the obligation type that external stakeholders are subject to, which requires notification within a fixed time, is 4,320 minutes and starts from the moment of detection; the limit is minute 4,506. **IR44:** The affected-subscriber count is an **estimate** and narrows over time: 48,000, 21,000, 9,400, 7,100, and 6,200. Because there is no data access log, the number never settles; the final value is also an estimate. **IR45:** Once a stakeholder has been notified, every time the estimate changes, a correction notice is required, and a correction adds an extra load equal to 0.35 of the stakeholder's prep person-hours. **IR46:** Uninformed subscriber-hours are the final estimate's subscriber count multiplied by the time between detection and the subscriber notification. **IR47:** Over-notified subscribers are the difference between the estimate at the notification moment and the final estimate. ```js // notification/model.mjs — communication and notification model: no real institution, // regulation or incident involved. Stakeholders, thresholds and the scope estimate are // data structures; the durations are fictional data. export const DETECTION = 186; // the minute the alert fired export const WINDOW = 4320; // the window, in minutes, of the obligation type // that requires notification within a fixed time // threshold: name -> the minute it occurs at (carried over from the previous lessons) export const THRESHOLD = { verification: 216, scope: 370, containment: 393, recovery: 1363, review: 17280 }; // stakeholder: name, internal/external, threshold, notification prep minutes, prep person-hours export const STAKEHOLDERS = [ ["shift lead", "internal", "verification", 5, 0.3], ["information security manager", "internal", "verification", 8, 0.5], ["service-owning teams", "internal", "verification", 12, 1.5], ["external response support", "external", "verification", 15, 1.0], ["executive leadership", "internal", "scope", 35, 2.0], ["legal department", "internal", "scope", 25, 1.5], ["notification-obligation contact", "external", "scope", 180, 8.0], ["business continuity team", "internal", "containment", 18, 1.0], ["corporate communications", "internal", "containment", 40, 3.0], ["enterprise customers", "external", "containment", 120, 6.0], ["affected subscribers", "external", "recovery", 260, 14.0], ["insurance provider", "external", "recovery", 95, 4.0], ]; // scope estimate: [minute, affected-subscriber count to notify at that moment] (data). // The number never firms up, because the previous lesson found no data access log. export const ESTIMATE = [[216, 48000], [370, 21000], [1363, 9400], [4506, 7100], [17280, 6200]]; export const estimate = (min) => [...ESTIMATE].reverse().find(([t]) => t <= min)?.[1] ?? ESTIMATE[0][1]; export const corrections = (min) => ESTIMATE.filter(([t]) => t > min).length; export const FINAL = ESTIMATE[ESTIMATE.length - 1][1]; // policy: the moment notification starts for a stakeholder export const POLICY = { "early": () => THRESHOLD.verification, "by threshold": (p) => THRESHOLD[p[2]], "at window's end": (p) => p[1] === "external" ? DETECTION + WINDOW - p[3] : THRESHOLD[p[2]], "late": () => THRESHOLD.review, }; export function measure(policyName) { const f = POLICY[policyName]; const rows = STAKEHOLDERS.map((p) => { const start = f(p), moment = start + p[3]; const external = p[1] === "external"; return { name: p[0], external, moment, delay: external ? Math.max(0, moment - (DETECTION + WINDOW)) : 0, corrections: corrections(moment), notified: estimate(moment), hours: p[4] }; }); const subscriber = rows.find((s) => s.name === "affected subscribers"); return { rows, lateExternal: rows.filter((s) => s.external && s.delay > 0).length, excess: Math.max(0, subscriber.notified - FINAL), notified: subscriber.notified, subscriberMoment: subscriber.moment, delay: rows.reduce((a, s) => a + s.delay, 0), corrections: rows.reduce((a, s) => a + s.corrections, 0), hours: rows.reduce((a, s) => a + s.hours * (1 + s.corrections * 0.35), 0), subscriberHours: Math.round(FINAL * (subscriber.moment - DETECTION) / 60) }; } ``` The measurement file first sorts stakeholders by threshold, then runs the four notification policies against the same incident. ```js // notification/measure.mjs — how many stakeholders there are, at what threshold notification // begins, how many corrections a notification made before scope is finalized produces, and // how many subscriber-hours the delay costs. import { DETECTION, WINDOW, THRESHOLD, STAKEHOLDERS, ESTIMATE, FINAL, POLICY, estimate, corrections, measure } from "./model.mjs"; const write = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("")); const t0 = process.hrtime.bigint(); const LIMIT = DETECTION + WINDOW; console.log(`${STAKEHOLDERS.length} stakeholders (${STAKEHOLDERS.filter((p) => p[1] === "external").length} external); ` + `detection at minute ${DETECTION}, notification window ${WINDOW} min, limit at minute ${LIMIT}`); console.log("\n1. stakeholders and their threshold-based notification moment"); const A = [-32, 9, 13, 10, 14, 13, 12]; write(A, "stakeholder", "type", "threshold", "prep", "notify moment", "in window", "notified"); for (const p of STAKEHOLDERS) { const moment = THRESHOLD[p[2]] + p[3]; write(A, p[0], p[1], p[2], p[3] + " min", moment, p[1] === "external" ? (moment <= LIMIT ? "yes" : "no") : "-", p[0] === "affected subscribers" ? estimate(moment) : "-"); } console.log("\n2. scope estimate (never firms up for lack of a data access log)"); const B = [-10, 10, 13, 18]; write(B, "minute", "estimate", "vs. final", "corrections left"); for (const [min, v] of ESTIMATE) write(B, min, v, (v / FINAL).toFixed(2) + "x", corrections(min)); console.log("\n3. four notification policies"); const C = [-18, 18, 12, 10, 14, 12, 12, 12]; write(C, "policy", "subscriber moment", "notified", "excess", "late external", "delay min", "corrections", "person-h"); const R = {}; for (const k of Object.keys(POLICY)) { const r = R[k] = measure(k); write(C, k, r.subscriberMoment, r.notified, r.excess, r.lateExternal, r.delay, r.corrections, r.hours.toFixed(1)); } console.log("\n4. the two sides of the decision"); const D = [-18, 21, 18, 21]; write(D, "policy", "uninformed sub-hours", "over-notified", "correction notices"); for (const [k, r] of Object.entries(R)) write(D, k, r.subscriberHours, r.excess, r.corrections); const e = R["early"], g = R["late"]; console.log(` early vs. late: ${g.subscriberHours - e.subscriberHours} subscriber-hours apart, against ` + `${e.excess - g.excess} over-notified subscribers and ${e.corrections - g.corrections} corrections`); console.log("\n5. external stakeholders past the window under the late policy"); for (const s of R["late"].rows.filter((x) => x.external && x.delay > 0)) write([-4, -32, 10, 12], "", s.name, s.moment, `+${s.delay} min`); const ms = Number(process.hrtime.bigint() - t0) / 1e6; // the raw measurement is not printed, only the band is checked console.log(`\nfour policies and ${STAKEHOLDERS.length} stakeholders within the 500 ms band: ${ms < 500 ? "yes" : "no"}`); ``` ``` 12 stakeholders (5 external); detection at minute 186, notification window 4320 min, limit at minute 4506 1. stakeholders and their threshold-based notification moment stakeholder type threshold prep notify moment in window notified shift lead internal verification 5 min 221 - - information security manager internal verification 8 min 224 - - service-owning teams internal verification 12 min 228 - - external response support external verification 15 min 231 yes - executive leadership internal scope 35 min 405 - - legal department internal scope 25 min 395 - - notification-obligation contact external scope 180 min 550 yes - business continuity team internal containment 18 min 411 - - corporate communications internal containment 40 min 433 - - enterprise customers external containment 120 min 513 yes - affected subscribers external recovery 260 min 1623 yes 9400 insurance provider external recovery 95 min 1458 yes - 2. scope estimate (never firms up for lack of a data access log) minute estimate vs. final corrections left 216 48000 7.74x 4 370 21000 3.39x 3 1363 9400 1.52x 2 4506 7100 1.15x 1 17280 6200 1.00x 0 3. four notification policies policy subscriber moment notified excess late external delay min corrections person-h early 476 21000 14800 0 0 46 95.0 by threshold 1623 9400 3200 0 0 38 82.6 at window's end 4506 7100 900 0 0 29 65.4 late 17540 6200 0 5 64540 0 42.8 4. the two sides of the decision policy uninformed sub-hours over-notified correction notices early 29967 14800 46 by threshold 148490 3200 38 at window's end 446400 900 29 late 1793247 0 0 early vs. late: 1763280 subscriber-hours apart, against 14800 over-notified subscribers and 46 corrections 5. external stakeholders past the window under the late policy external response support 17295 +12789 min notification-obligation contact 17460 +12954 min enterprise customers 17400 +12894 min affected subscribers 17540 +13034 min insurance provider 17375 +12869 min four policies and 12 stakeholders within the 500 ms band: yes ``` The numbers belong to the **measurement** class; their inputs are the assumptions above. #### Twelve Stakeholders The first table converts stakeholders' thresholds into notification moments. The seven internal stakeholders are informed between minutes 221 and 433; their prep times range from 5 to 40 minutes. For external stakeholders, prep stretches out noticeably: 180 minutes for the notification- obligation contact, 120 for enterprise customers, 260 for affected subscribers. This difference is not a delay — it is the fact that the notification itself is work: text gets written, legal reads it, a channel gets prepared. Under the by-threshold policy, all five external stakeholders stay inside the window; the latest notification goes to affected subscribers at minute 1,623, and the limit is minute 4,506. The window is not tight in this incident. What is tight is something else. The distribution of thresholds is also a decision. Three internal stakeholders are informed at the verification threshold — that is, the moment the alert is understood to be real; legal and executive leadership wait for the scope threshold. This order is not arbitrary: an executive notification made before scope is determined cannot say how many assets were affected, and it produces no decision in return. None of the external stakeholders sit before the verification threshold; the sole exception is external response support, and that is not a notification but a call. #### The Number That Never Settles The second table gives the scope estimate over time. At minute 216, the estimate is 48,000 — **7.74 times** the final estimate. It drops to 21,000 at minute 370, to 9,400 at minute 1,363, to 7,100 at minute 4,506, and settles at 6,200 at minute 17,280. The last column gives how many corrections come after each moment. This table's meaning was already established in the previous lesson: because the data access log was never turned on at all, the question of which subscriber records were read is unanswered. The number is therefore not a measurement but a narrowing estimate; 6,200 is not certain either, only the value that stops narrowing further. **Notification has to be made on the basis of a number that never settles**, and this turns the choice of notification moment into a question of estimate accuracy. #### Four Policies The third table runs the four policies. Under the early policy, every stakeholder is informed at the moment of verification; subscriber notification goes out at minute 476 with the count 21,000, 14,800 subscribers are over-notified, 46 corrections are required, and the total load is 95.0 person-hours. Under the by-threshold policy, subscriber notification shifts to minute 1,623, notified drops to 9,400 and over-notified to 3,200, and corrections fall to 38. The at-window's-end policy places external notifications right before the limit: minute 4,506, 7,100 subscribers, 900 over-notified, 29 corrections, and 65.4 person-hours. The late policy waits until the review is complete. The notified count is 6,200, over-notified is zero, and no corrections are needed at all — at 42.8 person-hours, it is the lowest. Its counterpart is in the fifth table: all five external stakeholders exceed the window, the total delay is **64,540 minutes**, and the largest overrun is with affected subscribers, at 13,034 minutes. The person-hours column is misleading on its own. Most of the early policy's 95.0 person-hours comes not from the notification itself but from the 46 corrections that follow it; each correction brings back a third of the stakeholder's prep load. As the policy is delayed, the correction count drops from 46 to 38, to 29, and to zero. The policy that looks cheapest in this column is the one that notifies affected subscribers nine days after the window; its cost is paid not in person-hours but in minutes. Choosing a notification policy by prep load alone means making a delayed notification look cheap. #### Two Sides of the Decision The fourth table places two columns side by side. Under the early policy, uninformed subscriber-hours is 29,967 and over-notified subscribers is 14,800. Under the late policy, over-notified is zero and uninformed subscriber-hours is **1,793,247**. The difference between the two extremes is 1,763,280 subscriber-hours against 14,800 over-notified subscribers and 46 corrections. The two policies in between show how this curve bends. Compared to early, the by-threshold policy adds 118,523 subscriber-hours to bring over-notified down from 14,800 to 3,200 — roughly ten subscriber-hours per subscriber. The at-window's-end policy adds a further 297,910 subscriber-hours to bring 3,200 down to 900; this time, a hundred thirty subscriber-hours per subscriber. **Accuracy gets more expensive as it goes.** The at-window's-end policy sits at the last moment the obligation's window allows, and past that point accuracy can only be increased by exceeding the window — which is exactly what the late policy does. The over-notified subscriber count is innocent enough as a number, but its counterpart is telling 14,800 people that their own data may have been seen; a later correction does not take back that first notification. Uninformed subscriber-hours is also a number, and its counterpart is the time the 6,200 people who were genuinely affected spent not knowing it. The model counts both and cannot convert one into the other; whoever makes the decision has to write down, by their own measure, which column weighs more. #### Summary - Five of the twelve stakeholders are external; internal notifications go out at minutes 221–433, external notifications go out at minutes 231–1,623 under the by-threshold policy, and prep times range from 5 to 260 minutes. - The window of the obligation type that requires notification within a fixed time is 4,320 minutes and the limit is minute 4,506; under the by-threshold policy, all five external stakeholders stay inside the window. - The affected-subscriber estimate narrows from 48,000 to 6,200 but never settles for lack of a data access log; notification is made on the basis of a number that has not settled. - The early policy over-notifies 14,800 subscribers and produces 46 corrections, with 29,967 uninformed subscriber-hours; the late policy never over-notifies but exceeds the window by 64,540 minutes across five external stakeholders and produces 1,793,247 subscriber-hours. - Accuracy gets more expensive as it goes: bringing over-notified down from 14,800 to 3,200 costs ten subscriber-hours per subscriber, and from 3,200 to 900 costs a hundred thirty subscriber-hours. #### Next Step The incident is closed: access was stopped, footholds were cleaned up, the service was brought back, and stakeholders were informed. Every decision measured across seven lessons was tied to a number. One could not be tied down, and it kept coming back in every lesson: most of the items that would prove **what happened** had already been deleted at the moment of containment. Assets were pulled off the network before a memory dump was taken, three configuration hooks could never be found again, two review questions stayed unanswered, and the subscriber count remained an estimate. The next topic takes on these items on their own terms: what shows the evidence's integrity, which items are volatile and vanish if not collected, how many items are lost when the collection order is wrong, and at how many items the chain of custody breaks. ## Forensic Examination Measuring the evidence item: evidence integrity and how many items the chain of custody breaks at, the verifiability of write-blocked copying and the items a partial copy misses, the items lost when order of volatility and collection order go wrong, merging three sources' timelines and the order that clock skew disrupts, the evidence value of a trace extracted from system structures, and presenting technical findings as a claim-to-evidence mapping in the legal context. ### Forensic Examination Principles Source: https://academia.sh/en/courses/security-operations/forensic-examination-principles Measuring evidence integrity and chain of custody in items: 28 evidence items, 78 transfers, and five record fields per transfer; 10 of 390 fields left blank puts 9 items in question, and hash mismatches put 3 more in question, leaving 12 of 28 items indefensible. If a single point in the field chain loses its record entirely, 13 items open up at once; the way a 2.6 percent record gap turns into a 43 percent item loss comes from the chain's structure. The previous topic closed the incident: containment was applied, the affected machines were pulled off the network, service came back. That closure left one question hanging — most of the items that would prove **what** the incident was had already been erased by the time containment happened. Memory was gone once the machine was powered off, the connection list was gone once the session dropped, temp files were gone once the system was reinstalled. A narrative can be built from what is left; whether that narrative is **testable** is a separate question. This topic measures that question, and the unit of measure is the **evidence item**. Three columns get counted: how many items are volatile and vanish if not collected, how many items the collection order costs, and how many items the chain of custody breaks at. This lesson builds the third column. In a fictional regional measurement network, unauthorized access was seen at a branch endpoint, and the examination covers six hosts. **Every number comes from a model.** Collecting evidence from a system that belongs to someone else — imaging it, dumping it, copying its records — is both illegal and unprofessional without written authorization and a defined scope. - **FE1.** An **evidence item** is a piece of data taken from a single source and bindable to a single integrity value: a disk image, a memory dump, a log slice, a configuration copy. - **FE2.** Every item has a **lifetime**: the time that passes before it is lost if not collected. An item whose lifetime does not exceed one hour is **volatile**. - **FE3.** The **chain of custody** is the record of transfers an item passes through from the moment it is collected to the moment it is analyzed. The chain consists of nodes; a transfer is the handover between two nodes. - **FE4.** Every transfer requires **five record fields**: from whom, to whom, when, in what medium, and which hash. The fields are filled in by hand and can be left blank. - **FE5.** An item is **defensible** only if the hash at the time of collection matches the hash at the time of analysis **and** no transfer in its chain has a missing field. Otherwise the item is **contestable** — it has not been shown to be wrong, but it cannot be shown to be right. - **FE6.** Item counts, durations, and the error rate are fictional; the random seed is visible. #### Item, Hash, and Transfer ```js // forensics/item.mjs — evidence item set, hash-based integrity, and chain of custody. A MODEL. // The regional measurement network is fictional; no real organization, incident, or machine name is used. import { createHash } from "node:crypto"; export const SEED = 20260731; const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; // class -> [lifetime(s): time before it is lost if not collected, collection duration(s)] export const CLASS = { "network-state": [120, 45], "process-list": [240, 60], "open-handle": [300, 55], "session-state": [600, 70], "memory-content": [900, 420], "temp-file": [3600, 180], "flow-record": [86400, 240], "disk-image": [604800, 5400], "configuration": [604800, 90], "remote-log": [2592000, 300], }; // host -> [chain of custody nodes, item classes it carries] export const HOST = { "branch-endpoint-3": [["collection", "packaging", "transit", "lab", "analysis"], ["network-state", "process-list", "open-handle", "session-state", "memory-content", "temp-file", "disk-image", "configuration"]], "field-laptop-14": [["collection", "packaging", "transit", "lab", "analysis"], ["network-state", "process-list", "session-state", "memory-content", "disk-image"]], "collector-2": [["collection", "transit", "analysis"], ["network-state", "process-list", "open-handle", "session-state", "memory-content", "disk-image", "configuration"]], "portal-frontend": [["collection", "transit", "analysis"], ["network-state", "process-list", "memory-content", "configuration"]], "central-log": [["intake", "analysis"], ["remote-log", "configuration"]], "flow-collector": [["intake", "analysis"], ["flow-record", "configuration"]], }; // The five record fields required at each transfer. export const FIELD = ["from", "to", "time", "medium", "hash"]; // Items opened without a write blocker: content unchanged, but the access timestamp changes. export const UNPROTECTED = ["branch-endpoint-3/temp-file", "collector-2/disk-image", "portal-frontend/configuration"]; export const digest = (s) => createHash("sha256").update(s).digest("hex").slice(0, 16); export function items() { const r = generator(SEED), list = []; for (const [host, [nodes, classes]] of Object.entries(HOST)) for (const c of classes) { const name = `${host}/${c}`; const content = `${name}|${Math.floor(r() * 1e9)}`; list.push({ name, host, class: c, nodes, lifetime: CLASS[c][0], duration: CLASS[c][1], collectionHash: digest(content), analysisHash: digest(content + (UNPROTECTED.includes(name) ? "|access-timestamp" : "")), transfer: nodes.slice(1).map((d, i) => { const record = Object.fromEntries(FIELD.map((a) => [a, true])); if (r() < 0.11) record[FIELD[Math.floor(r() * 5)]] = false; // field left blank return { point: `${nodes[i]}>${d}`, record }; }) }); } return list; } export const missingFields = (k) => k.transfer.reduce((a, t) => a + FIELD.filter((x) => !t.record[x]).length, 0); export const intact = (k) => k.collectionHash === k.analysisHash; export const defensible = (k) => intact(k) && missingFields(k) === 0; ``` The hash does exactly one job here: it **binds the item to itself**. The digest computed at collection time is recorded; it is recomputed at analysis time. If the two values match, the item is shown not to have changed between those two moments. If they do not match, the item is shown to have changed — not how or by whom, only **that** it changed. That is what the hash proves, and no more. Three items were opened without write protection. Nothing was written to their content; only the access timestamp was refreshed. The hash does not see this distinction, because it should not: an integrity value measures **difference**, not intent. Whether an item changed by accident or on purpose is a question for the chain, not for the hash. #### The Chain's Numbers ```js // forensics/measure.mjs — item, transfer, and defensibility count per host. import { items, FIELD, CLASS, missingFields, intact, defensible, SEED } from "./item.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const K = items(); const volatile = K.filter((k) => k.lifetime <= 3600).length; console.log(`model (seed ${SEED}): ${K.length} evidence items, ${Object.keys(CLASS).length} ` + `classes, ${FIELD.length} record fields per transfer`); console.log(`${volatile} items volatile (lifetime under one hour), ${K.length - volatile} items persistent`); const A = [-19, 6, 7, 11, 8, 16, 12]; console.log(""); y(A, "host", "item", "nodes", "transfers", "fields", "missing fields", "defensible"); for (const host of [...new Set(K.map((k) => k.host))]) { const g = K.filter((k) => k.host === host); const transfers = g.reduce((a, k) => a + k.transfer.length, 0); y(A, host, g.length, g[0].nodes.length, transfers, transfers * FIELD.length, g.reduce((a, k) => a + missingFields(k), 0), `${g.filter(defensible).length}/${g.length}`); } const transfers = K.reduce((a, k) => a + k.transfer.length, 0); y(A, "total", K.length, "-", transfers, transfers * FIELD.length, K.reduce((a, k) => a + missingFields(k), 0), `${K.filter(defensible).length}/${K.length}`); console.log(`\nhash mismatch in ${K.filter((k) => !intact(k)).length} items: ` + K.filter((k) => !intact(k)).map((k) => k.name).join(", ")); ``` ``` model (seed 20260731): 28 evidence items, 10 classes, 5 record fields per transfer 18 items volatile (lifetime under one hour), 10 items persistent host item nodes transfers fields missing fields defensible branch-endpoint-3 8 5 32 160 4 3/8 field-laptop-14 5 5 20 100 4 2/5 collector-2 7 3 14 70 1 5/7 portal-frontend 4 3 8 40 0 3/4 central-log 2 2 2 10 1 1/2 flow-collector 2 2 2 10 0 2/2 total 28 - 78 390 10 16/28 hash mismatch in 3 items: branch-endpoint-3/temp-file, collector-2/disk-image, portal-frontend/configuration ``` Two columns in the table must be read together. The field hosts follow a five-node chain: the item is collected, packaged, moved to the lab, received, and analyzed. Four transfers means twenty fields to fill in per item. At the central hosts the chain is three nodes, and for items pulled in remotely it is two nodes — there is no physical transport, so there are fewer fields to fill in. The result column reflects this structure directly. At the eight-item branch endpoint, the number of defensible items is three; at the two-item flow collector, two out of two are defensible. As the chain gets longer, the record load per item grows, and every extra field is one more chance to be left blank. **A longer chain does not protect the item more; it makes it breakable in more places.** #### A Single Missing Record ```js // forensics/gap.mjs — how many items a single missing record puts in question. import { items, FIELD, missingFields, intact } from "./item.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const K = items(); // For each transfer point: how many items pass through it; if its record drops entirely, all open up. const point = new Map(); for (const k of K) k.transfer.forEach((t, i) => { if (!point.has(t.point)) point.set(t.point, { passing: 0, next: 0 }); const n = point.get(t.point); n.passing += 1; n.next = k.transfer.length - i - 1; }); const A = [-24, 14, 14, 26]; y(A, "transfer point", "items passing", "next step", "opens up if record drops"); for (const [name, n] of [...point].sort((a, b) => b[1].passing - a[1].passing)) y(A, name, n.passing, n.next, `${n.passing} items (${Math.round(n.passing * 100 / K.length)}%)`); // Observed state: which field is missing, which question goes unanswered. const QUESTION = { from: "who had the item", to: "who received it", time: "can the order be proven", medium: "under what condition was it stored", hash: "can it be shown unchanged" }; console.log(""); const B = [-15, -37, 9, 7]; y(B, "missing field", "question left unanswered", "transfers", "items"); for (const a of FIELD) { const t = K.reduce((s, k) => s + k.transfer.filter((x) => !x.record[a]).length, 0); y(B, a, QUESTION[a], t, K.filter((k) => k.transfer.some((x) => !x.record[a])).length); } const fields = K.reduce((a, k) => a + k.transfer.length, 0) * FIELD.length; const empty = K.reduce((a, k) => a + missingFields(k), 0); const open = K.filter((k) => missingFields(k) > 0 || !intact(k)).length; console.log(`\n${empty} of ${fields} record fields are empty (${(empty * 100 / fields).toFixed(1)}%); ` + `this puts ${K.filter((k) => missingFields(k) > 0).length} items in question, and hash mismatches ` + `put ${K.filter((k) => !intact(k)).length} more in question`); console.log(`total ${open} / ${K.length} items contestable ` + `(${Math.round(open * 100 / K.length)}%), ${K.length - open} items defensible`); const worst = Math.max(...[...point.values()].map((n) => n.passing)); console.log(`if a single point's record drops entirely, the worst case is ${worst} items ` + `(${Math.round(worst * 100 / K.length)}%), added on top of the ${open} items already observed`); ``` ``` transfer point items passing next step opens up if record drops collection>packaging 13 3 13 items (46%) packaging>transit 13 2 13 items (46%) transit>lab 13 1 13 items (46%) lab>analysis 13 0 13 items (46%) collection>transit 11 1 11 items (39%) transit>analysis 11 0 11 items (39%) intake>analysis 4 0 4 items (14%) missing field question left unanswered transfers items from who had the item 3 3 to who received it 1 1 time can the order be proven 1 1 medium under what condition was it stored 2 2 hash can it be shown unchanged 3 3 10 of 390 record fields are empty (2.6%); this puts 9 items in question, and hash mismatches put 3 more in question total 12 / 28 items contestable (43%), 16 items defensible if a single point's record drops entirely, the worst case is 13 items (46%), added on top of the 12 items already observed ``` The last three lines are this lesson's result. **2.6 percent** of record fields are empty, and that puts **43 percent** of items in question. The gap between the two rates is not an arithmetic error; it is the chain's structure: a field is filled in one at a time, but an item is defended as a whole. If one of an item's twenty fields is empty, the verdict for the entire item is "cannot be shown unchanged." Integrity is not a divisible property. The first table shows where this fragility concentrates. Each of the four points in the field chain carries **thirteen items**; if any one of them loses its record entirely, thirteen items open up at once. This is not an item-to-record relationship; it is a **bottleneck** relationship: the same courier, the same intake desk, the same form. For the four items taken in remotely, the same risk measures four. The second table separates what goes unanswered by the type of missing field, and the columns are not equal. If the **hash** field is empty, the item cannot be shown to be unchanged; that makes the item directly unusable. If the **medium** field is empty, only the storage condition is unknown; the item is still bound by its hash, and that gap can later be closed by testimony. If a chain audit records these two cases in the same column, it confuses a closable gap with one that cannot be closed, and spends the examination's budget in the wrong place. #### The Cost of Shortening the Chain ```js // forensics/shorten.mjs — the link between chain length and expected open items. import { items, FIELD, missingFields } from "./item.mjs"; const K = items(), P = 0.11; // probability a field is left blank, per transfer const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const expect = (a) => 1 - (1 - P) ** a; // probability at least one field is left blank const A = [-20, 7, 11, 8, 10, 21]; y(A, "chain", "item", "transfers", "fields", "observed", "expected open items"); let tk = 0, ta = 0, tb = 0, tg = 0; for (const [name, n] of [["field (5 nodes)", 4], ["central (3 nodes)", 2], ["remote (2 nodes)", 1]]) { const g = K.filter((k) => k.transfer.length === n); const observed = g.filter((k) => missingFields(k) > 0).length; y(A, name, g.length, g.length * n, g.length * n * FIELD.length, observed, (g.length * expect(n)).toFixed(2)); tk += g.length; ta += g.length * n; tg += observed; tb += g.length * expect(n); } y(A, "total", tk, ta, ta * FIELD.length, tg, tb.toFixed(2)); // On-site analysis: field items are not transported, the chain drops to a single transfer. const s = K.filter((k) => k.transfer.length === 4).length; const a2 = s + K.filter((k) => k.transfer.length === 2).length * 2 + K.filter((k) => k.transfer.length === 1).length; const b2 = s * expect(1) + K.filter((k) => k.transfer.length === 2).length * expect(2) + K.filter((k) => k.transfer.length === 1).length * expect(1); y(A, "on-site analysis", tk, a2, a2 * FIELD.length, "-", b2.toFixed(2)); console.log(`transfers ${ta} -> ${a2} (${Math.round((1 - a2 / ta) * 100)}% fewer), ` + `expected open items ${tb.toFixed(2)} -> ${b2.toFixed(2)}`); ``` ``` chain item transfers fields observed expected open items field (5 nodes) 13 52 260 7 4.84 central (3 nodes) 11 22 110 1 2.29 remote (2 nodes) 4 4 20 1 0.44 total 28 78 390 9 7.57 on-site analysis 28 39 195 - 4.16 transfers 78 -> 39 (50% fewer), expected open items 7.57 -> 4.16 ``` The expected column tests the model against itself: with an 11 percent probability of a blank field per transfer, 4.84 of the field items are expected to open up; the observed number is 7. The deviation is normal for a small set, and that is exactly why chain audits are done item by item, not by average. The last row measures an option. If field items are not transported to the lab and analysis is done on-site, the chain drops to a single transfer: 78 transfers fall to 39, and expected open items fall from 7.57 to 4.16. The decision cannot be made without writing down what this gain costs. On-site analysis requires someone with that skill set to travel to the field; the time they spend traveling is added to every item's clock, and in the table, items whose lifetime is under two hundred seconds will not wait that long. **Shortening the chain protects the item; delay destroys the item** — the two are opposite ends of the same measure, and which one dominates can only be worked out with actual durations. #### Summary - An evidence item is a piece of data bindable to a single integrity value; in the fictional incident, 18 of 28 items are volatile, with a lifetime under one hour. - The hash binds the item to itself and shows only **that** it changed: three items have a mismatch, all three were opened without write protection, and only their access timestamp was refreshed. - The chain of custody is four transfers at field hosts, two at central hosts, and a single transfer for remote intake; five fields per transfer means 78 transfers and 390 fields in total. - 2.6 percent of fields being left blank puts 43 percent of items in question: a field is filled in one at a time, but an item is defended as a whole. - Each of the four points in the field chain carries 13 items; a single point losing its record opens up 46 percent of items at once. #### Next Step This lesson built the chain **around** the item: who carried it, when it was handed over, which hash was recorded. How the item itself was collected was never asked. In the table, three of the 28 items are disk images, and each carries a 5,400-second collection duration in the model — these are the most expensive items in the topic. The next lesson measures that copying in terms of **verifiability**. What proves the copy is identical to the source, which part of the source never makes it into the copy, how many items a partial copy misses, and how many items change during copying when the copy is taken from a running system. Three numbers come out separately; the first is established with a hash, the second with scope, and the third can only be seen by comparing two hashes. ### Disk Imaging Source: https://academia.sh/en/courses/security-operations/disk-imaging Measuring the verifiability of write-blocked copying: on a fictional 4,096-slice source, a selective directory copy takes 4 of 15 evidence items, a logical copy takes 8, a partition block takes 13, and a full block copy takes all 15. Because the source hash can be recomputed in an offline copy, 15 of 15 items are verified; in a copy taken from a running system, 303 of 4,096 slices change, all three hashes come out different, and the number of verified items drops to 9. The previous lesson built the chain around the item and noted that three of the 28 items are disk images, each carrying a 5,400-second collection duration in the model. The chain's first link, the **collection** step, passed there as a single box. This lesson opens that box and asks a single question: what proves the copy is identical to the source. The question is not as narrow as it looks. A hash compares two values; for the comparison to mean anything, **the second value has to be computable**. If the source does not change during copying, its hash can be recomputed afterward and compared with the image's. If the source changes, there is no fixed value to compare against, and the hash only binds the image to itself. This lesson works out three numbers separately: how many items are bound to the source by a hash, how many items a partial copy misses, and how many items change during copying. - **FE7.** The source is modeled as a 512 GiB storage volume divided into 4,096 **slices**. A slice here is a unit of measurement, not a file-system block; no real storage volume is read. - **FE8.** A **write blocker** does not pass through write requests directed at the source. The source does not change during or after copying. - **FE9.** A **forensic image** is a sequential copy of every slice within scope. The copying speed is 97 MiB/s, and it is fictional. - **FE10.** A copy is **verifiable** only if the source's hash can be recomputed after copying and compared with the image's hash. - **FE11.** On a running system, each zone changes at its own rate; the rate is the number of slices that change per 1,000 seconds, and it is fictional. - **FE12.** An evidence item occupies five consecutive slices; the item's hash is computed from the digests of those five slices. #### Source, Zone, and Scope ```js // forensics/disk.mjs — slice model of a 512 GiB source, its zones, and its evidence items. // A MODEL: no real disk is read; slice content comes from the generator. import { createHash } from "node:crypto"; export const SEED = 20260803, SLICE = 4096, MIB = 128, SPEED = 97; // slice 128 MiB, 97 MiB/s export const DURATION = Math.round(SLICE * MIB / SPEED); // full copy duration (s) const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; // zone -> [slice count, slices changed per 1,000 seconds (on a running system)] export const ZONE = { "partition-table": [8, 0], "filesystem-meta": [240, 9], "active-files": [2400, 26], "filesystem-journal": [128, 14], "swap-area": [512, 11], "unallocated-space": [760, 2], "off-partition-area": [48, 0], }; // Copying scope -> which zones it takes. export const SCOPE = { "selective directory": ["active-files"], // only the selected directories "logical copy": ["active-files", "filesystem-meta"], "partition block": ["active-files", "filesystem-meta", "filesystem-journal", "swap-area", "unallocated-space"], "full block": Object.keys(ZONE), }; // item -> [zone, whether inside the selected directory] export const ITEM = { "configuration-file": ["active-files", true], "service-definition": ["active-files", true], "application-log": ["active-files", true], "scheduled-task": ["active-files", true], "user-document": ["active-files", false], "cache-file": ["active-files", false], "deleted-installer-remnant": ["unallocated-space", false], "deleted-log-fragment": ["unallocated-space", false], "temp-file-residue": ["unallocated-space", false], "deleted-file-name": ["filesystem-meta", false], "timestamp-record": ["filesystem-meta", false], "log-entry": ["filesystem-journal", false], "swap-space-residue": ["swap-area", false], "off-partition-configuration": ["off-partition-area", false], "partition-table-record": ["partition-table", false], }; // Slices are laid out zone by zone; each item occupies five consecutive slices. export function source() { const r = generator(SEED), d = []; for (const [b, [n]] of Object.entries(ZONE)) for (let i = 0; i < n; i += 1) d.push({ no: d.length, zone: b, content: `${b}:${i}:${Math.floor(r() * 1e9)}` }); const item = {}; for (const [name, [b, selected]] of Object.entries(ITEM)) { const zoneSlices = d.filter((x) => x.zone === b).map((x) => x.no); const start = Math.floor(r() * (zoneSlices.length - 5)); item[name] = { zone: b, selected, slice: zoneSlices.slice(start, start + 5) }; } return { slice: d, item }; } export const digest = (s) => createHash("sha256").update(s).digest("hex").slice(0, 16); export const wholeHash = (d) => digest(d.map((x) => digest(x.content)).join("|")); export const itemHash = (d, k) => digest(k.slice.map((n) => digest(d[n].content)).join("|")); export const covered = (scope, name) => { const [b, selected] = ITEM[name]; return SCOPE[scope].includes(b) && (scope !== "selective directory" || selected); }; ``` The zone list is this lesson's real framework. A storage volume is not just files: file-system metadata carries the names and timestamps of deleted files, unallocated space carries the not-yet-overwritten remnants of deleted content, swap space carries fragments dropped out of running processes' memory, and the area outside the partition table is never seen by the file system at all. Most evidence items live in these zones. #### How Many Items Scope Misses ```js // forensics/scope.mjs — slices, duration, and items missed by copying scope. import { source, ZONE, SCOPE, ITEM, SLICE, MIB, SPEED, DURATION, covered } from "./disk.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const { slice } = source(); const selectedShare = 0.35; // selected directories are 35% of active files console.log(`model (seed 20260803): ${SLICE} slices x ${MIB} MiB = ` + `${SLICE * MIB / 1024} GiB, ${SPEED} MiB/s, full copy ${DURATION} s`); console.log(`${Object.keys(ZONE).length} zones, ${Object.keys(ITEM).length} evidence items`); const A = [-24, 8, 10, 12, 9, 9]; console.log(""); y(A, "scope", "slices", "GiB", "duration s", "items", "missed"); for (const name of Object.keys(SCOPE)) { let n = SCOPE[name].reduce((a, b) => a + ZONE[b][0], 0); if (name === "selective directory") n = Math.round(n * selectedShare); const taken = Object.keys(ITEM).filter((k) => covered(name, k)).length; y(A, name, n, (n * MIB / 1024).toFixed(1), Math.round(n * MIB / SPEED), taken, Object.keys(ITEM).length - taken); } const B = [-32, -22, 22, 17, 17, 12]; console.log(""); y(B, "item", "zone", "selective directory", "logical copy", "partition block", "full block"); for (const name of Object.keys(ITEM)) y(B, name, ITEM[name][0], ...Object.keys(SCOPE).map((k) => (covered(k, name) ? "yes" : "-"))); const missed = Object.keys(ITEM).filter((k) => !covered("partition block", k)); console.log(`\nitems missed by partition block: ${missed.join(", ")}`); console.log(`logical copy takes ${Math.round((ZONE["active-files"][0] + ZONE["filesystem-meta"][0]) * 100 / slice.length)}% of slices, takes ` + `${Object.keys(ITEM).filter((k) => covered("logical copy", k)).length} / ` + `${Object.keys(ITEM).length} items`); ``` ``` model (seed 20260803): 4096 slices x 128 MiB = 512 GiB, 97 MiB/s, full copy 5405 s 7 zones, 15 evidence items scope slices GiB duration s items missed selective directory 840 105.0 1108 4 11 logical copy 2640 330.0 3484 8 7 partition block 4040 505.0 5331 13 2 full block 4096 512.0 5405 15 0 item zone selective directory logical copy partition block full block configuration-file active-files yes yes yes yes service-definition active-files yes yes yes yes application-log active-files yes yes yes yes scheduled-task active-files yes yes yes yes user-document active-files - yes yes yes cache-file active-files - yes yes yes deleted-installer-remnant unallocated-space - - yes yes deleted-log-fragment unallocated-space - - yes yes temp-file-residue unallocated-space - - yes yes deleted-file-name filesystem-meta - yes yes yes timestamp-record filesystem-meta - yes yes yes log-entry filesystem-journal - - yes yes swap-space-residue swap-area - - yes yes off-partition-configuration off-partition-area - - - yes partition-table-record partition-table - - - yes items missed by partition block: off-partition-configuration, partition-table-record logical copy takes 64% of slices, takes 8 / 15 items ``` The scope table traces a curve, and the curve is not straight. The selective directory copy takes twenty percent of the source and brings back **four** of 15 items. The logical copy triples the area, and the item count doubles. The partition block copy adds fifty percent more area on top of the logical copy and raises the item count from 8 to **13**. The last step is the cheapest: 56 more slices, 74 more seconds, two more items. This last step is the most instructive row in the table. The partition block copy has taken 98.6 percent of the source and looks "nearly complete"; the two items it misses are the partition table record and the configuration in the off-partition area. A decision that measures scope **by ratio** treats the remaining 1.4 percent as negligible. A decision that measures scope **by item** sees that the same 1.4 percent carries two items. Evidence is counted by item, not by ratio. The second table shows the same result item by item. The deleted file name and the timestamp record are in metadata; a file copier sees these but does not read the deleted record. The deleted installer remnant, the deleted log fragment, and the temp-file residue are in unallocated space; no file copy can reach them, because the file system no longer binds these slices to a file. Counted by item, a file copy leaves **at best half, at worst three-quarters** of the evidence set outside its scope. #### Copying from a Running System ```js // forensics/live.mjs — verifiability of an offline copy versus a copy from a running system. import { source, ZONE, ITEM, SLICE, DURATION, wholeHash, itemHash } from "./disk.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const { slice, item } = source(); const start = slice.map((d) => ({ ...d })); // source state at the start of copying const r = (() => { let s = 20260804 >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; })(); // Offline copy: the write blocker does not change the source. const offline = start.map((d) => ({ ...d })); console.log(`offline copy: source hash ${wholeHash(start)}, image hash ` + `${wholeHash(offline)}, equal ${wholeHash(start) === wholeHash(offline) ? "yes" : "no"}`); // Running system: each zone changes at its own rate, copying advances slice by slice. const end = start.map((d) => ({ ...d })); const image = start.map((d) => ({ ...d })); const copyMoment = (no) => (no + 1) * DURATION / SLICE; const before = new Set(), after = new Set(); for (const [b, [n, rate]] of Object.entries(ZONE)) { const pool = start.filter((d) => d.zone === b).map((d) => d.no); for (let e = 0; e < Math.round(rate * DURATION / 1000); e += 1) { const no = pool[Math.floor(r() * n)], t = r() * DURATION; end[no].content += `|write${e}`; if (t < copyMoment(no)) { image[no].content = end[no].content; before.add(no); } else after.add(no); } } for (const no of before) after.delete(no); const A = [-22, 9, 10, 16, 17]; console.log(""); y(A, "zone", "slices", "changed", "before copy", "after copy"); for (const b of Object.keys(ZONE)) { const h = start.filter((d) => d.zone === b).map((d) => d.no); y(A, b, h.length, h.filter((n) => end[n].content !== start[n].content).length, h.filter((n) => before.has(n)).length, h.filter((n) => after.has(n)).length); } const changed = start.filter((d, i) => end[i].content !== d.content).length; y(A, "total", SLICE, changed, before.size, after.size); console.log(`\nrunning system: source at copy start ${wholeHash(start)}, source at copy ` + `end ${wholeHash(end)}, image ${wholeHash(image)}`); console.log(`all three differ; there is no single source state the image can be equated with`); const B = [-30, 16, 16, 12]; console.log(""); y(B, "item", "image=copy start", "image=copy end", "verified"); let clean = 0, neither = 0; for (const [name, k] of Object.entries(item)) { const a = itemHash(image, k) === itemHash(start, k); const b = itemHash(image, k) === itemHash(end, k); if (a && b) clean += 1; else y(B, name, a ? "yes" : "no", b ? "yes" : "no", "no"); if (!a && !b) neither += 1; } const N = Object.keys(ITEM).length; console.log(`${clean} of ${N} items are the same under both source states; ` + `${N - clean} items changed during copying, and ${neither} of those cannot be equated ` + `with either source state`); console.log(`items verified in the offline copy ${N}/${N}, in the running-system copy ${clean}/${N}`); ``` ``` offline copy: source hash 8db5b10c2d423fe9, image hash 8db5b10c2d423fe9, equal yes zone slices changed before copy after copy partition-table 8 0 0 0 filesystem-meta 240 43 4 39 active-files 2400 139 43 96 filesystem-journal 128 56 37 19 swap-area 512 54 43 11 unallocated-space 760 11 8 3 off-partition-area 48 0 0 0 total 4096 303 135 168 running system: source at copy start 8db5b10c2d423fe9, source at copy end 429ed9b9ddd32a58, image 8128d3063118f7f3 all three differ; there is no single source state the image can be equated with item image=copy start image=copy end verified scheduled-task yes no no cache-file no yes no deleted-file-name yes no no timestamp-record yes no no log-entry no no no swap-space-residue no yes no 9 of 15 items are the same under both source states; 6 items changed during copying, and 1 of those cannot be equated with either source state items verified in the offline copy 15/15, in the running-system copy 9/15 ``` The first line carries copying's evidentiary side by itself. Behind a write blocker the source does not change; after copying finishes, the source's hash is recomputed and compared with the image's. The two values are equal. What is proven is **that the image is a copy of the source**, and that is the strongest statement a hash can make. The same computation cannot be done on a running system. Copying takes 5,405 seconds, and in that time **303** of 4,096 slices change. Where the change concentrates directly reflects the zone rates: 56 of the file-system journal's 128 slices, 54 of swap space's 512 slices, only 11 of unallocated space's 760 slices. Unallocated space, the most valuable zone for evidence, is also the most stable among the data-bearing zones; the most active zones are the ones that turn over from the system's own operation. All three hashes come out different, and this result has only one reading: **there is no source state the image can be equated with.** The image is neither the source at the start of copying nor the source at the end; it carries the new content in 135 slices and the old content in 168. What results is a mixture that never existed at any single moment. The item table counts the cost of this: six of 15 items change during copying, and verified items drop from 15 to **9**. A warning is needed here. The model's "image = copy start" column **cannot be observed** in a real examination, because the source's state at the start of copying no longer exists. All that remains is the image's own hash, and that hash shows only that the image did not change afterward — not that it is identical to the source. In a live copy, the number of verified items is counted as zero, because it cannot be measured. This does not make a live copy the wrong move. On a system that cannot be shut down, it is the only option, and this lesson has already counted how many items scope brings back. What must be done is to write the condition under which the image was taken into the chain-of-custody record: the source was running, copying took this long, verification is limited to the image itself. A condition left unwritten later looks like **a gap that was never even noticed**. #### Summary - A copy's verifiability depends on whether the source's hash can be recomputed; behind a write blocker, the two hashes come out equal and 15 of 15 items are verified. - The scope curve is not straight: a selective directory brings back 4 items, a logical copy 8, a partition block 13, and a full block copy 15; the last step adds two items for 56 slices and 74 seconds. - A partition block copy that takes 98.6 percent of the source still misses two items; scope is measured by item, not by ratio. - Most evidence items are in zones that are not files: 3 in unallocated space, 2 in metadata, 1 in the file-system journal, 1 in swap space. - In a copy taken from a running system, 303 of 4,096 slices change, all three hashes come out different, and no source state remains that the image can be equated with; verified items drop from 15 to 9. #### Next Step One item never appeared in this lesson. The model's zone list has no memory, and rightly so: memory is not on disk, and no copying scope takes it. Swap-space residue carries fragments dropped out of memory, but that is memory's remnant, not memory itself — and that remnant is among the zones that change during copying. The next lesson works out this topic's most important number. Evidence items are arranged in an **order of volatility** by their lifetime, and it calculates how many items are lost when the collection order does not follow that order. The same 28 items, the same durations, three different orders: by volatility, host by host, and largest item first. The difference between them is not a matter of preference — it is directly the number of pieces of evidence lost. ### Memory Dump Analysis Source: https://academia.sh/en/courses/security-operations/memory-dump-analysis Establishing an order of volatility and counting the items a collection order costs: 28 evidence items are arranged into 10 volatility classes; with a single collector, the correct order loses 7 items, host-by-host collection loses 12, and largest-item-first collection loses 18. A three-person team following the correct order has 0 losses, a six-person team following the wrong order has 13; the wrong order is also the fastest collection to finish, because a lost item spends no time. The memory dump is relevant to 8 of 12 questions and answers 4 of them on its own. The previous lesson counted the source's seven zones and showed that most evidence items sit in zones that are not files. Memory was not on that list. Its absence is not an omission — it is the definition itself: memory is not on disk, and no copying scope reaches it. Swap-space residue carries fragments dropped out of memory, but it is only a fragment, and it is among the zones that change the most during copying. Memory opens a separate lesson in this topic because it changes the unit of measure. In disk imaging the question was "how many items does scope bring back," and scope was a decision that could be widened. In memory the question becomes "**when**," and time cannot be widened. This lesson measures that time: items are ordered by their lifetime, three different collection orders are run over the same set, and the item cost of each order is counted. The measurement is a **model**; **no memory-dump extraction tool or command is written**, what is measured is order and item count. - **FE13.** An item's **lifetime** is the time that passes before it is lost if collection has not begun. Lifetime comes from the item's class, and it is fictional. - **FE14.** The **order of volatility** is the ascending ordering of items by their lifetime: network state, process list, open handles, session state, memory content, temporary files, then disk and remote records. - **FE15.** Collection is **sequential**: a collector takes one item at a time and is occupied for that item's duration. As the team grows, the number of parallel queues grows. - **FE16.** An item is lost, and spends no time, if its lifetime runs out **before** collection begins on it. - **FE17.** Time zero is the moment the response team gains access to the hosts; detection and decision times are not counted in this lesson. - **FE18.** The item set is the same as the first lesson's set: six hosts, 28 items. #### Order of Volatility ```js // forensics/order.mjs — MODEL of order of volatility and the items lost by collection order. // No memory-extraction tool or command is written; what is measured is order and item count. // class -> [lifetime(s): time before it is lost if collection has not begun, collection duration(s)] export const CLASS = { "network-state": [120, 45], "process-list": [240, 60], "open-handle": [300, 55], "session-state": [600, 70], "memory-content": [900, 420], "temp-file": [3600, 180], "flow-record": [86400, 240], "remote-log": [2592000, 300], "disk-image": [604800, 5400], "configuration": [604800, 90], }; export const HOST = { "branch-endpoint-3": ["network-state", "process-list", "open-handle", "session-state", "memory-content", "temp-file", "disk-image", "configuration"], "field-laptop-14": ["network-state", "process-list", "session-state", "memory-content", "disk-image"], "collector-2": ["network-state", "process-list", "open-handle", "session-state", "memory-content", "disk-image", "configuration"], "portal-frontend": ["network-state", "process-list", "memory-content", "configuration"], "central-log": ["remote-log", "configuration"], "flow-collector": ["flow-record", "configuration"], }; export const SET = Object.entries(HOST).flatMap(([host, classes]) => classes.map((c) => ({ name: `${host}/${c}`, host, class: c, life: CLASS[c][0], duration: CLASS[c][1] }))); // Three orderings: by volatility, host by host (arrival order), largest item first. export const ORDER = { "by volatility": [...SET].sort((a, b) => a.life - b.life || a.duration - b.duration), "host by host": [...SET], "largest first": [...SET].sort((a, b) => b.duration - a.duration), }; // Collection: n collectors run in parallel; the next item in line goes to the first free collector. // An item is lost if collection starts on it after its lifetime has run out, and it spends no time. export function collect(order, n) { const free = Array(n).fill(0), lost = [], taken = []; for (const k of order) { const i = free.indexOf(Math.min(...free)), start = free[i]; if (start > k.life) lost.push({ ...k, start }); else { taken.push({ ...k, start }); free[i] = start + k.duration; } } return { taken, lost, duration: Math.max(...free) }; } ``` All three ordering candidates are habits actually encountered in practice. **Host-by-host** collection is the most natural: the team sits down at one machine, takes whatever is on it, then moves to the next. **Largest-first** collection looks like a precaution: if the most expensive item takes the longest, it should be started earliest. **Order of volatility**, by contrast, looks not at an item's value or size, but only at its lifetime. #### Items Lost When the Order Is Wrong ```js // forensics/measure.mjs — item-loss count for three orderings and two team sizes. import { CLASS, SET, ORDER, collect } from "./order.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const volatile = SET.filter((k) => k.life <= 3600); console.log(`model: ${SET.length} evidence items, ${Object.keys(CLASS).length} volatility classes, ` + `${volatile.length} items volatile, collecting all of them takes ${SET.reduce((a, k) => a + k.duration, 0)} s`); const A = [-16, 9, 9, 7, 14, 14, 14]; console.log(""); y(A, "class", "life s", "dur. s", "items", "1p/correct", "1p/host", "1p/largest"); const t = Object.fromEntries(Object.keys(ORDER).map((d) => [d, collect(ORDER[d], 1)])); for (const c of Object.keys(CLASS)) { const n = SET.filter((k) => k.class === c).length; y(A, c, CLASS[c][0], CLASS[c][1], n, ...Object.keys(ORDER).map((d) => `${t[d].lost.filter((k) => k.class === c).length} lost`)); } y(A, "total", "-", "-", SET.length, ...Object.keys(ORDER).map((d) => `${t[d].lost.length} lost`)); const B = [-16, 7, 9, 8, 14, 11]; console.log(""); y(B, "order", "people", "taken", "lost", "volatile lost", "end s"); for (const n of [1, 3, 6]) for (const d of Object.keys(ORDER)) { const s = collect(ORDER[d], n); y(B, d, n, s.taken.length, s.lost.length, s.lost.filter((k) => k.life <= 3600).length, s.duration); } ``` ``` model: 28 evidence items, 10 volatility classes, 18 items volatile, collecting all of them takes 19790 s class life s dur. s items 1p/correct 1p/host 1p/largest network-state 120 45 4 1 lost 3 lost 4 lost process-list 240 60 4 2 lost 3 lost 4 lost open-handle 300 55 2 1 lost 1 lost 2 lost session-state 600 70 3 0 lost 2 lost 3 lost memory-content 900 420 4 3 lost 3 lost 4 lost temp-file 3600 180 1 0 lost 0 lost 1 lost flow-record 86400 240 1 0 lost 0 lost 0 lost remote-log 2592000 300 1 0 lost 0 lost 0 lost disk-image 604800 5400 3 0 lost 0 lost 0 lost configuration 604800 90 5 0 lost 0 lost 0 lost total - - 28 7 lost 12 lost 18 lost order people taken lost volatile lost end s by volatility 1 21 7 7 18310 host by host 1 16 12 12 18020 largest first 1 10 18 18 17190 by volatility 3 28 0 0 6770 host by host 3 19 9 9 6640 largest first 3 10 18 18 5760 by volatility 6 28 0 0 5925 host by host 6 25 3 3 5705 largest first 6 15 13 13 5400 ``` The first table is this lesson's core, and its last row carries three numbers: **7, 12, 18**. The same 28 items, the same durations, the same single collector; only the order changes. In the order of volatility, seven items are lost; in host-by-host collection, twelve; in largest-item-first collection, eighteen. The last number is the entire volatile set: by the time the first 5,400-second disk image finishes, no item with a lifetime under an hour is still standing. The seven losses under the correct order also deserve a separate reading. Even though the order is the best one, one of four network-state items, two of four process-list items, and three of four memory-content items are lost. The reason is the single-person team: taking a network state takes 45 seconds, and the fourth of four hosts comes up in the queue at the 135th second, while its lifetime is 120. **The correct order minimizes loss, it does not zero it out**; what zeroes out loss is the number of parallel queues. The second table separates these two factors, and the comparison is not one-directional. When a three-person team works in the correct order, loss is **zero** — three people are enough for this fictional set. When a six-person team works in the wrong order, loss is **thirteen**. Twice the people does not fix the wrong order; order is more decisive than team size, because order is what consumes lifetime, and headcount only changes the rate of consumption. The last column is the most deceptive one. The collection that finishes fastest is the one that loses the most items: with a single-person team, "largest first" finishes at 17,190 seconds while the correct order takes 18,310 seconds. The difference is not an efficiency gain, it is an accounting illusion — **a lost item spends no time.** A collection finishing early tells you something only when read together with the number of items actually taken. #### Time to Reach a Host ```js // forensics/arrival.mjs — effect of host-access delay on loss under the correct order. import { SET, ORDER } from "./order.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const REMOTE = ["branch-endpoint-3", "field-laptop-14"]; // the two field hosts the team is traveling to // An item must be started after arriving at the host and before its lifetime runs out. function collect(order, n, delay) { const free = Array(n).fill(0), lost = []; for (const k of order) { const arrival = REMOTE.includes(k.host) ? delay : 0; const i = free.indexOf(Math.min(...free)), start = Math.max(free[i], arrival); if (start > k.life) lost.push(k); else free[i] = start + k.duration; } return lost; } const A = [-12, 9, 9, 12, 14]; y(A, "delay s", "people", "lost", "remote host", "central host"); for (const g of [0, 600, 1200, 2400]) for (const n of [3, 6]) { const k = collect(ORDER["by volatility"], n, g); y(A, g, n, k.length, k.filter((x) => REMOTE.includes(x.host)).length, k.filter((x) => !REMOTE.includes(x.host)).length); } const remoteVolatile = SET.filter((k) => REMOTE.includes(k.host) && k.life <= 3600).length; console.log(`\nvolatile items at the two remote hosts: ${remoteVolatile}; at a 2400-second delay, ` + `team size no longer changes the loss`); ``` ``` delay s people lost remote host central host 0 3 0 0 0 0 6 0 0 0 600 3 6 5 1 600 6 5 5 0 1200 3 9 9 0 1200 6 9 9 0 2400 3 9 9 0 2400 6 9 9 0 volatile items at the two remote hosts: 10; at a 2400-second delay, team size no longer changes the loss ``` The seventeenth claim defined time zero as "the moment the team gains access to the hosts" and used the same moment for every host. Two hosts are in the field; the team is traveling to them. A six-hundred-second trip raises loss from zero to **six** for a three-person team working in the correct order. At twelve hundred seconds, loss rises to **nine** and stays there: nine of the ten volatile items at the two field hosts run out their lifetime before the team arrives, and the only item still standing is the temp file, whose lifetime is 3,600 seconds. Two rows of this table also deserve a separate reading. At six hundred seconds, a three-person team loses six items and a six-person team loses five; the one item that differs is a central one, and it was left waiting in the queue of a collector held up by a remote host. After twelve hundred seconds, team size changes nothing, because the lost items were lost before anyone arrived. **Past a certain point, delay renders team size meaningless.** The first lesson calculated that shortening the chain reduces the expected open item count from 7.57 to 4.16, and noted that its cost was "the time it takes a qualified person to travel to the field." This table is that cost's number: a twelve-hundred-second trip is worth nine items. #### Questions Answerable Only from Memory ```js // forensics/source.mjs — which question is answered by which source. A MODEL. const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); // question -> [memory dump, disk image, remote log] 1: answers, 0: does not answer const QUESTION = { "which processes were running": [1, 0, 0], "what was the process's command line": [1, 0, 1], "which connections were active": [1, 0, 1], "content decrypted in memory": [1, 0, 0], "active session token": [1, 0, 0], "loaded kernel module": [1, 1, 0], "configuration in effect": [1, 0, 0], "configuration on disk": [0, 1, 0], "when did the file change": [0, 1, 0], "content of the deleted file": [0, 1, 0], "who logged in": [1, 1, 1], "who accessed which record": [0, 0, 1], }; const SOURCE = ["memory dump", "disk image", "remote log"]; const A = [-37, 13, 12, 12, 9]; y(A, "question", ...SOURCE, "sources"); for (const [s, v] of Object.entries(QUESTION)) y(A, s, ...v.map((x) => (x ? "yes" : "-")), v.reduce((a, x) => a + x, 0)); const only = SOURCE.map((_, i) => Object.values(QUESTION).filter((v) => v[i] === 1 && v.reduce((a, x) => a + x, 0) === 1).length); console.log(""); SOURCE.forEach((k, i) => console.log(`${k}: touches ` + `${Object.values(QUESTION).filter((v) => v[i]).length} questions, answers ${only[i]} on its own`)); console.log(`questions answered by two or three sources: ` + `${Object.values(QUESTION).filter((v) => v.reduce((a, x) => a + x, 0) > 1).length} / ` + `${Object.keys(QUESTION).length}`); ``` ``` question memory dump disk image remote log sources which processes were running yes - - 1 what was the process's command line yes - yes 2 which connections were active yes - yes 2 content decrypted in memory yes - - 1 active session token yes - - 1 loaded kernel module yes yes - 2 configuration in effect yes - - 1 configuration on disk - yes - 1 when did the file change - yes - 1 content of the deleted file - yes - 1 who logged in yes yes yes 3 who accessed which record - - yes 1 memory dump: touches 8 questions, answers 4 on its own disk image: touches 5 questions, answers 3 on its own remote log: touches 4 questions, answers 1 on its own questions answered by two or three sources: 4 / 12 ``` Only four of twelve questions are answered by more than one source. The remaining eight each have a single source, and if that source is not collected, the question **stays unanswered** — it cannot be inferred from another item. The memory dump is relevant to eight questions and answers four of them on its own; those four are also questions that only have an answer on a running system. There are two rows in the table that get mixed up with each other: **configuration in effect** and **configuration on disk**. They are separate questions and are answered by separate sources. The setting a process has read into memory differs from the disk file if that file has since been changed; which one held during an incident is exactly what is being asked. The disk image cannot answer this question, only the second one. A line in a report that says "the configuration was such-and-such" is an unmeasurable claim if it does not say which question it is the answer to. #### Summary - The order of volatility is the ascending ordering of items by their lifetime; in the fictional set, 18 of 28 items are volatile, and collecting all of them takes 19,790 seconds. - On the same set with a single collector, the correct order loses 7 items, host-by-host collection loses 12, and largest-item-first collection loses 18; the last figure is the entire volatile set. - The correct order minimizes loss, it does not zero it out: a 1,200-second arrival delay to the two field hosts raises loss from 0 to 9 for a team working in the correct order, and past that point, team size no longer changes the result. - Three people in the correct order produce 0 losses; six people in the wrong order produce 13 losses; order is more decisive than team size. - The collection that finishes fastest is the one that loses the most items, because a lost item spends no time; finish time only means something when read together with the number of items taken. - The memory dump is relevant to 8 of 12 questions and answers 4 of them on its own; only 4 of the questions can be answered by more than one source. #### Next Step This lesson ordered items by **when** they are collected. Once collected, each item carries its own timestamps: moments of file change, moments of login, moments of connection in the flow record. The next step of examination is to merge these timestamps onto a single timeline, and a silent assumption sits there — that every source's clock is the same. The next lesson tests that assumption. Events from three sources are merged into a single timeline, each source's clock skew is measured, and two numbers are worked out: how many events land in the wrong order, and how many events change position once the skew is corrected. A broken ordering is not a delay, it is a **cause-and-effect** error: two events read in the wrong order look like two events that caused each other. ### Timeline Creation Source: https://academia.sh/en/courses/security-operations/timeline-creation Merging the events of three sources onto a single timeline, and how clock skew breaks the ordering: raw merging of 54 events forming 1,431 pairs produces 35 inverted pairs, 33 of the 54 events land in the wrong position, and the largest skew is 243 seconds. Once skew is estimated and corrected using nineteen anchor events, inverted pairs drop to 1, wrong position drops to 2, and the correction changes the position of 34 events. All 35 of the 35 inverted pairs come from two different sources; skew never breaks a source's own internal order. The previous lesson measured **when items are collected**. Each collected item carries its own timestamps: moments of file change, moments of login, moments of connection in the flow record. The next step of examination is to merge these timestamps onto a single line; the result is called a **timeline**, and it is the backbone of the report. What a timeline carries is a claim of order. The sentence "the configuration changed first, then the external connection was established" is not a piece of time information, it is a **cause-and-effect** proposition; reversed, it says the opposite. The silent assumption the order rests on is that every source reads the same clock. This lesson tests that assumption: the events of three sources are merged, the number of events that land in the wrong order is counted, skew is estimated and corrected, and how many events the correction moves are measured. The events and clocks are **fictional**. - **FE19.** A **timeline** is the ordering of events from different sources onto a single axis. The information a timeline carries is not the time values, it is the **order**. - **FE20.** Each source's clock carries its own **skew**: a fixed offset and a drift that accumulates per second. Both are fictional. - **FE21.** An **anchor event** is the same event recorded by two sources at once. Skew is estimated only from anchor events. - **FE22.** The central log is the reference clock. Ordering is a relative question; absolute accuracy is not required, aligning the sources to one another is enough. - **FE23.** An **inverted pair** is a pair of events where one actually happened first, but the timeline shows the other one first. This is the measure of ordering error. - **FE24.** The true times are known in this model. They are not known in a real examination; the only foothold there is anchor events. #### Three Clocks, One Timeline ```js // forensics/time.mjs — MODEL of three sources' fictional event records and clock skew. // No real organization, address, or event name is written; the clocks are fictional. export const SEED = 20260805, N = 54, WINDOW = 7200; // a two-hour window const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; // source -> [fixed offset(s), drift(s/s)] the central log is the reference clock export const SOURCE = { "central-log": [0, 0], "endpoint-record": [-214, -0.0042], "flow-collector": [96, 0.0013], }; export const TYPE = ["login", "configuration-change", "external-connection", "process-start", "file-write", "privilege-escalation"]; // Reported time: the value read on the source's own clock. export const report = (k, t) => Math.round(t + SOURCE[k][0] + SOURCE[k][1] * t); // Events are generated with their true times; anchor events are recorded by two sources at once. export function events() { const r = generator(SEED), names = Object.keys(SOURCE), list = []; for (let i = 0; i < N; i += 1) { const t = Math.floor(r() * WINDOW), k = names[Math.floor(r() * 3)]; list.push({ no: i, t, source: k, type: TYPE[Math.floor(r() * TYPE.length)], anchor: k !== "central-log" && r() < 0.3 }); } list.sort((a, b) => a.t - b.t || a.no - b.no); const record = []; for (const o of list) { record.push({ ...o, reported: report(o.source, o.t) }); if (o.anchor) record.push({ ...o, source: "central-log", secondary: true, reported: report("central-log", o.t) }); } return record; } // Inverted pair: event pairs where a truly came first but b appears first on the timeline. export function invertedPairs(list) { let n = 0; for (let i = 0; i < list.length; i += 1) for (let j = i + 1; j < list.length; j += 1) if (list[i].t > list[j].t) n += 1; return n; } export const orderBy = (record, time) => [...record].sort((a, b) => time(a) - time(b) || a.no - b.no); ``` Skew consists of two components, and the two behave differently. **Fixed offset** is how many seconds a source's clock has been ahead or behind from the start; it shifts every record by the same amount. **Drift**, on the other hand, is the clock's accumulating error; the amount of shift grows over time. A fixed offset is corrected with a single number, drift cannot be corrected — it can only be measured and written down as residual error. #### Raw Timeline and Anchor-Based Correction ```js // forensics/timeline.mjs — raw merge, anchor-based correction, and remaining inverted pairs. import { SOURCE, N, events, invertedPairs, orderBy } from "./time.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const E = events(); const primary = E.filter((k) => !k.secondary); const anchors = E.filter((k) => k.anchor); const A = [-18, 8, 8, 15, 16]; y(A, "source", "records", "anchors", "true offset", "anchor estimate"); const estimate = {}; for (const k of Object.keys(SOURCE)) { const c = anchors.filter((x) => x.source === k && !x.secondary); const d = c.map((x) => x.reported - anchors.find((z) => z.no === x.no && z.secondary).reported); estimate[k] = d.length ? Math.round(d.reduce((a, b) => a + b, 0) / d.length) : 0; y(A, k, primary.filter((x) => x.source === k).length, c.length, SOURCE[k][0], estimate[k]); } const raw = orderBy(primary, (o) => o.reported); const fixed = orderBy(primary, (o) => o.reported - estimate[o.source]); const truth = orderBy(primary, (o) => o.t); const moved = (a, b) => a.filter((o, i) => b[i].no !== o.no).length; console.log(""); const B = [-24, 16, 16, 16]; y(B, "timeline", "inverted pairs", "wrong position", "largest skew"); const pairs = (N * (N - 1)) / 2; for (const [name, d, z] of [["raw merge", raw, (o) => o.reported], ["anchor-corrected", fixed, (o) => o.reported - estimate[o.source]], ["true time", truth, (o) => o.t]]) y(B, name, invertedPairs(d), moved(d, truth), Math.max(...d.map((o) => Math.abs(z(o) - o.t)))); console.log(`${N} events, ${pairs} pairs, ${anchors.length / 2} anchor events`); console.log(`inverted-pair rate in the raw timeline ${(invertedPairs(raw) * 100 / pairs).toFixed(1)}%, ` + `${(invertedPairs(fixed) * 100 / pairs).toFixed(1)}% after correction`); console.log(`the correction changes the timeline position of ${moved(raw, fixed)} events`); ``` ``` source records anchors true offset anchor estimate central-log 13 0 0 0 endpoint-record 25 12 -214 -230 flow-collector 16 7 96 100 timeline inverted pairs wrong position largest skew raw merge 35 33 243 anchor-corrected 1 2 15 true time 0 0 0 54 events, 1431 pairs, 19 anchor events inverted-pair rate in the raw timeline 2.4%, 0.1% after correction the correction changes the timeline position of 34 events ``` The gap between these two numbers is this lesson's first result. In the raw timeline, the inverted-pair rate is **2.4** percent and looks small; in the same timeline, **33** of 54 events are in the wrong position, that is, 61 percent of the events. These two ratios do not contradict each other: a skew of a few hundred seconds shifts most events by a few positions, but it only inverts pairs whose time gap is smaller than the skew. A reading that looks at the timeline and says "the events are roughly in the right order" rests on the 97.6 percent rate; the sentences actually used in a report, however, are built about events that are close to one another. Anchor-based correction brings the inverted-pair count down from 35 to **1** and the wrong-position count down from 33 to **2**. The one remaining inverted pair is a residual, and its cause is visible in the table: the endpoint record's true offset is −214, while the anchor estimate is −230. The sixteen-second bias comes from drift. A correction made with a single fixed number is accurate at the middle of the window and wrong at both ends; the largest remaining skew is 15 seconds. This number should be written in the report as the **resolution**: the order of two events closer than fifteen seconds cannot be proven with this timeline. The last line gives the scale of the correction: the position of **34 events** on the timeline changes. A narrative written before the correction would be a different text at thirty-four points after it. #### Anchor Count and the Shape of the Correction ```js // forensics/anchor.mjs — effect of anchor count and correction shape on the remaining error. import { SOURCE, events, invertedPairs, orderBy } from "./time.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const E = events(), primary = E.filter((o) => !o.secondary); // Anchor pair: (time read at reference source, difference between the two records). const pair = (k) => E.filter((o) => o.anchor && o.source === k && !o.secondary).map((o) => { const m = E.find((z) => z.no === o.no && z.secondary); return [m.reported, o.reported - m.reported]; }); const flat = (c) => [Math.round(c.reduce((a, [, d]) => a + d, 0) / c.length), 0]; // Linear fit: difference = offset + drift * time. function linear(c) { const n = c.length, st = c.reduce((a, [t]) => a + t, 0), sd = c.reduce((a, [, d]) => a + d, 0); const stt = c.reduce((a, [t]) => a + t * t, 0), std = c.reduce((a, [t, d]) => a + t * d, 0); const b = (n * std - st * sd) / (n * stt - st * st); return [(sd - b * st) / n, b]; } const back = ([a, b]) => (o) => (o.reported - a) / (1 + b); const A = [-8, 15, 12, 14, 14, 13]; y(A, "anchors", "flat estimate", "flat inv.", "flat skew", "linear inv.", "lin. skew"); for (const n of [2, 4, 8, 12]) { const p = {}, q = {}; for (const k of Object.keys(SOURCE)) { const c = pair(k).slice(0, n); p[k] = c.length ? flat(c) : [0, 0]; q[k] = c.length > 1 ? linear(c) : [0, 0]; } const d1 = orderBy(primary, (o) => back(p[o.source])(o)); const d2 = orderBy(primary, (o) => back(q[o.source])(o)); const skew = (m) => Math.round(Math.max(...primary.map((o) => Math.abs(back(m[o.source])(o) - o.t)))); y(A, n, p["endpoint-record"][0], invertedPairs(d1), skew(p), invertedPairs(d2), skew(q)); } console.log(`\ntrue offset ${SOURCE["endpoint-record"][0]} s, true drift ` + `${SOURCE["endpoint-record"][1]} s/s; endpoint-record has ${pair("endpoint-record").length} ` + `anchors, flow-collector has ${pair("flow-collector").length}`); ``` ``` anchors flat estimate flat inv. flat skew linear inv. lin. skew 2 -218 0 25 0 4 4 -221 0 22 0 1 8 -226 1 17 0 1 12 -230 1 15 0 1 true offset -214 s, true drift -0.0042 s/s; endpoint-record has 12 anchors, flow-collector has 7 ``` The second column goes the opposite way from what is expected. As the anchor count rises from two to twelve, the flat estimate shifts from −218 to −230, meaning it **moves away** from the true offset of −214. This is not a computation error: the flat estimate is the average of the whole window, and drift accumulates across the window. More anchors do not make a wrong model's estimate correct; they only make it wrong in a more stable way. The remaining skew shows the same thing — it falls from 25 seconds to 15 seconds and stays there. The two-coefficient correction gives a different result with the same anchor set. When the difference is fit as a linear function of time, the remaining skew falls to 4 seconds with even two anchors, and to 1 second after four anchors. The difference lies not in the amount of data, but in **the shape of the model**: correcting a clock that has drift with a single fixed number is representing a sloped line with a horizontal one. The inverted-pair columns carry no decision at this scale, and that has to be written down. In a number that has fallen from thirty-five to between zero and one, the difference between 0 and 1 is just which side a single pair falls on; the real information is in the remaining-skew column. A timeline is reported not as "no inverted pairs remain," but as "the order of events closer than fifteen seconds cannot be proven." The linear correction has one condition, and it sits where the table does not read: the anchor events must be **spread** across the window. If they are all clustered at one end of the window, the slope estimate becomes unstable, and the correction behaves worse than a flat correction at the other end. What is decisive is not the anchor count, but the anchors' distribution. #### Pairs That Invert ```js // forensics/pairs.mjs — pairs that invert in the raw timeline and the false inference they cause. import { events, orderBy } from "./time.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const primary = events().filter((k) => !k.secondary); const raw = orderBy(primary, (o) => o.reported); const inverted = []; for (let i = 0; i < raw.length; i += 1) for (let j = i + 1; j < raw.length; j += 1) if (raw[i].t > raw[j].t) inverted.push([raw[i], raw[j], raw[i].t - raw[j].t]); inverted.sort((a, b) => b[2] - a[2]); const CODE = { "central-log": "C", "endpoint-record": "E", "flow-collector": "F" }; const A = [-28, -28, 11, 13]; y(A, "appears first on timeline", "appears second on timeline", "raw gap", "true gap"); for (const [a, b] of inverted.slice(0, 5)) y(A, `${CODE[a.source]}/${a.type}`, `${CODE[b.source]}/${b.type}`, `${b.reported - a.reported} s`, `${b.t - a.t} s`); const different = inverted.filter(([a, b]) => a.source !== b.source).length; console.log(`\n${different} of ${inverted.length} inverted pairs come from two different sources, ` + `${inverted.length - different} from the same source`); console.log(`the largest inversion hides a true gap of ${inverted[0][2]} seconds`); ``` ``` appears first on timeline appears second on timeline raw gap true gap E/external-connection F/external-connection 18 s -306 s E/external-connection C/configuration-change 9 s -224 s E/configuration-change F/process-start 141 s -200 s E/process-start F/configuration-change 145 s -191 s E/external-connection F/login 156 s -179 s 35 of 35 inverted pairs come from two different sources, 0 from the same source the largest inversion hides a true gap of 306 seconds ``` The sentence before the last line gives this lesson's most useful rule: **35 of 35 inverted pairs come from two different sources, none of them from within the same source.** The reason is structural. All of a single source's records are read from the same clock; however wrong that clock is, it shifts its own records by the same amount and preserves their internal order. Order breaks down only **between** sources. This gives a distinction directly applicable to an examination. The order within a single source is independent of skew and needs no additional foundation. The order between two events from two sources, however, is a **claim**, and its foundation is only established once skew is measured. If a report writes these two with the same sentence pattern, the uncertainty the second one carries becomes invisible. The first row of the table is the concrete counterpart of this. The endpoint's external-connection record appears 18 seconds before the flow collector's external-connection record on the raw timeline; in truth it comes 306 seconds **after** it. An examination reading these two rows side by side would conclude that the endpoint connection caused the flow record — but the order is exactly reversed, and the two records are separated by an interval longer than five minutes. The error skew produces is not a delay, it is a **direction** error. #### Summary - The information a timeline carries is not time values, it is order; order builds a cause-and-effect proposition, and reversed, it says the opposite. - In the raw merge, 35 of 1,431 pairs turn out inverted (2.4 percent), but 33 of 54 events are in the wrong position; a low pair rate does not mean the timeline is correct. - The correction estimated from nineteen anchor events brings inverted pairs down to 1 and wrong position down to 2, and changes the timeline position of 34 events. - A fixed-offset estimate cannot correct drift: the true offset is −214 while the estimate is −230, and the largest remaining skew is 15 seconds; this number is the timeline's resolution. - Increasing the anchor count does not bring the flat estimate closer to the true offset (it goes from −218 to −230); a correction fit as a linear function of time brings the remaining skew down to 4 seconds with just two anchors. What is decisive is not the anchor count, but the model's shape and the anchors' spread across the window. - All of the inverted pairs come from two different sources; the order within a single source is unaffected by skew, while the order between sources cannot be claimed without measurement. #### Next Step In this lesson, events arrived as ready-made **records**: each one had a type, a source, and a time. In a real examination, part of the record set is not ready-made in this way; it is drawn out of structures the system keeps for its own operation. An entry in a configuration store, a filesystem metadata record, a shortcut, a cache entry — none of these was written to be an "event log," but each leaves a trace that can be ordered. The next lesson treats these structures in terms of their **evidentiary value**. Each structure answers a question, but which question it answers varies from structure to structure: some show directly that something happened, some only carry an inference, and some also arise from the machine's ordinary use. Any sentence written without counting this last category puts baseline noise into the report as a finding. ### Structure and Record Examination Source: https://academia.sh/en/courses/security-operations/structure-and-record-examination Measuring the evidentiary value of traces left by system structures: of 12 structures, 7 are in the direct class and 5 are in the inference class, and their bases range from 4 to 187 in a 268-machine fictional fleet. Of seven claims, 2 are distinctive, 4 are weak, and 1 is noise; the claim resting on a single structure is also true on 187 clean machines. A claim with three bases turns out weaker than a claim with two bases when all three bases are in the inference class, and the independence assumption makes the calculation a lower bound. The previous lesson took events as ready-made **records**: each one had a type, a source, and a time. A significant part of the trace collected in an examination does not arrive ready-made like that. The system writes to structures it keeps for its own operation — a configuration store entry, a file system metadata record, a shortcut, a cache entry. None of these was written to serve as a log, but each leaves a trace that can be ordered and read. This lesson treats those traces in terms of **evidentiary value**, and the unit of measure is a ratio: the number of clean machines carrying the same trace. The rarer a trace is, the more it distinguishes a claim; if a trace is already present on most machines, it does not distinguish the claim at all. Most sentences that go into a report sit somewhere between these two, and where they sit cannot be read unless it is written down. The fleet, the base numbers, and the claim list are **fictional**. - **FE25.** A **system structure** is a data structure that is not kept as an event record but leaves a trace because the system's operation requires it. - **FE26.** A structure is in the **direct** class only if it comes into being solely because the claimed event occurred. If it can also come into being for other reasons, it is in the **inference** class. - **FE27.** The **base** is the number of clean machines carrying the same trace. The fictional fleet is 268 machines; the base values are hand-assigned, and no real fleet was scanned. - **FE28.** A claim's **distinctiveness** is the expected number of clean machines carrying the same combination of traces. Below one is distinctive, between one and ten is weak, above ten is noise. - **FE29.** The expected number is calculated under the assumption that the traces are **independent** of one another. This assumption does not actually hold; the calculation is therefore a **lower bound**. - **FE30.** That the structures are present on the event machine is a given. This lesson measures not the finding, but the value of what is found. #### Structure, Question, and Base ```js // forensics/structure.mjs — traces from system structures and the fleet base MODEL. // Base numbers are hand-assigned fictional values; no real fleet was scanned. export const FLEET = 268; // number of machines counted as clean // structure -> [class, machines carrying the same trace in the clean fleet, question it answers] export const STRUCTURE = { "deleted-file-name": ["direct", 4, "the file existed on this machine"], "recycle-bin-metadata": ["direct", 47, "the file was deleted"], "application-cache": ["direct", 9, "the binary was executed"], "scheduled-task-definition": ["direct", 6, "the item runs automatically"], "configuration-store-entry": ["direct", 12, "the item runs at startup"], "removable-media-record": ["direct", 22, "the media was connected to this machine"], "installation-record": ["direct", 63, "the software was installed"], "shortcut-record": ["inference", 31, "the file was accessed"], "recent-items-list": ["inference", 58, "the file was listed"], "file-timestamp": ["inference", 96, "the file changed within the window"], "share-directory-entry": ["inference", 74, "a network share was connected"], "user-profile-stamp": ["inference", 187, "the user logged in"], }; // Claim -> structures it rests on. Claims are sentences intended for the report. export const CLAIM = { "the file existed on the machine and was deleted": ["deleted-file-name", "recycle-bin-metadata"], "the item runs on every startup": ["configuration-store-entry", "scheduled-task-definition"], "the binary was executed on this machine": ["application-cache", "shortcut-record"], "data was copied to removable media": ["removable-media-record", "file-timestamp", "recent-items-list"], "the software was installed on the machine": ["installation-record", "application-cache"], "a user opened the file": ["recent-items-list", "shortcut-record", "user-profile-stamp"], "this user was logged in at that time": ["user-profile-stamp"], }; // Expected clean-machine count carrying the same trace combination (traces assumed independent). export const expected = (basis) => FLEET * basis.reduce((a, s) => a * STRUCTURE[s][1] / FLEET, 1); export const directCount = (basis) => basis.filter((s) => STRUCTURE[s][0] === "direct").length; export const classify = (b) => (b < 1 ? "distinctive" : b < 10 ? "weak" : "noise"); ``` The third field in the table carries the **question each structure answers**, and this is the field most often confused when writing a claim. The deleted-file-name record answers "the file existed on this machine," not "who brought the file here." Recycle-bin metadata says "the file was deleted," not "the file was hidden." The application cache says "the binary was executed," not "the binary did such-and-such." Any sentence that goes beyond the question a structure answers is an extension with no basis, and it cannot be written into any column of the table. The difference between the two classes looks at **when the structure comes into being**. An application cache entry does not come into being unless the binary was executed; the entry's presence directly shows the execution. A file timestamp, on the other hand, changes when the file is modified, when it is copied, and when it is restored from backup; the timestamp falling within the event window is an inference. The same distinction applies to the shortcut record: a shortcut can come into being both when a user opens the file and when a script touches it. #### A Claim's Distinctiveness ```js // forensics/value.mjs — base per structure, distinctiveness per claim, and dependence on a single structure. import { FLEET, STRUCTURE, CLAIM, expected, directCount, classify } from "./structure.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const structures = Object.keys(STRUCTURE), claims = Object.keys(CLAIM); console.log(`model: ${structures.length} structures, ${claims.length} claims, a ${FLEET}-machine ` + `clean fleet; ${structures.filter((k) => STRUCTURE[k][0] === "direct").length} structures direct, ` + `${structures.filter((k) => STRUCTURE[k][0] === "inference").length} structures inference`); const A = [-28, -10, 8, 7, 16]; console.log(""); y(A, "structure", "class", "base", "share", "claims"); for (const k of structures) y(A, k, STRUCTURE[k][0], STRUCTURE[k][1], `${Math.round(STRUCTURE[k][1] * 100 / FLEET)}%`, claims.filter((i) => CLAIM[i].includes(k)).length); const B = [-49, 8, 10, 15, 14]; console.log(""); y(B, "claim", "basis", "direct", "clean machine", "result"); for (const i of claims) { const b = expected(CLAIM[i]); y(B, i, CLAIM[i].length, directCount(CLAIM[i]), b.toFixed(2), classify(b)); } const unsupported = claims.filter((i) => directCount(CLAIM[i]) === 0); console.log(`\nclaims without a direct basis: ${unsupported.length} (${unsupported.join("; ")})`); console.log(`distinctive ${claims.filter((i) => expected(CLAIM[i]) < 1).length}, ` + `weak ${claims.filter((i) => classify(expected(CLAIM[i])) === "weak").length}, ` + `noise ${claims.filter((i) => classify(expected(CLAIM[i])) === "noise").length}`); // Dependence on a single structure: which claim changes class if a structure is removed. console.log(""); const C = [-28, 12, 19, 14]; y(C, "if structure removed", "affected", "left unsupported", "class dropped"); for (const k of structures) { const affected = claims.filter((i) => CLAIM[i].includes(k)); if (!affected.length) continue; const orphaned = affected.filter((i) => CLAIM[i].length === 1); const dropped = affected.filter((i) => CLAIM[i].length > 1 && classify(expected(CLAIM[i].filter((z) => z !== k))) !== classify(expected(CLAIM[i]))); y(C, k, affected.length, orphaned.length, dropped.length); } ``` ``` model: 12 structures, 7 claims, a 268-machine clean fleet; 7 structures direct, 5 structures inference structure class base share claims deleted-file-name direct 4 1% 1 recycle-bin-metadata direct 47 18% 1 application-cache direct 9 3% 2 scheduled-task-definition direct 6 2% 1 configuration-store-entry direct 12 4% 1 removable-media-record direct 22 8% 1 installation-record direct 63 24% 1 shortcut-record inference 31 12% 2 recent-items-list inference 58 22% 2 file-timestamp inference 96 36% 1 share-directory-entry inference 74 28% 0 user-profile-stamp inference 187 70% 2 claim basis direct clean machine result the file existed on the machine and was deleted 2 2 0.70 distinctive the item runs on every startup 2 2 0.27 distinctive the binary was executed on this machine 2 1 1.04 weak data was copied to removable media 3 1 1.71 weak the software was installed on the machine 2 2 2.12 weak a user opened the file 3 0 4.68 weak this user was logged in at that time 1 0 187.00 noise claims without a direct basis: 2 (a user opened the file; this user was logged in at that time) distinctive 2, weak 4, noise 1 if structure removed affected left unsupported class dropped deleted-file-name 1 0 1 recycle-bin-metadata 1 0 1 application-cache 2 0 2 scheduled-task-definition 1 0 1 configuration-store-entry 1 0 1 removable-media-record 1 0 1 installation-record 1 0 0 shortcut-record 2 0 1 recent-items-list 2 0 1 file-timestamp 1 0 0 user-profile-stamp 2 1 0 ``` The last column ranges from 0.27 to 187, and this range is the lesson's result. The sentence at the bottom — "this user was logged in at that time" — rests on a single structure, and that structure's base is **187** of the 268 machines. The sentence is very probably true and distinguishes nothing: seven out of every ten machines in the fleet carry the same trace. A claim being true and a claim showing something are separate qualities, and in a report only the second one does any work. The fourth and sixth rows should be read together. "Data was copied to removable media" rests on three bases and its expected number is 1.71; "a user opened the file" also rests on three bases and its expected number is 4.68. What makes the difference is not the number of bases but their **class**: in the first, the removable-media record is in the direct class and its base is 8 percent; in the second, all three bases are inference, and the weakest one has a base of 70 percent. **Adding a basis does not increase distinctiveness; adding a rare basis does.** The independence assumption must also be stated here. The product accepts that the traces come into being independently of each other. In reality, the shortcut record and the recent-items list are most often born of the same action; if one is present, the probability that the other is also present is higher than the product gives. That is, the real number is **larger** than the calculated one, and the 4.68 read from the table is a lower bound. The independence assumption always makes a claim look stronger than it is; an analysis that does not state this is biased in its own favor. The third table counts the cost of losing a structure. The application cache is a basis for two claims, and its loss **downgrades the class of both claims**; it is the single most expensive structure. The user profile stamp is also a basis for two claims, but its loss downgrades neither class — one is already noise, and the other stays in the same class without it. Two structures can be a basis for the same number of claims, and the weight they carry differs by exactly the difference in their bases. The quietest row in the table is the **share directory entry**: it has been found, it has been classified, and it is a basis for no claim at all. The set of traces collected and the set of traces used are not the same set, and that difference is the source of the gap between a report's length and its value. #### A Structure's Lifespan and the Cost of Delay ```js // forensics/lifespan.mjs — the time before a structure is overwritten, and the effect of delay on claims. import { FLEET, STRUCTURE, CLAIM, expected, classify } from "./structure.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); // structure -> [number of entries it holds, entries added per day under ordinary use] 0 = never overwritten const CYCLE = { "deleted-file-name": [2400, 180], "recycle-bin-metadata": [500, 12], "application-cache": [1024, 96], "scheduled-task-definition": [1, 0], "configuration-store-entry": [1, 0], "removable-media-record": [64, 3], "installation-record": [1, 0], "shortcut-record": [149, 22], "recent-items-list": [20, 14], "file-timestamp": [1, 0.35], "share-directory-entry": [32, 9], "user-profile-stamp": [1, 1], }; const lifespan = (k) => (CYCLE[k][1] ? CYCLE[k][0] / CYCLE[k][1] : Infinity); const readable = (k, day) => day <= lifespan(k); const A = [-28, 10, 10, 12, 10]; y(A, "structure", "entries", "daily", "life days", "base"); for (const k of Object.keys(STRUCTURE)) y(A, k, CYCLE[k][0], CYCLE[k][1], CYCLE[k][1] ? lifespan(k).toFixed(1) : "continuous", STRUCTURE[k][1]); const B = [-10, 14, 14, 8, 10, 14]; console.log(""); y(B, "delay", "readable", "distinctive", "weak", "noise", "unsupported"); for (const day of [1, 3, 7, 14, 30, 90]) { const remaining = Object.fromEntries(Object.keys(CLAIM).map((i) => [i, CLAIM[i].filter((k) => readable(k, day))])); const count = (s) => Object.values(remaining).filter((d) => d.length && classify(expected(d)) === s).length; y(B, `${day} day${day === 1 ? "" : "s"}`, Object.keys(STRUCTURE).filter((k) => readable(k, day)).length, count("distinctive"), count("weak"), count("noise"), Object.values(remaining).filter((d) => !d.length).length); } const permanent = Object.keys(STRUCTURE).filter((k) => lifespan(k) === Infinity); console.log(`\n${permanent.length} of the ${Object.keys(STRUCTURE).length} structures are never overwritten; ` + `fleet ${FLEET} machines, base values do not change`); ``` ``` structure entries daily life days base deleted-file-name 2400 180 13.3 4 recycle-bin-metadata 500 12 41.7 47 application-cache 1024 96 10.7 9 scheduled-task-definition 1 0 continuous 6 configuration-store-entry 1 0 continuous 12 removable-media-record 64 3 21.3 22 installation-record 1 0 continuous 63 shortcut-record 149 22 6.8 31 recent-items-list 20 14 1.4 58 file-timestamp 1 0.35 2.9 96 share-directory-entry 32 9 3.6 74 user-profile-stamp 1 1 1.0 187 delay readable distinctive weak noise unsupported 1 day 12 2 4 1 0 3 days 9 2 2 2 1 7 days 7 2 2 1 2 14 days 5 1 0 3 3 30 days 4 1 0 2 4 90 days 3 1 0 1 5 3 of the 12 structures are never overwritten; fleet 268 machines, base values do not change ``` Most structures are **cyclical**: they hold a fixed number of entries and drop the oldest one when a new one arrives. Lifespan is the ratio of these two numbers, and in the table it ranges from one day to continuous. Three structures are never overwritten at all; these are definitional records, and they stay until deleted. When two columns of the first table are read together, an inverse relationship emerges. The structure with the shortest lifespan — the user profile stamp — is also the structure with the highest base: its lifespan is one day, its base is 187. The more often a structure is written to, the sooner it is overwritten and the more clean machines it is also found on. A trace that disappears quickly is the least distinguishing trace even when it has not yet disappeared. The second table carries the course's measurement axis into this lesson. As delay rises from one day to ninety days, the number of readable structures goes from 12 to 3, and the number of claims left without a basis goes from 0 to **5**. The number of claims never changes; the only thing that changes is whether the structures carrying those claims are still there. The fourteen-day row also deserves attention: the number of claims in the weak class is **zero**. The claims have not improved; because the structures carrying the bases in between have been erased by the cycle, the claims have been thrown to the two extremes — they have either stayed distinctive by resting on permanent structures, or they have fallen to noise, left with only a high-base structure. **Delay does not just reduce evidence, it also distorts its distribution.** #### Summary - A structure is in the direct class if it comes into being only because the claimed event occurred, and in the inference class if it can also come into being for other reasons; 7 of the 12 structures are direct, and their bases range from 4 to 187 in the 268-machine fleet, that is, from 1 percent to 70 percent. - Of seven claims, 2 are distinctive, 4 are weak, and 1 is noise; the noise claim rests on a single inference structure and is also true on 187 clean machines. - The number of bases does not determine distinctiveness: a claim with three inference bases (4.68) turns out weaker than a claim with one direct and two inference bases (1.71). - The independence assumption makes the calculation a lower bound and always biases it in the claim's favor; traces born of the same action are found together. - The loss of a structure is measured as a number: losing the application cache downgrades the class of two claims, while losing the user profile stamp downgrades neither. - Nine of the structures are cyclical and their lifespans range from 1 to 41.7 days; when delay rises from one day to ninety days, readable structures go from 12 to 3, and claims left without a basis go from 0 to 5. #### Next Step This topic's five lessons produced five separate numbers: how many items the chain breaks at, how many items scope brings in, how many items order costs, how many events skew reverses, and how many clean machines a trace is also found on. None of these is a report sentence yet. A report consists of claims built on top of these numbers, and the numbers do not appear in the report unless the link between each claim and each item is written down one by one. The final lesson measures this mapping and closes the course. Every claim intended for the report is linked to the evidence item it rests on; three columns are counted separately — how many claims are supported directly by an item, how many rest on an inference, and how many have no evidence at all. What the technical finding means in a legal context depends exactly on how these three numbers are written. ### Reporting Source: https://academia.sh/en/courses/security-operations/reporting Measuring technical findings as a claim-to-evidence mapping: in a fictional 20-claim report draft built on 22 items collected by a three-person shift, 10 claims come out directly supported, 9 are inference, and 1 is unsupported. Seven of the 9 claims counted as inference drop there because of chain-of-custody gaps, and 2 are already inference. Perfect collection yields 11 direct claims, while host-by-host collection with a one-person team drops that to 7 and raises unsupported claims to 3. This topic's five lessons each produced a separate number: how many items the chain broke at, how many items scope brought in, how many items order cost, how many events the drift reversed, on how many clean machines a single trace was also found. None of these is a report sentence. A report consists of **claims**, and each claim is an opinion, not a number, until the item it rests on is written down. This lesson measures that mapping. A fictional twenty-claim draft is built on top of the item set from the first three lessons — the same 28 items, the same chain gaps, the same collection model. Three columns are counted separately: how many claims are directly supported by an item, how many rest on inference, and how many have no evidence at all. - **FE31.** **Report draft** is the list of claims planned to be written at the end of the examination. The twenty claims are fictional. - **FE32.** A claim is **directly supported** only if at least one of its basis items was collected, that item is defensible (its chain is intact, its hash matches), and it directly carries the claim. - **FE33.** If a claim's basis was collected but the item's chain is broken, or the claim reaches the item only through inference, the claim is **inference**. - **FE34.** If none of a claim's basis items were collected, the claim is **unsupported**. - **FE35.** The item set, chain gaps, and collection model are the same as the previous lessons'; since the generator seed did not change, the numbers come from exactly the same set. #### Report Draft and the Mapping Rule ```js // forensics/report.mjs — the MODEL for mapping report claims to evidence items. // The item set, chain failures, and collection order are the same as the model in the first // three lessons. const generator = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; export const CLASS = { "network-state": [120, 45], "process-list": [240, 60], "open-handle": [300, 55], "session-state": [600, 70], "memory-content": [900, 420], "temp-file": [3600, 180], "flow-record": [86400, 240], "remote-log": [2592000, 300], "disk-image": [604800, 5400], "configuration": [604800, 90], }; const D5 = ["collection", "packaging", "transit", "lab", "analysis"]; const D3 = ["collection", "transit", "analysis"], D2 = ["intake", "analysis"]; export const HOST = { "branch-endpoint-3": [D5, ["network-state", "process-list", "open-handle", "session-state", "memory-content", "temp-file", "disk-image", "configuration"]], "field-laptop-14": [D5, ["network-state", "process-list", "session-state", "memory-content", "disk-image"]], "collector-2": [D3, ["network-state", "process-list", "open-handle", "session-state", "memory-content", "disk-image", "configuration"]], "portal-frontend": [D3, ["network-state", "process-list", "memory-content", "configuration"]], "central-log": [D2, ["remote-log", "configuration"]], "flow-collector": [D2, ["flow-record", "configuration"]], }; const UNPROTECTED = ["branch-endpoint-3/temp-file", "collector-2/disk-image", "portal-frontend/configuration"]; // The same generator sequence as the first lesson: one draw per item, one (or two) draws per // transfer. export function items() { const r = generator(20260731), list = []; for (const [host, [nodes, classes]] of Object.entries(HOST)) for (const c of classes) { const name = `${host}/${c}`; r(); let missing = 0; for (let i = 1; i < nodes.length; i += 1) if (r() < 0.11) { r(); missing += 1; } list.push({ name, host, class: c, lifetime: CLASS[c][0], duration: CLASS[c][1], defensible: missing === 0 && !UNPROTECTED.includes(name) }); } return list; } // The collection from the third lesson: n people, the given order, arrival delay to remote hosts. const REMOTE = ["branch-endpoint-3", "field-laptop-14"]; export function collect(order, n, delay) { const free = Array(n).fill(0), taken = new Set(); for (const k of order) { const i = free.indexOf(Math.min(...free)); const start = Math.max(free[i], REMOTE.includes(k.host) ? delay : 0); if (start <= k.lifetime) { taken.add(k.name); free[i] = start + k.duration; } } return taken; } export const ORDER = { "by volatility": (K) => [...K].sort((a, b) => a.lifetime - b.lifetime || a.duration - b.duration), "host by host": (K) => [...K], }; // Report draft: claim -> [basis items, does the basis directly carry the claim] export const CLAIM = { "an unauthorized session was opened at the branch endpoint": [["central-log/remote-log", "branch-endpoint-3/session-state"], true], "an outbound connection was established from the branch endpoint": [["flow-collector/flow-record", "branch-endpoint-3/network-state"], true], "a new process ran at the branch endpoint": [["branch-endpoint-3/process-list", "branch-endpoint-3/memory-content"], true], "the process wrote a file": [["branch-endpoint-3/open-handle", "branch-endpoint-3/disk-image"], true], "the written file was later deleted": [["branch-endpoint-3/disk-image"], true], "the item was set to run at startup": [["branch-endpoint-3/configuration"], true], "the temp files at the branch endpoint were cleared": [["branch-endpoint-3/temp-file"], true], "a connection was established to the collector": [["flow-collector/flow-record", "collector-2/network-state"], true], "a session was opened at the collector": [["collector-2/session-state", "central-log/remote-log"], true], "a process was started at the collector": [["collector-2/process-list", "collector-2/memory-content"], true], "subscriber data was read from the collector": [["collector-2/disk-image", "collector-2/open-handle"], true], "the collector's configuration did not change": [["collector-2/configuration"], true], "the same session was used on the field laptop": [["field-laptop-14/session-state", "field-laptop-14/memory-content"], true], "the field laptop accessed the collector": [["field-laptop-14/network-state", "flow-collector/flow-record"], true], "a file was modified on the field laptop": [["field-laptop-14/disk-image"], true], "a token was used at the portal frontend": [["portal-frontend/memory-content", "central-log/remote-log"], true], "the portal frontend's setting was changed": [["portal-frontend/configuration"], true], "all the accesses are from the same person": [["central-log/remote-log"], false], "data was transmitted to a third party": [[], false], "this same access never happened before the incident": [["central-log/remote-log", "flow-collector/flow-record"], false], }; export function classify(claim, taken, items) { const [basis, direct] = CLAIM[claim]; const held = basis.filter((d) => taken.has(d)); if (!held.length) return "unsupported"; if (!direct) return "inference"; return held.some((d) => items.find((k) => k.name === d).defensible) ? "direct" : "inference"; } ``` The mapping rule has three inputs: whether the item was collected, whether the item's chain is intact, and whether the item directly carries the claim. The first two were measured in previous lessons; the third is the examiner's judgment, and it is filled in **while the claim is written** — if it is filled in afterward, the strength of the basis gets chosen to fit the claim. #### The Class of the Twenty Claims ```js // forensics/mapping.mjs — the claim-to-evidence mapping and collection order's counterpart in the report. import { items, collect, ORDER, CLAIM, classify } from "./report.mjs"; const y = (g, ...s) => console.log(s.map((v, i) => (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join("").trimEnd()); const K = items(), claims = Object.keys(CLAIM); const main = collect(ORDER["by volatility"](K), 3, 600); // three-person shift, 600 s travel console.log(`model: ${K.length} evidence items, ${K.filter((k) => k.defensible).length} ` + `defensible; ${claims.length} claims in the report draft`); console.log(`three-person shift, order of volatility, 600-second arrival: ${main.size} items taken`); const A = [-65, 9, 8, 13, 11]; console.log(""); y(A, "claim", "basis", "held", "class", "reason"); for (const i of claims) { const [d, direct] = CLAIM[i]; const held = d.filter((x) => main.has(x)); const s = classify(i, main, K); y(A, i, d.length, held.length, s, s === "direct" ? "-" : s === "unsupported" ? "no item" : direct ? "chain" : "claim"); } const count = (taken) => ["direct", "inference", "unsupported"].map((s) => claims.filter((i) => classify(i, taken, K) === s).length); const [d0, c0, k0] = count(main); const chain = claims.filter((i) => CLAIM[i][1] && classify(i, main, K) === "inference").length; console.log(`\ndirectly supported ${d0}, inference ${c0}, unsupported ${k0} (total ${claims.length})`); console.log(`of the ${c0} claims counted as inference, ${chain} dropped because of chain gaps, ` + `${c0 - chain} were already inference`); const B = [-22, 7, 9, 10, 10, 13]; console.log(""); y(B, "collection", "people", "taken", "direct", "inference", "unsupported"); for (const [name, people, delay] of [["by volatility", 3, 0], ["by volatility", 3, 600], ["by volatility", 1, 0], ["host by host", 3, 0], ["host by host", 1, 0]]) { const a = collect(ORDER[name](K), people, delay); y(B, `${name}${delay ? " +600 s" : ""}`, people, a.size, ...count(a)); } ``` ``` model: 28 evidence items, 16 defensible; 20 claims in the report draft three-person shift, order of volatility, 600-second arrival: 22 items taken claim basis held class reason an unauthorized session was opened at the branch endpoint 2 2 inference chain an outbound connection was established from the branch endpoint 2 1 direct - a new process ran at the branch endpoint 2 1 inference chain the process wrote a file 2 1 direct - the written file was later deleted 1 1 direct - the item was set to run at startup 1 1 inference chain the temp files at the branch endpoint were cleared 1 1 inference chain a connection was established to the collector 2 2 direct - a session was opened at the collector 2 2 direct - a process was started at the collector 2 2 direct - subscriber data was read from the collector 2 2 inference chain the collector's configuration did not change 1 1 direct - the same session was used on the field laptop 2 2 direct - the field laptop accessed the collector 2 1 direct - a file was modified on the field laptop 1 1 direct - a token was used at the portal frontend 2 1 inference chain the portal frontend's setting was changed 1 1 inference chain all the accesses are from the same person 1 1 inference claim data was transmitted to a third party 0 0 unsupported no item this same access never happened before the incident 2 2 inference claim directly supported 10, inference 9, unsupported 1 (total 20) of the 9 claims counted as inference, 7 dropped because of chain gaps, 2 were already inference collection people taken direct inference unsupported by volatility 3 28 11 8 1 by volatility +600 s 3 22 10 9 1 by volatility 1 21 8 10 2 host by host 3 19 9 10 1 host by host 1 16 7 10 3 ``` The result sits in two lines: of the twenty claims, ten are directly supported, nine are inference, and one is unsupported. The real information is in the line below. **Seven** of the nine claims counted as inference land there not because they are inherently inference, but because the chain of custody of the item they rest on is broken. The items are collected, they are on hand, their contents are read — they are not defensible. This is the report's counterpart to the 2.6 percent blank-record-field rate measured in the first lesson: **seven sentences that could have been directly supported drop one class down.** The reason column separates three defects, each with a different remedy. Rows marked "chain" are a **record** problem, and some fraction of them can be closed after the fact. The two rows marked "claim" are a **logic** problem: the sentence "all the accesses are from the same person" does not rise by adding any item, because an item shows a session, not an identity. The single row marked "no item" is a **scope** problem. The table below gives collection decisions their report counterpart. Under the perfect condition — three people, the correct order, no delay — all 28 items are taken and the number of directly supported claims is **11**. If the same team spends six hundred seconds traveling, that drops to 10; if the team shrinks to one person, to 8; if collection goes host by host and the team is down to one person, it drops to **7**, and the number of unsupported claims rises from one to three. A single ordering decision costs four report sentences and two unanswered questions. #### Claim, Inference, and What Goes Unwritten How the three columns are written into the report matters as much as measuring them. If the class is not written down, the reader gives every sentence the same weight; an inference sentence sits next to a directly supported one and cannot be told apart from it. That distinction is the writer's job, not the reader's. An inference sentence can go into the report; it cannot go in without stating that it is inference. An unsupported claim is not deleted — it moves under an "asked and unanswered" heading. That the question was asked, and why it could not be answered, carries decision weight: if it is not known whether the data left the network, writing down that it is not known is not the same thing as not writing it at all. The items collected and the items that go into the report also differ: 28 items were collected, 22 remained on hand, 23 items were cited as a basis, and 18 of those were on hand. An unused item is not a defect, but the report's length is measured not by the items collected but by the items **mapped**. The methodology section is itself a list of items: hash values, chain-of-custody records, collection order, arrival delay, the schedule's remaining drift. They are written **so that the same result can be reproduced**. The last boundary concerns the legal context, and it is drawn in a single sentence: the examination states **what the item shows**, not how the act should be characterized. Characterization is another profession's job; the moment a technical report takes its place, it opens even its correct technical part to dispute. This is also why the report's scope, its written authorization, and the list of systems examined go on the first page. #### Summary - A report consists of claims, and each claim is mapped to the item it rests on; the mapping rule's three inputs are whether the item was collected, is defensible, and directly carries the claim. - In the twenty-claim draft, with 22 items, 10 claims are directly supported, 9 are inference, and 1 is unsupported. - Seven of the 9 claims counted as inference dropped there because of chain-of-custody gaps; the items are on hand, they cannot be defended. - The three defects have separate remedies: a chain gap is a record problem, remaining inference is a logic problem, and the absence of an item is a scope problem. - Perfect collection yields 11 direct claims; host-by-host collection with a one-person team drops that to 7 and raises unsupported claims to 3. #### Course Wrap-Up | Lesson | Initial record / source | Steps to decision | Cost of delay | |---|---|---|---| | Security Operations Center | 273 alerts, 3-person shift | 26 cases, 1 handover point | 64 alerts unseen, median 987 minutes | | Log Aggregation and Normalization | 3 sources, 1-8 minute delay | 8-field schema, 2 intermediate tables | 9 redundant fields, 404 records actorless | | Security Information and Event Management | 3,083 raw events | 12 candidates, 10 alerts, 43 minutes | 2 of 5 campaigns never appear | | Network Flow Data | 2,979 flows, 7 fields | 4 metrics, 43 flags | 4 of 10 questions unanswered, 8.8 GB outage | | False Positives and False Negatives | 900 candidates, 20 items | 4 attributes, 1 threshold | at threshold 72, 7 missed, 42 extra items | | Detection Engineering | 26 techniques, 8 tactics | 11 rules, 2 versions | 2 techniques uncovered, 513 minutes review | | Threat Hunting | 12,910 records, 6 hypotheses | 2-3 elimination steps | 2 items in 1,691 minutes, 846 minutes/item | | Threat Intelligence | 900 indicators, 12 definitions | 90-day window | 828 dead indicators, 55 contacts missed | | Response Automation | 8 procedures, 158 items/day | 3 modes, 1,215 minutes manual work | 2.65 wrong actions/day, 457 minutes | | Preparation | 12 preparation items | 6-step chain | unprepared: 470 minutes, 13 items | | Identification | 30 assets, 20 logged | 7 criteria, 3-73 minutes | legitimate work interrupted at 13 assets | | Containment | 8 short, 4 long actions | 7 containment patterns | late decision: 5 extra items, 372 minutes | | Eradication | 22 points, 9 assets | 5 discovery methods, 506 minutes | 5 points missed, 8 reinfections in 60 days | | Recovery | 7 rollback points | 4-stage verification | dirty backup: 1,347 minutes, 9.5 days of data | | Lessons Learned | 16 questions, 9 record sources | 12 improvement items | 5 questions unanswered, 136 person-hours | | Communication and Notification Obligations | 12 stakeholders, 5 external | 4,320-minute window | late notification: 64,540 minutes over | | Forensic Examination Principles | 28 items, 6 hosts | 4 transfers, 5 record fields | 12 items contestable | | Disk Imaging | 15 items, 7 zones | 4 copy scopes | 6 items unverifiable on a live copy | | Memory Dump Analysis | 28 items, 18 volatile | 10-class order of volatility | 9 items in 1,200 seconds | | Timeline Creation | 3 sources, 54 events | 19 anchors, 2 coefficients | 33 events misplaced | | Structure and Record Examination | 12 structures, base 4-187 | 7 claims, 3 classes | 5 claims without basis at 90 days | | Reporting | 20 claims, 22 items | 3-class mapping | 7 claims drop at the chain | The fourth column has no empty cell. Each of the twenty-two lessons wrote the cost of delay as a duration or an item count, and in none did it come out to zero. The course's rule is read from this column: **the output of a security operation is not the number of alerts, but the time between detection and containment; if that time is not measured, the pipeline is not considered built.** The alert count measures workload; it does not prove a built pipeline. The second and third columns are the rule's two halves. A record that was never dropped in the first place cannot be recovered by any step; every extra step, in turn, is both an elimination and a delay. In none of the twenty-two lessons did both improve at once. This pipeline rested on a single silent assumption, which the last six lessons used most plainly: that logs, images, and memory dumps sit on **machines you can reach**. The collector's disk could be imaged; the branch endpoint could be visited. But part of the systems now run on hardware that is no longer yours; there, no image can be taken, and the first link in the chain of custody is in someone else's hands. **M24/K09 Cloud and Virtualization Security** begins with exactly this question.