Endpoint Detection and Response

The previous lesson’s alert mode cut no legitimate work, and so it looked cheap. For an item to be visible, a record of it must remain, and that record does not exist on its own: it must be collected on the endpoint, moved somewhere, and retained until someone looks at it. This lesson’s subject is the layer that produces those records.

Three numbers are measured. The first is coverage: how many of the eight foothold point types appear in at least one telemetry source, and how many appear in none. The second is volume: how many bytes each source produces per day. The third is the window: given a fixed retention budget, how many days back the collected data reaches. The three are not independent — the only way to grow coverage is to add sources, and every source shortens the window. Which of the course’s two columns this layer is written into also becomes clear in this lesson: the number of points it closes is zero.

• EP13. Seven sources, event rates, event sizes, and the retention budget belong to a fictional regional measurement network; the 432 endpoints consist of four classes. What is measured is the model built with node.
• EP14. Event rates are a fixed average per endpoint. In reality servers produce far more events than field laptops, and the rate fluctuates through the day; the average does not change the direction of the window calculation.
• EP15. A point type is either visible in a source or it is not. Partial visibility — seeing only that the point ran, not that it was written — was not modeled; this makes coverage look better than it is.
• EP16. Compression, sampling, and moving data to a cold tier were not modeled. All three extend the window and all three have their own cost; the shape of the curve does not change.
• EP17. Detection delays are a fictional list: the number of days that pass before past events are noticed. The real distribution has a long tail.
• EP18. The telemetry agent running on the endpoint is itself software and adds an item to the inventory; this item is counted, the agent’s own security is not measured.

Sources and What They See

The model consists of two tables. The first holds each telemetry source’s daily event count per endpoint and byte size per event; the second holds which foothold point type leaves a trace in which source. The second table encodes a technical fact: a point type becomes visible only when the action that writes it or runs it is recorded.

// telemetry.mjs — endpoint telemetry sources, visibility matrix, and retention budget; model
export const ENDPOINTS = 432;                // fictional fleet: field laptop, branch endpoint, aggregator, server
export const BUDGET = 120 * 1024 ** 3;       // retention budget, bytes (fictional)
// source -> [events/endpoint/day, bytes/event]
export const SOURCE = {
  'configuration change log': [140, 320],
  'script block log': [260, 4096],
  'module load log': [9800, 384],
  'file modification log': [31000, 256],
  'process creation log': [2400, 512],
  'network connection log': [5600, 224],
  'logon log': [18, 448],
};
// foothold point type -> sources that see it
export const VISIBILITY = {
  'startup-entry': ['configuration change log', 'process creation log'],
  'service-definition': ['configuration change log', 'process creation log'],
  'scheduled-task': ['configuration change log', 'process creation log'],
  'login-script': ['script block log', 'process creation log', 'file modification log'],
  'load-path-hook': ['module load log', 'file modification log'],
  'legitimate-binary-attach': ['file modification log'],
  'event-subscription': [],
  'media-autorun': [],
};
export const POINT = Object.keys(VISIBILITY);
export const daily = (k) => SOURCE[k][0] * SOURCE[k][1] * ENDPOINTS;
export const visible = (set) => POINT.filter((n) => VISIBILITY[n].some((k) => set.includes(k)));
export const window = (set) => BUDGET / set.reduce((a, k) => a + daily(k), 0);
export const toGiB = (b) => (b / 1024 ** 3).toFixed(2);
// fictional: days that pass before past events are noticed
export const DELAY = [3, 7, 12, 18, 21, 26, 34, 45, 60, 90];

// run1.mjs — source volumes and the visibility matrix: which point appears in which source
import { SOURCE, VISIBILITY, POINT, ENDPOINTS, BUDGET, daily, toGiB } from './telemetry.mjs';

const sources = Object.keys(SOURCE);
console.log(`${ENDPOINTS} endpoints, ${sources.length} telemetry sources, ${POINT.length} foothold point types; ` +
  `retention budget ${toGiB(BUDGET)} GiB\n`);
console.log(`${'source'.padEnd(31)} ${'evt/ep/day'.padStart(11)} ${'bytes/evt'.padStart(9)} ` +
  `${'GiB/day'.padStart(8)} ${'points seen'.padStart(13)}`);
for (const k of sources)
  console.log(`${k.padEnd(31)} ${String(SOURCE[k][0]).padStart(11)} ${String(SOURCE[k][1]).padStart(9)} ` +
    `${toGiB(daily(k)).padStart(8)} ${String(POINT.filter((n) => VISIBILITY[n].includes(k)).length).padStart(13)}`);
const total = sources.reduce((a, k) => a + daily(k), 0);
console.log(`${'total'.padEnd(31)} ${''.padStart(11)} ${''.padStart(9)} ${toGiB(total).padStart(8)}`);

console.log(`\n${'foothold point type'.padEnd(26)} ${'count'.padStart(6)}  sources that see it`);
for (const n of POINT)
  console.log(`${n.padEnd(26)} ${String(VISIBILITY[n].length).padStart(6)}  ${VISIBILITY[n].join(', ') || '-'}`);
const dark = POINT.filter((n) => VISIBILITY[n].length === 0);
console.log(`\npoint types seen by no source: ${dark.length}/${POINT.length} (${dark.join(', ')})`);
console.log(`sources that see no point type: ` +
  sources.filter((k) => !POINT.some((n) => VISIBILITY[n].includes(k))).join(', '));

432 endpoints, 7 telemetry sources, 8 foothold point types; retention budget 120.00 GiB

source                           evt/ep/day bytes/evt  GiB/day   points seen
configuration change log                140       320     0.02             3
script block log                        260      4096     0.43             1
module load log                        9800       384     1.51             1
file modification log                 31000       256     3.19             3
process creation log                   2400       512     0.49             4
network connection log                 5600       224     0.50             0
logon log                                18       448     0.00             0
total                                                     6.16

foothold point type         count  sources that see it
startup-entry                   2  configuration change log, process creation log
service-definition              2  configuration change log, process creation log
scheduled-task                  2  configuration change log, process creation log
login-script                    3  script block log, process creation log, file modification log
load-path-hook                  2  module load log, file modification log
legitimate-binary-attach        1  file modification log
event-subscription              0  -
media-autorun                   0  -

point types seen by no source: 2/8 (event-subscription, media-autorun)
sources that see no point type: network connection log, logon log

Two numbers stand out immediately, and both surprise anyone expecting the opposite. Two of the eight point types appear in no source at all. Persistent event subscription and removable media’s autorun entry fall outside telemetry; the first because it is written into the management framework’s own record space, the second because it sits on the attached medium, not the endpoint. These two point types were the cheapest to close in the previous lesson — and when they are not closed, there is no visibility standing in to take their place either.

The second number runs the other way: two sources see no foothold point at all. The network connection log produces half a GiB a day and makes not a single point visible. This does not mean the source is unnecessary; once an item is caught, only that source says where it connected and which endpoints it talked to. But by the foothold-point measure its contribution is zero, and it still takes its share of the budget.

The volume column gives the third surprising number. The process creation log sees the most points, four, and the configuration change log sees three; the second is one of the two cheapest sources, and the two together sit just above half a GiB a day. The most expensive source, the file modification log, also sees three point types but produces 3.19 GiB a day; two of the three it sees are already seen by other sources. There is no clean proportion between coverage and volume.

As Coverage Grows, the Window Shrinks

// run2.mjs — adds sources by point-gain-per-volume ratio; the window and the detections that fall outside it
import { SOURCE, POINT, DELAY, daily, visible, window, toGiB } from './telemetry.mjs';

const remaining = Object.keys(SOURCE), chosen = [];
console.log(`${'source added'.padEnd(31)} ${'new points'.padStart(10)} ${'visible'.padStart(7)} ` +
  `${'GiB/day'.padStart(7)} ${'window days'.padStart(11)} ${'window detections'.padStart(18)}`);
while (remaining.length) {
  let best = null;
  for (const k of remaining) {
    const gain = visible([...chosen, k]).length - visible(chosen).length;
    const ratio = gain / daily(k);
    if (!best || ratio > best.ratio || (ratio === best.ratio && daily(k) < daily(best.k))) best = { k, gain, ratio };
  }
  chosen.push(best.k);
  remaining.splice(remaining.indexOf(best.k), 1);
  const w = window(chosen), volume = chosen.reduce((a, k) => a + daily(k), 0);
  console.log(`${best.k.padEnd(31)} ${String(best.gain).padStart(10)} ${`${visible(chosen).length}/${POINT.length}`.padStart(7)} ` +
    `${toGiB(volume).padStart(7)} ${w.toFixed(1).padStart(11)} ${`${DELAY.filter((g) => g <= w).length}/${DELAY.length}`.padStart(18)}`);
}
const four = Object.keys(SOURCE).slice(0, 4);
console.log(`\nfour-source setup: visible ${visible(four).length}/${POINT.length}, ` +
  `window ${window(four).toFixed(1)} days, detections in window ${DELAY.filter((g) => g <= window(four)).length}/${DELAY.length}`);
const all = Object.keys(SOURCE);
console.log(`seven-source setup: visible ${visible(all).length}/${POINT.length}, ` +
  `window ${window(all).toFixed(1)} days, detections in window ${DELAY.filter((g) => g <= window(all)).length}/${DELAY.length}`);
console.log(`\nbudget needed for a 21-day window with seven sources: ` +
  `${toGiB(21 * all.reduce((a, k) => a + daily(k), 0))} GiB (` +
  `${(21 * all.reduce((a, k) => a + daily(k), 0) / (120 * 1024 ** 3)).toFixed(2)}x the current budget)`);
console.log(`foothold points closed: 0 — this layer does not close points, it makes them visible`);
console.log(`points the telemetry agent adds to the endpoint: 2 (service-definition, load-path-hook)`);

source added                    new points visible GiB/day window days  window detections
configuration change log                 3     3/8    0.02      6657.6              10/10
script block log                         1     4/8    0.45       268.8              10/10
module load log                          1     5/8    1.96        61.2               9/10
file modification log                    1     6/8    5.15        23.3               5/10
logon log                                0     6/8    5.16        23.3               5/10
process creation log                     0     6/8    5.65        21.2               5/10
network connection log                   0     6/8    6.16        19.5               4/10

four-source setup: visible 6/8, window 23.3 days, detections in window 5/10
seven-source setup: visible 6/8, window 19.5 days, detections in window 4/10

budget needed for a 21-day window with seven sources: 129.27 GiB (1.08x the current budget)
foothold points closed: 0 — this layer does not close points, it makes them visible
points the telemetry agent adds to the endpoint: 2 (service-definition, load-path-hook)

The add order was set by point gain per byte, and the curve exhausts itself in four steps. The configuration change log alone makes three point types visible and costs 0.02 GiB a day — a window of six thousand six hundred fifty-seven days, meaning the budget is practically unlimited. After the fourth step, no source brings a new point: coverage stays at six, and the remaining three sources add only volume. The window drops from twenty-three days to nineteen and a half.

These last three rows are not a list of waste, and reading them that way misuses the measurement. The process creation log is the source that sees the most point types; the reason it comes late in the order is that all four points it sees are already seen by cheaper sources. But once an item is noticed, only that source says what started what; the context to be measured in the fifth lesson is exactly this. The network connection log and the logon log belong to the same class. The foothold-point measure does not show these three sources’ value; what it shows is that these three sources cannot be defended on coverage grounds.

The bottom lines give the lesson’s real tension. When all seven sources are collected together, the window is nineteen and a half days; four of the ten fictional detection delays fit inside this window. For an event noticed after twenty-one days, the endpoint’s record has already been deleted — detection worked, there is no evidence. The same coverage is achieved with four sources and a twenty-three-day window, and five of the ten detections stay inside it. In this model, collecting less with the same budget means seeing more.

The last three lines record this layer’s place in the course’s two columns. The number of foothold points it closes is zero; it makes six of the eight point types visible and never sees two of them at all. On top of that, it needs a service definition and a load-path hook on the endpoint for its own agent: it adds two points to the inventory. The visibility layer’s cost is not only storage — it is two items of the very thing it measures.

Depth of Visibility

The coverage column says whether a point is visible or not; it does not say how many sources it is visible from. The difference matters: a point tied to a single source becomes invisible when that source goes silent, and silencing a source is a single configuration item on the endpoint.

// run3.mjs — depth of visibility: how many sources see a point type, how many points depend on a single source
import { SOURCE, VISIBILITY, POINT, visible, window } from './telemetry.mjs';

const FOUR = Object.keys(SOURCE).slice(0, 4), ALL = Object.keys(SOURCE);
const depth = (set, n) => VISIBILITY[n].filter((k) => set.includes(k)).length;
console.log(`${'foothold point type'.padEnd(26)} ${'4 sources'.padStart(13)} ${'7 sources'.padStart(13)}`);
for (const n of POINT)
  console.log(`${n.padEnd(26)} ${String(depth(FOUR, n)).padStart(13)} ${String(depth(ALL, n)).padStart(13)}`);
for (const [label, set] of [['four sources', FOUR], ['seven sources', ALL]]) {
  const g = visible(set);
  console.log(`\n${label}: visible ${g.length}/${POINT.length}, window ${window(set).toFixed(1)} days`);
  console.log(`  points on a single source : ${g.filter((n) => depth(set, n) === 1).length}`);
  console.log(`  points on two or more     : ${g.filter((n) => depth(set, n) >= 2).length}`);
  const one = g.filter((n) => depth(set, n) === 1)
    .map((n) => VISIBILITY[n].find((k) => set.includes(k)));
  const critical = [...new Set(one)].map((k) => [k, one.filter((x) => x === k).length])
    .sort((a, b) => b[1] - a[1])[0];
  console.log(`  source that darkens the most points if silenced: ${critical[0]} (${critical[1]} points)`);
}

foothold point type            4 sources     7 sources
startup-entry                          1             2
service-definition                     1             2
scheduled-task                         1             2
login-script                           2             3
load-path-hook                         2             2
legitimate-binary-attach               1             1
event-subscription                     0             0
media-autorun                          0             0

four sources: visible 6/8, window 23.3 days
  points on a single source : 4
  points on two or more     : 2
  source that darkens the most points if silenced: configuration change log (3 points)

seven sources: visible 6/8, window 19.5 days
  points on a single source : 1
  points on two or more     : 5
  source that darkens the most points if silenced: file modification log (1 points)

The two setups give the same coverage with different depths. In the four-source setup, four of the six visible points depend on a single source, and three of them on the same source: when the configuration change log is silenced, startup entry, service definition, and scheduled task go dark all at once. In the seven-source setup, the number of points tied to a single source drops to one. The three extra sources brought nothing in the coverage column; in the depth column they raise three points from one source to two, and one point to three.

This changes how the previous table is read. “A source that brings no new point” and “a source that is worthless” are not the same thing. The three measures contradict each other: coverage saturates at the fourth source, depth peaks at the seventh, and the window shortens with every addition. No setup in the model makes all three best at once. The decision is made by writing down how much of which measure will be given up — and that writing is this very table.

Summary

• 2 of the 8 foothold point types appear in no telemetry source (persistent event subscription, removable media entry). These two were the cheapest points to close in the previous lesson.
• 2 sources see no point type (network connection log, logon log) and together cost 0.50 GiB a day. Their value lies in response context, not coverage.
• When sorted by point gain per byte, coverage reaches 6/8 at 4 sources and stalls; the remaining 3 sources raise the daily volume from 5.15 GiB to 6.16 GiB without bringing a new point.
• With a 120 GiB budget, the window is 23.3 days with 4 sources and 19.5 days with 7. The 21-day fictional detection delay falls outside the window in the second case: detection works, no evidence is found. Holding the same window with 7 sources needs 1.08 times the budget.
• The number of points this layer closes is 0; it makes 6 points visible and adds 2 points to the endpoint for its own agent. Visibility is not closing, and its cost is not only storage.

Next Step

This lesson’s telemetry recorded what happened on the endpoint, but only one source saw the relationship between endpoints, and that source could not be defended on coverage grounds. The connection between endpoints itself was measured somewhere else: the box standing at the network’s edge could count the traffic crossing between zones. There is a set that measurement cannot see — the connection between two endpoints sitting in the same zone never crosses the edge at all. The next lesson’s question is this: how many connection paths exist between endpoints on the same network, how many of them does the perimeter rule never see, and how many of those paths does a filter sitting on the host side close while cutting how much legitimate work along with them.