--- title: 'Application-Level Anomaly Detection' source: 'https://academia.sh/en/courses/runtime-security/application-level-anomaly-detection' course: 'Runtime Security and Secrets Management' language: en updated: '2026-10-07T04:31:05+00:00' license: 'CC BY-SA 4.0' objectives: - 'Runs the anomaly score against a role baseline on a behavior log and measures the flagging delay in minutes' - 'Counts together the records that left during the delay and the legitimate session that was falsely flagged' - 'Compares how many records earlier the same leak could have been closed at the design stage' --- # Application-Level Anomaly Detection Running seven sessions scored against a role baseline through a forty-five-minute behavior log and comparing the flagging threshold across four settings: the delay between the minute the deviation began and the minute the flag was raised, the number of records that left during that delay, the score gap between the slowly deviating insider threat session and the legitimate analyst session in the billing period, and how many records earlier the same leak could have been closed at the design stage. The previous four lessons looked at the request: the rule set at the shape, the rate counter at the number, the thresholds at the account and the address, the headers at the response. All four made their decision on a single request or a single response. The item that no setting has been able to stop since the first lesson slipped through for exactly this reason: the requests of the out-of-workflow bulk download were flawless one by one, authenticated, and authorized. The only thing wrong was **the shape the requests of the same session drew together**. This lesson moves the defense out of the request and into the behavior of the session. The measure changes too: instead of requests stopped, **delay** is counted: how many minutes after the deviation began the flag is raised, and how much data leaves during those minutes. This layer does not block a request; it flags a session late or early. ## Baseline and Anomaly Score Everything below is a model built with `node`; no real session log is read, and no method of taking over a session is written. Taken-over sessions are represented only by behavior vectors: requests per minute, unique endpoints, records, and the number of first-seen endpoints. - **AD21.** The seven sessions, three role baselines, and forty-five-minute window belong to the fictional measurement network. The baseline is given as the mean of thirty days of normal behavior. - **AD22.** The score derives from the ratio of each feature to its baseline, and the deviation share per feature is capped at two units. Without the cap, a single outlier would flag the session instantly; real scoring systems have the same kind of saturation. - **AD23.** The flag is raised when the score total of the last five minutes crosses the threshold. The stacking window is fixed; the only thing that changes is the threshold. - **AD24.** Two of the legitimate sessions are deliberately unusual: the analyst in the billing period has a record volume rising to two and a half times the baseline, and the new-hire field technician has a high number of first-seen endpoints. Both are candidates for a false flag. - **AD25.** The "earlier closing stage" column was marked by hand for each bad session. ```js // behavior.mjs — forty-five-minute session behavior log and anomaly scoring; a model export const DURATION = 45; export const FEATURES = ['requests/min', 'unique endpoints', 'records/min', 'first-seen endpoints']; export const WEIGHT = [1, 1.5, 2, 2.5]; export const CAP = 2; // cap on the deviation share per feature // role baseline: mean taken from thirty days of normal behavior export const BASELINE = { subscriber: [4, 3, 12, 0.2], field: [9, 5, 40, 0.4], analyst: [6, 7, 300, 0.3], }; // [session, role, true class, deviation minute, deviation multipliers, minutes to full multiplier, earlier stage] export const SESSIONS = [ ['subscriber-3391', 'subscriber', 'legit', Infinity, [1, 1, 1, 1], 1, '-'], ['field-12, shift', 'field', 'legit', Infinity, [1, 1, 1, 1], 1, '-'], ['analyst-4, billing period', 'analyst', 'legit', 0, [1.2, 1.1, 2.6, 1.3], 4, '-'], ['field-58, new hire', 'field', 'legit', 0, [0.8, 1.8, 0.9, 6.0], 3, '-'], ['subscriber-8842, taken over', 'subscriber', 'bad', 12, [3.5, 2.5, 22, 5], 3, 'code'], ['analyst-9, insider threat', 'analyst', 'bad', 8, [1.4, 1.3, 2.2, 1.5], 12, 'design'], ['field-31, stolen token', 'field', 'bad', 20, [2.2, 4.4, 3.5, 9], 2, 'none'], ]; // the minute-by-minute feature vector of a session export function trace(o) { const [, role, , onset, mult, ramp] = o, base = BASELINE[role], d = []; for (let t = 0; t < DURATION; t++) { const f = t < onset ? 0 : Math.min(1, (t - onset + 1) / ramp); d.push(base.map((b, i) => b * (1 + (mult[i] - 1) * f))); } return d; } export const score = (v, role) => v.reduce((t, x, i) => t + WEIGHT[i] * Math.min(CAP, Math.max(0, x / BASELINE[role][i] - 1)), 0); export const stack = (o, win = 5) => { const p = trace(o).map((v) => score(v, o[1])); return p.map((_, t) => p.slice(Math.max(0, t - win + 1), t + 1).reduce((x, y) => x + y, 0)); }; // the session is flagged when the score total of the last `win` minutes crosses the threshold export function flag(o, threshold, win = 5) { const d = trace(o), p = d.map((v) => score(v, o[1])); for (let t = 0; t < DURATION; t++) { const sum = p.slice(Math.max(0, t - win + 1), t + 1).reduce((x, y) => x + y, 0); if (sum >= threshold) { const start = Number.isFinite(o[3]) ? o[3] : 0; const records = d.slice(start, t + 1).reduce((x, v) => x + v[2], 0); return { minute: t, delay: t - start, records: Math.round(records) }; } } return null; } ``` ## With How Many Minutes of Delay ```js // run-flagging.mjs — the same behavior log, with the flagging threshold as the only change import { DURATION, SESSIONS, trace, score, stack, flag } from './behavior.mjs'; const s = (x, n) => String(x).padStart(n); console.log(`${SESSIONS.length} sessions, ${DURATION} minutes, 3 role baselines\n`); console.log(`${'session'.padEnd(30)}${s('role', 11)}${s('class', 7)}${s('deviation min', 15)}` + `${s('peak minute score', 20)}${s('peak stack', 17)}${s('total records', 14)}${s('earlier stage', 15)}`); for (const o of SESSIONS) { const d = trace(o), p = d.map((v) => score(v, o[1])); console.log(`${o[0].padEnd(30)}${s(o[1], 11)}${s(o[2], 7)}` + `${s(Number.isFinite(o[3]) ? o[3] : '-', 15)}${s(Math.max(...p).toFixed(2), 20)}` + `${s(Math.max(...stack(o)).toFixed(1), 17)}` + `${s(Math.round(d.reduce((x, v) => x + v[2], 0)), 14)}${s(o[6], 15)}`); } console.log(`\n${'threshold'.padEnd(11)}${s('flagged bad', 18)}${s('missed bad', 16)}` + `${s('falsely flagged', 20)}${s('mean delay min', 21)}${s('records out during delay', 26)}`); const B = SESSIONS.filter((o) => o[2] === 'bad'), L = SESSIONS.filter((o) => o[2] === 'legit'); for (const threshold of [40, 30, 20, 12]) { const fb = B.filter((o) => flag(o, threshold)), fl = L.filter((o) => flag(o, threshold)); const g = fb.map((o) => flag(o, threshold)); console.log(`${String(threshold).padEnd(11)}${s(`${fb.length}/${B.length}`, 18)}` + `${s(B.length - fb.length, 16)}${s(`${fl.length}/${L.length}`, 20)}` + `${s((g.reduce((x, r) => x + r.delay, 0) / g.length).toFixed(1), 21)}` + `${s(g.reduce((x, r) => x + r.records, 0), 26)}`); } for (const threshold of [30, 20]) { console.log(`\nthreshold ${threshold}, session by session:`); console.log(` ${'session'.padEnd(30)}${s('class', 7)}${s('flag min', 15)}${s('delay min', 12)}` + `${s('records out during delay', 26)}`); for (const o of SESSIONS) { const r = flag(o, threshold); console.log(` ${o[0].padEnd(30)}${s(o[2], 7)}${s(r ? r.minute : 'not flagged', 15)}` + `${s(r ? r.delay : '-', 12)}${s(r && o[2] === 'bad' ? r.records : '-', 26)}`); } } ``` ``` 7 sessions, 45 minutes, 3 role baselines session role class deviation min peak minute score peak stack total records earlier stage subscriber-3391 subscriber legit - 0.00 0.0 540 - field-12, shift field legit - 0.00 0.0 1800 - analyst-4, billing period analyst legit 0 4.30 21.5 34380 - field-58, new hire field legit 0 6.20 31.0 1624 - subscriber-8842, taken over subscriber bad 12 13.25 66.3 8604 code analyst-9, insider threat analyst bad 8 4.50 22.5 24840 design field-31, stolen token field bad 20 13.20 66.0 4250 none threshold flagged bad missed bad falsely flagged mean delay min records out during delay 40 2/3 1 0/4 3.0 1314 30 2/3 1 1/4 2.0 910 20 3/3 0 2/4 4.7 7106 12 3/3 0 2/4 3.3 4556 threshold 30, session by session: session class flag min delay min records out during delay subscriber-3391 legit not flagged - - field-12, shift legit not flagged - - analyst-4, billing period legit not flagged - - field-58, new hire legit 5 5 - subscriber-8842, taken over bad 14 2 540 analyst-9, insider threat bad not flagged - - field-31, stolen token bad 22 2 370 threshold 20, session by session: session class flag min delay min records out during delay subscriber-3391 legit not flagged - - field-12, shift legit not flagged - - analyst-4, billing period legit 6 6 - field-58, new hire legit 3 3 - subscriber-8842, taken over bad 13 1 276 analyst-9, insider threat bad 20 12 6600 field-31, stolen token bad 21 1 230 ``` The two rows of the first table should be read side by side. The legitimate analyst in the billing period has a peak five-minute stack of 21.5, and the insider threat session has 22.5. The gap is one unit, and everything the scoring sees is made of this gap. Both are in the same role, working at several times the baseline volume; one is producing an end-of-period report, the other is exfiltrating. The scoring cannot separate these two sessions, because the difference is not in the behavior but in **the intent**: the same limit as the rule set of the first lesson, one layer up. The threshold table gives the consequence. At threshold 40, no legitimate session is flagged and the mean delay is 3 minutes, but the insider threat is never caught: its stack never reaches 40. At threshold 30 the table looks even better, with the delay falling to 2 minutes and the records leaving during the delay to 910, but the insider threat escapes again and the new-hire field technician is falsely flagged. The first setting that catches the insider threat is 20, and there three of the four numbers get worse at once: falsely flagged sessions rise to 2, the mean delay to 4.7 minutes, and the records leaving during the delay to 7,106. The two session-by-session lists show where the delay comes from. The taken-over subscriber session and the stolen token session deviate fast and are flagged in one or two minutes; the records they let out during the delay lie between 230 and 540. The insider threat, on the other hand, is flagged after 12 minutes, and in that time it downloads 6,600 records, 93% of the total delay cost of the three bad sessions. The slowly deviating session is both the one caught last and the one that takes out the most data; delay and damage grow in the same direction. ## Where the Delay Belongs ```js // stage.mjs — at which earlier stage, and by how many records, the same leak would be closed sooner import { DURATION, SESSIONS, BASELINE, trace, flag } from './behavior.mjs'; const s = (x, n) => String(x).padStart(n); const find = (name) => SESSIONS.find((o) => o[0].startsWith(name)); const insider = find('analyst-9'), period = find('analyst-4'), T = BASELINE.analyst[2]; const cum = (o) => trace(o).reduce((a, v) => (a.push((a.at(-1) || 0) + v[2]), a), []); const excess = (o, t) => Math.round(cum(o)[t] - T * (t + 1)); const capAt = (o, cap) => { const k = cum(o), t = k.findIndex((x) => x >= cap); return t < 0 ? { label: 'not cut', t: DURATION - 1 } : { label: `min ${t}`, t }; }; console.log(`analyst role baseline ${T} records/min; ${DURATION} minutes of normal work ${T * DURATION} records`); console.log(`insider session downloaded ${Math.round(cum(insider).at(-1))} records in total,` + ` above baseline ${excess(insider, DURATION - 1)}\n`); console.log(`${'measure'.padEnd(34)}${s('stage', 16)}${s('above-baseline records', 24)}${s('legit cost', 34)}`); const e20 = flag(insider, 20), e40 = flag(insider, 40); for (const [a, b, c, d] of [ ['no measure', '-', excess(insider, DURATION - 1), '0'], ['anomaly threshold 40', 'runtime', e40 ? excess(insider, e40.minute) : excess(insider, DURATION - 1), 'no false flag'], ['anomaly threshold 20', 'runtime', excess(insider, e20.minute), `billing analyst flagged at min ${flag(period, 20).minute}`], ['per-role record cap 18000', 'design', excess(insider, capAt(insider, 18000).t), `billing analyst cut at ${capAt(period, 18000).label}`], ['bulk export moved to queue job', 'design', 0, 'bulk job goes to approval'], ]) console.log(`${a.padEnd(34)}${s(b, 16)}${s(c, 24)}${s(d, 34)}`); console.log(`\n${'cap'.padEnd(8)}${s('insider cut', 13)}${s('above-baseline records', 24)}` + `${s('billing analyst cut', 21)}${s('legit records cut', 19)}`); for (const cap of [13500, 18000, 25000]) { const a = capAt(insider, cap), b = capAt(period, cap); console.log(`${String(cap).padEnd(8)}${s(a.label, 13)}${s(excess(insider, a.t), 24)}${s(b.label, 21)}` + `${s(Math.round(cum(period).at(-1) - cum(period)[b.t]), 19)}`); } console.log(`\n${'bad session'.padEnd(30)}${s('threshold 20 delay', 20)}${s('earlier stage', 15)}${s('closes there', 15)}`); const CLOSES = { code: 'second factor', design: 'workflow', none: 'none' }; for (const o of SESSIONS.filter((x) => x[2] === 'bad')) console.log(`${o[0].padEnd(30)}${s(`${flag(o, 20).delay} min`, 20)}${s(o[6], 15)}${s(CLOSES[o[6]], 15)}`); ``` ``` analyst role baseline 300 records/min; 45 minutes of normal work 13500 records insider session downloaded 24840 records in total, above baseline 11340 measure stage above-baseline records legit cost no measure - 11340 0 anomaly threshold 40 runtime 11340 no false flag anomaly threshold 20 runtime 2700 billing analyst flagged at min 6 per-role record cap 18000 design 7740 billing analyst cut at min 23 bulk export moved to queue job design 0 bulk job goes to approval cap insider cut above-baseline records billing analyst cut legit records cut 13500 min 27 5220 min 18 20280 18000 min 34 7740 min 23 16380 25000 not cut 11340 min 32 9360 bad session threshold 20 delay earlier stage closes there subscriber-8842, taken over 1 min code second factor analyst-9, insider threat 12 min design workflow field-31, stolen token 1 min none none ``` The top table shows that a crude design measure can be **worse** than runtime scoring. A per-role record cap of 18,000 lowers the records the insider threat takes out above baseline only from 11,340 to 7,740, while the anomaly threshold of 20 stops the same item at 2,700. The middle table explains why the cap stays so weak: the cap would have to be lowered enough to cut the insider threat early, but a cap of 13,500 cuts the billing-period analyst at minute 18 and leaves 20,280 legitimate records outside. A fixed number cannot be tuned in a role whose workload is seasonal. What can be tuned is the workflow itself. When bulk export is taken off the interactive endpoint and moved to an approved queue job, the number of records a session can take out above baseline is zero: the insider threat session cannot produce that volume, because the interactive endpoint does not have that capability. The legitimate cost is also smaller: the billing analyst's bulk job is not cut, it goes to approval and is delayed. This is a decision given once in the design; it does not require adjusting a threshold every day. The bottom table separates the three bad sessions. The taken-over subscriber session is caught with a one-minute delay, but there was already a cheaper option: the third lesson had lowered the same takeover from seventy to five with the second factor. The insider threat is caught with a 12-minute delay, and the right place is the workflow. For the stolen token session, there is no earlier stage: the token is valid, authentication is successful, and no design or code decision invalidates a valid credential. The share only this layer can cover is that single row, and even that row carries one minute of delay. ## Summary - The legitimate analyst in the billing period has a peak five-minute stack of 21.5, and the insider threat session has 22.5. The difference the scoring sees between the two sessions is one unit; the separation lies not in the behavior but in the intent. - Thresholds 40 and 30 flag no legitimate session or only one, and miss the insider threat. The first setting that catches the insider threat is 20, and there falsely flagged sessions rise to 2/4, the mean delay to 4.7 minutes, and the records leaving during the delay to 7,106. - The two fast-deviating sessions were flagged in 1-2 minutes and let out 230-540 records during the delay. The slowly deviating insider threat was flagged in 12 minutes and let out 6,600 records: 93% of the total delay cost. Delay and damage grow in the same direction. - A crude design measure can be worse than runtime: a record cap of 18,000 stopped the above-baseline leak at 7,740, while anomaly threshold 20 stopped it at 2,700. A cap of 13,500 cuts the insider threat at minute 27, but it also cuts 20,280 legitimate records of the billing analyst. - Moving bulk export to an approved queue job zeroes the above-baseline records and does not cut the legitimate work, only delays it. Of the three bad sessions, one closes at the code stage (second factor) and one at the design stage (workflow); for the stolen token session, there is no earlier stage. ## Next Step This topic counted in five lessons what is caught at the request layer and what is falsely blocked: the rule set, the rate limit, the account and address thresholds, header coverage, and session behavior. In each lesson a class of items was written as "not here, it would have been closed earlier", and in each lesson there was also one of the opposite kind: a class that can be closed only here. This lesson's stolen token session is exactly of that class, and the question it leaves goes beyond the whole topic. That session was flagged, but what required it to be flagged was that the token **was still valid**. The defenses of all five lessons rest on the same assumption: the keys, tokens, and credentials behind them are valid, and as long as they stay valid, these layers have to look at behavior. Where those credentials are kept, who can read them, and for how long they stay valid was never asked. The next topic asks this, and its first measure is lifetime: the time a secret stays valid is itself the window it leaves open.