Dynamic Application Security Testing

The previous lesson counted static analysis’s field of view and showed that 62 of the 163 open points stayed outside it at every threshold. Most of these were points whose outcome is decided at runtime: whether a request matches the caller’s authorization, whether a header is actually sent, could not be read from the source text. There is a way to reach these points — bring the application up and request it from the outside.

Dynamic application security testing is the check that looks for vulnerabilities by sending requests to a running application. It does not look at source code, so it is independent of language and framework; in exchange, it only sees what it can reach. This lesson measures that limit: how many endpoints are reached, how many are not, what escapes at the endpoints that are reached, and how many pipeline minutes all of this costs.

• AC7. The surface is the 96 endpoints of the subscriber portal and the meter intake intake service. The inventory is fictional and is a model; no request is sent to any target on this page.
• AC8. Every endpoint has two independent properties. Discovery is how the address is found: a page link, an address only the client-side script constructs, or an interface declaration. Access is the condition for getting a valid response from the endpoint: anonymous, session-required, appearing only within a particular flow sequence, or requiring an admin role.
• AC9. A configuration is the capability set given to the scanner. Three are compared, and each adds a single capability on top of the previous one.
• AC10. Every vulnerability has a detectable value: whether the tool would catch it once it reaches that endpoint. The value is a property of the vulnerability itself, independent of the configuration; adding a capability therefore changes the vulnerabilities found only through access.
• AC11. 0.05 model minutes are spent per request. The durations are data, not a measurement performed on this page.
• AC12. A finding that has been reviewed once and marked a false positive enters the suppression list and never stops the build again in subsequent runs.

The Surface Reached and Not Reached

// surface.mjs — the surface a tool scanning a running application can and cannot reach.
// THIS IS A MODEL; the endpoint inventory is fictional, no request is sent to any target.
export const SEED = 20260317, ENDPOINT_COUNT = 96, REQUEST_MIN = 0.05;
const rng = (t) => { let s = t >>> 0;
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };
const r = rng(SEED);

// discovery: how the endpoint's address is found. access: the condition for getting a
// valid response from the endpoint.
const DISCOVERY = [["link", 0.52], ["client", 0.27], ["declaration", 0.21]];
const ACCESS = [["anonymous", 0.30], ["session", 0.42], ["sequence", 0.16], ["admin", 0.12]];
// Vulnerability class: detectRate = the probability the tool catches this class once it
// reaches an endpoint that has it.
export const CLASS = [
  { name: "config header", share: 0.22, detectRate: 0.95, requests: 4 },
  { name: "reflected input", share: 0.26, detectRate: 0.85, requests: 26 },
  { name: "error message leak", share: 0.18, detectRate: 0.75, requests: 12 },
  { name: "access control", share: 0.20, detectRate: 0.15, requests: 18 },
  { name: "workflow rule", share: 0.14, detectRate: 0.05, requests: 22 },
];
const pick = (table, u) => { let t = 0;
  for (const [name, p] of table) { t += p; if (u < t) return name; } return table.at(-1)[0]; };
const pickClass = (u) => { let t = 0;
  for (const c of CLASS) { t += c.share; if (u < t) return c; } return CLASS.at(-1); };

export const ENDPOINTS = Array.from({ length: ENDPOINT_COUNT }, (_, i) => {
  const discovery = pick(DISCOVERY, r()), access = pick(ACCESS, r());
  const vuln = r() < 0.34 ? pickClass(r()) : null;
  // detectable is the vulnerability's own property: does the tool catch it even once it
  // reaches the endpoint.
  const detectable = vuln ? r() < vuln.detectRate : false;
  return { i, discovery, access, vuln, detectable,
    requests: 6 + (vuln ? vuln.requests : 14), falsePositive: r() < 0.13 };
});

// Three configurations: each adds one capability on top of the previous one.
export const CONFIGURATION = [
  { name: "anonymous browsing", discovery: ["link"], access: ["anonymous"] },
  { name: "browsing with session", discovery: ["link", "client"], access: ["anonymous", "session"] },
  { name: "session + recorded flow", discovery: ["link", "client", "declaration"],
    access: ["anonymous", "session", "sequence"] },
];
export const reachableFor = (y) =>
  ENDPOINTS.filter((u) => y.discovery.includes(u.discovery) && y.access.includes(u.access));

export function run(y) {
  const reached = reachableFor(y);
  const requests = reached.reduce((a, u) => a + u.requests, 0);
  const real = reached.filter((u) => u.vuln && u.detectable);
  const falsePos = reached.filter((u) => u.falsePositive);
  const openEndpoints = ENDPOINTS.filter((u) => u.vuln);
  return { name: y.name, reached, requests, real, falsePos,
    unreachable: openEndpoints.filter((u) => !reached.includes(u)).length,
    unseen: openEndpoints.length - real.length, min: Math.round(requests * REQUEST_MIN) };
}

const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0]))).join("");

if (import.meta.url === `file://${process.argv[1]}`) {
  const openEndpoints = ENDPOINTS.filter((u) => u.vuln);
  console.log(`model (seed ${SEED}): ${ENDPOINT_COUNT} endpoints, ${openEndpoints.length} with a real vulnerability; ` +
    `${ENDPOINTS.filter((u) => u.access === "admin").length} endpoints require an admin role`);
  const y1 = gen([23, 9, 8, 8, 9, 9, 12, 14, 11]);
  console.log("\n" + y1(["configuration", "reached", "reqs", "found", "real", "false+",
    "unseen", "unreachable", "scan min"]));
  const result = CONFIGURATION.map(run);
  for (const o of result) {
    console.log(y1([o.name, o.reached.length, o.requests, o.real.length + o.falsePos.length,
      o.real.length, o.falsePos.length, o.unseen, o.unreachable, o.min]));
  }
  const full = result.at(-1);
  const y2 = gen([23, 7, 11, 10, 14, 16]);
  console.log("\nby vulnerability class in the widest configuration");
  console.log(y2(["vuln class", "open", "reached", "found", "unreachable", "reached, missed"]));
  for (const c of CLASS) {
    const l = openEndpoints.filter((u) => u.vuln.name === c.name);
    const e = l.filter((u) => full.reached.includes(u));
    const b = full.real.filter((u) => u.vuln.name === c.name).length;
    console.log(y2([c.name, l.length, e.length, b, l.length - e.length, e.length - b]));
  }
  console.log(`\nwhat adding capability buys: reached endpoints ${result[0].reached.length} -> ` +
    `${result[1].reached.length} -> ${full.reached.length}, real vulnerabilities found ` +
    `${result[0].real.length} -> ${result[1].real.length} -> ${full.real.length}, ` +
    `scan ${result[0].min} -> ${result[1].min} -> ${full.min} minutes`);
}

model (seed 20260317): 96 endpoints, 28 with a real vulnerability; 12 endpoints require an admin role

configuration            reached    reqs   found     real   false+      unseen   unreachable   scan min
anonymous browsing            13     278       2        2        0          26            25         14
browsing with session         50    1066       9        5        4          23            19         53
session + recorded flow       84    1772      17       10        7          18             5         89

by vulnerability class in the widest configuration
vuln class                open    reached     found   unreachable reached, missed
config header                6          3         3             3               0
reflected input              5          5         5             0               0
error message leak           3          3         2             0               1
access control               7          7         0             0               7
workflow rule                7          5         0             2               5

what adding capability buys: reached endpoints 13 -> 50 -> 84, real vulnerabilities found 2 -> 5 -> 10, scan 14 -> 53 -> 89 minutes

The first row is a scan’s most common setup: give it the application’s address and let it browse. It reaches 13 of 96 endpoints. 25 of the 28 vulnerabilities sit at an endpoint that is unreachable at that point, and the report closes with two true findings. This report is not wrong; it says nowhere that it is incomplete. Someone reading the result “two findings” has no way of knowing that 86 percent of the surface was never tried.

The second row gives the scanner a valid session and the addresses constructed on the client side: reached endpoints climb to 50, real findings to 5. The third row adds the interface declaration and the recorded flow sequence — reaching the 19 endpoints that only become accessible in a particular sequence. Reached endpoints reach 84, real findings 10. What multiplies the number of vulnerabilities found by five is not the tool’s capability, but the access it is given. The 12 admin endpoints are never scanned in any row, because the model never gives the scanner an admin identity; this is where a scope decision turns directly into a false negative.

The second table splits the 18 unseen vulnerabilities into two reasons, and the split carries a decision. Five go unseen because they are unreachable — three config-header vulnerabilities and two workflow-rule vulnerabilities sit at the admin endpoint. These can be closed by widening scope. Thirteen go unseen despite being reached; widening scope does nothing for these. All seven of the access-control class’s vulnerabilities are reached, and none is found: telling that a request is unauthorized requires sending the same request with two different identities and comparing the responses; a single-identity browse cannot do this. The workflow-rule class is similar — seeing that a step can be skipped requires knowing what the expected sequence is, and the scanner does not know it.

Read together with the previous lesson, the table completes itself. Static analysis never saw the config-header class at all; the tool here finds three of its six. The access-control class, by contrast, closes with zero findings in both checks. Even the combination of two checks leaves one class empty, and which class that is only becomes visible once the coverage list is written down.

Placement in the Pipeline

The scan takes 89 model minutes. That duration determines its place in the pipeline, and the place determines how many builds later the finding reaches the developer.

// placement.mjs — the same scan is placed in the pipeline three different ways: full on
// every build, only the changed endpoints, full nightly. Durations are MODEL minutes, not a measurement.
import { CONFIGURATION, reachableFor, run, REQUEST_MIN, SEED } from "./surface.mjs";

const rng = (t) => { let s = t >>> 0;
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };
const r = rng(SEED + 11);
const BUILDS = 60, NIGHTLY_EVERY = 12;             // nightly run happens every 12 builds
const REACHED = reachableFor(CONFIGURATION.at(-1));
const FULL_MIN = run(CONFIGURATION.at(-1)).min;

// Each build touches a handful of endpoints. A vulnerability is born in one build; the
// build that gives birth to it does not have to touch that endpoint itself, because a
// shared-library change shows up at another endpoint.
const touched = Array.from({ length: BUILDS }, () => new Set(
  Array.from({ length: 1 + Math.floor(r() * 3) }, () => REACHED[Math.floor(r() * REACHED.length)])));
const BORN = REACHED.filter((u) => u.vuln && u.detectable).map((u) => {
  const y = Math.floor(r() * BUILDS);
  if (r() < 0.65) touched[y].add(u);              // the change touches that endpoint directly
  return { endpoint: u, born: y };
});

// A placement says which endpoint set is scanned in which build.
const PLACEMENT = [
  { name: "full on every build", gate: true, scan: (y) => REACHED, runs: () => true },
  { name: "changed endpoints", gate: true, scan: (y) => [...touched[y]], runs: () => true },
  { name: "full nightly", gate: false, scan: (y) => REACHED, runs: (y) => y % NIGHTLY_EVERY === NIGHTLY_EVERY - 1 },
];

function measure(z) {
  const seen = new Set(), delay = [], blockedBuilds = new Set(), falseBuilds = new Set();
  let min = 0;
  for (let y = 0; y < BUILDS; y++) {
    if (!z.runs(y)) continue;
    const set = z.scan(y);
    min += set.reduce((a, u) => a + u.requests, 0) * REQUEST_MIN;
    for (const u of set) {
      if (u.falsePositive && !seen.has(u)) { seen.add(u); falseBuilds.add(y); }
      const d = BORN.find((x) => x.endpoint === u && x.born <= y && !seen.has(x));
      if (d) { seen.add(d); delay.push(y - d.born); blockedBuilds.add(y); }
    }
  }
  const found = delay.length;
  return { name: z.name, min: Math.round(min), found, missed: BORN.length - found,
    avg: found ? (delay.reduce((a, b) => a + b, 0) / found).toFixed(1) : "-",
    worst: found ? Math.max(...delay) : "-",
    blocked: z.gate ? blockedBuilds.size : 0,
    falseBlocked: z.gate ? [...falseBuilds].filter((y) => !blockedBuilds.has(y)).length : 0 };
}

const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0]))).join("");
const y = gen([20, 11, 10, 9, 13, 13, 8, 14]);
console.log(`${BUILDS} builds, full scan ${FULL_MIN} model minutes, ` +
  `${BORN.length} vulnerabilities visible in the pipeline; delay is a build count`);
console.log(y(["placement", "total min", "found", "missed", "avg delay", "worst delay",
  "blocked", "false blocked"]));
for (const z of PLACEMENT) {
  const o = measure(z);
  console.log(y([o.name, o.min, o.found, o.missed, o.avg, o.worst, o.blocked, o.falseBlocked]));
}
const [a, b, c] = PLACEMENT.map(measure);
console.log(`\nfull scan takes ${a.min} minutes per build and delivers ${a.found} vulnerabilities in ` +
  `the same build; nightly scan drops to ${c.min} minutes, ${(100 * (1 - c.min / a.min)).toFixed(1)}% ` +
  `less, and delivers the same ${c.found} vulnerabilities an average of ${c.avg} builds later`);
console.log(`changed-endpoint scan finds ${b.found} vulnerabilities in ${b.min} minutes; ` +
  `${b.missed} vulnerabilities are never found because they sit at an endpoint that is never scanned`);

60 builds, full scan 89 model minutes, 10 vulnerabilities visible in the pipeline; delay is a build count
placement             total min     found   missed    avg delay  worst delay blocked false blocked
full on every build        5316        10        0          0.0            0      10             0
changed endpoints           126         7        3          2.7           19       7             6
full nightly                443        10        0          6.0           11       0             0

full scan takes 5316 minutes per build and delivers 10 vulnerabilities in the same build; nightly scan drops to 443 minutes, 91.7% less, and delivers the same 10 vulnerabilities an average of 6.0 builds later
changed-endpoint scan finds 7 vulnerabilities in 126 minutes; 3 vulnerabilities are never found because they sit at an endpoint that is never scanned

All three rows handle the same scan, the same surface, and the same ten vulnerabilities; where they differ is cost. The full scan on every build delivers all ten vulnerabilities in the very build where they were born — zero delay — and in exchange takes 5,316 model minutes. The nightly run finds the same ten vulnerabilities with the same completeness, takes 91.7 percent fewer minutes out of the pipeline, and never stops a single build. What it pays is written in a single column: an average delay of 6 builds, 11 builds in the worst case. A vulnerability can ship to production for eleven builds after it is born.

The third row is cheap and incomplete. Scanning only the changed endpoints stays at 126 minutes but never finds three of the ten vulnerabilities. The reason is written in the second code block’s comment: a vulnerability is not always born in a change that touches its own endpoint. A change to a shared library can break another endpoint’s behavior; that endpoint never changes again for sixty builds, and the changed-endpoint scan never visits it. A check that looks at the changed file cannot see a vulnerability born at a place that did not change.

The last column holds a surprise. The cheapest placement is the one that blocks the most builds for false positives: 6 builds. The full scan surfaces all seven false positives on its first run; all seven are cleared at once and enter the suppression list. The changed-endpoint scan spreads the same seven findings across weeks, surfacing them one at a time, and each one blocks a separate build. The total false-positive count is the same; the trace it leaves is not.

Summary

• A tool scanning a running application only sees what it can reach: anonymous browsing reaches 13 of 96 endpoints and finds 2 vulnerabilities; adding a session and a recorded flow brings that to 84 endpoints and 10 vulnerabilities.
• The 18 unseen vulnerabilities split into two reasons, and the split carries a decision: 5 because they are unreachable (closed by widening scope), 13 despite being reached (widening scope has no effect).
• All seven of the access-control class’s vulnerabilities are reached, and none is found; the same class closed with zero findings in static analysis too — even the combination of both checks leaves it empty.
• Placement in the pipeline changes not which vulnerabilities are found but the delay and the cost: a full scan on every build costs 5,316 minutes with zero delay; a nightly run costs 443 minutes with an average delay of 6 builds.
• Scanning only the changed endpoints drops to 126 minutes but never sees three of the ten vulnerabilities, because a vulnerability born from a shared-component change sits at an endpoint that is never touched.
• The cost of a false positive depends on placement: the same seven findings are cleared all at once by the full scan, but block six separate builds when cleared by the changed-endpoint scan.

Next Step

Both checks worked from the same assumption: what gets scanned is the code the team wrote. But most of a running process comes from components written elsewhere, and the vulnerabilities in those components are already known — someone has already found them, recorded them, and published them. So the job is not finding, it is matching: there is a dependency graph on one side and a catalog of records on the other, and the question is which record actually affects which component. This match has its own false positives and its own false negatives. The next lesson matches the dependency graph against the known-vulnerability catalog, separates directly and transitively affected components, and counts why a vulnerability in a component that is never invoked belongs in a different bucket.