External Report Intake

Up to this point all the logs and alerts have come from within the organization itself. A vulnerability in an application is often heard of from somewhere else: someone looking from the outside finds it and reports it. The external report intake and bug bounty program is the structure these reports enter. The reporter’s process was measured in the Security Tooling and Penetration Testing course; it is not repeated here, it becomes an input. This lesson measures the side that operates the flow, and its unit is the day: how many days from report to verification, how many days from verification to fix, how many of the incoming reports are duplicates.

• DR22. The report flow is fictional: 340 reports in 180 days, 120 vulnerabilities, five finding classes. The numbers come from a generator whose seed is visible.
• DR23. The same vulnerability is reported more than once. The first report of a vulnerability is unique, the later ones are duplicates; a report that falls under no vulnerability is out of scope.
• DR24. The triage queue works in arrival order; the capacity is consumed by the number of reports, not by their kind.
• DR25. Verification is done in two ways: if the required fields are in a single joinable set, from the log in one day; if not, by manual reproduction in three to nine days depending on the class.
• DR26. Every report raises a second question: has this vulnerability been exploited before. The log reaches back retention days from the verification day, and the time spent in the queue is subtracted from this window. There are three outcomes: definite, partial (it was possible to look back only as far as the window) and unanswered (the field does not exist).
• DR27. The schemas and the retention days come as data from the model of the earlier lessons. The reward is a fictional unit; the durations are not a measurement made on this page.

Incoming Reports, Duplicates and the Triage Queue

// report.mjs — the external report intake and bug bounty program are measured from the side that
// operates the flow: how many reports arrive, how many are duplicates, how many days pass until
// triage. IT IS A MODEL.
export const SEED = 20260611, VULN_COUNT = 120, REPORT_COUNT = 340, PERIOD = 180, TRIAGE = 2;
const rng = (t) => { let s = t >>> 0;
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };
const r = rng(SEED);

// visibility: how many times the vulnerability will be reported. fields: the log fields the
// exploitation question needs.
export const CLASSES = [
  { name: "configuration", share: 0.26, visibility: 3.4, manual: 3, fix: 2, reward: 8,
    fields: ["path", "status"] },
  { name: "input validation", share: 0.21, visibility: 2.2, manual: 5, fix: 3, reward: 15,
    fields: ["trace", "path", "method"] },
  { name: "authorization bypass", share: 0.20, visibility: 1.0, manual: 8, fix: 6, reward: 40,
    fields: ["actor", "action", "authz_decision"] },
  { name: "data leak", share: 0.18, visibility: 1.4, manual: 9, fix: 8, reward: 55,
    fields: ["actor", "action", "object"] },
  { name: "session management", share: 0.15, visibility: 0.8, manual: 7, fix: 5, reward: 30,
    fields: ["session", "action"] },
];
const pickClass = (u) => { let t = 0;
  for (const s of CLASSES) { t += s.share; if (u < t) return s; } return CLASSES.at(-1); };

// Vulnerability = the real item behind a report; the same vulnerability is reported in proportion
// to its visibility.
export const VULNS = Array.from({ length: VULN_COUNT }, () => {
  const s = pickClass(r());
  return { cls: s.name, weight: s.visibility * (0.3 + 1.4 * r()),
    age: Math.round(8 + 340 * r() ** 1.6) };
});
const TOTAL = VULNS.reduce((t, a) => t + a.weight, 0);
const pickVuln = (u) => { let t = 0;
  for (let i = 0; i < VULN_COUNT; i++) { t += VULNS[i].weight / TOTAL; if (u < t) return i; }
  return VULN_COUNT - 1; };

// When the channel opens, the volume piles up at the front: arrival day PERIOD * u^2.2. 18% is out of scope.
export const REPORTS = Array.from({ length: REPORT_COUNT }, (_, i) => ({
  i, day: Math.floor(PERIOD * r() ** 2.2), vuln: r() < 0.18 ? null : pickVuln(r()),
})).sort((a, b) => a.day - b.day);
const seen = new Set();
for (const b of REPORTS) {
  b.kind = b.vuln === null ? "out-of-scope" : seen.has(b.vuln) ? "duplicate" : "unique";
  seen.add(b.vuln);
}
export const count = (x) => REPORTS.filter((b) => b.kind === x).length;

// Triage queue: `rate` items per day, in arrival order; the capacity is consumed by the count, not the kind.
export function triage(rate = TRIAGE, write = false) {
  const queue = [], reviewed = new Array(REPORT_COUNT);
  let peak = 0, j = 0;
  for (let g = 0; j < REPORT_COUNT || queue.length; g++) {
    while (j < REPORT_COUNT && REPORTS[j].day === g) queue.push(REPORTS[j++]);
    peak = Math.max(peak, queue.length);
    for (let k = 0; k < rate && queue.length; k++) {
      const b = queue.shift();
      reviewed[b.i] = g;
      if (write) b.reviewed = g;
    }
  }
  return { peak, last: Math.max(...reviewed),
    wait: REPORTS.reduce((t, b) => t + (reviewed[b.i] - b.day), 0) / REPORT_COUNT };
}

export const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0]))).join("");

if (import.meta.url === `file://${process.argv[1]}`) {
  const noise = count("duplicate") + count("out-of-scope");
  console.log(`model (seed ${SEED}): ${REPORT_COUNT} reports in ${PERIOD} days, ${VULN_COUNT} vulnerabilities, ` +
    `${REPORTS.filter((b) => b.day < 30).length} reports in the first 30 days`);
  console.log(`unique ${count("unique")}, duplicate ${count("duplicate")}, out-of-scope ` +
    `${count("out-of-scope")}: ${((noise / REPORT_COUNT) * 100).toFixed(0)}% are not new vulnerabilities`);
  const y = gen([22, 10, 11, 11, 10]);
  console.log("\n" + y(["class", "reports", "unique", "duplicate", "mean age"]));
  for (const s of CLASSES) {
    const l = REPORTS.filter((b) => b.vuln !== null && VULNS[b.vuln].cls === s.name);
    const be = l.filter((b) => b.kind === "unique");
    console.log(y([s.name, l.length, be.length, l.length - be.length,
      (be.reduce((a, b) => a + VULNS[b.vuln].age, 0) / be.length).toFixed(0)]));
  }
  const z = gen([10, 9, 15, 12]);
  console.log("\n" + z(["capacity", "peak", "mean wait", "last review"]));
  for (const rate of [1, 2, 3, 5]) {
    const t = triage(rate);
    console.log(z([`${rate}/day`, t.peak, t.wait.toFixed(1), t.last]));
  }
}

model (seed 20260611): 340 reports in 180 days, 120 vulnerabilities, 149 reports in the first 30 days
unique 91, duplicate 190, out-of-scope 59: 73% are not new vulnerabilities

class                    reports     unique  duplicate  mean age
configuration                163         35        128       151
input validation              44         20         24       120
authorization bypass          35         17         18       153
data leak                     24         11         13       136
session management            15          8          7       108

capacity       peak      mean wait last review
1/day           174          116.1         339
2/day           104           31.2         177
3/day            70           10.4         177
5/day            49            2.3         177

Of the three hundred and forty reports, ninety-one carry a new vulnerability. The remaining 249 items — 190 duplicates and 59 out of scope — are 73% of the queue. This rate comes not from the channel’s poor quality but from its structure: the channel is open to everyone, everyone who sees the same vulnerability reports it, and who reported earlier is not visible from the outside.

The class table writes down where the pile-up is. The configuration class produces 163 reports and only 35 of them are a new vulnerability; in the session management class there are 15 reports and 8 unique vulnerabilities. The share of a finding class in the channel measures not how many vulnerabilities that class leaves in the organization, but how readily it is seen from the outside. The composition of the queue is a photograph not of the organization’s vulnerability distribution but of the external visibility distribution.

The last column prepares the second half of this lesson. The mean age of the reported vulnerabilities is between 108 and 153 days: when a vulnerability is reported, it has already been there for months. The question “has this vulnerability been exploited before” requires looking back over a history longer than a hundred days; the retention windows of the earlier lessons were between 45 and 65 days.

The queue table gives the third number. At a triage capacity of two items per day the queue rises to 104 items and a report waits 31.2 days on average. When capacity drops to one, the last report is reviewed on day 339: the period is 180 days, so the channel cannot catch up with its own period. The cause of the pile-up is not the mean volume; 149 reports arrive in the first 30 days, and 44% of the period total falls in the first sixth. Opening the channel creates a load curve; when capacity is chosen according to the mean arrival, it clogs at the peak, and the clog is not one-time but permanent.

Verification, the Exploitation Question and the Fix Queue

// verification.mjs — unique reports are run against four log schemas: is verification done from
// the log or manually, is the exploitation question answered, when is the fix. IT IS A MODEL.
import { REPORTS, VULNS, CLASSES, triage, count, gen, REPORT_COUNT } from "./report.mjs";

const K1 = ["time", "trace", "actor", "action", "outcome", "path", "method", "status", "client_ip"];
const K2 = ["time", "table", "object", "operation"];
const EXTRA = ["authz_decision", "version", "session", "old_value"];
export const SCHEMAS = [
  { name: "base", retention: 65, factor: 1.00, extra: 0, sets: [K1, K2] },
  { name: "+trace", retention: 64, factor: 1.02, extra: 1, sets: [[...K1, ...K2]] },
  { name: "+decision+version", retention: 62, factor: 1.04, extra: 3,
    sets: [[...K1, ...K2, ...EXTRA.slice(0, 2)]] },
  { name: "+session+value", retention: 45, factor: 1.45, extra: 5, sets: [[...K1, ...K2, ...EXTRA]] },
];
export const CAPACITY = 2;   // work-days set aside per day for fixes
const hasFields = (schema, a) => schema.sets.some((c) => a.every((x) => c.includes(x)));

// Verification is 1 day from the log; without a record it is the class's manual reproduction days. In
// the exploitation question the log reaches back `retention` days; the `duration` spent in the queue
// is subtracted from the window.
export function run(schema, rate = 2) {
  triage(rate, true);
  const items = REPORTS.filter((b) => b.kind === "unique").map((b) => {
    const a = VULNS[b.vuln], s = CLASSES.find((x) => x.name === a.cls);
    const fromLog = hasFields(schema, s.fields);
    const duration = b.reviewed - b.day + (fromLog ? 1 : s.manual);
    return { b, s, fromLog, duration, verified: b.day + duration,
      exploit: !fromLog ? "unanswered" : a.age <= schema.retention - duration ? "definite" : "partial" };
  }).sort((x, y) => x.verified - y.verified);

  // Fix queue: CAPACITY work-days are added to the pool each day, the one with the higher reward takes its turn first.
  const waiting = [];
  let j = 0, pool = 0;
  for (let g = items[0].verified; j < items.length || waiting.length; g++) {
    while (j < items.length && items[j].verified <= g) waiting.push(items[j++]);
    pool = waiting.length ? pool + CAPACITY : CAPACITY;
    waiting.sort((x, y) => y.s.reward - x.s.reward);
    while (waiting.length && pool >= waiting[0].s.fix)
      { const k = waiting.shift(); pool -= k.s.fix; k.fixed = g; }
  }
  const n = items.length, sum = (f) => items.reduce((t, k) => t + f(k), 0);
  const tally = (x) => items.filter((k) => k.exploit === x).length;
  return { n, items, fromLog: items.filter((k) => k.fromLog).length,
    verifyDays: sum((k) => k.duration) / n, totalDays: sum((k) => k.fixed - k.b.day) / n,
    fixDays: sum((k) => k.fixed - k.verified) / n, reward: sum((k) => k.s.reward),
    definite: tally("definite"), partial: tally("partial"), unanswered: tally("unanswered") };
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const y = gen([18, 11, 10, 14, 12, 10, 9, 12]);
  console.log(y(["schema", "retention", "from log", "verification", "fix", "definite", "partial",
    "unanswered"]));
  const results = SCHEMAS.map((s) => {
    const o = run(s);
    console.log(y([s.name, `${s.retention} days`, `${o.fromLog}/${o.n}`,
      `${o.verifyDays.toFixed(1)} days`, `${o.fixDays.toFixed(1)} days`,
      o.definite, o.partial, o.unanswered]));
    return o;
  });
  const [a, , c, d] = results, h5 = SCHEMAS.map((s) => run(s, 5));
  const el = a.items.filter((k) => !k.fromLog), o = (l) => l.reduce((t, k) => t + k.s.reward, 0);
  console.log(`\nfrom report to fix rises from ${a.totalDays.toFixed(1)} days to ` +
    `${d.totalDays.toFixed(1)} days; the definite answer peaks at ${c.definite} in the ${SCHEMAS[2].name} schema ` +
    `and is ${d.definite} in the ${SCHEMAS[3].name} schema`);
  console.log(`when capacity rises from 2 to 5, definite answers are ${h5.map((x) => x.definite).join("/")} ` +
    `instead of ${results.map((x) => x.definite).join("/")}: without adding a field, ` +
    `${h5[3].definite - d.definite} more reports get a definite answer`);
  console.log(`the reward of the ${el.length} manually verified reports is ${o(el)} fictional units, ` +
    `of the ${a.n - el.length} verified from the log ${a.reward - o(el)}`);
}

schema              retention  from log  verification         fix  definite  partial  unanswered
base                  65 days     55/91     28.6 days   57.1 days        15       40          36
+trace                64 days     66/91     27.6 days   60.8 days        18       48          25
+decision+version     62 days     83/91     26.3 days   63.6 days        20       63           8
+session+value        45 days     91/91     25.8 days   64.2 days        13       78           0

from report to fix rises from 85.7 days to 89.9 days; the definite answer peaks at 20 in the +decision+version schema and is 13 in the +session+value schema
when capacity rises from 2 to 5, definite answers are 18/21/24/24 instead of 15/18/20/13: without adding a field, 11 more reports get a definite answer
the reward of the 36 manually verified reports is 1525 fictional units, of the 55 verified from the log 580

The verification column looks like a triumph of log design: the reports verified from the log rise from 55 to 91, and the mean verification falls from 28.6 days to 25.8 days. The gain is 2.8 days. The same schemas had shortened the response 4.17 times in the previous lesson; here the ratio is 1.11, because the dominant term of the duration is not the verification work but the wait in the queue. A log field that shortens a job shortens the total time, when the job is behind a queue, only as much as the queue leaves it.

The fix column goes the opposite way: from 57.1 days to 64.2 days, and the total time from report to fix from 85.7 to 89.9 days. As verification speeds up, items enter the fix queue earlier, while the queue capacity is fixed; the day that was gained is carried to the next queue and waited out there. The bottleneck is not the log design but the fix capacity of two work-days a day.

The last three columns write the measure of this course. The unanswered questions fall from 36 to zero as the schema widens: in the widest schema the exploitation question can be asked for all 91 reports. But the definite answer rises from 15 to 20 and then falls to 13, below even the base schema. The widest schema leaves no question unanswered and gives the fewest definite answers.

The mechanism is in DR26: the condition of a definite answer is that the age of the vulnerability fits in the window of retention minus the elapsed time. In the widest schema the retention is 45 and the mean duration is 25.8 days; a window of about 19 days remains, while the mean age of the reported vulnerabilities is between 108 and 153 days. Every day spent in the queue is subtracted from the retention window. This is the bill, in the external report flow, of the design that trades volume against retention: in return for 5 extra fields, 1.45 times the daily volume and 20 days shorter retention, the definite answer falls by two units.

The third output line plays the same number with a second lever. When the triage capacity is raised from two to five, the definite-answer series becomes 18/21/24/24 instead of 15/18/20/13; this increase in the widest schema is gained without adding any log field and without paying any volume. Triage capacity looks like a service-level decision; when measured, it turns out to be a logging decision.

The reward side writes the last column of the cost. In the base schema the reward total of the 36 manually verified reports is 1,525 fictional units, and that of the 55 verified from the log is 580 units. The high-reward classes are the ones that cannot be verified from the log; the cheap and often-reported configuration class is closed in a day. The verification load and the reward budget go in the same direction: the most expensive findings are the slowest to verify. The 249 items that receive no reward consume 73% of the triage capacity; the real cost of the bounty program is not in the unit paid but in the line held by the items that are not paid.

Summary

• The external report intake is not a detection pipeline but a queue: in the model, of 340 reports, 91 carry a new vulnerability, 190 are duplicates and 59 are out of scope.
• The composition of the queue reflects not the organization’s vulnerability distribution but the external visibility distribution: the configuration class fills half of the channel with 163 reports, behind which there are only 35 vulnerabilities.
• Capacity cannot be chosen according to the mean arrival: 149 reports arrive in the first 30 days, at a capacity of two items per day the queue rises to 104 items and the mean wait becomes 31.2 days.
• Adding log fields lowers verification from 28.6 days to 25.8 days, but the time from report to fix rises from 85.7 days to 89.9 days: the day that was gained is carried to the fix queue, whose capacity is fixed.
• The days spent in the queue are subtracted from the retention window; while the widest schema lowers the unanswered questions to zero, it lowers the definite answer from 20 to 13, below the 15 of the base schema.
• The cost is measured on two levers: while 5 extra fields, 1.45 times the volume and 20 days of shorter retention reduce the definite answer, raising triage capacity from two to five carries the same number to 24 without adding a field.

Next Step

Throughout this topic a log was designed, an alert threshold was chosen, a breach was responded to, the trace of short-lived workloads was measured and an external report flow was set up. Every lesson asked the same question: how many of the questions to be asked after the incident can be answered, and what does it cost to make them answerable. One question remains, and the organization itself does not ask it: what will show someone looking from the outside that all this practice is being done. The next topic measures this — how many practices an audit control item corresponds to in engineering terms, for how many of them the evidence is generated automatically in the pipeline, and how many are collected by hand after the incident.