Web Application Firewall

The previous course scanned the code, inventoried the components, and signed the build output; all of those checks stopped before deployment. What protects the application from the moment it starts running was never asked. This topic answers that question at the request layer, and the first lesson sets up the measure itself.

Runtime is the most expensive stage at which to catch an item: the item is in production, a real subscriber is sending the request, and the defense has milliseconds to decide. For that reason the sentence “a web application firewall should be installed” carries no decision by itself. What carries the decision is three counts, and all three must be read from the same run: how many bad requests were blocked, how many bad requests got through, how many legitimate requests were falsely blocked. When the third is not written down, the first two measure nothing.

What the Rule Set Looks At

The regional measurement network’s subscriber portal and field application stand in the same production environment. A web application firewall is placed in front of them and started with a ready-made rule set: each rule looks for a particular signal in the request and writes a score when it matches, and if the request’s total score reaches the threshold, the request is blocked. Everything below is a model built with node; no real traffic is generated, and no real payload or evasion string is written.

• AD1. The eleven request classes, their daily counts, the five endpoints, and the ten rules belong to the fictional measurement network.
• AD2. The true class of a request (legitimate or bad) is known to the model. In production that class is known only after review; the model assumes it is known so that falsely blocked requests can be counted.
• AD3. The set of signals a class carries is fixed; every request in the same class gets the same score. In reality scores spread across individual requests, and this spread softens the choice of threshold without changing its direction.
• AD4. A rule matches when all the tags it looks for are present. Signal tags name a surface feature the defense sees, not a syntax.
• AD5. The “earlier closing stage” column was marked by hand for each bad class: whether the item could have been closed in design, in code, or at no earlier stage at all.

// requests.mjs — one day of requests and a ready-made rule set; a model, not real traffic
// [name, true class, daily count, signal tags, endpoint, earlier closing stage]
export const CLASSES = [
  ['subscriber query form', 'legit', 84000, [], '/portal/query', '-'],
  ['field app bulk upload', 'legit', 5400, ['long-body', 'encoded-field'], '/field/upload', '-'],
  ['report filter, free text', 'legit', 2600, ['quote', 'keyword'], '/portal/report', '-'],
  ['invoice attachment file name', 'legit', 900, ['path-separator'], '/portal/bill', '-'],
  ['internal monitoring probe', 'legit', 1440, ['tool-signature'], '/health', '-'],
  ['concatenation in query field', 'bad', 320, ['quote', 'keyword', 'comment-marker'], '/portal/query', 'code'],
  ['script tag in form field', 'bad', 210, ['tag-opening', 'event-attribute'], '/portal/query', 'code'],
  ['traversal in attachment name', 'bad', 96, ['path-separator', 'parent-dir'], '/portal/bill', 'code'],
  ['quote sequence under cover', 'bad', 260, ['quote', 'keyword'], '/portal/report', 'code'],
  ['component flaw with no patch', 'bad', 40, ['malformed-body'], '/field/upload', 'none'],
  ['out-of-workflow bulk download', 'bad', 1200, [], '/portal/report', 'design'],
];

// [rule name, tags looked for (all must be present), score]
export const RULES = [
  ['quote and keyword', ['quote', 'keyword'], 3],
  ['comment marker', ['comment-marker'], 2],
  ['tag opening', ['tag-opening'], 3],
  ['event attribute', ['event-attribute'], 2],
  ['path separator', ['path-separator'], 2],
  ['parent directory', ['parent-dir'], 3],
  ['malformed body', ['malformed-body'], 4],
  ['known tool signature', ['tool-signature'], 2],
  ['encoded field', ['encoded-field'], 2],
  ['long body', ['long-body'], 1],
];

// exclusions = [endpoint, rule name] pairs; at that endpoint that rule writes no score at all
export const score = (s, exclusions = []) => RULES.reduce((t, [name, find, p]) =>
  t + (find.every((e) => s[3].includes(e)) && !exclusions.some(([u, k]) => u === s[4] && k === name) ? p : 0), 0);

export function measure(threshold, exclusions = []) {
  let blocked = 0, missed = 0, falsely = 0;
  for (const s of CLASSES) {
    const stopped = score(s, exclusions) >= threshold;
    if (s[1] === 'bad') stopped ? (blocked += s[2]) : (missed += s[2]);
    else if (stopped) falsely += s[2];
  }
  return { blocked, missed, falsely };
}

A rule set arriving ready-made does not mean the rules know this application. The rules look at a syntactic surface, and that surface is one that legitimate work also passes through: a subscriber typing free text into a report filter and a request attempting concatenation in a query field can carry the same two signals. The rule cannot tell them apart, because the difference is not in the shape of the request but in its intent and context; the only thing the rule sees is shape.

The Three Counts Move Together

The threshold is the only real setting of the rule set. In the run below, the rule set, the request set, and the scores stay fixed; the only thing that changes is the threshold.

// run.mjs — the same daily set, only the threshold changes
import { CLASSES, RULES, score, measure } from './requests.mjs';

const s = (x, n) => String(x).padStart(n);
const bad = CLASSES.filter((k) => k[1] === 'bad');
const T = (l) => l.reduce((t, k) => t + k[2], 0);
console.log(`${CLASSES.length} request classes, ${RULES.length} rules; ${T(CLASSES)} requests per day` +
  ` (${T(bad)} bad / ${T(CLASSES) - T(bad)} legit)\n`);
console.log(`${'request class'.padEnd(31)}${s('class', 6)}${s('daily', 7)}${s('score', 6)}` +
  `${s('endpoint', 15)}${s('early stage', 13)}`);
for (const k of CLASSES)
  console.log(`${k[0].padEnd(31)}${s(k[1], 6)}${s(k[2], 7)}${s(score(k), 6)}${s(k[4], 15)}${s(k[5], 13)}`);

console.log(`\n${'threshold'.padEnd(11)}${s('blocked', 11)}${s('missed', 8)}${s('falsely blocked', 16)}` +
  `${s('false/blocked', 19)}`);
for (const e of [6, 5, 4, 3, 2]) {
  const r = measure(e);
  console.log(`${String(e).padEnd(11)}${s(r.blocked, 11)}${s(r.missed, 8)}${s(r.falsely, 16)}` +
    `${s(r.blocked ? (r.falsely / r.blocked).toFixed(1) : '-', 19)}`);
}

11 request classes, 10 rules; 96466 requests per day (2126 bad / 94340 legit)

request class                   class  daily score       endpoint  early stage
subscriber query form           legit  84000     0  /portal/query            -
field app bulk upload           legit   5400     3  /field/upload            -
report filter, free text        legit   2600     3 /portal/report            -
invoice attachment file name    legit    900     2   /portal/bill            -
internal monitoring probe       legit   1440     2        /health            -
concatenation in query field      bad    320     5  /portal/query         code
script tag in form field          bad    210     5  /portal/query         code
traversal in attachment name      bad     96     5   /portal/bill         code
quote sequence under cover        bad    260     3 /portal/report         code
component flaw with no patch      bad     40     4  /field/upload         none
out-of-workflow bulk download     bad   1200     0 /portal/report       design

threshold      blocked  missed falsely blocked      false/blocked
6                    0    2126               0                  -
5                  626    1500               0                0.0
4                  666    1460               0                0.0
3                  926    1200            8000                8.6
2                  926    1200           10340               11.2

The five rows are a tuning curve, and each row has to be read on its own. At threshold 6 the rule set blocks nothing: no class reaches a total score of 6, which means a layer that is installed but inert cannot be told apart from zero in day-to-day measurement. At threshold 5 three classes are blocked and 626 bad requests are cut; falsely blocked requests are zero. Lowering the threshold to 4 adds only 40 requests, and falsely blocked stays zero; four is the setting that costs nothing in this set.

The real decision is in the third row. Lowering the threshold to 3 blocks 260 more bad requests and, in exchange, cuts 8,000 legitimate requests. The ratio is in the last column: 8.6 legitimate requests cut for every bad request blocked. Nor are the ones cut random; they are the field application’s bulk upload and the free text typed into the report filter, which means two real functions of the system. At threshold 2 the blocked bad requests do not increase at all, and the cut requests increase by 2,340: past a certain point, tightening only starts to cut legitimate work.

The missed column never drops below 1,200 at any setting. The reason is not a choice of threshold but a limit of sight: the out-of-workflow bulk download carries no signal at all. The requests are flawless in shape, authenticated, and authorized; the only thing wrong is volume and context. A rule set that looks at syntax cannot see this at any threshold.

The Price of Writing Exclusions

Holding the threshold at 3 and closing the false blocking one case at a time is the most common answer in production: the endpoint that was cut and the rule that cut it are written into an exclusion list, and at that endpoint that rule writes no score. Three exclusions are enough: two rules at the field upload endpoint and the quote rule at the report endpoint.

// exclusions.mjs — the price of closing false blocking at threshold 3 with exclusions
import { CLASSES, score, measure } from './requests.mjs';

const s = (x, n) => String(x).padStart(n);
const EXCLUSIONS = [['/field/upload', 'long body'], ['/field/upload', 'encoded field'],
  ['/portal/report', 'quote and keyword']];

const SETTINGS = [['threshold 4, no exclusions', 4, []], ['threshold 3, no exclusions', 3, []],
  ['threshold 3, 3 exclusions', 3, EXCLUSIONS]];
console.log(`${'setting'.padEnd(27)}${s('blocked', 11)}${s('missed', 8)}${s('falsely blocked', 16)}${s('tracked items', 15)}`);
for (const [name, e, m] of SETTINGS) {
  const r = measure(e, m);
  console.log(`${name.padEnd(27)}${s(r.blocked, 11)}${s(r.missed, 8)}${s(r.falsely, 16)}${s(10 + m.length, 15)}`);
}

console.log(`\nthe surface each exclusion opens at its endpoint:`);
for (const [u, k] of EXCLUSIONS) {
  const passed = CLASSES.filter((x) => x[4] === u && x[1] === 'bad' && score(x) >= 3 && score(x, EXCLUSIONS) < 3);
  console.log(`  ${u.padEnd(15)}${k.padEnd(26)}${passed.length ? `passed: ${passed[0][0]} (${passed[0][2]})` : 'no bad request passed'}`);
}

setting                        blocked  missed falsely blocked  tracked items
threshold 4, no exclusions         666    1460               0             10
threshold 3, no exclusions         926    1200            8000             10
threshold 3, 3 exclusions          666    1460               0             13

the surface each exclusion opens at its endpoint:
  /field/upload  long body                 no bad request passed
  /field/upload  encoded field             no bad request passed
  /portal/report quote and keyword         passed: quote sequence under cover (260)

The first and third rows are identical in all three counts: 666, 1,460, 0. In other words, the threshold tightened with three exclusions gives exactly the same result as the loose threshold with no exclusions written at all. The difference is only in the last column: the number of tracked items has gone from ten to thirteen. Three more items, three separate justifications, three lines to be reviewed again at every endpoint change.

The sublist gives the reason. An exclusion switches off one rule at one endpoint and covers every request that passes through that endpoint. The quote exclusion at the report endpoint rescued the legitimate filter text, and in the same stroke it rescued the 260 bad requests that pass through the same endpoint: the three-point rule no longer writes anything there. The two exclusions at the field upload endpoint did not produce this side effect, because the only bad class at that endpoint is caught by a different rule. The difference between the two cannot be known in advance; it becomes visible only when the set of requests the exclusion covers is counted. An exclusion list is therefore not a setting but an inventory: unless it is tracked separately which rule each line silences at which endpoint, and what else passes through that endpoint, the list becomes, over time, more decisive than the rule set itself.

Where the Blocked Volume Belongs

The first two columns say what this layer does. The third column is the real question of the course: could the blocked volume already have been closed at a cheaper stage.

// stage.mjs — how much of the blocked item could have been closed at a cheaper stage
import { CLASSES, score } from './requests.mjs';

const s = (x, n) => String(x).padStart(n);
const bad = CLASSES.filter((k) => k[1] === 'bad');
const STAGES = ['code', 'design', 'none'];

console.log(`${'threshold'.padEnd(11)}${STAGES.map((a) => s(a, 10)).join('')}${s('only here %', 17)}`);
for (const e of [5, 4, 3]) {
  const d = bad.filter((k) => score(k) >= e);
  const t = d.reduce((x, k) => x + k[2], 0);
  const share = STAGES.map((a) => d.filter((k) => k[5] === a).reduce((x, k) => x + k[2], 0));
  console.log(`${String(e).padEnd(11)}${share.map((p) => s(p, 10)).join('')}` +
    `${s(t ? (100 * share[2] / t).toFixed(1) : '-', 17)}`);
}

console.log(`\nbad classes never blocked at any threshold:`);
for (const k of bad.filter((x) => score(x) < 2))
  console.log(`  ${k[0].padEnd(31)}${s(k[2], 7)}  no signal, earlier stage: ${k[5]}`);
console.log(`\nclasses that can be closed only at runtime:`);
for (const k of bad.filter((x) => x[5] === 'none'))
  console.log(`  ${k[0].padEnd(31)}${s(k[2], 7)}  score ${score(k)}, stops at threshold 4`);

threshold        code    design      none      only here %
5                 626         0         0              0.0
4                 626         0        40              6.0
3                 886         0        40              4.3

bad classes never blocked at any threshold:
  out-of-workflow bulk download     1200  no signal, earlier stage: design

classes that can be closed only at runtime:
  component flaw with no patch        40  score 4, stops at threshold 4

The table shows the layer’s real share. Of the 666 requests blocked at threshold 4, 626, or 94%, come from classes that can be closed at the code stage: query concatenation closes with a parameterized query, the script tag with output encoding, directory traversal with path validation, and each of the three is written once and holds for all requests. This layer catches the same items again every day, on every request.

The remaining 6% is something else. For a flaw in a component that has no patch yet, there is no earlier stage: the item was not in the design, it was not written into the code, and at the moment it showed up in the dependency inventory no fixed release existed. The only thing that blocks those 40 requests is this layer, and that is exactly the reason the rule set exists. The measure of the layer is not the total number of blocks but the number blocked only here.

The sublist tells the opposite direction. The out-of-workflow bulk download is not blocked at any threshold; this item is caught neither by the rules nor by the threshold, because it is an authorization and workflow decision, which means it belongs to the design stage. Even though the rule set never sees this item, the daily dashboard will still show 666 blocks. Having the layer installed does not widen the class of items it covers.

Summary

• A runtime defense is measured with three counts, and all three are read from the same run: bad requests blocked, bad requests missed, legitimate requests falsely blocked. When the third is not written down, the first two measure nothing.
• Lowering the threshold from 5 to 4 raised blocked requests from 626 to 666 and left falsely blocked at zero; going down to 3 blocked 260 more requests and cut 8,000 legitimate requests (8.6 cut per blocked request), and going down to 2 added 2,340 more cuts without adding any gain.
• Threshold 3 with three exclusions gave the same three counts as threshold 4 with no exclusions (666, 1,460, 0); the only difference is that the tracked items went from 10 to 13. One of the exclusions also covered the 260 bad requests passing through the same endpoint.
• Missed requests never dropped below 1,200 at any threshold: the out-of-workflow bulk download carries no signal. A rule that looks at syntax sees shape, not context.
• Of the 666 requests blocked at threshold 4, 626 come from classes that can be closed once at the code stage. The part that can be closed only at runtime is 40 requests per day, which is 6% — a component flaw with no patch.

Next Step

This lesson’s rule set looked at a single request and decided: is there a signal, does the score reach the threshold. The 1,200 missed requests were missed for exactly this reason: none of them was wrong on its own. The wrongness of the out-of-workflow bulk download is not inside the request but in the number and timing of requests coming from the same client. To see it, the defense has to read not a request but a counter: how many requests per window, tied to which key, under which threshold. The next lesson runs that counter in three different window shapes and asks the same three counts there, especially of the second, because the most expensive side of a rate limit is not the bad request it blocks but the request it cuts from legitimate peak load.