Security Headers Coverage

The previous three lessons measured defenses that look at the request: the rule set looked at the shape of the request, the rate counter at the number of requests, and the thresholds at the account and the address. All three decided whether to stop a request or let it through. There is one more decision made at runtime, and it stops no request: the security headers sent along with the response determine what the browser is allowed to do.

What the headers do was measured inside the application in an earlier course and is not repeated here. This lesson’s measure is different: when header configuration is taken out of the application and moved to the edge layer in front of them, how many applications and how many endpoints are covered at once, and what is left outside coverage. This is a defense that produces no “caught requests” count; its measure is coverage, and when coverage is not counted, a configuration file gets confused with real protection.

Where the Header Is Written

The regional measurement network’s production environment is modeled with node; no real server is brought up, and no real response is produced.

• AD16. The fourteen endpoints, nine applications, six headers, and daily request counts belong to the fictional measurement network. Some of the endpoints pass through the shared edge layer, and some are called directly.
• AD17. The “browser surface” field says whether the endpoint returns to a browser a document or a resource the browser loads. Security headers do something only on browser surface; written on an API, they change no behavior.
• AD18. Management at the edge writes all six headers to every endpoint that passes through the edge. On an endpoint that does not pass through the edge, only the header the application wrote itself remains.
• AD19. The content security policy is run in report-only mode: the resources that would be blocked are counted, not actually blocked. Whether the resources are legitimate is known in the model.
• AD20. The “earlier closing stage” column was marked by hand for each header.

// endpoints.mjs — endpoints in production, six security headers, and page resources; a model
// [endpoint, application, path, browser surface?, daily requests, headers the application wrote itself]
export const ENDPOINTS = [
  ['subscriber portal, page', 'subscriber portal', 'edge', true, 84000, 2],
  ['subscriber portal, API', 'subscriber portal', 'edge', false, 41000, 1],
  ['subscriber portal, authentication', 'subscriber portal', 'edge', true, 6800, 2],
  ['field app, page', 'field app', 'edge', true, 12000, 0],
  ['field app, file upload', 'field app', 'edge', false, 5400, 0],
  ['field app, mobile backend', 'field app', 'edge', false, 22000, 0],
  ['billing page', 'billing interface', 'edge', true, 9000, 3],
  ['report page', 'reporting', 'edge', true, 2600, 1],
  ['health probe', 'subscriber portal', 'edge', false, 1440, 0],
  ['admin interface', 'admin interface', 'direct', true, 900, 0],
  ['legacy subscriber portal', 'legacy portal', 'direct', true, 3100, 1],
  ['static asset host', 'asset host', 'direct', true, 62000, 0],
  ['measurement collection API', 'measurement collection', 'direct', false, 150000, 0],
  ['billing service API', 'billing service', 'direct', false, 38000, 0],
];

// [header, class of item it stops, earlier closing stage]
export const HEADERS = [
  ['content security policy', 'unauthorized script execution', 'code'],
  ['framing protection', 'page embedding', 'none'],
  ['transport layer enforcement', 'plaintext connection', 'none'],
  ['content type pinning', 'type sniffing', 'code'],
  ['referrer information policy', 'identifier leaked in transit', 'code'],
  ['browser capability restriction', 'unwanted device access', 'none'],
];

// number of headers written on an endpoint after edge management
export const written = (e) => (e[2] === 'edge' ? HEADERS.length : e[5]);
// a header does something only on browser surface
export const effective = (e) => (e[3] ? written(e) : 0);

// [page, resource, location, legitimate?, daily loads]
export const RESOURCES = [
  ['subscriber portal, page', 'app script', 'same-origin', true, 84000],
  ['subscriber portal, page', 'stylesheet', 'asset-host', true, 84000],
  ['subscriber portal, page', 'inline measurement script', 'inline', true, 84000],
  ['subscriber portal, page', 'extra script from a dependency', 'third-party', false, 84000],
  ['subscriber portal, authentication', 'form script', 'same-origin', true, 6800],
  ['subscriber portal, authentication', 'inline field validation', 'inline', true, 6800],
  ['field app, page', 'map component', 'asset-host', true, 12000],
  ['field app, page', 'inline status script', 'inline', true, 12000],
  ['billing page', 'table script', 'same-origin', true, 9000],
  ['billing page', 'font', 'asset-host', true, 9000],
  ['report page', 'charting library', 'asset-host', true, 2600],
  ['report page', 'injected inline script', 'inline', false, 2600],
];

export const POLICIES = [
  ['same origin only', (r) => r[2] === 'same-origin'],
  ['plus asset host', (r) => ['same-origin', 'asset-host'].includes(r[2])],
  ['plus inline, blanket', (r) => ['same-origin', 'asset-host', 'inline'].includes(r[2])],
  ['plus inline, hashed', (r) => ['same-origin', 'asset-host'].includes(r[2]) ||
    (r[2] === 'inline' && r[3])],
];

How Coverage Is Counted

// coverage.mjs — endpoints, requests, and cells covered by managing headers at the edge
import { ENDPOINTS, HEADERS, written, effective } from './endpoints.mjs';

const s = (x, n) => String(x).padStart(n);
const T = (f) => ENDPOINTS.filter(f).reduce((t, e) => t + e[4], 0);
const N = (f) => ENDPOINTS.filter(f).length;
console.log(`${ENDPOINTS.length} endpoints, ${new Set(ENDPOINTS.map((e) => e[1])).size} applications, ${HEADERS.length} headers` +
  ` = ${ENDPOINTS.length * HEADERS.length} cells; ${T(() => true)} requests per day\n`);

console.log(`${'endpoint'.padEnd(33)}${s('path', 10)}${s('browser', 10)}${s('daily', 8)}` +
  `${s('own written', 15)}${s('after edge', 16)}${s('effective', 11)}`);
for (const e of ENDPOINTS)
  console.log(`${e[0].padEnd(33)}${s(e[2], 10)}${s(e[3] ? 'yes' : 'no', 10)}${s(e[4], 8)}` +
    `${s(e[5], 15)}${s(written(e), 16)}${s(effective(e), 11)}`);

const after = ENDPOINTS.reduce((t, e) => t + written(e), 0), eff = ENDPOINTS.reduce((t, e) => t + effective(e), 0);
console.log(`\n${'measure'.padEnd(38)}${s('before', 8)}${s('after edge', 16)}`);
for (const [a, b, c] of [
  ['written header cells', ENDPOINTS.reduce((t, e) => t + e[5], 0), after],
  ['effective cells on browser surface', ENDPOINTS.reduce((t, e) => t + (e[3] ? e[5] : 0), 0), eff],
  ['endpoints writing headers', N((e) => e[5] > 0), N((e) => written(e) > 0)],
]) console.log(`${a.padEnd(38)}${s(b, 8)}${s(c, 16)}`);

console.log(`\n${'how coverage is counted'.padEnd(38)}${s('covered', 10)}${s('total', 8)}${s('%', 8)}`);
for (const [a, b, c] of [
  ['by endpoint count', N((e) => e[2] === 'edge'), ENDPOINTS.length],
  ['by request volume', T((e) => e[2] === 'edge'), T(() => true)],
  ['by request volume on browser surface', T((e) => e[2] === 'edge' && e[3]), T((e) => e[3])],
  ['by browser cell count', eff, N((e) => e[3]) * HEADERS.length],
]) console.log(`${a.padEnd(38)}${s(b, 10)}${s(c, 8)}${s((100 * b / c).toFixed(1), 8)}`);

console.log(`\nbrowser surface left outside coverage (what bringing it to the edge would gain):`);
for (const e of ENDPOINTS.filter((x) => x[3] && x[2] !== 'edge'))
  console.log(`  ${e[0].padEnd(24)}${s(e[4], 8)} requests/day, ${s(HEADERS.length - e[5], 2)} cells gained`);
console.log(`\nconfigured but ineffective cells (on endpoints without a browser surface): ${after - eff}`);

14 endpoints, 9 applications, 6 headers = 84 cells; 438240 requests per day

endpoint                               path   browser   daily    own written      after edge  effective
subscriber portal, page                edge       yes   84000              2               6          6
subscriber portal, API                 edge        no   41000              1               6          0
subscriber portal, authentication      edge       yes    6800              2               6          6
field app, page                        edge       yes   12000              0               6          6
field app, file upload                 edge        no    5400              0               6          0
field app, mobile backend              edge        no   22000              0               6          0
billing page                           edge       yes    9000              3               6          6
report page                            edge       yes    2600              1               6          6
health probe                           edge        no    1440              0               6          0
admin interface                      direct       yes     900              0               0          0
legacy subscriber portal             direct       yes    3100              1               1          1
static asset host                    direct       yes   62000              0               0          0
measurement collection API           direct        no  150000              0               0          0
billing service API                  direct        no   38000              0               0          0

measure                                 before      after edge
written header cells                        10              55
effective cells on browser surface           9              31
endpoints writing headers                    6              10

how coverage is counted                  covered   total       %
by endpoint count                              9      14    64.3
by request volume                         184240  438240    42.0
by request volume on browser surface      114400  180400    63.4
by browser cell count                         31      48    64.6

browser surface left outside coverage (what bringing it to the edge would gain):
  admin interface              900 requests/day,  6 cells gained
  legacy subscriber portal    3100 requests/day,  5 cells gained
  static asset host          62000 requests/day,  6 cells gained

configured but ineffective cells (on endpoints without a browser surface): 24

The gain of edge management is written in the first summary table, and it is not small: written header cells rise from 10 to 55, so a single configuration change fills 45 cells at once. Inside the application, this work would have been done nine times in nine separate codebases, and five applications had never done it. This is the real advantage of the edge layer: coverage is taken out of decisions the application teams would each make separately.

The second table gives the same coverage with four separate counts, and the four produce different numbers: 64.3% by endpoint count, 42.0% by request volume, 63.4% by request volume on browser surface, and 64.6% by browser cell count. Four different coverage rates can be written for the same layer, and which one is reported is a choice. The request volume count comes out low because the measurement collection API carries 150,000 requests a day, yet that endpoint is not called from a browser; on that endpoint nothing changes whether a header is present or not. So the meaningful count is the third: how much of the request volume on browser surface is covered.

The lower list gives the real finding. Three endpoints are left outside coverage, and one of them is the static asset host, which carries 62,000 requests a day. Because it does not appear as an “application” in the application inventory, it is never counted in coverage tables; yet the browser loads resources from it, and those resources carry no header. The admin interface and the legacy subscriber portal add up to 4,000 requests, which is 6% of the volume outside coverage. This list sets the order of the work of widening coverage, and an endpoint that never entered the inventory is the largest item in line.

The last line counts the flaw in the opposite direction: 24 cells are configured but ineffective. All six headers are written on the API endpoints, they show green in the audit list, and they change no behavior. The number of configurations is not a measure of protection.

The Page a Strict Policy Breaks

Coverage produces no false positive; the content of the header does. When the content security policy is tightened, what gets blocked is not only the unauthorized script but the page’s own resources.

// policy.mjs — run the content security policy in report-only mode and count the legitimate resources it breaks
import { ENDPOINTS, HEADERS, RESOURCES, POLICIES, written } from './endpoints.mjs';

const s = (x, n) => String(x).padStart(n);
const covered = new Set(ENDPOINTS.filter((e) => e[3] && written(e) === HEADERS.length).map((e) => e[0]));
const R = RESOURCES.filter((r) => covered.has(r[0]));
console.log(`policy applies only on the ${covered.size} browser endpoints managed from the edge;` +
  ` these endpoints hold ${R.length} resources\n`);

console.log(`${'policy'.padEnd(24)}${s('blocked legit', 18)}${s('broken pages', 15)}` +
  `${s('blocked bad', 17)}${s('passed bad', 12)}`);
for (const [name, allow] of POLICIES) {
  const blocked = R.filter((r) => !allow(r)), m = blocked.filter((r) => r[3]);
  console.log(`${name.padEnd(24)}${s(m.length, 18)}${s(new Set(m.map((r) => r[0])).size, 15)}` +
    `${s(blocked.filter((r) => !r[3]).length, 17)}${s(R.filter((r) => allow(r) && !r[3]).length, 12)}`);
}

const inline = R.filter((r) => r[2] === 'inline' && r[3]);
const allow2 = POLICIES[1][1], afterCode = R.filter((r) => r[2] !== 'inline' || !r[3]);
console.log(`\ninline scripts that must be written to the hash list: ${inline.length}`);
console.log(`with inline scripts removed at the code stage, the "plus asset host" policy:` +
  ` blocked legit ${afterCode.filter((r) => r[3] && !allow2(r)).length},` +
  ` passed bad ${afterCode.filter((r) => !r[3] && allow2(r)).length}`);

console.log(`\n${'header'.padEnd(33)}${'item stopped'.padEnd(32)}${s('earlier stage', 15)}`);
for (const [name, item, stage] of HEADERS)
  console.log(`${name.padEnd(33)}${item.padEnd(32)}${s(stage, 15)}`);
const none = HEADERS.filter((h) => h[2] === 'none');
console.log(`\nheaders that can be closed only at this layer ${none.length}/${HEADERS.length};` +
  ` endpoints where they are effective ${ENDPOINTS.filter((e) => e[3] && written(e) === HEADERS.length).length},` +
  ` ${ENDPOINTS.filter((e) => e[3] && written(e) === HEADERS.length).reduce((t, e) => t + e[4], 0)} requests per day`);

policy applies only on the 5 browser endpoints managed from the edge; these endpoints hold 12 resources

policy                       blocked legit   broken pages      blocked bad  passed bad
same origin only                         7              5                2           0
plus asset host                          3              3                2           0
plus inline, blanket                     0              0                1           1
plus inline, hashed                      0              0                2           0

inline scripts that must be written to the hash list: 3
with inline scripts removed at the code stage, the "plus asset host" policy: blocked legit 0, passed bad 0

header                           item stopped                      earlier stage
content security policy          unauthorized script execution              code
framing protection               page embedding                             none
transport layer enforcement      plaintext connection                       none
content type pinning             type sniffing                              code
referrer information policy      identifier leaked in transit               code
browser capability restriction   unwanted device access                     none

headers that can be closed only at this layer 3/6; endpoints where they are effective 5, 114400 requests per day

The four rows are a curve of strictness. The strictest policy blocks two bad resources, and alongside them it blocks seven legitimate resources, breaking all five pages; every browser endpoint covered is broken. Once the asset host is allowed, the broken pages fall to three, and the bad resources are still two. The third row is the escape route seen most often in production: inline scripts are allowed wholesale, broken pages fall to zero, and the injected inline script on the report page passes with the same permission. This setting, which zeroes the broken pages, drops the bad resources the policy blocks from two to one.

The fourth row gives both at once: when inline scripts are allowed through a hash list, broken pages are zero and the bad resources blocked are two. The price is the list itself. Three inline scripts have to be written to the hash list, and if the list is not updated when one of these scripts changes, the page breaks. The policy is no longer a header value but an inventory that has to be checked at every release of the code, a burden of the same kind as the exclusion list in the first lesson.

One line below answers the course’s question. When inline scripts are removed at the code stage, the “plus asset host” policy blocks zero legitimate resources and lets zero bad resources through: no hash list is needed. The header gives the same result only in return for continuous maintenance. The lower table spreads this distinction across the six headers: the item stopped by three of them can be closed at the code stage, and three have no earlier stage. These three determine the browser’s behavior and cannot be stated from application code; this is the share only this layer still covers, and it holds on five endpoints, at 114,400 requests a day.

Summary

• Managing headers at the edge raised written cells from 10 to 55 and endpoints writing headers from 6 to 10; coverage no longer depends on the separate decisions of the application teams.
• The same coverage gives four different rates with four separate counts: 64.3% by endpoint count, 42.0% by request volume, 63.4% by request volume on browser surface, and 64.6% by browser cell count. The meaningful one is the third; the others also count endpoints the browser never sees.
• Three browser endpoints remained outside coverage, and of their 66,000 requests, 62,000 belong to the static asset host that does not appear as an “application” in the inventory. On the other side, 24 cells are configured but ineffective: headers are written on API endpoints and do nothing.
• At its strictest setting, the content security policy blocked 2 bad resources and also blocked 7 legitimate resources, breaking all 5 pages. Allowing inline scripts wholesale zeroed the breakage but dropped the bad resources blocked from 2 to 1; the hash list gave both (0 broken, 2 blocked) and in return brought a maintenance inventory of 3 lines.
• When inline scripts are removed at the code stage, the policy gives 0 broken and 0 bad resources passed, without a hash list. The item stopped by three of the six headers closes at the code stage; the three headers that can be closed only at this layer hold on 5 endpoints, at 114,400 requests a day.

Next Step

This lesson’s measure was coverage, and coverage was provided by the values sent along with a response. The previous three lessons looked at the shape, the number, and the identity of the request. The limit shared by all four is the same: the decision is made on a single request or a single response. Yet the item that no setting has been able to stop since the first lesson, the out-of-workflow bulk download, looked flawless in each individual request; the only thing wrong was the shape the requests of the same session drew together. The next lesson takes the defense out of the request and looks at the behavior of the session, flags the session that deviates from the baseline with an anomaly score, and puts the main question there: how many minutes after the deviation began.