---
title: 'Server Security and Going to Production'
source: 'https://academia.sh/en/courses/backend-production'
language: en
updated: '2026-08-23T07:00:25+00:00'
license: 'CC BY-SA 4.0'
lessons: 16
outcomes:
- 'Builds defense layers at both the application and the server level'
- 'Applies a zero-downtime release strategy'
- 'Grounds scaling decisions in measurement'
---
# Server Security and Going to Production
The point where the application opens to the outside: enforcing the untrusted-data boundary with input validation and output encoding, the concatenation decision at the root of injection-class vulnerabilities, configuring cross-origin access correctly on the server side, transport layer hardening, storing, rotating, and restricting access to a secret, dependency risk, the audit log, abuse defense, reverse proxy configuration, process managers, the cost difference between the two scaling axes, stateless design, the effect of carrying a session, zero-downtime release, the ordering of backward-compatible migrations, and resource estimation from load test data.
## Application Security
Where untrusted data meets the application: enforcing validation and encoding at the boundary, the string-concatenation decision at the root of injection-class vulnerabilities, configuring cross-origin access correctly on the server, security headers and transport-layer hardening, storing and rotating secrets, managing dependency risk with a version policy, the audit log for traceability, and abuse defense against bots, brute force, and enumeration.
### Input Validation and Encoding
Source: https://academia.sh/en/courses/backend-production/input-validation-and-encoding
Enforcing the untrusted-data boundary: the false-accept and false-reject counts of the allowlist versus the blocklist over a set of twenty-two requests, the wasted-side-effect cost of stopping validation at the endpoint, the service, or the store, the number of values a single encoder breaks across three output contexts, and the field a single-line setting silently opens.
The Observability and Reliability course made the loan system visible and made it withstand
failure. Across that course every incoming request was valid: the load generator sent bodies
matching the contract, and the chaos trials broke the infrastructure, not the request. How far a
request that does **not** conform to the contract can travel inside the system was never
measured.
This lesson's subject is that boundary. The split loan system has three external endpoints, and
every body arriving at them is **untrusted data** until proven otherwise. The vulnerabilities are
not covered — the client side belongs to Frontend Quality, the scanning side to Non-Functional
Testing — here, **the implementation of the defense** is measured.
#### The Untrusted-Data Boundary
The contract is written first: which endpoint accepts which field, in which shape. Whether each
request should be accepted or rejected comes from the loan system's rules and is labeled by hand.
**AS1 (assumption):** the labels are complete and correct; the counts are relative to this label
set and say nothing about a request shape outside it.
```js
// requests.mjs — the loan system's three external endpoints, field contract, and hand-labeled request set
const O = "POST /loans", K = "GET /catalog", N = "POST /member/note";
export const SCHEMA = {
[O]: { memberId: { t: "str", pattern: /^U-\d{5}$/, required: 1 },
isbn: { t: "str", pattern: /^\d{13}$/, required: 1 },
days: { t: "num", min: 1, max: 30 } },
[K]: { q: { t: "str", max: 64, required: 1 },
sort: { t: "str", options: ["title", "author", "year"] },
page: { t: "num", min: 1, max: 500 } },
[N]: { memberId: { t: "str", pattern: /^U-\d{5}$/, required: 1 },
note: { t: "str", max: 120, required: 1 } },
};
// [endpoint, body, legitimate?] — the legitimate value is a hand-assigned decision, not measured.
export const REQUESTS = [
[O, { memberId: "U-10432", isbn: "9789750718533", days: 21 }, 1],
[O, { memberId: "U-10432", isbn: "9789750718533" }, 1],
[K, { q: "lost time" }, 1],
[K, { q: "O'Neill", sort: "author" }, 1],
[K, { q: "pencil", page: 3 }, 1],
[N, { memberId: "U-10432", note: "cover worn" }, 1],
[N, { memberId: "U-77120", note: "5 < 7 pages missing on return" }, 1],
[O, { memberId: "U-00001", isbn: "9786050000000", days: 1 }, 1],
[K, { q: "history -- 20th century" }, 1],
[N, { memberId: "U-31005", note: "book; left with staff" }, 1],
[O, { memberId: "U-10432", isbn: "978-975-07-1853-3" }, 1], // hyphenated legitimate form
[O, { memberId: "u-10432", isbn: "9789750718533" }, 0], // lowercase
[O, { memberId: "U-10432", isbn: "978975071853" }, 0], // twelve digits
[O, { memberId: "U-10432", isbn: "9789750718533", days: 400 }, 0],
[O, { memberId: "U-10432", isbn: "9789750718533", days: "21" }, 0],
[O, { memberId: ["U-10432"], isbn: "9789750718533" }, 0],
[O, { memberId: "U-10432", isbn: "9789750718533", penalty: 0 }, 0],
[K, { q: "k".repeat(200) }, 0],
[K, { q: "pencil", sort: "book.internal_note" }, 0],
[K, { q: "pencil", page: -2 }, 0],
[N, { memberId: "U-10432" }, 0],
[N, { memberId: "U-10432", note: "ok", role: "admin" }, 0],
].map(([endpoint, body, m], i) => ({ no: i + 1, endpoint, body, expected: m ? "accept" : "reject" }));
export const countBy = (e) => REQUESTS.filter((i) => i.expected === e).length;
```
The out-of-contract requests carry no malicious payload; they carry the wrong shape, the wrong
type, and extra fields.
#### Two Decision Shapes
The **allowlist** passes nothing except what the contract writes down; the **blocklist** rejects
named bad shapes and passes everything else. The `unknownField` setting produces a third row.
```js
// validator.mjs — two decision shapes looking at the same request set
import { SCHEMA } from "./requests.mjs";
// Allowlist: nothing passes except what the contract writes down.
// unknownField = "reject" | "pass" (the second is this lesson's silent setting)
export function allowlist(endpoint, body, { unknownField = "reject" } = {}) {
const fields = SCHEMA[endpoint];
if (!fields) return { decision: "reject", reason: "undefined endpoint" };
const reject = (n) => ({ decision: "reject", reason: n });
for (const [name, v] of Object.entries(body)) {
const k = fields[name];
if (!k) { if (unknownField === "reject") return reject(`unknown field ${name}`); continue; }
if (k.t === "str") {
if (typeof v !== "string") return reject(`${name} is not a string`);
if (k.pattern && !k.pattern.test(v)) return reject(`${name} does not match the pattern`);
if (k.max && v.length > k.max) return reject(`${name} is too long`);
if (k.options && !k.options.includes(v)) return reject(`${name} is outside the options`);
} else {
if (!Number.isInteger(v)) return reject(`${name} is not an integer`);
if (v < k.min || v > k.max) return reject(`${name} is out of range`);
}
}
for (const [name, k] of Object.entries(fields))
if (k.required && body[name] === undefined) return reject(`${name} is missing`);
return { decision: "accept", reason: "" };
}
// Blocklist: named bad patterns are rejected, everything else passes.
const BANNED = ["<", ">", "'", '"', "--", ";", "..", "\\"];
export function blocklist(endpoint, body) {
for (const [name, v] of Object.entries(body)) {
const m = typeof v === "string" ? v : String(v ?? "");
for (const b of BANNED) if (m.includes(b)) return { decision: "reject", reason: `${name} carries banned token ${b}` };
if (m.length > 256) return { decision: "reject", reason: `${name} is too long` };
}
return { decision: "accept", reason: "" };
}
// False accept: a request that should have been rejected was accepted.
// False reject: a request that should have been accepted was rejected.
export function measure(requests, decide) {
const r = { falseAccept: 0, falseReject: 0, escaped: [], snagged: [] };
for (const i of requests) {
const c = decide(i.endpoint, i.body).decision;
if (c === i.expected) continue;
if (i.expected === "reject") { r.falseAccept++; r.escaped.push(i.no); }
else { r.falseReject++; r.snagged.push(i.no); }
}
return r;
}
```
```js
// compare.mjs — the same 22 requests, three decision shapes; the third is the first two's silent setting
import { REQUESTS, SCHEMA, countBy } from "./requests.mjs";
import { allowlist, blocklist, measure } from "./validator.mjs";
const extra = (i) => Object.keys(i.body).filter((a) => !SCHEMA[i.endpoint][a]); // field not in the contract
const fieldCount = Object.values(SCHEMA).reduce((t, a) => t + Object.keys(a).length, 0);
console.log(`${Object.keys(SCHEMA).length} endpoints, ${fieldCount} field contract, ${REQUESTS.length} requests: ` +
`${countBy("accept")} legitimate, ${countBy("reject")} out of contract\n`);
const SHAPE = [
["allowlist", (endpoint, b) => allowlist(endpoint, b)],
["allowlist (pass)", (endpoint, b) => allowlist(endpoint, b, { unknownField: "pass" })],
["blocklist", blocklist],
];
console.log(`${"decision shape".padEnd(22)}${"false accept".padStart(13)}${"false reject".padStart(14)}${"extra-field record".padStart(20)}${"error".padStart(6)}`);
for (const [name, f] of SHAPE) {
const r = measure(REQUESTS, f);
let a = 0, err = 0;
for (const i of REQUESTS) {
try { if (f(i.endpoint, i.body).decision === "accept" && extra(i).length) a++; } catch { err++; }
}
console.log(name.padEnd(22) + String(r.falseAccept).padStart(13) + String(r.falseReject).padStart(14) +
String(a).padStart(20) + String(err).padStart(6));
console.log(` passed out of contract: ${r.escaped.join(", ") || "none"}`);
console.log(` legitimate snagged : ${r.snagged.map((n) => `${n} (${f(REQUESTS[n - 1].endpoint, REQUESTS[n - 1].body).reason})`).join(", ") || "none"}`);
}
```
```
3 endpoints, 8 field contract, 22 requests: 11 legitimate, 11 out of contract
decision shape false accept false reject extra-field record error
allowlist 0 1 0 0
passed out of contract: none
legitimate snagged : 11 (isbn does not match the pattern)
allowlist (pass) 2 1 2 0
passed out of contract: 17, 22
legitimate snagged : 11 (isbn does not match the pattern)
blocklist 11 4 2 0
passed out of contract: 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22
legitimate snagged : 4 (q carries banned token '), 7 (note carries banned token <), 9 (q carries banned token --), 10 (note carries banned token ;)
```
The blocklist passed **all** eleven out-of-contract requests: the list only recognizes tokens it
knows by name, and neither the twelve-digit identifier nor the extra field carries any of them.
The same list rejected four legitimate requests — an apostrophe in an author's name, a less-than
sign in a note, two dashes in a search string, a semicolon. **The blocklist loses on both columns
at once.**
**AS2 (assumption):** the allowlist's contract was written by looking at the shape of the
legitimate requests in the set; the zero false accepts is partly a result of that closeness. The
residue is the eleventh request: the hyphenated identifier is legitimate, and the contract does
not recognize it. **This is the allowlist's cost** — every unanticipated legitimate shape is a
false reject.
The middle row is this lesson's first silent setting. With the `unknownField` policy set to
`pass`, the thrown-error count is still zero, but two requests are accepted and land in the store
carrying two fields the contract does not know: a penalty value and a role value. No code reads
these fields today.
#### Where the Boundary Sits
A loan request passes through three steps: recorded at the endpoint, reserved in the catalog at
the service, and written to the store. The first two are side effects that must be undone.
**AS3 (assumption):** the side effect is measured by step count, not seconds; every step's
duration is treated as constant here.
```js
// boundary.mjs — the same validator, five different boundaries; what is measured is where the side effect stops
import { REQUESTS, SCHEMA } from "./requests.mjs";
import { allowlist } from "./validator.mjs";
const STEPS = ["endpoint", "service", "store"]; // side effect: request record, catalog reservation, write
function run(request, placement, recorded) {
let sideEffect = 0;
for (const a of STEPS) {
if (a === placement && recorded.includes(request.endpoint) &&
allowlist(request.endpoint, request.body).decision === "reject") return { sideEffect, written: false };
sideEffect++;
}
return { sideEffect, written: true };
}
const ALL = Object.keys(SCHEMA);
const MISSING = ALL.filter((u) => u !== "POST /member/note"); // one endpoint is waiting to be wired in
const invalid = REQUESTS.filter((i) => i.expected === "reject");
const legitimate = REQUESTS.filter((i) => i.expected === "accept");
console.log(`${invalid.length} out-of-contract requests, ${legitimate.length} legitimate requests, ${STEPS.length}-step pipeline`);
console.log(`\n${"placement".padEnd(20)}${"covered endpoints".padStart(18)}${"stopped".padStart(9)}${"invalid written".padStart(17)}${"wasted side effect".padStart(20)}${"legitimate written".padStart(20)}`);
for (const [name, at, recorded] of [["endpoint", "endpoint", ALL], ["endpoint (missing)", "endpoint", MISSING],
["service", "service", ALL], ["store", "store", ALL], ["none", "-", []]]) {
const b = invalid.map((i) => run(i, at, recorded)), m = legitimate.map((i) => run(i, at, recorded));
console.log(name.padEnd(20) + String(recorded.length).padStart(18) +
String(b.filter((r) => !r.written).length).padStart(9) +
String(b.filter((r) => r.written).length).padStart(17) +
String(b.reduce((t, r) => t + (r.written ? 0 : r.sideEffect), 0)).padStart(20) +
String(m.filter((r) => r.written).length).padStart(20));
}
const passed = invalid.filter((i) => run(i, "endpoint", MISSING).written).map((i) => i.no);
console.log(`\nendpoint POST /member/note not recorded -> out-of-contract records that reach the store: ${passed.length} (request ${passed.join(", ")})`);
console.log(`errors raised by the application at this placement: 0`);
```
```
11 out-of-contract requests, 11 legitimate requests, 3-step pipeline
placement covered endpoints stopped invalid written wasted side effect legitimate written
endpoint 3 11 0 0 10
endpoint (missing) 2 9 2 0 10
service 3 11 0 11 10
store 3 11 0 22 10
none 0 0 11 0 11
endpoint POST /member/note not recorded -> out-of-contract records that reach the store: 2 (request 21, 22)
errors raised by the application at this placement: 0
```
All three full placements stop the same number of requests; where they diverge is **how much work
is wasted**. Validation at the endpoint stops with zero side effect; at the service it wastes
eleven request records; at the store it wastes twenty-two steps — eleven of them a catalog
reservation set aside and then put back.
The second row is the second silent setting. Per-endpoint registration means three write sites,
and a fourth endpoint needs a fourth write site added by hand. Skip it, and the application still
runs, no test breaks, and two out-of-contract records land silently in the store. A single
pipeline reduces this risk to one point.
The `legitimate written` column says one last thing: at every validated placement, only **ten** of
the eleven legitimate requests reach the store. The cost of a false reject is an unrecorded loan
transaction.
#### Output Encoding
A value that passes validation is not safe; it is **contract-compliant**. A member note accepts
free text, because a quote, a comma, and a less-than sign legitimately appear in a reader's note.
This value is written to three places: a markup body, inside an attribute's quotes, and a
comma-separated export line. The measure is one question: **does the value come back the same
after being placed raw and read back?**
```js
// encoding.mjs — legitimate values that passed validation, three separate output contexts
// Measure: is a value the same after being placed raw and read back with the container's own rule?
const VALUES = ["cover worn", "5 < 7 pages missing on return", 'member "urgent" note dropped',
"shelf 3, floor 2", "first line\nsecond line", 'donation, "used" 5 books'];
const body = (s) => s.replace(/&/g, "&").replace(//g, ">");
const attribute = (s) => body(s).replace(/"/g, """).replace(/\n/g, "
");
const quoted = (s) => `"${s.replace(/"/g, '""')}"`;
const decode = (s) => s.replace(/"/g, '"').replace(/
/g, "\n")
.replace(/</g, "<").replace(/>/g, ">").replace(/&/g, "&");
const split = (row) => { // quote-aware field splitter
const fields = []; let s = "", q = false;
for (let i = 0; i < row.length; i++) {
const c = row[i];
if (q && c === '"' && row[i + 1] === '"') { s += '"'; i++; }
else if (c === '"') q = !q;
else if (!q && (c === "," || c === "\n")) { fields.push(s); s = ""; }
else s += c;
}
return [...fields, s];
};
const CONTEXTS = [
["body", (x) => `${x}`, body, (k) => {
const end = k.indexOf("<", "".length); // field is read up to the next container marker
return k.startsWith("") && k.slice(end) === "" ? decode(k.slice("".length, end)) : null;
}],
["attribute", (x) => ``, attribute,
(k) => { const m = /title="([^"]*)"/.exec(k); return m ? decode(m[1]) : null; }],
["quoted", (x) => `U-10432,${x},2026-03-04`, quoted,
(k) => { const a = split(k); return a.length === 3 ? a[1] : null; }],
];
const attempt = (encoder) => {
const list = [];
for (const [name, wrap, correct, read] of CONTEXTS)
VALUES.forEach((v, i) => {
if (read(wrap(encoder(v, correct))) !== v) list.push(`${name}/${i + 1}`);
});
return list;
};
console.log(`${VALUES.length} values x ${CONTEXTS.length} contexts = ${VALUES.length * CONTEXTS.length} placements\n`);
console.log(`${"encoder".padEnd(24)}${"not readable back".padStart(18)} where`);
for (const [name, f] of [["no encoding", (v) => v], ["single encoder (body)", body],
["context-aware", (v, correct) => correct(v)]]) {
const l = attempt(f);
console.log(`${name.padEnd(24)}${String(l.length).padStart(18)} ${l.join(" ") || "-"}`);
}
```
```
6 values x 3 contexts = 18 placements
encoder not readable back where
no encoding 7 body/2 attribute/3 attribute/6 quoted/3 quoted/4 quoted/5 quoted/6
single encoder (body) 7 attribute/3 attribute/6 quoted/2 quoted/3 quoted/4 quoted/5 quoted/6
context-aware 0 -
```
The single encoder gained nothing: seven of the eighteen placements still fail to read back, and
**the list changed**. The body context was fixed, but the second value broke in the quoted context
— the less-than sign in the note turned into an escape sequence and now sits as raw text in the
export line. This is the third silent outcome: the file opens, the lines are read, and the note
inside it is no longer the note that was written.
The correct implementation encodes **not on input, but on write**; a value encoded on input gets
encoded twice when it reaches a second context.
#### Summary
- On the same twenty-two requests, the allowlist produced 0 false accepts and 1 false reject; the
blocklist produced 11 false accepts and 4 false rejects. What it passed carried no malicious
token, only the wrong shape, the wrong type, and extra fields.
- The allowlist's cost is its single false reject: a legitimate identifier written with a hyphen
was rejected, and a loan transaction went unrecorded.
- Validation stopped at the endpoint wasted 0 steps; at the service, 11; at the store, 22 — the
number of requests stopped was the same in all three; only the cost differs.
- A single encoder was not enough for three contexts: 7 of 18 placements failed to read back, and
as one context was fixed another broke. Encoding is done separately in every output context.
- All three wrong settings are silent, and the thrown-error total is zero: the `unknownField`
policy sent 2 extra fields to the store, a forgotten write site let 2 out-of-contract records
through, and the wrong-context encoder corrupted one line. Per-endpoint registration repeats
this at 3 points; a single pipeline stops it at 1.
#### Next Step
The validation boundary guarantees the shape of the body. What it does not guarantee is **how
that value is used** inside the system. A search string, a sort field, or a notification template
that fully conforms to the contract stops being data and becomes part of another language the
moment it is placed into a query string, a command line, or a template body by concatenation. The
next lesson shows these three places share the same root and measures a single decision: how many
different outcomes the same request set produces when a value is concatenated into text versus
passed as a separate operand.
### Injection-Class Vulnerabilities
Source: https://academia.sh/en/courses/backend-production/injection-class-vulnerabilities
Measuring query, command, and template injection from a single root: how many unexpected results ten legitimate search inputs produce in a concatenated query, how many of six report labels break when passed as a single string, how many of five note texts change when written into a template body, and how many of twelve call sites use concatenation.
The previous lesson established the shape of the boundary: whether the body's fields, types, and
ranges conform to the contract. A value that passes the contract is now **correctly shaped**, but
it is still text, and it will be written somewhere inside the system. This lesson measures that
moment of writing.
Three separate classes are counted — query, command, and template — and all three come down to
the same decision: **was the value concatenated into that context's text, or passed as a separate
operand?** No malicious input is used anywhere in the measurement. Every input is text that really
appears in the library catalog: author names with an apostrophe, staff notes with curly braces,
report labels with spaces. What is measured is **how many legitimate inputs produce an unexpected
result**.
**AS4 (assumption):** the two setups are expected to behave the same. The separate-operand path is
taken as the yardstick; every result the concatenated path gives that differs from it — an error
as well as a different row — counts as an unexpected result.
#### Query Context
The catalog search endpoint is the most frequently called place in the loan system. The same
search is written with two setups and asked with ten legitimate inputs, one at a time.
```js
// data.mjs — the loan catalog and legitimate search inputs that really appear in real catalogs
import { DatabaseSync } from "node:sqlite";
export const db = new DatabaseSync(":memory:");
db.exec(`CREATE TABLE book (isbn TEXT PRIMARY KEY, title TEXT, author TEXT, year INTEGER)`);
const insert = db.prepare(`INSERT INTO book VALUES (?, ?, ?, ?)`);
for (const s of [
["9789750718533", "Lost Time", "M. Proust", 1913],
["9786050000000", "History of Rock 'n' Roll", "J. O'Neill", 1998],
["9789750000001", "100% Cotton", "A. Saenz", 2004],
["9789750000002", "L'Etranger", "A. Camus", 1942],
["9789750000003", "Snow_ Pattern", "H. Yilmaz", 2011],
["9789750000004", "History -- 20th Century", "S. Aydin", 2019],
["9789750000005", 'Essay "Two"', "N. Reed", 2020],
["9789750000006", "Pencil and Paper", "B. Demir", 1995],
]) insert.run(...s);
// None of these are an attack; all are legitimate search strings that appear in the library catalog.
export const SEARCHES = ["Pencil", "O'Neill", "100%", "Snow_", "--", 'Two"',
"Rock 'n' Roll", "Camus", "L'Etranger", "Reed's"];
// Sort field: the contract expects a field name.
export const SORT = ["title", "year", "author", "year desc", "book.internal_note"];
```
```js
// query.mjs — the same search, two setups: separate operand and string concatenation
import { db, SEARCHES, SORT } from "./data.mjs";
const attempt = (f) => { try { return { ok: 1, v: f() }; } catch (e) { return { ok: 0, v: e.message.slice(0, 34) }; } };
const operand = (q) => db.prepare(`SELECT isbn FROM book WHERE title LIKE ? OR author LIKE ?`)
.all(`%${q}%`, `%${q}%`).map((r) => r.isbn);
const concatenated = (q) => db.prepare(
`SELECT isbn FROM book WHERE title LIKE '%${q}%' OR author LIKE '%${q}%'`).all().map((r) => r.isbn);
console.log(`${SEARCHES.length} legitimate search inputs, two setups\n`);
console.log(`${"input".padEnd(16)}${"operand".padStart(9)}${"concatenated".padStart(14)} diff`);
let diverged = 0, errored = 0;
for (const q of SEARCHES) {
const a = attempt(() => operand(q)), b = attempt(() => concatenated(q));
const same = a.ok && b.ok && String(a.v) === String(b.v);
if (!same) diverged++;
if (!b.ok) errored++;
console.log(JSON.stringify(q).padEnd(16) + (a.ok ? String(a.v.length) : "error").padStart(9) +
(b.ok ? String(b.v.length) : "error").padStart(14) + " " + (same ? "-" : b.ok ? "different row" : b.v));
}
console.log(`\nunexpected result in the concatenated setup: ${diverged}/${SEARCHES.length} (${errored} of them errors)`);
// A field name cannot be passed as an operand; concatenation is unavoidable here.
const ALLOWED = { title: "title", year: "year", author: "author" };
const sortBy = (field, allowlisted) => {
if (allowlisted && !ALLOWED[field]) return "reject";
const s = allowlisted ? ALLOWED[field] : field;
return attempt(() => db.prepare(`SELECT isbn FROM book ORDER BY ${s}`).all()[0].isbn);
};
console.log(`\n${"sort input".padEnd(20)}${"concatenation".padStart(18)}${"allowlist".padStart(18)}`);
for (const field of SORT) {
const b = sortBy(field, false), i = sortBy(field, true);
const format = (r) => (r === "reject" ? "reject" : r.ok ? r.v : "error");
console.log(field.padEnd(20) + format(b).padStart(18) + format(i).padStart(18));
}
```
```
10 legitimate search inputs, two setups
input operand concatenated diff
"Pencil" 1 1 -
"O'Neill" 1 error near "Neill": syntax error
"100%" 1 1 -
"Snow_" 1 1 -
"--" 1 1 -
"Two\"" 1 1 -
"Rock 'n' Roll" 1 error near "n": syntax error
"Camus" 1 1 -
"L'Etranger" 1 error near "Etranger": syntax error
"Reed's" 0 error near "s": syntax error
unexpected result in the concatenated setup: 4/10 (4 of them errors)
sort input concatenation allowlist
title 9789750000001 9789750000001
year 9789750718533 9789750718533
author 9789750000002 9789750000002
year desc 9789750000005 reject
book.internal_note error reject
```
Four inputs deviate, and what all four have in common is that they carry an apostrophe. These four
readers' searches **do not work** — the loan system cannot search for an author's name. This is
the most common production effect of an injection class: not a breach, a broken function.
The second table is the actual point. The sort field cannot be passed as an operand, because it is
itself part of the query text; concatenation is unavoidable there. The `year desc` input is
accepted **without an error** by concatenation and returns a different row. `book.internal_note`
errors. The allowlist rejects both. **The one place where concatenation is mandatory is the one
place where the deviation is silent** — and the only remedy there is the previous lesson's
allowlist.
#### Command Context
The report service hands the export job off to a separate process. The label can be passed two
ways: as a separate argument, or inside a string concatenated into the command line.
```js
// echo.mjs — report process: prints the arguments it received as-is
console.log(JSON.stringify(process.argv.slice(2)));
```
```js
// command.mjs — a report label is passed to the child process two ways: separate argument and single string
import { execFileSync, execSync } from "node:child_process";
// All of these are legitimate report labels written by library staff.
const LABELS = ["2026-03", "daily report", "section $shelf", "O'Neill list", "second hand", "lost-found"];
const read = (f) => { try { return JSON.parse(f()); } catch { return null; } };
const operand = (e) => read(() => execFileSync("node", ["echo.mjs", e], { encoding: "utf8" }));
const concatenated = (e) => read(() => execSync(`node echo.mjs ${e}`, { encoding: "utf8", stdio: ["pipe", "pipe", "ignore"] }));
console.log(`${LABELS.length} legitimate report labels, two passing methods\n`);
console.log(`${"label".padEnd(19)}${"separate argument".padStart(22)}${"single string".padStart(26)}`);
let broken = 0;
for (const e of LABELS) {
const a = operand(e), b = concatenated(e);
const same = b && b.length === 1 && b[0] === e;
if (!same) broken++;
console.log(e.padEnd(19) + JSON.stringify(a).padStart(22) + (b ? JSON.stringify(b) : "process error").padStart(26));
}
console.log(`\nseparate argument: 0/${LABELS.length} broken, single string: ${broken}/${LABELS.length} broken`);
```
```
6 legitimate report labels, two passing methods
label separate argument single string
2026-03 ["2026-03"] ["2026-03"]
daily report ["daily report"] ["daily","report"]
section $shelf ["section $shelf"] ["section"]
O'Neill list ["O'Neill list"] process error
second hand ["second hand"] ["second","hand"]
lost-found ["lost-found"] ["lost-found"]
separate argument: 0/6 broken, single string: 4/6 broken
```
Same root, different language. The label with a space splits into two arguments, the double space
collapses to one, and the label with an apostrophe never starts the process at all. The third row
is this section's silent case: the `section $shelf` label runs without an error and arrives at the
child process **as `section`**. The report is generated, the file is written, no one notices
anything; only half the label is missing.
#### Template Context
Notification text is produced from a template. The note a staff member writes is either given to
the template **as data**, or written into the template's body and then processed.
```js
// template.mjs — the notification text is built two ways: data as an operand, data into the template body
const TEMPLATE = "Dear {name}, {isbn} is due {due}. Note: {note}";
const FILL = (text, data) => text.replace(/\{(\w+)\}/g, (_, k) => data[k] ?? "");
// All legitimate note text written by staff by hand.
const NOTES = ["cover worn", "return date written as {due}", "shelf {location} not found",
"$5 fee", "registered under {name}"];
const BASE = { name: "A. Reed", isbn: "9789750718533", due: "2026-03-25" };
// Separate operand: the template is fixed, the data is substituted once and never scanned again.
const operand = (note) => FILL(TEMPLATE, { ...BASE, note });
// Concatenation: the data is written into the template body first, then the body is processed.
const concatenated = (note) => FILL(TEMPLATE.replace("{note}", note), { ...BASE, note });
const noteField = (m) => m.split("Note: ")[1];
console.log(`${NOTES.length} legitimate note texts, two setups\n`);
console.log(`${"note entered".padEnd(36)}${"note printed by the concatenated setup".padEnd(40)}diff`);
let broken = 0;
for (const note of NOTES) {
const b = concatenated(note), same = b === operand(note);
if (!same) broken++;
console.log(note.padEnd(36) + noteField(b).padEnd(40) + (same ? "-" : "changed"));
}
console.log(`\nnotification whose note field is not printed as entered: concatenated ${broken}/${NOTES.length}, operand 0/${NOTES.length}`);
```
```
5 legitimate note texts, two setups
note entered note printed by the concatenated setup diff
cover worn cover worn -
return date written as {due} return date written as 2026-03-25 changed
shelf {location} not found shelf not found changed
$5 fee $5 fee -
registered under {name} registered under A. Reed changed
notification whose note field is not printed as entered: concatenated 3/5, operand 0/5
```
Three of the five notes turned into a different text in the notification the reader receives. None
of them errored. One of them **went empty** because it carried an unknown key; the record still
exists, but it is not in the notification. On the separate-operand path the template is fixed and
the data is substituted once and never scanned again — so all five come out exactly as entered.
#### Counting the Concatenation
The decision is not made in one place; it is made separately at every call site. The split
system's call sites are counted.
```js
// calls.mjs — a count of the call sites in the split system and the measure of the one that was missed
import { db, SEARCHES } from "./data.mjs";
// [service, call site, context, setup]
const CALLS = [
["catalog", "search", "query", "operand"],
["catalog", "sort", "query", "concatenation + allowlist"],
["catalog", "getByIsbn", "query", "operand"],
["loan", "issue", "query", "operand"],
["loan", "overdue", "query", "operand"],
["loan", "searchArchive", "query", "concatenation"], // the one call site that was missed
["member", "addNote", "query", "operand"],
["member", "history", "query", "operand"],
["notification", "format", "template", "operand"],
["notification", "bulkSend", "template", "operand"],
["report", "export", "command", "operand"],
["report", "archive", "command", "operand"],
];
const count = (k) => CALLS.filter(k).length;
console.log(`${CALLS.length} call sites, ${new Set(CALLS.map((c) => c[0])).size} services`);
for (const b of ["query", "command", "template"])
console.log(` ${b.padEnd(9)} ${count((c) => c[2] === b)} sites, ` +
`${count((c) => c[2] === b && c[3].startsWith("concatenation"))} of them concatenation`);
console.log(` unprotected concatenation: ${count((c) => c[3] === "concatenation")} ` +
`(${CALLS.filter((c) => c[3] === "concatenation").map((c) => c[0] + "." + c[1]).join(", ")})`);
// The missed call site is actually run: the same search inputs, the only difference is the setup.
const searchArchive = (q) => db.prepare(`SELECT isbn FROM book WHERE title LIKE '%${q}%'`).all();
let diverged = 0;
const deviations = [];
for (const q of SEARCHES) {
try { searchArchive(q); } catch { diverged++; deviations.push(q); }
}
console.log(`\nat the one call site that was missed, ${diverged} of ${SEARCHES.length} inputs gave an unexpected result`);
console.log(`what the deviations have in common: all carry an apostrophe -> ${deviations.every((q) => q.includes("'"))}`);
console.log(`a test set built with ${SEARCHES.filter((q) => !q.includes("'")).length} inputs carrying no apostrophe never sees this site`);
```
```
12 call sites, 5 services
query 8 sites, 2 of them concatenation
command 2 sites, 0 of them concatenation
template 2 sites, 0 of them concatenation
unprotected concatenation: 1 (loan.searchArchive)
at the one call site that was missed, 4 of 10 inputs gave an unexpected result
what the deviations have in common: all carry an apostrophe -> true
a test set built with 6 inputs carrying no apostrophe never sees this site
```
Two of the twelve call sites use concatenation: one is the sort field, protected by an allowlist;
the other was missed. The system runs, every endpoint responds, and no test is red. The last line
says why: the deviation only shows up on inputs carrying an apostrophe, and a test set built with
plain names never sees that site.
**Where the setting sits, and how often it repeats:** this decision does not live in a
configuration file; it is made separately at each of the twelve call sites. For it to become
changeable from a single place, the query, command, and template calls would all need to pass
through a single wrapper — then the decision drops from 12 points to 3.
#### Summary
- All three classes come down to one decision: was the value concatenated into the context's text,
or passed as a separate operand. No malicious input was used anywhere in the measurement.
- 4 of ten legitimate search inputs produced an unexpected result in the concatenated query; all
four carried an apostrophe and all four resulted in an error — not a breach, a broken search.
- The sort field cannot be passed as an operand; at this one place where concatenation is
mandatory, the deviation is silent: `year desc` was accepted without an error and returned a
different row. The allowlist rejected it.
- 4 of six report labels broke when passed as a single string; `section $shelf` ran without an
error and reached the child process cut in half. 3 of five notes changed when written into the
template body, and one went empty. On the separate-operand path, breakage was zero across all
three contexts.
- 2 of twelve call sites use concatenation; one is protected by an allowlist, one was missed. The
missed site deviates on 4 of 10 inputs, and a test set built with the remaining 6 never sees it
at all. The decision is repeated at 12 points.
#### Next Step
The two lessons so far looked at the request's **body**: whether the fields are correctly shaped,
whether values are concatenated into text. Neither asked where the request came from. Yet requests
to the loan system's endpoints can arrive not only from its own interface but from other pages
open in the browser, and some of these requests carry the reader's session along with them. The
next lesson measures the server's one statement on this — how much it permits a request from a
given origin — and counts where the setting lives, how many origins a wildcard value opens the
door to, and why a wrong setting on a request carrying credentials fails silently, without an
error.
### Cross-Origin Resource Sharing
Source: https://academia.sh/en/courses/backend-production/cross-origin-resource-sharing
The server-side configuration of cross-origin sharing: how many of seven origins each of four settings lets the response be used for, how many origins a wildcard setting opens, how a wildcard setting on a credentialed request returns 200 on the server while being silently dropped on the client, the round-trip cost of a preflight request, and how many endpoints carry the policy.
The previous two lessons looked at the request's body. This lesson looks at **where the request
came from** and measures the one thing the server can say about it: the response headers.
The client side of cross-origin sharing — which request the browser blocks and why, the effect of
the policy on the page — was built in the Frontend Quality course and is not repeated here. What
this lesson measures is **the server-side configuration**: which layer the setting sits in, how
many origins a wildcard value opens the door to, what a wrong setting does on a request carrying
credentials, and what a preflight request costs.
The server's job is small: it looks at the incoming origin and either adds a few response headers
or does not. The decision is made by the **client**. This distinction is the point every
measurement in this lesson rests on.
**AS5 (assumption):** the client's rule was written here, for this lesson. The correctness of the
server setting is measured by whether the response is used according to that rule; the measurement
is the outcome of the rule's implementation, not of a real client.
#### The Server's One Statement
```js
// policy.mjs — the server's one statement on cross-origin sharing: response headers
export const SETTING = {
origins: ["https://loans.library", "https://panel.library"], // "*" means wildcard
methods: ["GET", "POST"],
headers: ["content-type", "x-loan-trace"],
credentials: false,
preflightMaxAge: 600,
};
export function headers(request, setting) {
const h = {};
if (!request.origin) return h; // same-origin request; no statement needed
const wildcard = setting.origins === "*";
if (!wildcard && !setting.origins.includes(request.origin)) return h; // silence = reject
h["access-control-allow-origin"] = wildcard ? "*" : request.origin;
if (setting.credentials) h["access-control-allow-credentials"] = "true";
if (!wildcard) h["vary"] = "origin"; // the response varies by origin
if (request.method === "OPTIONS") {
h["access-control-allow-methods"] = setting.methods.join(",");
h["access-control-allow-headers"] = setting.headers.join(",");
h["access-control-max-age"] = String(setting.preflightMaxAge);
}
return h;
}
```
```js
// client.mjs — the decision a compliant client makes by reading the response headers
// The rule is written here for this lesson; the server setting's correctness is measured against it.
export function decide(request, h) {
if (!request.origin) return { used: true, reason: "same origin" };
const allow = h["access-control-allow-origin"];
if (!allow) return { used: false, reason: "no allow header" };
if (allow !== "*" && allow !== request.origin) return { used: false, reason: "origin mismatch" };
if (request.credentials) {
if (allow === "*") return { used: false, reason: "wildcard + credentials" };
if (h["access-control-allow-credentials"] !== "true")
return { used: false, reason: "no credentials permission" };
}
return { used: true, reason: "" };
}
```
The policy layer's rejection is not an error, it is **silence**: it adds no header at all to an
origin it does not recognize. The server still returns 200, and the body is still written. This
design decision is the source of every silent outcome in the rest of the lesson.
#### The Origins a Wildcard Setting Opens
Seven origins are taken: the loan system's own interface, the internal panel, the same name with a
different scheme and port, a subdomain, an unrelated origin, and a same-origin request that
carries no origin header at all.
```js
// origin.mjs — seven origins, four settings; what the server returns versus what the client uses
import { SETTING, headers } from "./policy.mjs";
import { decide } from "./client.mjs";
const ORIGINS = [
null, // same origin (no header)
"https://loans.library", // its own interface
"https://panel.library", // internal panel
"http://loans.library", // different scheme
"https://loans.library:8443", // different port
"https://sub.loans.library", // subdomain
"https://other.site", // unrelated
];
const SETTINGS = [
["list, no credentials", { ...SETTING }],
["wildcard, no credentials", { ...SETTING, origins: "*" }],
["wildcard, credentials", { ...SETTING, origins: "*", credentials: true }],
["list, credentials", { ...SETTING, credentials: true }],
];
const passes = (setting, origin, credentials) => {
const request = { origin, method: "GET", credentials };
return decide(request, headers(request, setting)).used;
};
console.log(`${ORIGINS.length} origins (1 same origin + ${ORIGINS.length - 1} external origins), ${SETTINGS.length} settings`);
console.log(`\n${"setting".padEnd(24)}${"server 2xx".padStart(11)}${"client used: no credentials".padStart(29)}${"credentials".padStart(13)}`);
for (const [name, setting] of SETTINGS) {
const k0 = ORIGINS.filter((o) => passes(setting, o, false)).length;
const k1 = ORIGINS.filter((o) => passes(setting, o, true)).length;
console.log(name.padEnd(24) + String(ORIGINS.length).padStart(11) + String(k0).padStart(29) + String(k1).padStart(13));
}
const wildcardOpen = ORIGINS.filter((o) => o && passes({ ...SETTING, origins: "*" }, o, false));
const listOpen = ORIGINS.filter((o) => o && passes(SETTING, o, false));
console.log(`\nexternal origins the list setting opens: ${listOpen.length} (${listOpen.join(", ")})`);
console.log(`external origins the wildcard setting opens in this set: ${wildcardOpen.length}; no limit outside the set`);
console.log(`\nrequests carrying credentials under the 'wildcard, credentials' setting:`);
for (const o of ORIGINS.slice(1, 4)) {
const request = { origin: o, method: "GET", credentials: true };
const h = headers(request, { ...SETTING, origins: "*", credentials: true });
const c = decide(request, h);
console.log(` ${o.padEnd(30)} server 200 + ${Object.keys(h).length} headers -> client: ${c.used ? "used" : "dropped (" + c.reason + ")"}`);
}
```
```
7 origins (1 same origin + 6 external origins), 4 settings
setting server 2xx client used: no credentials credentials
list, no credentials 7 3 1
wildcard, no credentials 7 7 1
wildcard, credentials 7 7 1
list, credentials 7 3 3
external origins the list setting opens: 2 (https://loans.library, https://panel.library)
external origins the wildcard setting opens in this set: 6; no limit outside the set
requests carrying credentials under the 'wildcard, credentials' setting:
https://loans.library server 200 + 2 headers -> client: dropped (wildcard + credentials)
https://panel.library server 200 + 2 headers -> client: dropped (wildcard + credentials)
http://loans.library server 200 + 2 headers -> client: dropped (wildcard + credentials)
```
The `server 2xx` column is seven in all four settings. The server rejects no request; the setting
only decides whether the client uses the response. The list setting is open to two external
origins; the wildcard setting is open to six in this set, and **there is no limit outside the
set** — every origin whose name was never written down gets in, scheme difference, port
difference, and subdomain included. The difference between wildcard and list in this row is 2
against 6; in reality it is 2 against unlimited.
The third row is this lesson's silent setting. `credentials` is turned on, the server sends both
headers, the request returns 200 — and the client **drops** the response. The `wildcard,
credentials` row is identical to the `wildcard, no credentials` row: turning the setting on
changed nothing. In a log pile, these two settings cannot be told apart; both cases have 200, no
error, no latency. Only the internal panel fails to work, and the reason is invisible on the
server side.
The correct setting is the last row: an origin list **and** credentials permission together. The
three credentialed requests are used only there.
#### The Cost of Preflight
The policy layer is measured in a real process. A request carrying a custom header requires a
**preflight** request before it is sent; the preflight's result can be cached for a lifetime.
```js
// server.mjs — local process listening on the loan endpoints; the policy layer sits in front of the endpoints
import http from "node:http";
import { headers } from "./policy.mjs";
export const ENDPOINTS = ["/catalog", "/loans", "/member", "/report", "/archive"];
export function start(setting, covered = ENDPOINTS) {
const counter = { request: 0, preflight: 0, bytes: 0 };
const s = http.createServer((q, y) => {
counter.request++;
if (q.method === "OPTIONS") counter.preflight++;
const request = { origin: q.headers.origin ?? null, method: q.method };
const h = covered.includes(q.url) ? headers(request, setting) : {};
const body = q.method === "OPTIONS" ? "" : JSON.stringify({ endpoint: q.url, status: "ok" });
counter.bytes += body.length +
Object.entries(h).reduce((t, [k, v]) => t + k.length + v.length + 4, 0);
y.writeHead(q.method === "OPTIONS" ? 204 : 200, { "content-type": "application/json", ...h });
y.end(body);
});
return new Promise((c) => s.listen(0, "127.0.0.1", () =>
c({ port: s.address().port, counter, close: () => s.close() })));
}
```
```js
// measurement.mjs — real requests: the cost of preflight and the measure of the layer's coverage
import { start, ENDPOINTS } from "./server.mjs";
import { SETTING } from "./policy.mjs";
import { decide } from "./client.mjs";
const ORIGIN = "https://panel.library";
const N = 6;
// Compliant client: a request with a custom header asks for a preflight first, and the result is cached for its lifetime.
async function send(port, endpoint, { custom, cache, credentials = false }) {
const base = `http://127.0.0.1:${port}`;
if (custom && !(cache && cache.has(endpoint))) {
const o = await fetch(base + endpoint, { method: "OPTIONS", headers: { origin: ORIGIN } });
if (cache) cache.set(endpoint, Number(o.headers.get("access-control-max-age") ?? 0));
}
const headers = { origin: ORIGIN, ...(custom ? { "x-loan-trace": "1" } : {}) };
const y = await fetch(base + endpoint, { headers });
const h = Object.fromEntries(y.headers);
return { status: y.status, decision: decide({ origin: ORIGIN, credentials }, h) };
}
const { port, counter, close } = await start(SETTING);
const run = async (name, option) => {
const before = { ...counter };
for (let i = 0; i < N; i++) await send(port, "/catalog", option);
console.log(`${name.padEnd(28)}${String(counter.request - before.request).padStart(8)}` +
`${String(counter.preflight - before.preflight).padStart(12)}${String(counter.bytes - before.bytes).padStart(9)}`);
};
console.log(`${N} real requests, one endpoint\n`);
console.log(`${"request shape".padEnd(28)}${"round trip".padStart(8)}${"preflight".padStart(12)}${"bytes".padStart(9)}`);
await run("plain (no preflight needed)", { custom: false });
await run("custom header, no cache", { custom: true });
await run("custom header, cached", { custom: true, cache: new Map() });
console.log(`preflight lifetime: ${SETTING.preflightMaxAge} s -> 1 preflight per endpoint per lifetime`);
close();
// Layer coverage: the policy is registered on four of five endpoints.
const MISSING = ENDPOINTS.filter((u) => u !== "/archive");
const second = await start({ ...SETTING, credentials: true }, MISSING);
console.log(`\npolicy registered on ${MISSING.length}/${ENDPOINTS.length} endpoints\n`);
console.log(`${"endpoint".padEnd(10)}${"server status".padStart(14)}${"client".padStart(9)} reason`);
let dropped = 0;
for (const endpoint of ENDPOINTS) {
const r = await send(second.port, endpoint, { custom: false, credentials: true });
if (!r.decision.used) dropped++;
console.log(endpoint.padEnd(10) + String(r.status).padStart(14) +
(r.decision.used ? "used" : "dropped").padStart(9) + " " + (r.decision.reason || "-"));
}
console.log(`\n${dropped} of ${ENDPOINTS.length} endpoints had their response dropped by the client; errors returned by the server: 0`);
second.close();
```
```
6 real requests, one endpoint
request shape round trip preflight bytes
plain (no preflight needed) 6 0 618
custom header, no cache 12 6 1770
custom header, cached 7 1 810
preflight lifetime: 600 s -> 1 preflight per endpoint per lifetime
policy registered on 4/5 endpoints
endpoint server status client reason
/catalog 200 used -
/loans 200 used -
/member 200 used -
/report 200 used -
/archive 200 dropped no allow header
1 of 5 endpoints had their response dropped by the client; errors returned by the server: 0
```
The cost of preflight is a number independent of the run: six custom requests without a cache make
**twelve** round trips, twice as many. Once the lifetime setting takes effect, the count drops to
seven — one preflight per endpoint. Policy bytes nearly triple too: 618 against 1770.
This cost belongs only to requests with a custom header. The first row shows this: six requests
without a custom header produce no preflight at all. **The way to avoid preflight is not to loosen
the setting, but to simplify the shape of the request.**
#### Where the Setting Sits
The last table is the layer question. The policy is registered on four of five endpoints; the
`/archive` endpoint was forgotten. That endpoint **accepts** requests, writes the body, and
returns 200. The response is dropped on the client, and there is no trace of it on the server
side: the error count is zero, the status code is the same as the other four endpoints.
How many places the setting is repeated in directly determines this risk. If the policy is
registered separately on every endpoint handler, there are five write sites, and a sixth endpoint
needs a sixth write site added by hand. If the policy sits as a single layer in front of the
endpoints, there is one write site and a new endpoint is covered automatically — the `covered`
list inside `server.mjs` is exactly the difference between these two choices.
#### Summary
- The server's one statement on cross-origin sharing is the response headers; its rejection is not
an error, it is silence. In all four settings, the server returned 2xx to all seven of the
seven requests.
- The origin list is open to 2 external origins; the wildcard setting is open to 6 origins in the
measured set and to an unlimited number outside it — scheme difference, port difference, and
subdomain included.
- Adding credentials permission to the wildcard setting changed nothing: the `wildcard,
credentials` row was identical to `wildcard, no credentials`. The server sent both headers,
returned 200, and the client dropped the response. The correct setting uses an origin list
together with credentials permission.
- The cost of preflight is independent of the run: 6 custom requests without a cache make 12 round
trips and 1770 bytes; with the lifetime setting, 7 round trips and 810 bytes; 6 requests without
a custom header make 0 preflights.
- With the policy registered on 4 of 5 endpoints, the response from the skipped endpoint was
dropped on the client and the server showed 0 errors. Per-endpoint registration repeats at 5
points; a single layer in front of the endpoints stops at 1 point and covers a new endpoint
automatically.
#### Next Step
The cross-origin sharing setting was nothing more than a set of response headers, and the
measurement showed that the real question is **which layer** produces the header. The same
question applies to the other headers a response carries: the ones that say how the connection
should be established, how the content should be interpreted, and what the browser may load can
just as well be scattered endpoint by endpoint or placed from a single site. The next lesson
counts these headers: how many endpoints they repeat on, how many files moving them to a single
layer touches, what behavior a missing header silently leaves open, and where the point at which
the encrypted connection terminates fits into this picture.
### Security Headers and TLS
Source: https://academia.sh/en/courses/backend-production/security-headers-and-tls
The server-side measure of transport-layer hardening: how many write sites six security headers occupy across ten endpoints, the eleven files touched by migrating to a single layer, the number of obligations a missing header and a written-but-ineffective value drop without producing an error, and the false accepts and false rejects a forwarded protocol header produces under three settings.
The previous lesson showed the server's one statement on cross-origin sharing is the response
headers, and the real question is **which layer produces the header**. This lesson carries that
question to the response's other headers and measures three things: how many endpoints repeat
these headers and how many files a move to a single layer touches; what a missing or wrongly
written header drops without an error; and where the application reads its "this connection is
encrypted" assumption from, and what happens when that source is forged.
#### The Header's Contract
A security header is a one-sentence obligation the server gives a compliant client; when it is
missing, what exists is not an error but the absence of an obligation. What is measured is the
sentence itself: is it written, and if so, does its value actually construct it.
```js
// hardening.mjs — the header set
const define = (value, ifMissing, rule) => ({ value, ifMissing, effective: (v) => rule.test(v.trim()) });
export const SET = {
"strict-transport-security":
define("max-age=31536000; includeSubDomains", "the first request can be tried unencrypted", /max-age=\s*[1-9]\d*/),
"content-security-policy":
define("default-src 'self'; frame-ancestors 'none'", "no resource or frame restriction", /default-src(?![^;]*\*)/),
"x-content-type-options":
define("nosniff", "content type can be guessed", /^nosniff$/i),
"referrer-policy":
define("no-referrer", "the address and query string leak outward", /^(no-referrer|same-origin|strict-origin)$/i),
"x-frame-options":
define("DENY", "no embedding restriction", /^(deny|sameorigin)$/i),
"permissions-policy":
define("geolocation=(), camera=()", "no capability restriction", /=\(\)/),
};
export function audit(b) {
const missing = [], ineffective = [];
for (const [name, k] of Object.entries(SET)) {
if (b[name] === undefined) missing.push(name);
else if (!k.effective(b[name])) ineffective.push(name);
}
return { missing, ineffective, sound: Object.keys(SET).length - missing.length - ineffective.length };
}
```
The `effective` field carries the lesson's second measure: a header can be absent, present and
effective, or **present but ineffective**. The third is counted separately since it is confused
with the second at a glance — present in the list, name correct, only the value constructs no
obligation.
**AS6 (assumption):** the client's obligation is written as a contract here, not as browser
behavior; the web side was built in Frontend Quality. What is measured is the response itself: the
headers read with `node:http`.
#### How Many Endpoints It Repeats On
The loan system splits into four services exposing ten endpoints; today the headers are written
separately inside every endpoint handler.
```js
// system.mjs — endpoints and two deviations
export const SERVICES = {
catalog: ["search", "book", "recommend"],
loan: ["borrow", "extend", "return"],
member: ["profile", "penalty"],
report: ["daily", "monthly"],
};
export const ENDPOINTS = Object.entries(SERVICES).flatMap(([s, us]) => us.map((u) => `/${s}/${u}`));
export const MISSING = { "/report/monthly": 2 }; // last two headers not written
export const WRONG = { "/loan/extend": ["x-content-type-options", "no-sniff"] };
```
```js
// distribution.mjs — write sites and migration
import { mkdirSync, writeFileSync, readFileSync, readdirSync, rmSync } from "node:fs";
import { SET } from "./hardening.mjs";
import { ENDPOINTS, MISSING, WRONG } from "./system.mjs";
const D = new URL("./ep/", import.meta.url).pathname;
rmSync(D, { recursive: true, force: true });
mkdirSync(D);
// each endpoint writes its own
for (const endpoint of ENDPOINTS) {
const [wh, wv] = WRONG[endpoint] ?? [];
const lines = Object.entries(SET)
.slice(0, Object.keys(SET).length - (MISSING[endpoint] ?? 0))
.map(([h, k]) => ` y.setHeader(${JSON.stringify(h)}, ${JSON.stringify(h === wh ? wv : k.value)});`);
writeFileSync(D + endpoint.slice(1).replace("/", "-") + ".mjs",
`export function handle(q, y) {\n${lines.join("\n")}\n y.end(JSON.stringify({ endpoint: ${JSON.stringify(endpoint)} }));\n}\n`);
}
const scan = () => readdirSync(D).map((f) => {
const matches = [...readFileSync(D + f, "utf8").matchAll(/setHeader\("([^"]+)",\s"([^"]*)"\)/g)];
return { f, record: matches.length, sound: matches.filter(([, h, v]) => SET[h]?.effective(v)).length };
});
const before = scan();
const total = Object.keys(SET).length;
console.log(`${before.length} endpoint files, ${total} headers`);
console.log(`write sites: ${before.length} record lines: ${before.reduce((t, d) => t + d.record, 0)}`);
console.log(`files whose set is complete and effective: ${before.filter((d) => d.sound === total).length}/${before.length}`);
for (const d of before.filter((d) => d.sound !== total))
console.log(` ${d.f.padEnd(20)} record ${d.record}, effective ${d.sound}`);
// migration: lines are deleted, a layer is written
let touched = 0;
for (const { f } of before) {
const old = readFileSync(D + f, "utf8"), updated = old.replace(/^\s*y\.setHeader\(.*\n/gm, "");
if (updated !== old) { writeFileSync(D + f, updated); touched++; }
}
const layer = Object.entries(SET)
.map(([h, k]) => ` y.setHeader(${JSON.stringify(h)}, ${JSON.stringify(k.value)});`).join("\n");
writeFileSync(D + "layer.mjs", `export function harden(q, y) {\n${layer}\n}\n`);
touched++;
const after = scan().filter((d) => d.f !== "layer.mjs");
console.log(`\nfiles touched by the migration: ${touched} (${before.length} endpoints + 1 layer)`);
console.log(`write sites after migration: 1 record lines remaining at endpoints: ${after.reduce((t, d) => t + d.record, 0)}`);
console.log(`header lines to write when a new endpoint is added: ${total} per endpoint, 0 in a single layer`);
```
```
10 endpoint files, 6 headers
write sites: 10 record lines: 58
files whose set is complete and effective: 8/10
loan-extend.mjs record 6, effective 5
report-monthly.mjs record 4, effective 4
files touched by the migration: 11 (10 endpoints + 1 layer)
write sites after migration: 1 record lines remaining at endpoints: 0
header lines to write when a new endpoint is added: 6 per endpoint, 0 in a single layer
```
Ten endpoints, ten write sites. The record-line count that should be sixty is fifty-eight:
`/report/monthly` fell two headers short. At `/loan/extend`, the value was written as `no-sniff`
— the header is there, name correct, value an undefined token. Eight of ten files are complete.
Both deviations trace back to the write-site count: six headers written in ten separate places
means ten opportunities to deviate, and none announces itself.
The cost of migration is one-time: **eleven files**. After it, there is one write site, and the
header line count for an eleventh endpoint is zero; per-endpoint registration demands six lines
per new endpoint.
#### Missing Header and Ineffective Value
How many places it was written was measured in the files; what shows up in the response is a
separate question, read from a real process.
```js
// measurement.mjs — response headers
import http from "node:http";
import { SET, audit } from "./hardening.mjs";
import { ENDPOINTS, MISSING, WRONG } from "./system.mjs";
const full = () => Object.fromEntries(Object.entries(SET).map(([h, k]) => [h, k.value]));
const SETTINGS = [
["per-endpoint record", (endpoint) => {
const b = full(), [wh, wv] = WRONG[endpoint] ?? [];
for (const h of Object.keys(b).slice(Object.keys(b).length - (MISSING[endpoint] ?? 0))) delete b[h];
if (wh) b[wh] = wv;
return b;
}],
["single layer", () => full()],
["single layer, wrong value", () => ({ ...full(), "strict-transport-security": "max-age=0" })],
];
const start = (produce) => new Promise((c) => {
const s = http.createServer((q, y) => {
y.writeHead(200, { "content-type": "application/json", ...produce(q.url) });
y.end(JSON.stringify({ endpoint: q.url }));
});
s.listen(0, "127.0.0.1", () => c(s));
});
const req = (port, endpoint) => new Promise((c) =>
http.get({ host: "127.0.0.1", port, path: endpoint }, (y) => { y.resume(); c(y); }));
console.log(`${ENDPOINTS.length} endpoints, ${Object.keys(SET).length} headers, ${ENDPOINTS.length} requests per setting\n`);
console.log(`${"setting".padEnd(25)}${"200".padStart(5)}${"errors".padStart(7)}${"bytes".padStart(7)}` +
`${"sound endpoints".padStart(19)}${"missing".padStart(9)}${"ineffective".padStart(13)}`);
const detail = [];
for (const [name, produce] of SETTINGS) {
const s = await start(produce);
let [ok200, bytes, soundEndpoints, missing, ineffective] = [0, 0, 0, 0, 0];
for (const endpoint of ENDPOINTS) {
const y = await req(s.address().port, endpoint);
if (y.statusCode === 200) ok200++;
for (const h of Object.keys(SET)) if (y.headers[h]) bytes += h.length + y.headers[h].length + 4;
const d = audit(y.headers);
if (d.sound === Object.keys(SET).length) soundEndpoints++;
else detail.push([name, endpoint, d]);
missing += d.missing.length; ineffective += d.ineffective.length;
}
s.close();
console.log(name.padEnd(25) + String(ok200).padStart(5) + "0".padStart(7) + String(bytes).padStart(7) +
`${soundEndpoints}/${ENDPOINTS.length}`.padStart(19) + String(missing).padStart(9) + String(ineffective).padStart(13));
}
console.log(`\nendpoints whose set fell short, and the silent outcome:`);
for (const [name, endpoint, d] of detail.slice(0, 4)) {
const note = ["missing", "ineffective"].flatMap((t) => d[t].map((h) => `${t} ${h} -> ${SET[h].ifMissing}`));
console.log(` ${name.padEnd(27)}${endpoint.padEnd(17)}${note.join("; ")}`);
}
console.log(` ... in the "single layer, wrong value" setting, the same line repeats on ${ENDPOINTS.length} of ${ENDPOINTS.length} endpoints`);
```
```
10 endpoints, 6 headers, 10 requests per setting
setting 200 errors bytes sound endpoints missing ineffective
per-endpoint record 10 0 2591 8/10 2 1
single layer 10 0 2660 10/10 0 0
single layer, wrong value 10 0 2400 0/10 0 10
endpoints whose set fell short, and the silent outcome:
per-endpoint record /loan/extend ineffective x-content-type-options -> content type can be guessed
per-endpoint record /report/monthly missing x-frame-options -> no embedding restriction; missing permissions-policy -> no capability restriction
single layer, wrong value /catalog/search ineffective strict-transport-security -> the first request can be tried unencrypted
single layer, wrong value /catalog/book ineffective strict-transport-security -> the first request can be tried unencrypted
... in the "single layer, wrong value" setting, the same line repeats on 10 of 10 endpoints
```
All thirty of thirty requests returned 200; the error count is zero across all three settings. The
only quantitative difference is hardening bytes: 2591, 2660, 2400 — and **the setting producing the
fewest bytes is the one whose set is complete on zero of ten endpoints.** Response size gives no
ranking of correctness.
`per-endpoint record` carries the file-level deviation into the response: two missing obligations,
one ineffective value, eight sound endpoints. `single layer` zeroes it out. The third row is the
other face of a single layer: the header is present on ten of ten endpoints, name correct, shape
valid, but with `max-age` at zero it constructs no obligation — one character in one place dropped
the fully compliant count from 10/10 to 0/10, status code and body unchanged.
This does not undo the decision to reduce write sites to one; it says where the measurement must
sit. A single layer ends the deviation and concentrates the value risk; catching both requires an
audit that runs not next to the configuration but **on the response itself**.
#### The Encrypted-Connection Assumption
The first header in the set differs from the others: it is meaningful only over an encrypted
connection, and the application cannot directly see that the connection is encrypted. TLS
termination happens not in the application but in the reverse proxy in front of it; which layer it
sits in and its cost were measured in The Traffic Layer course and are not repeated here. The
only information left to the application is a header the proxy forwards.
**AS7 (assumption):** local connections are unencrypted; a request through the proxy counts as
encrypted, one arriving directly as unencrypted. What is measured is how accurately the
application guesses this. **AS8 (assumption):** the local socket in the third setting is the
measurable counterpart of "the application is reachable from the network only through the proxy."
```js
// transport.mjs — the encrypted-connection assumption
import http from "node:http";
import { tmpdir } from "node:os";
import { rmSync } from "node:fs";
import { SET } from "./hardening.mjs";
const HEADER = "x-forwarded-proto";
const local = (port) => ({ host: "127.0.0.1", port });
const listen = (s, address) => new Promise((c) => s.listen(address, () => c(s)));
// app: redirects the unencrypted
const app = (rule) => (q, y) => {
const p = (q.headers[HEADER] ?? "").split(",").map((s) => s.trim()).filter(Boolean)
.at(rule === "first" ? 0 : -1);
if (p !== "https") return y.writeHead(308, { location: "https://loans.library" + q.url }).end();
y.writeHead(200, { "strict-transport-security": SET["strict-transport-security"].value });
y.end(JSON.stringify({ endpoint: q.url }));
};
// proxy: terminates and forwards
function proxy(target, shape) {
const s = http.createServer((q, y) => {
const incoming = q.headers[HEADER], chain = shape === "appending" && incoming ? `${incoming}, https` : "https";
const i = http.request({ ...target, path: q.url, headers: { ...q.headers, [HEADER]: chain } },
(c) => { y.writeHead(c.statusCode, c.headers); c.pipe(y); });
i.end();
});
return listen(s, local(0));
}
const listenApp = (rule, address) => {
if (typeof address === "string") rmSync(address, { force: true });
return listen(http.createServer(app(rule)), address);
};
const sendRequest = (target, sent) => new Promise((c) => {
const i = http.get({ ...target, path: "/loan/borrow", headers: sent ? { [HEADER]: sent } : {} },
(y) => { y.resume(); c(y); });
i.on("error", () => c({ statusCode: 0 }));
});
// a TCP address known to be closed
const closed = await listen(http.createServer(), local(0));
const closedPort = closed.address().port;
closed.close();
const REQUESTS = ["proxy", "direct"].flatMap((y) => [null, "https", "http"].map((g) => [y, g]));
const socket = tmpdir() + "/loans.sock";
const SETTINGS = [
["appending proxy, first value", "first", "appending", null],
["appending proxy, last value", "last", "appending", null],
["overwriting proxy, socket only", "last", "overwriting", socket],
];
console.log(`${REQUESTS.length} requests: 3 through the proxy (actually: encrypted), 3 direct (actually: unencrypted)\n`);
console.log(`${"setting".padEnd(34)}${"false accept".padStart(13)}${"false reject".padStart(14)}` +
`${"transport header/6".padStart(20)}${"unreachable/3".padStart(15)}`);
for (const [name, rule, shape, address] of SETTINGS) {
const application = await listenApp(rule, address ?? local(0));
const v = await proxy(address ? { socketPath: address } : local(application.address().port), shape);
let [falseAccept, falseReject, transportHeader, unreachable] = [0, 0, 0, 0];
for (const [path, sent] of REQUESTS) {
const encrypted = path === "proxy";
const y = await sendRequest(local(encrypted ? v.address().port
: address ? closedPort : application.address().port), sent);
if (y.statusCode === 0) { unreachable++; continue; }
if (y.headers["strict-transport-security"]) transportHeader++;
if (!encrypted && y.statusCode === 200) falseAccept++;
if (encrypted && y.statusCode === 308) falseReject++;
}
console.log(name.padEnd(34) + String(falseAccept).padStart(13) + String(falseReject).padStart(14) +
String(transportHeader).padStart(20) + String(unreachable).padStart(15));
v.close(); application.close();
}
// the cost of a false reject
const application = await listenApp("first", local(0));
const v = await proxy(local(application.address().port), "appending");
const statuses = [];
for (let i = 0; i < 5; i++) statuses.push((await sendRequest(local(v.address().port), "http")).statusCode);
console.log(`\nin the "first value" setting, an encrypted request carrying a reversed header: 5 attempts -> ${statuses.join(", ")}`);
console.log(`5xx count returned by the application: 0; no error in the log`);
v.close(); application.close();
```
```
6 requests: 3 through the proxy (actually: encrypted), 3 direct (actually: unencrypted)
setting false accept false reject transport header/6 unreachable/3
appending proxy, first value 1 1 3 0
appending proxy, last value 1 0 4 0
overwriting proxy, socket only 0 0 3 3
in the "first value" setting, an encrypted request carrying a reversed header: 5 attempts -> 308, 308, 308, 308, 308
5xx count returned by the application: 0; no error in the log
```
All three settings see the same six requests. The first setting reads the chain's **first**
value, which the client can write: an unencrypted impersonator is accepted (false accept), a
genuinely encrypted request turned back (false reject). The second setting reads the **last**
value, written by the layer closest to the application; the false reject drops to zero, but the
false accept remains, since on a direct request the chain holds one client-written value — "last"
reads the same as "first" there.
The third setting changes two things at once: the proxy overwrites what the client sent instead of
forwarding it, and the application listens only on a local socket, not the network. The three
direct requests cannot connect, both error shapes drop to zero, and the transport header lands on
exactly the three responses that deserve it. The outcome is a **path** rule more than a header
rule: trusting the forwarded header means trusting the layer writing it is the only entry point.
The cost of the false reject is silent too. In the first setting, an encrypted request with a
reversed header returned 308 on five of five attempts; a client following the redirect reproduces
the same request and gets the same response back. The application's 5xx count is zero — no one
searching logs for an error finds this loop. The same decision binds the previous section: the
transport header is written only when the connection counts as encrypted, and in the first
setting one of those three responses went to an unencrypted connection.
#### Summary
- Per-endpoint writing has 10 write sites and 58 record lines; two deviations turned up in 2
files. Migrating to a single layer touched 11 files, cutting write sites to 1 and new-endpoint
lines to 0.
- Across three settings, 30 of 30 requests returned 200, 0 errors; hardening bytes were 2591 /
2660 / 2400, and **the fewest-byte setting is the one whose set is complete on 0 of 10
endpoints** — response size gives no order of correctness.
- A written-but-ineffective value is counted separately from a missing header: one wrong value in
a single layer dropped the compliant-endpoint count from 10/10 to 0/10, status code and body
unchanged — why the audit runs on the **response**, not the configuration.
- The forwarded protocol header's first-value rule produced 1 false accept and 1 false reject; the
last-value rule zeroed only the false reject, since on a direct request the client writes the
chain's only value.
- An overwriting proxy plus a local-socket-only listener zeroed both: 3 direct requests could not
connect, and the header was written only on the 3 responses that deserved it. The false reject
leaves no trace — an encrypted request with a reversed header returned 308 on 5 of 5 attempts, a
5xx count of 0.
#### Next Step
The transport side was hardened here: headers gathered into a single layer, their values auditable
from the response, the encrypted-connection assumption tied to a single path. But every value used
was written into the code as plain text. The path the proxy reaches the application by, the values
the audit expects, and the keys the encrypted connection rests on all sit in the same place: the
source files. The next lesson counts these values — how many remain in the code, how many services
a change touches, and how many seconds the window lasts where the old and new value are both valid
during rotation.
### Secrets Management
Source: https://academia.sh/en/courses/backend-production/secrets-management
Three measures of a secret in production: how many requests verify with which version during the window where two keys are valid together while rotating, how long a leaked value stays valid when the cleanup step is skipped, and the false-accept and false-reject counts of a scan that looks for a value hardcoded in the source.
The previous lesson hardened the transport: an incoming request now enters through an encrypted
channel, and the headers telling the browser what it may do are set server-side. With both ends
closed, what remains is what the channel carries. The loan system's seven services prove who they
are to each other with a **signing key**; the catalog ID, the payment key, and the notification
token belong to the same class. These are **secrets**, and the transport layer protects them only
on the network — not where they sit.
Distributing these values to the **test environment** was measured in the Secrets and Data
Management lesson: which value enters, how much scope it takes, and how many points it leaks from
in a run's artifacts. Production asks a different question. There the value is already in, its
scope already wide, and no one takes it back after a run. What is measured here is three
quantities: how many places the value **stands in**, how many requests break while **rotating**,
and, once it leaks, **how long** it keeps working.
**AS10 — seven services share a single signing-key ring.** Both the issuer and the verifier of a
ticket read from the same ring; the ring is versioned, and the store keeps the signing and
verifying version sets apart. **AS11 — a service ticket is valid for 120 seconds**, and a service
does not refresh it until it expires. **AS12 — the new value reaches services one at a time, 45
seconds apart**; the last of the seven reads it 270 seconds after the rotation moment.
#### Where the Value Stands
The store gives verification the versions to accept as a set **bound to time**: only the old
version before the rotation moment, both through the window, only the new one after it closes.
```js
// vault.mjs — the secret store: versioned key ring, the signing and verification set
import { createHmac, timingSafeEqual } from 'node:crypto';
export const TICKET_LIFETIME = 120; // AS11: a service ticket is valid for 120 seconds
export const ROTATION_MOMENT = 300; // the moment the new version is produced
export const ROLLOUT_INTERVAL = 45; // AS12: services read the new value 45 seconds apart
export const HORIZON = 900;
export const SERVICES = ['catalog', 'loan', 'membership', 'notification', 'penalty', 'search', 'reporting'];
// The store keeps two sets apart: the signer has one version, the verifier one or two.
export function openVault(window) {
const key = { 1: 'v1-ring-4c9a1d', 2: 'v2-ring-77e5b0' };
const closing = window === Infinity ? Infinity : ROTATION_MOMENT + window;
return {
verifying: (t) => (t < ROTATION_MOMENT ? [1] : t < closing ? [1, 2] : [2]),
sign: (version, body) => createHmac('sha256', key[version]).update(body).digest('base64url'),
verify(t, body, signature) {
for (const version of this.verifying(t)) {
const expected = Buffer.from(this.sign(version, body));
const incoming = Buffer.from(signature);
if (expected.length === incoming.length && timingSafeEqual(expected, incoming)) return version;
}
return 0;
},
};
}
// How many places the value stands: every service reads it in both its process and its scheduled job.
export const PLACEMENT = {
distributed: { locations: SERVICES.length * 2, reads: 0 },
centralized: { locations: 1, reads: SERVICES.length },
};
```
The last lines meet this course's third question: how many places the setting repeats in. Standing
separately in every service's process and scheduled job puts it in fourteen places; read from a
single store at run time, it is in one place with seven read points. The difference shows up in
the cost of rotation.
#### The Rotation Window
The run below stands up a real verification service and walks a 900-second timeline in
five-second steps. Time is virtual: every request reports the time it carries in a header, so
counts do not depend on the wall clock. Window width is swept, and each width runs twice — once
with services refreshing tickets in the ordinary order, once with **the worst alignment**, where
each service refreshes its last old ticket one second before reading the new value.
```js
// rotation.mjs — overlap-window scan: which key verified, how many requests were rejected
import { createServer } from 'node:http';
import { openVault, TICKET_LIFETIME, ROTATION_MOMENT, ROLLOUT_INTERVAL, HORIZON, SERVICES, PLACEMENT } from './vault.mjs';
let vault = openVault(0);
const server = createServer((request, response) => {
const t = Number(request.headers['x-time']);
const body = request.headers['x-ticket'];
const issued = Number(body.split('.')[1]);
const version = vault.verify(t, body, request.headers['x-signature'] ?? '');
const [status, result] = version === 0 ? [401, 'sig'] : t >= issued + TICKET_LIFETIME ? [401, 'expired'] : [200, version];
response.writeHead(status, { 'content-type': 'application/json' });
response.end(JSON.stringify({ result }));
});
server.listen(8951, '127.0.0.1');
const BASE = 'http://127.0.0.1:8951/loan';
// Service i reads the new value at ROTATION_MOMENT + i*ROLLOUT_INTERVAL; it holds its ticket until it expires.
const readMoment = (i) => ROTATION_MOMENT + i * ROLLOUT_INTERVAL;
const LAST_READ = readMoment(SERVICES.length - 1);
// align: each service's last v1 ticket is refreshed one second before it reads the new value (worst case).
async function run(window, align = false) {
vault = openVault(window);
const ticket = SERVICES.map(() => null);
const aligned = SERVICES.map(() => false);
const counter = { 1: 0, 2: 0, sig: 0, expired: 0 };
let lastReject = null;
for (let t = 0; t <= HORIZON; t += 5) {
const i = (t / 5) % SERVICES.length;
const worst = align && !aligned[i] && ticket[i] !== null && t >= readMoment(i);
if (ticket[i] === null || t >= ticket[i].issued + TICKET_LIFETIME || worst) {
const issued = worst ? readMoment(i) - 1 : t;
const version = issued < readMoment(i) ? 1 : 2;
const body = `${SERVICES[i]}.${issued}`;
ticket[i] = { version, body, signature: vault.sign(version, body), issued };
aligned[i] ||= worst;
}
const y = await fetch(BASE, { headers: { 'x-time': String(t), 'x-ticket': ticket[i].body, 'x-signature': ticket[i].signature } });
const { result } = await y.json();
counter[result] += 1;
if (y.status === 401) lastReject = t;
}
// A leaked old version tried at the horizon: does the value still open the door.
const body = `search.${HORIZON}`;
const leaked = await fetch(BASE, { headers: { 'x-time': String(HORIZON), 'x-ticket': body, 'x-signature': vault.sign(1, body) } });
return { ...counter, lastReject, leaked: leaked.status };
}
const s = (x, n) => String(x).padStart(n);
console.log(`${'window'.padEnd(12)}${s('v1', 6)}${s('v2', 6)}${s('reject', 9)}${s('last reject', 12)}`
+ `${s('worst reject', 13)}${s('leaked v1', 10)}`);
for (const window of [0, 100, 200, 300, 400, Infinity]) {
const r = await run(window);
const k = await run(window, true);
const label = window === Infinity ? 'no cleanup' : `${window} s`;
console.log(`${label.padEnd(12)}${s(r[1], 6)}${s(r[2], 6)}${s(r.sig + r.expired, 9)}`
+ `${s(r.lastReject ?? '-', 12)}${s(k.sig + k.expired, 13)}${s(r.leaked, 10)}`);
}
server.close();
const required = LAST_READ - ROTATION_MOMENT + TICKET_LIFETIME;
console.log(`\nlast read ${LAST_READ} s, ticket lifetime ${TICKET_LIFETIME} s -> arithmetic lower bound ${required} s`);
for (const [label, y] of Object.entries(PLACEMENT)) {
console.log(`${label}: the value stands in ${y.locations} places, ${y.reads} reads at run time, `
+ `4 rotations a year make ${y.locations * 4} manual steps`);
}
const day = 24 * 60;
for (const period of [90, 30, 7]) {
console.log(`rotation period ${period} days -> average validity of a leaked key ${period / 2} days, `
+ `worst case ${period} days; a ticket leaked at the same moment ${TICKET_LIFETIME / 60} minutes (${(period * day / (TICKET_LIFETIME / 60)).toFixed(0)}x shorter)`);
}
```
```
window v1 v2 reject last reject worst reject leaked v1
0 s 60 85 36 555 51 401
100 s 80 85 16 555 31 401
200 s 90 85 6 555 15 401
300 s 96 85 0 - 3 401
400 s 96 85 0 - 0 401
no cleanup 96 85 0 - 0 200
last read 570 s, ticket lifetime 120 s -> arithmetic lower bound 390 s
distributed: the value stands in 14 places, 0 reads at run time, 4 rotations a year make 56 manual steps
centralized: the value stands in 1 places, 7 reads at run time, 4 rotations a year make 4 manual steps
rotation period 90 days -> average validity of a leaked key 45 days, worst case 90 days; a ticket leaked at the same moment 2 minutes (64800x shorter)
rotation period 30 days -> average validity of a leaked key 15 days, worst case 30 days; a ticket leaked at the same moment 2 minutes (21600x shorter)
rotation period 7 days -> average validity of a leaked key 3.5 days, worst case 7 days; a ticket leaked at the same moment 2 minutes (5040x shorter)
```
#### Reading the Window
The first row is rotation without a window, and it is this course's clearest example of the rule.
The moment the new version is produced, verification stops accepting the old one; **there is no
configuration error anywhere** — store, signature, services all fine. Still, 36 of 181 requests
are rejected, and not at the rotation moment: they run **all the way to 255 seconds after** it,
last reject at the 555th second. The cause is a service that does not drop its held ticket before
it expires; the last to read still holds one signed with the old version until the 570th second.
As the window widens, rejects drop from 36 to 16, to 6, to zero — but note which width reaches
zero: 300 seconds sufficed here, yet the same width rejects three requests under the worst
alignment. The run-independent quantity sits below the table: last-read moment minus rotation
moment, plus ticket lifetime — 270 + 120 = **390 seconds**. The measured value (300) gives only
that run's alignment, not the correct setting; a team picking the window from a measurement will
drop requests again the moment the schedule slips, and those drops, arriving four minutes after
the rotation, will not trace back to the cause.
The last row lies in the opposite direction. In the **no-cleanup** setup, verification accepts
both versions forever: drops zero, signature errors zero, dashboard clean, rotation looking
"successful." An attempt at the 900th second with the leaked old value still gets **200** — the
other five rows get 401. Producing the new version does not invalidate the old one; the **cleanup
step** does, and skipping it worsens no measure at all. That is the whole point of rotation, and
it fails silently at exactly that step.
#### The Lifetime of a Leaked Value
When a leaked value is noticed, the question is not "how long did it work" but "how long will it
keep working," and the answer depends on the rotation period. Since the leak can fall anywhere
within the period, expected exposure is half the period and the worst case is the whole period:
forty-five days in a ninety-day period, fifteen in a thirty-day one, three and a half in a
seven-day one. This shows what shortening the period buys — not closing the door, but **how long
it stays open**.
The second half of the same lines shows the real lever. If what leaks is a ticket produced with
the key rather than the key itself, its lifetime is 120 seconds — sixty-four thousand times
shorter than a ninety-day period. Holding the long-lived value in few places and deriving a
short-lived ticket from it shortens the exposure window by four orders of magnitude, independent
of rotation frequency. The placement rows write out the price: fourteen locations means fifty-six
manual steps a year for four rotations, any of which can be missed; one location means four steps,
with seven services reading it at run time.
#### The Value Left in the Code
All the arithmetic above assumes the value stands in **configuration**. A value sitting in the
code does not go invalid on rotation — the old value keeps signing until the code ships, so the
last-read moment shifts back by the release cycle's length, and the required window grows with
it. This is why scanning for a hardcoded value is part of the rotation mechanism.
```js
// scan.mjs — scanning for a value hardcoded in the source: false accept and false reject
// [file, line, is it really a secret]
const source = [
['loan/ticket.mjs', "const SIGNING_KEY = 'v2-ring-77e5b0';", true],
['report/export.mjs', "const STORE_PASSWORD = 'k3M-pW9-zQ4-vB7-nR2';", true],
['notification/send.mjs', "const config = { upstream: 'T-4f8a1c9d2b6e0357a9c4f1b8' };", true],
['search/index.mjs', "const refresh = 'Zq8Rv3Nw6Lp1Ke5Ty9Xu2Bd7Hm4';", true],
['membership/test.mjs', "const TEST_TOKEN = 'sb-9d31-77af-2c60';", true],
['catalog/record.mjs', "const CATALOG_ID = 'ac-catalog-public-record';", false],
['loan/config.mjs', "const KEY_NAME = 'signing-key-v2';", false],
['ui/icon.mjs', "const ICON = 'iVBORw0KGgoAAAANSUhEUgAAAAgAAAAI';", false],
['penalty/config.mjs', "const secretDir = './data/tmp/penalty';", false],
['notification/sample.mjs', "const TOKEN_PLACEHOLDER = 'CHANGE-ME-DURING-SETUP';", false],
['membership/identity.mjs', "const memberId = '9f2c1a4e-77b0-4d31-8c6a-2e5b90f14d73';", false],
];
const value = (line) => (line.match(/'([^']+)'/) ?? [, ''])[1];
const name = (line) => (line.match(/(?:const|let)\s+(\w+)|(\w+)\s*:/) ?? [, ''])[1] ?? '';
function entropy(d) {
const count = {};
for (const c of d) count[c] = (count[c] ?? 0) + 1;
return -Object.values(count).reduce((a, n) => a + (n / d.length) * Math.log2(n / d.length), 0);
}
const NAME_PATTERN = /key|secret|passphrase|token|signature|password/i;
const ALLOWED = [/^ac-/, /^CHANGE-ME/];
const rules = {
'name pattern': (s) => NAME_PATTERN.test(name(s)) && value(s).length >= 8,
'entropy': (s) => value(s).length >= 20 && entropy(value(s)) >= 3.6,
'union': (s) => rules['name pattern'](s) || rules.entropy(s),
'union + allowlist': (s) => rules.union(s)
&& !ALLOWED.some((r) => r.test(value(s))) && !value(s).includes('/'),
};
const s = (x, n) => String(x).padStart(n);
console.log(`${'rule'.padEnd(24)}${s('hits', 7)}${s('caught', 11)}${s('false accept', 14)}${s('false reject', 14)}`);
for (const [rname, rule] of Object.entries(rules)) {
const hits = source.filter(([, line]) => rule(line));
const correct = hits.filter(([, , secret]) => secret).length;
console.log(`${rname.padEnd(24)}${s(hits.length, 7)}${s(`${correct}/5`, 11)}`
+ `${s(5 - correct, 14)}${s(hits.length - correct, 14)}`);
}
console.log('\nreal secrets the name pattern missed:');
for (const [file, line, secret] of source) {
if (secret && !rules['name pattern'](line)) {
console.log(` ${file.padEnd(24)} name=${name(line).padEnd(10)} `
+ `length=${s(value(line).length, 2)} entropy=${entropy(value(line)).toFixed(2)}`);
}
}
console.log('harmless strings the entropy rule falsely rejected:');
for (const [file, line, secret] of source) {
if (!secret && rules.entropy(line)) {
console.log(` ${file.padEnd(24)} length=${s(value(line).length, 2)} `
+ `entropy=${entropy(value(line)).toFixed(2)} allowed=${ALLOWED.some((r) => r.test(value(line)))}`);
}
}
```
```
rule hits caught false accept false reject
name pattern 6 3/5 2 3
entropy 5 2/5 3 3
union 10 5/5 0 5
union + allowlist 7 5/5 0 2
real secrets the name pattern missed:
notification/send.mjs name=config length=26 entropy=4.09
search/index.mjs name=refresh length=27 entropy=4.75
harmless strings the entropy rule falsely rejected:
catalog/record.mjs length=24 entropy=3.61 allowed=true
notification/sample.mjs length=22 entropy=3.75 allowed=true
membership/identity.mjs length=36 entropy=4.00 allowed=false
```
The name pattern alone finds three of the five real values. The two it misses sit at the end of
the list by name: the variable is named `config` and `refresh`. A scan relying on a naming
convention cannot see a line that does not follow it, and whoever hardcoded a secret has already
not followed it. Entropy catches these two but is worse alone: two of five. Short, regular-shaped
values — a nineteen-character password, a hyphen-split token — stay under the threshold.
The union of the two finds all five, at a cost of five false rejects. Three are eliminable by
rule: the open catalog ID's prefix, the setup placeholder, the directory path carrying a slash.
The remaining two cannot be eliminated — one is a constant holding the key's **name**, the other a
member ID, and both look exactly like a secret. The trade-off here runs through the whole course:
a false accept costs a value stuck in the code, unrotatable; a false reject costs a two-line
manual review. Tuning the threshold to the second cost means accepting the first.
#### Summary
- The three measures of a secret in production are how many places it stands in, how many
requests it breaks while rotating, and how long it works once it leaks; the leak surface
measured in the test environment is already at its widest here.
- Rotation without a window rejected 36 of 181 requests, running until 255 seconds after the
rotation; no component raised an error.
- The required window is read from arithmetic, not measurement: last-read moment minus rotation
moment plus ticket lifetime, 390 seconds here. This run's 300 seconds also gave zero rejects;
the worst alignment gave three.
- Skipping the cleanup step keeps dropped requests at zero, the dashboard clean, and a leaked old
key still gets 200 at the 900th second; only the cleanup step completes a rotation.
- Holding the key in few places and deriving a 120-second ticket from it shortens a leaked value's
lifetime sixty-four thousand times against a ninety-day period; fourteen locations cost
fifty-six manual steps a year, one location costs four.
- In the hardcoded-value scan, the name pattern caught three of five, entropy two of five; their
union caught five of five for five false rejects — three eliminable by rule, two not.
#### Next Step
Every measurement here stayed inside code we wrote ourselves: we produced the key, chose the
ticket lifetime, set the scan's threshold. But most of the code serving a request is not ours.
When the loan service's process comes up, hundreds of files load, and few are lines the team
wrote; each is pinned to a version number that changes one day. The next lesson treats this
dependency as a configuration decision — counting the trade-off between upgrade frequency and the
accumulating risk versus the versions that break, by running three policies over one dependency
tree.
### Dependency Risk
Source: https://academia.sh/en/courses/backend-production/dependency-risk
A version policy is a configuration decision: five policies are run over the same dependency tree and the accumulated exposure days and broken upgrades are counted; the silent outcome of a loose version range is measured by how many packages diverge between two deployments of the same source version.
Every measurement in the previous lesson stayed inside code we wrote ourselves. When the loan
service's process comes up, most of the loaded files are not lines the team wrote; the tree holds
twenty-four packages, and only eight of their names appear in a manifest file. The remaining
sixteen arrive **transitively**: each is a dependency of a dependency, no one chose it, and no one
wrote its version down.
**Scanning** these dependencies was measured in the Dependency and Component Scanning lesson; the question there
was what the tool finds. Here, what the tool finds is assumed, and the next question is asked:
what happens when a finding arrives depends on a rule. That rule is the **version policy**, and it
is a configuration decision — it is written where a manifest file sets a version constraint, and
its result shows up in a trade-off the team never sees: upgrading more often reduces accumulated
risk and raises the number of broken versions.
**No real package name, version number, or vulnerability record is used.** The tree, the release
calendar, and the record set below are produced with our own generator; the seed is visible and
the run is repeatable.
#### The Tree and the Record Set
**AS13 — the tree has 24 packages: eight direct, sixteen transitive, with a 360-day horizon.**
Every package has a release interval and a break rate: how much of its releases break our code.
**AS15 — a vulnerability record takes between 0 and 90 days to reach us.** The record reaches us
not on the day the vulnerability begins to exist, but on the day it is reported; the exposure
window is counted from the day it exists to the day it closes.
```js
// tree.mjs — our own dependency tree, our own release calendar, our own record set
export const HORIZON = 360; // days
export const SEED = 8617;
let state = SEED;
const rand = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648;
const pick = (arr) => arr[Math.floor(rand() * arr.length)];
// depth 1: package named in our manifest; 2 and 3: transitive, not named.
const DEPTH = [...Array(8).fill(1), ...Array(10).fill(2), ...Array(6).fill(3)];
const INTERVAL = [14, 30, 45, 60, 90]; // release interval (days)
const BREAK_RATE = [0, 0.08, 0.18, 0.35]; // fraction of releases that break us
export const packages = DEPTH.map((depth, i) => {
const interval = pick(INTERVAL);
const breakRate = pick(BREAK_RATE);
// transitive package: the fix cannot be installed until the parent package cuts a new release.
const upstreamDelay = depth === 1 ? 0 : Math.round(pick(INTERVAL) * rand());
const releases = [{ day: 0, breaking: false }];
for (let g = Math.round(rand() * interval); g <= HORIZON; g += interval) {
if (g > 0) releases.push({ day: g, breaking: rand() < breakRate });
}
return { name: `p${String(i + 1).padStart(2, '0')}`, depth, interval, upstreamDelay, releases };
});
// A record: a vulnerability present in a package. day: day it started existing; fixedBy: the
// release that closes it; known: day the record reaches us. reachable: our code path touches that surface.
export const records = [];
for (let k = 0; k < 11; k++) {
const p = Math.floor(rand() * packages.length);
const day = 20 + Math.floor(rand() * (HORIZON - 80));
const fixedBy = packages[p].releases.findIndex((y) => y.day >= day);
if (fixedBy <= 0) continue;
const notice = Math.round(rand() * 90); // AS15: record delay 0-90 days
records.push({ package: p, day, fixedBy, known: day + notice, notice, reachable: rand() < 0.55 });
}
// The constraint sits separately in each service's own manifest.
export const SERVICES = ['catalog', 'loan', 'membership', 'notification', 'penalty', 'search', 'reporting'];
export const constraintLocation = packages.map((p) => (p.depth === 1 ? 1 + Math.floor(rand() * SERVICES.length) : 0));
export const readyDay = (p, i) => packages[p].releases[i].day + packages[p].upstreamDelay;
```
#### Five Policies
**AS14 — code releases happen every fourteen days.** The policies differ at exactly one point: on
which days, and for which packages, the tree changes. The fifth is not a policy but a shape of
constraint — when the version range is left loose, the tree changes on its own at release time,
without anyone touching a file.
```js
// policy.mjs — five version policies on the same tree: open days versus broken upgrades
import { packages, records, constraintLocation, readyDay, SERVICES, HORIZON, SEED } from './tree.mjs';
const DEPLOY = 14; // AS14: release day; the loose range only changes the tree on these days
const policies = {
'fixed': { day: () => false },
'immediate': { day: () => true },
'monthly': { day: (g) => g % 30 === 0 },
'security only': { day: () => true, selective: true },
'loose range': { day: (g) => g % DEPLOY === 0, fileless: true },
};
function run(pol) {
const installed = packages.map(() => 0);
const open = records.map(() => 0);
let upgrades = 0, broken = 0, intervention = 0, brokenDay = 0, mostInDay = 0;
const snapshot = {};
for (let day = 1; day <= HORIZON; day++) {
let dailyUpgrades = 0, dailyBroken = 0;
if (pol.day(day)) {
for (let p = 0; p < packages.length; p++) {
// Selective policy upgrades a package only once its record has reached us.
if (pol.selective && !records.some((k) => k.package === p && day >= k.known && installed[p] < k.fixedBy)) continue;
let target = installed[p];
while (target + 1 < packages[p].releases.length && readyDay(p, target + 1) <= day) target += 1;
if (target === installed[p]) continue;
for (let i = installed[p] + 1; i <= target; i++) if (packages[p].releases[i].breaking) dailyBroken += 1;
dailyUpgrades += 1;
installed[p] = target;
}
}
upgrades += dailyUpgrades;
broken += dailyBroken;
if (dailyUpgrades > 0) intervention += 1;
if (dailyBroken > 0) brokenDay += 1;
mostInDay = Math.max(mostInDay, dailyBroken);
records.forEach((k, i) => { if (day >= k.day && installed[k.package] < k.fixedBy) open[i] += 1; });
if (day === 180 || day === 194) snapshot[day] = [...installed];
}
const reachable = records.map((k, i) => (k.reachable ? open[i] : 0));
return {
intervention, upgrades, broken, brokenDay, mostInDay,
openDays: reachable.reduce((a, x) => a + x, 0),
allDays: open.reduce((a, x) => a + x, 0),
longest: Math.max(...reachable),
fileless: pol.fileless ? upgrades : 0,
diff: snapshot[180].filter((v, i) => v !== snapshot[194][i]).length,
};
}
const s = (x, n) => String(x).padStart(n);
console.log(`seed ${SEED}: ${packages.length} packages, ${records.length} records, horizon ${HORIZON} days`);
console.log(`${'policy'.padEnd(17)}${s('interv.', 10)}${s('upgrades', 11)}${s('broken', 9)}`
+ `${s('broken day', 11)}${s('most', 8)}${s('open day', 10)}${s('longest', 9)}`);
const result = {};
for (const [name, pol] of Object.entries(policies)) {
const r = (result[name] = run(pol));
console.log(`${name.padEnd(17)}${s(r.intervention, 10)}${s(r.upgrades, 11)}${s(r.broken, 9)}`
+ `${s(r.brokenDay, 11)}${s(r.mostInDay, 8)}${s(r.openDays, 10)}${s(r.longest, 9)}`);
}
const g = result['loose range'];
console.log(`\nin the loose range, all ${g.fileless} upgrades arrived without touching a manifest file;`
+ ` the day-180 and day-194 deployments of the same source version diverge on ${g.diff} packages`);
console.log(`the same divergence is ${result.immediate.diff} packages under immediate, ${result.fixed.diff} under fixed`);
const direct = records.filter((k) => packages[k.package].depth === 1);
const delay = records.reduce((a, k) => a + packages[k.package].upstreamDelay, 0);
const notice = records.reduce((a, k) => a + k.notice, 0) / records.length;
console.log(`${direct.length} of the records are in a direct package, ${records.length - direct.length} transitive;`
+ ` ${records.filter((k) => k.reachable).length} are reachable`);
console.log(`total upstream wait for transitive records: ${delay} days; average record delay: ${notice.toFixed(1)} days`);
const rank = (field) => Object.keys(result).sort((a, b) => result[a][field] - result[b][field]).join(' < ');
console.log(`open days with the reachability filter removed: `
+ Object.entries(result).map(([name, r]) => `${name} ${r.allDays}`).join(', '));
console.log(`ranking, filtered: ${rank('openDays')}`);
console.log(`ranking, unfiltered: ${rank('allDays')}`);
console.log(`depth-1 packages: ${packages.filter((p) => p.depth === 1).length}, `
+ `total constraints in service manifests: ${constraintLocation.reduce((a, x) => a + x, 0)} `
+ `(${SERVICES.length} services); most files one package touches: ${Math.max(...constraintLocation)}`);
```
```
seed 8617: 24 packages, 11 records, horizon 360 days
policy interv. upgrades broken broken day most open day longest
fixed 0 0 0 0 0 860 269
immediate 237 330 58 54 3 113 43
monthly 12 214 58 12 8 195 65
security only 14 15 12 7 6 178 67
loose range 25 320 56 24 6 147 49
in the loose range, all 320 upgrades arrived without touching a manifest file; the day-180 and day-194 deployments of the same source version diverge on 14 packages
the same divergence is 13 packages under immediate, 0 under fixed
7 of the records are in a direct package, 4 transitive; 5 are reachable
total upstream wait for transitive records: 51 days; average record delay: 42.8 days
open days with the reachability filter removed: fixed 2181, immediate 349, monthly 520, security only 534, loose range 422
ranking, filtered: immediate < loose range < security only < monthly < fixed
ranking, unfiltered: immediate < loose range < monthly < security only < fixed
depth-1 packages: 8, total constraints in service manifests: 39 (7 services); most files one package touches: 7
```
#### Reading the Trade-Off
The first row is the cost of not upgrading, and it reads zero in the broken-version column. The
tree stays as it was on day one, no release wakes anyone, no test breaks — and the reachable
records stay open for a total of **860 days**, with the longest single one at 269. This row is
this course's rule at its most expensive: a fixed tree is the quietest configuration, and its
quiet is paid for in accumulated risk.
The second row is the other extreme. Daily upgrading brings open days down from 860 to **113**,
roughly sevenfold. Its cost is 237 intervention days and 58 broken versions. The distribution of
the breaks matters: they spread across 54 separate days, with at most three landing on a single
day. Finding which upgrade a break came from is cheap in this distribution.
The third row is where intuition is wrong. Monthly batch upgrading **does not reduce** the number
of broken versions at all — still 58. Batching does not eliminate breaks, it only compresses them
into twelve days, with eight breaks landing on a single day. What is gained is intervention count
(12 instead of 237); what is lost is open days (195 instead of 113, eighty-two more) and the ease
of tracing a break to its source. Batching upgrades does not reduce the cost of breaking; it piles
it onto a single day.
The fourth row upgrades only the package that has a record, bringing intervention down to fifteen
upgrades. Broken versions drop to twelve, because most of the tree is never touched. Against that,
open days are 178 — sixty-five more than daily upgrading — and the longest single window, at 67
days, is the worst of any policy. The cause is in the line below the table: average record delay
is **42.8 days**. A policy that acts on a record has to wait for the record to arrive; a policy
that does not wait has already installed the fix. The security-only policy is cheap, and its
ceiling is not in its own hands.
#### The Days No Policy Can Reach
Part of the open days sits where no policy can reach. Four of the eleven records are in a
transitive package; even once the fix ships, the upstream package cannot be installed until its
parent cuts a new release, and that wait totals **51 days**. These days are the same in all five
rows — a version constraint does not bind a package that is not in a manifest file. Combined with
the 42.8-day record delay, part of what the table shows is a product of the calendar, not the
policy.
The reachability filter carries a warning of its own. Removing it doubles or triples the numbers
(immediate goes from 113 to 349, fixed from 860 to 2181), and the two extremes stay in place — but
the two middle policies swap places. Filtered, monthly looks worse than security-only; unfiltered,
it looks better. Separating the surface our code path actually touches is therefore not a detail
but a decision that flips the ranking.
#### Where the Range Sits
The last row belongs to this course's rule. A loose version range looks good in the numbers: 147
open days, 56 broken versions, 25 interventions. Yet all 320 of those upgrades arrived **without
touching a manifest file**. Its one measurable consequence is the line below the table: the
day-180 and day-194 deployments of the same source version diverge on **fourteen** of twenty-four
packages. The two deployments carry the same code, run with a different tree, and there is no
record anywhere of the difference. Rolling back the code does not roll back the tree.
Under daily upgrading, the same divergence is thirteen packages — nearly the same number. The
difference is the record of the divergence: there, every upgrade is a file change; here, none is.
Under the fixed policy, divergence is zero, and that is exactly why 860 open days get paid.
Where the constraint sits is counted in the last line: eight direct packages are written as a
total of **39 constraints** across seven services' manifest files, and a single package can appear
in as many as seven separate files at once. Responding to one record means finding all seven of
the seven files; an upgrade that finds six and misses one raises no error — it just leaves one
service on the old version.
#### Summary
- Never upgrading produced zero breaks and 860 open days; the longest single window ran 269 days.
The cost of a quiet configuration here is accumulated risk.
- Daily upgrading brought open days down to 113; its cost was 237 intervention days and 58 broken
versions spread across 54 days.
- Monthly batch upgrading did not reduce the broken-version count (still 58); it piled them into
twelve days, up to eight in a single day, and raised open days to 195.
- The security-only policy ran on fifteen upgrades but, because of a 42.8-day average record
delay, pushed the longest window to 67 days; its ceiling is not in its own hands.
- Transitive packages' upstream wait is 51 days and closes under no policy; the reachability
filter swaps the order of the two middle policies.
- In the loose range, all 320 upgrades arrived without touching a file, and two deployments of the
same source version diverged on fourteen packages; the numbers look good while the tree has no
record.
#### Next Step
Two lessons measured two configuration decisions, and both left a question open. Who rotated a
key, when, with which version; which day the tree changed when a package was upgraded; which
deployment the fourteen diverging packages in the loose range arrived in — none of these questions
has an answer sitting inside the system. The application's own logs write the request and the
error, not the decision. The next lesson takes on this gap: it counts which events get logged, how
to verify a log has not been altered afterward, and how much personal data the log itself carries.
### Audit Logging
Source: https://academia.sh/en/courses/backend-production/audit-logging
Four measures of the traceability requirement: which decision points get logged and, as a result, how many questions can be answered, what the hash chain catches — a change to the body — and what it cannot — cutting the tail — the personal data the log itself carries, and the effect of retention on volume.
The previous two lessons measured two configuration decisions, and both left a question open:
who, and when. The application's own logs write the request and the error; they do not write
which staff member waived which member's penalty. The **audit log** fills this gap, and it is
separated by purpose from the **audit trail** in the Case Studies course: there, the trail was a
data-structure decision for reconstructing state; here, the log is kept to **show, after the
fact**, who was responsible for a decision, and so it must not have been altered itself.
Traceability is measured with four questions. Which events are logged — how many of the system's
decision points are written. Can the log prove it has not been altered afterward. What does the
log itself carry. And how long is it kept. All four are configuration decisions, and the system
raises no error when they are set wrong.
#### Decision Points
**AS16 — the loan system has twelve decision points**, and their daily volumes sit in the table
below; the total is 3,012 events. **AS17 — every record carries four fields:** kind, actor,
target, result. The store sits on `node:sqlite`, and every record's digest also covers the
previous one's.
```js
// log.mjs — the audit log store: decision points, hash chain, verification
import { createHash } from 'node:crypto';
import { DatabaseSync } from 'node:sqlite';
export const SEED = 4231;
let state = SEED;
const rand = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648;
const pick = (n) => Math.floor(rand() * n);
// Decision points: [kind, daily count, changes state?, actor class, target class]
export const KINDS = [
['LOGIN', 1200, false, 'member', 'self'], ['LOGIN_REJECTED', 60, false, 'member', 'self'],
['VIEW', 900, false, 'staff', 'member'], ['LOAN', 400, true, 'member', 'catalog'],
['RETURN', 380, true, 'member', 'catalog'], ['CATALOG_EDIT', 45, true, 'staff', 'catalog'],
['PENALTY_WAIVE', 12, true, 'staff', 'member'], ['EXPORT', 4, false, 'staff', 'member'],
['ROLE_CHANGE', 5, true, 'staff', 'member'], ['MEMBER_DELETE', 3, true, 'staff', 'member'],
['ROLLOUT', 2, true, 'staff', 'system'], ['ROTATION', 1, true, 'staff', 'system'],
];
export const MEMBERS = 40, STAFF = 6, CATALOG_SIZE = 60;
export function generateDay() {
const events = [];
for (const [kind, count, writes, actorClass, targetClass] of KINDS) {
for (let i = 0; i < count; i++) {
const actor = actorClass === 'member' ? `U-${1 + pick(MEMBERS)}` : `G-${1 + pick(STAFF)}`;
const target = targetClass === 'self' ? actor
: targetClass === 'member' ? `U-${1 + pick(MEMBERS)}`
: targetClass === 'catalog' ? `K-${800 + pick(CATALOG_SIZE)}` : 'system';
events.push({ kind, actor, target, writes, seq: rand(), result: kind === 'LOGIN_REJECTED' ? 'reject' : 'accept' });
}
}
return events.sort((a, b) => a.seq - b.seq);
}
// Hash chain: every record's digest also covers the previous one's digest.
export const digest = (seq, previous, body) =>
createHash('sha256').update(`${seq}|${previous}|${body}`).digest('hex').slice(0, 16);
export function openLog(events, alias = null) {
const db = new DatabaseSync(':memory:');
db.exec('create table log (seq integer primary key, kind text, actor text, target text, result text, body text, digest text)');
const insert = db.prepare('insert into log values (?, ?, ?, ?, ?, ?, ?)');
let previous = 'start';
events.forEach((o, i) => {
const [actor, target] = alias ? [alias(o.actor), alias(o.target)] : [o.actor, o.target];
const body = JSON.stringify({ kind: o.kind, actor, target, result: o.result });
previous = digest(i + 1, previous, body);
insert.run(i + 1, o.kind, actor, target, o.result, body, previous);
});
return db;
}
// Verification walks the chain from the start; returns the seq of the first mismatch, else 0.
export function verify(db) {
let previous = 'start';
for (const k of db.prepare('select * from log order by seq').all()) {
previous = digest(k.seq, previous, k.body);
if (previous !== k.digest) return k.seq;
}
return 0;
}
```
#### Three Logging Policies
There are three common ways to choose which events to log, and all three look defensible: writing
only the operations that change state, writing only rejected authorization requests, writing every
decision point. The way to measure the difference between them is not volume but **how many
questions can be answered**. The run below asks eight accountability questions against all three
logs as real queries; the questions' targets are read from the log itself.
```js
// measurement.mjs — three log policies, what the chain catches and what it does not, the log's own privacy
import { createHash } from 'node:crypto';
import { generateDay, openLog, verify, digest, KINDS, SEED } from './log.mjs';
const events = generateDay();
const full = openLog(events);
const one = (s) => full.prepare(s).get();
const penaltyMember = one("select target from log where kind='PENALTY_WAIVE'").target;
const deletedMember = one("select target from log where kind='MEMBER_DELETE'").target;
const catalogItem = one("select target from log where kind='CATALOG_EDIT'").target;
const topViewer = one("select actor from log where kind='VIEW' group by actor order by count(*) desc").actor;
const questions = [
[`who waived ${penaltyMember}'s penalty`, `select actor from log where kind='PENALTY_WAIVE' and target='${penaltyMember}'`],
[`how many distinct members did ${topViewer} view`, `select distinct target from log where kind='VIEW' and actor='${topViewer}'`],
[`who edited ${catalogItem}'s record`, `select actor from log where kind='CATALOG_EDIT' and target='${catalogItem}'`],
['which account had more than four failed logins', "select target from log where kind='LOGIN_REJECTED' group by target having count(*) > 4"],
['who rotated the signing key', "select actor from log where kind='ROTATION'"],
['who started the rollout', "select actor from log where kind='ROLLOUT'"],
[`who accessed ${penaltyMember}'s data`, `select distinct actor from log where target='${penaltyMember}' and kind in ('VIEW','EXPORT')`],
[`who accessed deleted member ${deletedMember}'s record`, `select distinct actor from log where target='${deletedMember}' and kind='VIEW'`],
];
const policies = {
'writes only': (o) => o.writes,
'rejects only': (o) => o.result === 'reject',
'all decisions': () => true,
};
const s = (x, n) => String(x).padStart(n);
console.log(`seed ${SEED}: ${KINDS.length} decision points, ${events.length} events/day`);
console.log(`${'policy'.padEnd(20)}${s('records', 8)}${s('bytes', 8)}${s('answered', 17)}${s('unanswered', 18)}`);
const answer = {};
for (const [name, filter] of Object.entries(policies)) {
const db = openLog(events.filter(filter));
const bytes = db.prepare('select sum(length(body)) as b, count(*) as n from log').get();
answer[name] = questions.map(([, sql]) => db.prepare(sql).all().length > 0);
const count = answer[name].filter(Boolean).length;
console.log(`${name.padEnd(20)}${s(bytes.n, 8)}${s(bytes.b, 8)}${s(`${count}/8`, 17)}`
+ `${s(answer[name].map((v, i) => (v ? '' : i + 1)).filter(Boolean).join(',') || '-', 18)}`);
}
console.log(questions.map(([name], i) => ` ${i + 1}. ${name}`).join('\n'));
// Chain: change the body, refresh the digest too, truncate the tail.
const corrupted = openLog(events);
const victim = 500;
const old = corrupted.prepare('select body, digest from log where seq=?').get(victim);
const updated = old.body.replace('"result":"accept"', '"result":"reject"');
corrupted.prepare('update log set body=? where seq=?').run(updated, victim);
console.log(`\nverification while the chain is clean: ${verify(full)}`);
console.log(`record ${victim}'s body was changed -> first mismatch at seq ${verify(corrupted)}`);
const previous = corrupted.prepare('select digest from log where seq=?').get(victim - 1).digest;
corrupted.prepare('update log set digest=? where seq=?').run(digest(victim, previous, updated), victim);
console.log(`digest refreshed too -> first mismatch at seq ${verify(corrupted)}`);
const truncated = openLog(events);
truncated.prepare('delete from log where seq > ?').run(events.length - 25);
console.log(`last 25 records deleted -> verification ${verify(truncated)} (0 = no mismatch)`);
for (const anchor of [Infinity, 500, 100, 10]) {
const silent = anchor === Infinity ? events.length : events.length % anchor;
const label = anchor === Infinity ? 'no anchor ' : `anchor every ${s(anchor, 3)}`;
console.log(`${label} -> ${s(silent, 4)} records can be silently deleted in this run, worst case ${anchor === Infinity ? silent : anchor - 1}`);
}
// The log's own privacy.
const personal = full.prepare("select count(*) as n from log where actor like 'U-%' or target like 'U-%'").get().n;
const history = full.prepare("select actor, count(*) as n from log where kind='LOAN' group by actor order by n desc").all();
const salted = (salt) => (d) => (d.startsWith('U-') ? `T-${createHash('sha256').update(salt + d).digest('hex').slice(0, 8)}` : d);
const pseudonymous = openLog(events, salted('period-1'));
const pseudonymousHistory = pseudonymous.prepare("select actor, count(*) as n from log where kind='LOAN' group by actor order by n desc").all();
const period2 = new Set(history.map((g) => salted('period-2')(g.actor)));
const shared = pseudonymousHistory.filter((g) => period2.has(g.actor)).length;
console.log(`\nrecords identifying a person: ${personal}/${events.length} (${(100 * personal / events.length).toFixed(1)}%)`);
console.log(`members whose loan history can be built: with open identity ${history.length}, longest list ${history[0].n} records`);
console.log(`same measure on the pseudonymous log: ${pseudonymousHistory.length} members, longest list ${pseudonymousHistory[0].n} records`);
console.log(`shared pseudonym across two periods when the period salt changes: ${shared}`);
const bytes = full.prepare('select sum(length(body)) as b from log').get().b;
for (const day of [30, 180, 3650]) {
console.log(`retention ${s(day, 4)} days -> ${s(events.length * day, 9)} records, `
+ `${s((bytes * day / 1048576).toFixed(1), 7)} MiB, person-days ${history.length * day}`);
}
```
```
seed 4231: 12 decision points, 3012 events/day
policy records bytes answered unanswered
writes only 848 56159 4/8 2,4,7,8
rejects only 60 4412 1/8 1,2,3,5,6,7,8
all decisions 3012 194726 8/8 -
1. who waived U-12's penalty
2. how many distinct members did G-1 view
3. who edited K-834's record
4. which account had more than four failed logins
5. who rotated the signing key
6. who started the rollout
7. who accessed U-12's data
8. who accessed deleted member U-27's record
verification while the chain is clean: 0
record 500's body was changed -> first mismatch at seq 500
digest refreshed too -> first mismatch at seq 501
last 25 records deleted -> verification 0 (0 = no mismatch)
no anchor -> 3012 records can be silently deleted in this run, worst case 3012
anchor every 500 -> 12 records can be silently deleted in this run, worst case 499
anchor every 100 -> 12 records can be silently deleted in this run, worst case 99
anchor every 10 -> 2 records can be silently deleted in this run, worst case 9
records identifying a person: 2964/3012 (98.4%)
members whose loan history can be built: with open identity 40, longest list 18 records
same measure on the pseudonymous log: 40 members, longest list 18 records
shared pseudonym across two periods when the period salt changes: 0
retention 30 days -> 90360 records, 5.6 MiB, person-days 1200
retention 180 days -> 542160 records, 33.4 MiB, person-days 7200
retention 3650 days -> 10993800 records, 677.8 MiB, person-days 146000
```
#### The Answer to Eight Questions
The policy that logs only writes keeps 848 of the daily 3,012 events and answers four of eight
questions. The four left unanswered are all the same class: each asks about a **read**. How many
members' records a staff member viewed, who accessed a member's data, who viewed a deleted
member's record before deletion — none of these change state, so none is written. Asked about the
traceability of access to personal data, this policy is structurally unable to answer, and nothing
in its configuration flags that.
The policy logging only rejected requests is cheapest at 60 records and answers one of eight —
only the question on failed logins. Its silent assumption is that abuse shows up in rejected
requests. Yet a staff member with authorization viewing nine hundred records is never rejected,
and never appears in this log at all.
Logging every decision point gives eight of eight, at roughly three and a half times the volume
(194,726 bytes against 56,159 here). Where the difference comes from matters: 2,100 of the 3,012
events are logins and views — the expensive part of traceability is exactly what the writes-only
policy skips. Counting decision points thus comes before any argument about volume — which
questions can be answered is set by which events are written.
#### What the Chain Does Not Catch
If a log is kept to show accountability, it must be able to show it has not been altered
afterward. The hash chain provides this only in part. When the result field in record 500's body
is changed from `accept` to `reject`, verification stops at exactly that point. If whoever changed
it also recomputes that record's digest, the mismatch shifts to the next record (501): because
every digest covers the one before it, fixing a single record requires rewriting every record
after it.
Cutting the tail is different. When the last twenty-five records are deleted, verification returns
**zero** — no mismatch at all. The remaining chain is intact, because the chain shows only its own
**internal consistency**; it does not know how many records ought to exist. The only way to know a
record was deleted is to write the chain's digest at some point outside the log, under a separate
responsibility. The anchor period measures this gap: no anchor lets all 3,012 of the day's records
be silently deleted; an anchor every five hundred caps the worst case at 499; every ten, at 9. The
wrong configuration here is the record's **absence**, and an absent record breaks no verification.
#### The Log's Own Risk
**98.4%** of this day's records identify a person. An audit log, by definition, writes who did
what; the moment it is collected, it becomes the system's densest set of personal data. From the
loan records, the full reading history of all forty members can be built, with the longest list at
eighteen records. In a library system, this is the list of what members have read, and it keeps
standing in the audit log even after it has been deleted from the loan service's own database
following a return.
Pseudonymizing the member ID with a salted hash changes this measure **not at all**: still forty
members, still an eighteen-record longest list. A pseudonym hides identity, not the **link**; the
same pseudonym keeps tying all of one person's records together. The only setting that breaks the
link is changing the salt per period, and its measure is in the last line: the two periods share
zero pseudonyms. The cost sits there too — once the period salt is discarded, the seventh question
cannot be asked of past periods, because the member's pseudonym for that period can no longer be
recomputed.
Retention is the last setting, and it grows two measures at once. Thirty days comes to 5.6 MiB and
1,200 person-days; ten years comes to 677.8 MiB and 146,000 person-days. The volume side is cheap
— even ten years of log stays under a gigabyte, so extending retention raises no alarm anywhere.
What grows is the other half of the cost: an audit log kept for ten years is a data set holding ten
years of reading history in one place, and when it leaks, what leaks is not a list of requests but
a list of people.
#### Summary
- The audit log is separated by purpose from the audit trail: the trail reconstructs state, the
log shows a decision's accountable party, and so must also prove its own immutability.
- Logging only writes left four of eight questions unanswered, and every unanswered one asked
about a read; logging only rejected requests answered one of eight.
- Logging every decision point gave eight of eight and multiplied volume roughly three and a half
times; the entire difference comes from login and view records.
- The hash chain caught a body change at seq 500, a change with a refreshed digest at seq 501; it
never caught the deletion of the last twenty-five records. The anchor period is the measure of
this silent gap: 3,012 with no anchor, worst case 499 with an anchor every five hundred.
- 98.4% of records identify a person, and the full reading history of all forty members can be
built; pseudonymization did not change this number, only a per-period salt zeroed the link
between two periods.
- Extending retention to ten years comes to 677.8 MiB and 146,000 person-days; the volume is too
small to raise an alarm, and person-days is the real cost of the retention decision.
#### Next Step
The audit log asks afterward and answers well, but it stops nothing. This day's log holds twelve
hundred logins, sixty failed logins, and nine hundred views; the log writes who they came from,
not that someone **should not reach** that count. In a forty-member system, sixty failed logins
piling onto a single account and sixty spread across forty accounts produce the same log volume,
and the two are entirely different things. The next lesson takes on this distinction: who the
limit is applied to, how the response itself gives away whether an account exists, and how long
lockout keeps the legitimate account out.
### Abuse Defense
Source: https://academia.sh/en/courses/backend-production/abuse-defense
Two configuration decisions in abuse defense: which key the limit is applied to (client, account, resource) and whether the response gives away whether a record exists; the attempts lockout stops and the legitimate members it locks out are counted in the same run.
The audit log asks afterward and answers well, but it stops nothing. Part of what reaches the loan
system's exposed endpoints neither forces a vulnerability nor breaks a rule; it is **too much** —
attempts searching for an account's password, requests polling member IDs in sequence, a client
pulling the entire catalog. The mechanism that stops these is rate limiting, and its
algorithm and placement were measured in earlier courses in this curriculum; both are **taken as
input** here. This lesson measures two different decisions: **who** the limit is applied to, and
**what** the application answers.
#### Who the Limit Is Applied To
**AS18 — traffic has four components:** forty members' ordinary requests (twelve sharing the
library branch's common egress address), two hundred distributed attempts targeting a single
account, three hundred identity probes from a single address, and two thousand record pulls from
one authenticated account. The limit value and the algorithm are the same across all five schemes;
the only thing that changes is which field the counter is bound to.
```js
// key.mjs — what the limit applies to: client, account, target
const LIMIT = 20; // requests per window; algorithm and placement are taken as input
const requests = [];
const add = (n, y) => { for (let i = 0; i < n; i++) requests.push(y(i)); };
// Good traffic: 40 members, each 1 login (unauthenticated) + 5 authenticated requests; 12 share the branch address.
for (let u = 1; u <= 40; u++) {
const address = u <= 12 ? 'A-branch' : `A-member-${u}`;
add(1, () => ({ address, account: null, target: `U-${u}`, bad: false }));
add(5, () => ({ address, account: `U-${u}`, target: `U-${u}`, bad: false }));
}
// Distributed brute force: 200 separate addresses, one account.
add(200, (i) => ({ address: `A-distributed-${i}`, account: null, target: 'U-17', bad: true }));
// Enumeration: one address, 300 separate targets.
add(300, (i) => ({ address: 'A-scan', account: null, target: `U-${1000 + i}`, bad: true }));
// Scraping: one authenticated account, 2000 separate resources.
add(2000, (i) => ({ address: 'A-scrape', account: 'U-39', target: `K-${i}`, bad: true }));
// A scheme binds a request to one or more keys and a limit for each key.
const schemes = {
'client': (r) => [['a', r.address, LIMIT]],
'account': (r) => (r.account ? [['h', r.account, LIMIT]] : []),
'target': (r) => [['t', r.target, LIMIT]],
'combined': (r) => [['a', r.address, LIMIT], ['t', r.target, LIMIT], ...(r.account ? [['h', r.account, LIMIT]] : [])],
'layered': (r) => (r.account ? [['h', r.account, LIMIT]] : [['a', r.address, LIMIT], ['t', r.target, LIMIT]]),
};
function measure(scheme, normalize = (d) => d, stream = requests) {
const counter = new Map();
let stoppedBad = 0, passedBad = 0, falseReject = 0;
for (const r of stream) {
let drop = false;
for (const [kind, raw, limit] of scheme(r)) {
const key = `${kind}:${normalize(raw)}`;
const n = (counter.get(key) ?? 0) + 1;
counter.set(key, n);
if (n > limit) drop = true;
}
if (r.bad) drop ? stoppedBad++ : passedBad++;
else if (drop) falseReject++;
}
return { stoppedBad, passedBad, falseReject };
}
const s = (x, n) => String(x).padStart(n);
const bad = requests.filter((r) => r.bad).length;
console.log(`good requests ${requests.length - bad}, bad requests ${bad}, limit ${LIMIT}`);
console.log(`${'key'.padEnd(13)}${s('stopped', 12)}${s('passed', 7)}${s('false reject', 14)}`);
for (const [name, scheme] of Object.entries(schemes)) {
const r = measure(scheme);
console.log(`${name.padEnd(13)}${s(r.stoppedBad, 12)}${s(r.passedBad, 7)}${s(r.falseReject, 14)}`);
}
// The same 200 attempts, if the target name is given in ten different spellings.
const spellings = ['U-17', 'u-17', ' U-17', 'U-017', 'U-17 ', 'u-017', 'U_17', 'u_17', 'U-17\t', 'U-0017'];
const spread = requests.map((r) => (r.bad && r.target === 'U-17'
? { ...r, target: spellings[requests.indexOf(r) % spellings.length] } : r));
const normalize = (d) => d.trim().replace(/_/g, '-').toUpperCase().replace(/^U-0+/, 'U-');
console.log('\nthe same 200 attempts, if the target name is given in ten different spellings (layered scheme):');
for (const [name, n] of [['raw key', (d) => d], ['normalized', normalize]]) {
const r = measure(schemes.layered, n, spread);
console.log(` ${name.padEnd(18)} stopped ${r.stoppedBad}, passed ${r.passedBad}, false reject ${r.falseReject}`);
}
```
```
good requests 240, bad requests 2500, limit 20
key stopped passed false reject
client 2260 240 52
account 1985 515 0
target 186 2314 0
combined 2451 49 52
layered 2446 54 0
the same 200 attempts, if the target name is given in ten different spellings (layered scheme):
raw key stopped 2266, passed 234, false reject 0
normalized stopped 2446, passed 54, false reject 0
```
The first three rows say each key sees exactly one shape of abuse. The client key stops the
probing and the scraping (2,260), never sees the distributed attempt, and drops fifty-two
legitimate requests from the branch's shared address. The account key's false reject is zero but
it never applies to the five hundred unauthenticated requests. The target key alone catches the
distributed attempt, stopping only a hundred and eighty-six requests.
The last two rows are the decision itself. Applying all three keys at once stops 2,451 requests
and still drops the branch's fifty-two. The layered scheme stops five fewer requests (2,446) and
brings the false reject to **zero**: an authenticated request binds to the narrowest key, the
account; an unauthenticated one binds to both address and target. Same limit, same algorithm, same
placement — the difference lies only in which field the counter looks at.
The final block's last two lines are this decision's silent flaw. If the key comes from an
identifier carried in the request and is **not normalized**, two hundred attempts probing the same
account under ten spellings spread across ten separate counters: stopped drops from 2,446 to
2,266, meaning a hundred and eighty attempts pass under the limit. There is no error anywhere; the
counter counts correctly, the limit applies correctly, it just counts ten separate things.
#### The Response Itself
The limit stops too many requests. Enumeration's problem is that **too few requests** can also be
enough: if forty probes tell you which of forty identities exist, the information has already
gotten out before the limit ever engages. What decides this is the shape of the response. The run
below actually stands up three versions of the same endpoint and probes forty identities,
alternating in order.
```js
// response.mjs — does the response shape enable enumeration: signature difference and duration difference
import { createServer } from 'node:http';
import { createHash } from 'node:crypto';
const ROUNDS = 60000; // fixed work the validator does: digest rounds
const THRESHOLD = 5; // ms; fixed threshold for separating the two sets (see text)
const doWork = () => { let d = 'x'; for (let i = 0; i < ROUNDS; i++) d = createHash('sha256').update(d).digest('hex'); };
const members = new Set(Array.from({ length: 20 }, (_, i) => `U-${i + 1}`));
const RECEIVED = '{"status":"request received"}';
const variants = {
'separate response': (v) => { if (v) doWork(); return v ? [202, '{"status":"code sent"}'] : [404, '{"error":"no record"}']; },
'same response': (v) => { if (v) doWork(); return [202, RECEIVED]; },
'same response + fixed work': () => { doWork(); return [202, RECEIVED]; },
};
const server = createServer((request, response) => {
const [status, body] = variants[request.headers['x-variant']](members.has(request.headers['x-identity']));
response.writeHead(status, { 'content-type': 'application/json' });
response.end(body);
});
server.listen(8952, '127.0.0.1');
const BASE = 'http://127.0.0.1:8952/password-reset';
// Attempt order alternates: existing and non-existing identities in turn, so drift affects both
// sets at once and the duration difference comes only from the branch's own work.
const identities = Array.from({ length: 20 }, (_, i) => [`U-${i + 1}`, `U-${i + 21}`]).flat();
const attempt = async (variant, identity) => {
const t = performance.now();
const y = await fetch(BASE, { method: 'POST', headers: { 'x-variant': variant, 'x-identity': identity } });
const body = await y.text();
return { duration: performance.now() - t, signature: `${y.status}|${body}`, real: members.has(identity) };
};
for (let i = 0; i < 5; i++) await attempt('same response + fixed work', 'U-1'); // warm-up
const s = (x, n) => String(x).padStart(n);
console.log(`${'variant'.padEnd(27)}${s('sig real/fake', 14)}${s('sig disjoint', 13)}`
+ `${s('by duration', 14)}${s('digest rounds', 16)}`);
for (const variant of Object.keys(variants)) {
const measurements = [];
for (const k of identities) measurements.push(await attempt(variant, k));
const sigs = (real) => new Set(measurements.filter((o) => o.real === real).map((o) => o.signature));
const [realSigs, fakeSigs] = [sigs(true), sigs(false)];
const disjoint = [...realSigs].every((x) => !fakeSigs.has(x));
const correct = measurements.filter((o) => (o.duration > THRESHOLD) === o.real).length;
const work = variant === 'same response + fixed work' ? `${ROUNDS}/${ROUNDS}` : `${ROUNDS}/0`;
console.log(`${variant.padEnd(27)}${s(`${realSigs.size}/${fakeSigs.size}`, 14)}${s(String(disjoint), 13)}`
+ `${s(`${correct}/40`, 14)}${s(work, 16)}`);
}
server.close();
console.log(`\nduration threshold ${THRESHOLD} ms is fixed; 20/40 is the random-guess level`);
console.log(`digest-round count is independent of the run, durations are not`);
```
```
variant sig real/fake sig disjoint by duration digest rounds
separate response 1/1 true 40/40 60000/0
same response 1/1 false 40/40 60000/0
same response + fixed work 1/1 false 20/40 60000/60000
duration threshold 5 ms is fixed; 20/40 is the random-guess level
digest-round count is independent of the run, durations are not
```
The first version splits the response in two: 202 and "code sent" for an existing identity, 404
and "no record" for one that does not exist. The signature sets are disjoint, so all forty of the
forty probes are correctly classified. This is not a vulnerability, it is a **response-shape
decision**, and it is usually made to help the user.
The second version returns the same body and status code on both branches; the signature
difference closes. The duration difference stays open, though, and all forty probes are again
correctly classified, since the validator only runs on an existing identity. The run-independent
quantity is the last column: sixty thousand digest rounds on one branch, zero on the other. Here
the medians came to roughly 19.9 ms against 1.5 ms; the ratio is machine-dependent, its existence
is not.
The third version does the work on both branches, and duration-based classification falls to the
random-guess level (20/40). Equalizing the response alone is not enough; **the work done must be
equalized too**. So the first two versions produce no error at all — all three run, all three
return a correct answer, and the difference between them is measurable only by probing.
#### The False Reject of Lockout
The first defense that comes to mind against brute force is locking the account after a few
consecutive errors. **AS19 — the number of failed attempts in a login session** comes from the
distribution above, and thirty days for forty members is generated. **AS20 — the attacker makes an
attempt every three seconds**, with a nine-hundred-second measurement window.
```js
// lockout.mjs — the false-reject cost of lockout against brute force
const SEED = 9137;
let state = SEED;
const rand = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648;
const WINDOW = 900; // s: measurement window
const PACE = 3; // s: seconds between two attacker attempts
const THRESHOLD = 5; // consecutive failed attempts
const LOCKOUT = 900; // s: lockout duration
const MEMBERS = 40, DAYS = 30;
// AS19: distribution of failed attempts in a login session (the legitimate member's own error)
const DISTRIBUTION = [[0, 0.72], [1, 0.15], [2, 0.06], [3, 0.03], [4, 0.02], [6, 0.015], [8, 0.005]];
const sessions = [];
for (let i = 0; i < MEMBERS * DAYS; i++) {
let x = rand(), errors = 0;
for (const [n, p] of DISTRIBUTION) { if ((x -= p) < 0) { errors = n; break; } }
sessions.push(errors);
}
const falseReject = sessions.filter((h) => h >= THRESHOLD).length;
// Backoff waits 2^(n-1) s before the nth attempt; how many attempts fit in the window.
let backoff = 0;
for (let t = 0, n = 1; t + 2 ** (n - 1) <= WINDOW; n++) { t += 2 ** (n - 1); backoff = n; }
const policies = [
['no limit', WINDOW / PACE, 0, 0, 0, 0],
['lockout, 15 min', THRESHOLD, falseReject, 0, MEMBERS, LOCKOUT],
['lockout, permanent', THRESHOLD, falseReject, falseReject, MEMBERS, -1],
['backoff', backoff, 0, 0, 0, 2 ** (backoff - 1)],
];
const s = (x, n) => String(x).padStart(n);
console.log(`seed ${SEED}: ${sessions.length} login sessions (${MEMBERS} members x ${DAYS} days), window ${WINDOW} s`);
console.log(`${'policy'.padEnd(19)}${s('attempts passed', 16)}${s('legit locked', 15)}`
+ `${s('staff unlock', 14)}${s('targeted lockouts', 18)}${s('legit member wait', 20)}`);
for (const [name, passed, locked, unlock, targeted, wait] of policies) {
console.log(`${name.padEnd(19)}${s(passed, 16)}${s(locked, 15)}${s(unlock, 14)}${s(targeted, 18)}`
+ `${s(wait === -1 ? 'staff unlocks' : `${wait} s`, 20)}`);
}
console.log(`\ntargeted lockout locks out ${MEMBERS} members with ${MEMBERS * THRESHOLD} requests`);
console.log(`sessions with ${THRESHOLD}+ consecutive errors: ${falseReject}/${sessions.length}`
+ ` (${(100 * falseReject / sessions.length).toFixed(1)}%); all of them a legitimate member`);
```
```
seed 9137: 1200 login sessions (40 members x 30 days), window 900 s
policy attempts passed legit locked staff unlock targeted lockouts legit member wait
no limit 300 0 0 0 0 s
lockout, 15 min 5 30 0 40 900 s
lockout, permanent 5 30 30 40 staff unlocks
backoff 9 0 0 0 256 s
targeted lockout locks out 40 members with 200 requests
sessions with 5+ consecutive errors: 30/1200 (2.5%); all of them a legitimate member
```
Lockout does its job: in nine hundred seconds, attempts passing drop from three hundred to five.
Its cost shows up in two places at once. First, thirty legitimate member sessions a month — 2.5%
of twelve hundred — belong to no attacker; under permanent lockout, all thirty become a staff
task. Second, and heavier: lockout hands anyone who knows an account name a **kill switch**. Five
attempts against each of forty accounts — two hundred requests — locks out all forty members,
without knowing a single password.
Backoff allows nine attempts in the same window — four more than lockout, thirty-three times fewer
than no limit — and locks out no legitimate member at all. The remaining cost is the last column:
the real owner of a targeted account waits 256 seconds after the attacker burns through the
attempts. So the delay produces a false reject too, but a finite one needing no staff work. The
choice between the two defenses is not "which is more secure" but **who a false reject lands on,
and for how long**.
#### Summary
- The limit's algorithm and placement are this lesson's input; the decision measured is which key
the limit is applied to, and each key sees only one shape of abuse.
- The client key stopped 2,260 bad requests and dropped 52 legitimate ones over the shared
address; the layered scheme stopped 2,446 and brought the false reject to zero — an
authenticated request bound to the account, an unauthenticated one to the address and target.
- Without normalizing the key, two hundred attempts probing the same account under ten spellings
spread across ten counters, and stopped requests dropped from 2,446 to 2,266; the counter works
correctly, it just counts ten separate things.
- Giving a separate response to an existing versus a non-existing identity gave away all forty of
the forty probes; equalizing the response closed the signature difference, but the duration
difference stayed forty for forty, because the work happened on only one branch.
- Doing the work on both branches brought classification down to the random-guess level (20/40);
the run-independent measure is equalizing the digest-round count on both branches.
- Lockout brought passing attempts from 300 to 5 and locked out 30 legitimate member sessions a
month; two hundred requests can lock out all forty members. Backoff allowed nine attempts,
locked no one out, and left the legitimate member a finite 256-second wait.
#### Next Step
Every defense built across this topic sits **inside the application**. The validator, the encoder,
the header layer, the secret store, the audit log, and the counter — all run in the loan service's
own process, and all assume the request the application sees is really the request that arrived.
In production this assumption does not hold on its own. The application does not stand alone: a
reverse proxy meets the connection in front of it, a process manager keeps it up behind it, and
both configurations change the request the application sees — client address, body size, headers,
even how many process copies are running. Which address this lesson's counters count depends on
that too. The next lesson takes on the layer in front of the application and measures the same
rule there: a wrong configuration raises no error, it shows up as a difference in the measure.
## Server and Deployment Topology
Where the application stands in production and how it changes: TLS termination and header forwarding at the reverse proxy, the restart policy and resource limits of process managers, the cost difference between horizontal and vertical scaling, the stateless design that scaling requires as a precondition, the effect of session placement and stickiness, zero downtime in blue-green and canary releases, how backward-compatible migration steps relate to release order, and capacity estimation from load-test data.
### Reverse Proxy Configuration
Source: https://academia.sh/en/courses/backend-production/reverse-proxy-configuration
What the reverse proxy leaves out for the application: the client address and scheme information lost after TLS termination, restored through header forwarding; the rate limit collapsing into a single bucket while forwarding is off, and the audit log showing a single address; the wrong record returning successfully when the path is resolved once too often at the proxy; and how the body-size limit sitting separately at two layers changes the effective value and the log.
The previous topic built every defense inside the application: the validator, the allowlist,
the secrets store, the audit log, the rate limiter. Each one read the request and made a
decision, and all of them leaned on one assumption — **the request the application sees is the
request the client sent.**
In production this assumption does not hold. The application never stands alone: a reverse
proxy sits in front of it, a process manager sits underneath it. Both touch the request and
both are configured, and their configuration changes what the application sees. What a reverse
proxy is was established in the Load Balancer and Reverse Proxy lesson; this lesson measures
its **settings** instead.
#### The Gap Termination Leaves
TLS termination happens at the proxy: the encrypted connection ends there, and the connection
between the proxy and the application carries no encryption. This takes two pieces of
information away from the application. The first is the **client address**: the other end of
the connection the application sees is no longer the client, it is the proxy. The second is the
**scheme**: the connection reaching the application is unencrypted, and nothing in it records
whether the original request arrived encrypted.
Neither is lost — it is **forwarded**: the proxy adds two headers to the request. Forwarding is
a setting, and off, the application raises no error; it just reads the wrong value.
- **SD1.** On a single machine, every client shares the same loopback address. At the proxy,
the member address is derived from the source-port block; in a real deployment it comes
directly from the connection.
- **SD2.** The lesson does not measure encryption itself, only the information loss
termination leaves behind.
- **SD3.** The rate limit is five requests per identity in a single-run window; the audit log
writes each request under the identity it resolved.
The loan application takes identity from the header first, and from the connection if there
is none.
```js
// topology/app.mjs — loan application: reads identity, scheme, and body limit from the request.
// Identity comes first from the "x-forwarded-for" header, otherwise from the connection itself.
import { createServer } from "node:http";
const [port, bodyLimit] = [Number(process.argv[2]), Number(process.argv[3] ?? 16384)];
const LIMIT = 5; // requests per identity
const CATALOG = { "KTP/17": "Regional History", "KTP%2F17": "Regional History Supplement" };
const counter = new Map();
const stats = { requests: 0, passed: 0, stopped: 0, identity: new Set(), bytesRead: 0 };
function resolveIdentity(req) {
const forwarded = req.headers["x-forwarded-for"];
return forwarded ?? req.socket.remoteAddress; // no header: the other end of the connection
}
if (Number.isInteger(port) === false) console.log("usage: node topology/app.mjs [body-limit]");
else createServer((req, res) => {
res.sendDate = false;
const path = new URL(req.url, "http://local").pathname;
if (path === "/measure") {
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify({ ...stats, identity: stats.identity.size }));
return;
}
const identity = resolveIdentity(req);
stats.requests += 1;
stats.identity.add(identity);
const n = (counter.get(identity) ?? 0) + 1;
counter.set(identity, n);
if (n > LIMIT) {
stats.stopped += 1;
res.writeHead(429, { "x-identity": identity }).end("limit");
return;
}
stats.passed += 1;
if (req.headers["x-forwarded-proto"] !== "https") {
res.writeHead(308, { location: path, "x-identity": identity }).end("to secure scheme");
return;
}
if (path.startsWith("/book/")) {
const bookId = decodeURIComponent(path.split("/")[2] ?? "");
const title = CATALOG[bookId];
if (title === undefined) res.writeHead(404, { "x-identity": identity }).end("not found");
else res.writeHead(200, { "x-record": title, "x-identity": identity }).end(title);
return;
}
if (req.method === "POST") {
let bytes = 0;
req.on("data", (p) => {
bytes += p.length;
stats.bytesRead += p.length;
if (bytes > bodyLimit) { res.writeHead(413, { "x-layer": "app" }).end("too large"); req.destroy(); }
});
req.on("end", () => { if (res.writableEnded === false) res.writeHead(200, { "x-identity": identity }).end(String(bytes)); });
return;
}
res.writeHead(200, { "x-identity": identity }).end("loan");
}).listen(port, "127.0.0.1");
```
The proxy takes four settings from one string: header forwarding, scheme forwarding, path
handling, body limit — each a configuration line in a real product, here a single string
field.
```js
// topology/proxy.mjs — reverse proxy. Terminates TLS itself, connects to the backend unencrypted.
// Setting string: headers=on|off,scheme=on|off,path=passthrough|resolve,body=
import { createServer, request } from "node:http";
const [port, target, configText] = [Number(process.argv[2]), Number(process.argv[3]), process.argv[4] ?? ""];
const OPTS = Object.fromEntries(configText.split(",").filter(Boolean).map((p) => p.split("=")));
const counters = { received: 0, forwarded: 0, aborted: 0, bytesRead: 0 };
// SD1: on a single machine every client shares the same loopback address. The member address is
// derived from the source-port block; in a real deployment this value comes directly from the connection.
const MEMBER_IP = { 412: "10.4.0.11", 413: "10.4.0.12", 414: "10.4.0.13", 415: "10.4.0.14" };
const clientAddress = (s) => MEMBER_IP[Math.floor(s.remotePort / 100)] ?? s.remoteAddress;
if (Number.isInteger(port) === false || Number.isInteger(target) === false) {
console.log("usage: node topology/proxy.mjs ");
} else createServer((clientReq, clientRes) => {
clientRes.sendDate = false;
if (clientReq.url === "/proxymeasure") { clientRes.writeHead(200).end(JSON.stringify(counters)); return; }
counters.received += 1;
const headers = { ...clientReq.headers };
delete headers["x-forwarded-for"]; // discard any identity claim from outside
delete headers["x-forwarded-proto"];
if (OPTS.headers === "on") headers["x-forwarded-for"] = clientAddress(clientReq.socket);
if (OPTS.scheme === "on") headers["x-forwarded-proto"] = "https";
const bodyLimit = Number(OPTS.body ?? Infinity);
const path = OPTS.path === "resolve" ? decodeURIComponent(clientReq.url) : clientReq.url;
let bytes = 0;
const upstreamReq = request({ port: target, path, method: clientReq.method, headers }, (upstreamRes) => {
clientRes.writeHead(upstreamRes.statusCode, upstreamRes.headers);
upstreamRes.pipe(clientRes);
});
counters.forwarded += 1;
upstreamReq.on("error", () => {
counters.aborted += 1;
if (clientRes.headersSent === false) clientRes.writeHead(502, { "x-layer": "proxy" }).end("backend disconnected");
});
clientReq.on("data", (p) => {
bytes += p.length;
counters.bytesRead += p.length;
if (bytes > bodyLimit) {
if (clientRes.headersSent === false) clientRes.writeHead(413, { "x-layer": "proxy" }).end("too large");
upstreamReq.destroy();
clientReq.destroy();
return;
}
upstreamReq.write(p);
});
clientReq.on("end", () => upstreamReq.end());
clientReq.on("error", () => upstreamReq.destroy());
}).listen(port, "127.0.0.1");
```
Four members connect from separate source-port blocks: A sends ten requests, B, C, and D send
two each — sixteen total, against a per-identity limit of five.
```js
// topology/client.mjs — members connect from separate source ports; identity is pinned to the connection.
// Scenarios: rate | scheme | path | body. Usage: node topology/client.mjs