---
title: 'Secure Coding'
source: 'https://academia.sh/en/courses/secure-coding'
language: en
updated: '2026-09-26T03:02:38+00:00'
license: 'CC BY-SA 4.0'
lessons: 17
outcomes:
- 'Enforces the untrusted data boundary explicitly in code and measures the number of covered points'
- 'Closes common vulnerability classes with language- and framework-independent defenses'
- 'Shows the misuse of cryptography and secret management with concrete numbers'
---
# Secure Coding
How many points in a codebase a defensive pattern closes on its own and how many stay uncovered: the difference between allowlist and denylist validation against a new input type, choosing context-aware encoding wrongly in four contexts, the point that a parameterized query interface does not cover, template and expression injection, file upload checks, outbound requests reaching the internal network, insecure deserialization, timing and enumeration leaks in authentication implementation, session fixation, broken object level authorization, business logic flaws, bypassing rules with concurrent requests, cryptographic misuse, cryptographically secure generators, secrets left in repository history, error message leaks and transitive dependency depth.
## Input and Output Security
Counting the points in the codebase where externally supplied data enters and measuring how far the defense's coverage reaches: the entry point inventory, the point that allowlist-based validation closes on its own, escaping across four output contexts, the identifier position the parameterized query interface cannot cover, user data that reaches a template, file upload inspection layers, outbound request targets that reach the internal network, and the trusted type set in deserialization.
### Input Validation Patterns
Source: https://academia.sh/en/courses/secure-coding/input-validation-patterns
Measuring how many points validation covers: a probe that breaks its own declaration is sent to each of the fifteen field points across the regional measurement network's five entry points, the number of points closed by allowlist-based validation, by the denylist, and by a hand-written check per endpoint is counted, and how coverage changes when a new endpoint and a new field type are added to the inventory is measured.
The Secure Development Lifecycle course counted which threats arose at the design stage
and which countermeasures were planned for them. Those countermeasures shared one
property: each was a decision, not a place. Once a decision turns into code, it does not
live in a single place — at every entry point in the codebase, it either holds or it does
not, independently.
This lesson counts those places. What it measures is not how validation is written — the
Server Security and Going to Production course measured the validation function itself.
What is measured here is **coverage**: how many points there are, how many the defense
covers, and how many it leaves uncovered. The unit of measure for this course is the final
count.
#### Point Inventory
The regional measurement network is fictional: the subscriber portal, the meter reading
intake service, the field application, and the report generator. The unit of the
inventory is not the endpoint but the **point**: one field of one endpoint. If an
endpoint has three fields, that endpoint carries three points, because the defense
exists — or does not — per field.
**IO1 (assumption):** the inventory is complete. Every count in this lesson is relative
to this inventory and says nothing about an entry point that is not in it.
```js
// points.mjs — entry points of the regional measurement network (fictional model)
// Point = one field of one endpoint. The declaration keeps each point's constraint in one place.
export const DECLARATION = {
"POST /subscriber": { subscriberNo: ["code", /^A-\d{6}$/], name: ["text", 60],
tariff: ["set", ["residential", "commercial", "agricultural"]] },
"GET /invoice": { subscriberNo: ["code", /^A-\d{6}$/], period: ["code", /^\d{4}-\d{2}$/],
page: ["number", [1, 200]] },
"POST /reading": { meterNo: ["code", /^S-\d{7}$/], value: ["number", [0, 999999]],
source: ["set", ["field", "remote"]] },
"POST /work-order": { subscriberNo: ["code", /^A-\d{6}$/], type: ["set", ["disconnection", "connection", "measurement"]],
description: ["text", 200] },
"GET /report": { period: ["code", /^\d{4}-\d{2}$/], region: ["set", ["north", "south"]],
format: ["set", ["summary", "detailed"]] },
};
export const points = (d = DECLARATION) =>
Object.entries(d).flatMap(([endpoint, fields]) => Object.keys(fields).map((fieldName) => [endpoint, fieldName]));
const VALID = { code: (r) => (r.test("A-100432") ? "A-100432" : r.test("S-2200145") ? "S-2200145" : "2026-03"),
text: () => "measurement note", set: (s) => s[0], number: ([lo]) => lo, list: (s) => [s[0]] };
export const validRecord = (endpoint, d = DECLARATION) =>
Object.fromEntries(Object.entries(d[endpoint]).map(([fieldName, [type, spec]]) => [fieldName, VALID[type](spec)]));
// Probe: breaks only a single point's declaration; the other points in the same record stay valid.
const VIOLATION = { code: () => "A-1", text: (n) => "x".repeat(n + 1), set: () => "unknown",
number: ([, hi]) => hi + 1, list: () => ["unknown"] };
export const probe = (endpoint, fieldName, d = DECLARATION) =>
({ ...validRecord(endpoint, d), [fieldName]: VIOLATION[d[endpoint][fieldName][0]](d[endpoint][fieldName][1]) });
```
The probe concept carries the whole measurement. The only way to ask whether a point is
covered is to send a value that breaks **that point's own declaration** and check whether
the record is rejected. The values in the probe are harmless: a short identifier, one
character too much text, a word that is not in the set, a number one above the upper
bound.
**IO2 (assumption):** a point counts as covered when its probe is rejected. Since the
probe breaks only that one point, the rejection decision can be attributed to it.
#### Three Defense Forms
Three forms are laid over the same inventory. **Allowlist-based validation** derives its
decision from the declaration: an endpoint with no declaration is rejected, and so is a
field that is not in the declaration. **Denylist-based validation** rejects named shapes
and never reads the endpoint's declaration at all. The third is scattered defense: a
hand-written check per endpoint.
```js
// validator.mjs — three defense forms looking at the same set of points
import { DECLARATION } from "./points.mjs";
// Type table: the constraint types the allowlist recognizes. A new type is added here.
export const TYPE = {
code: (v, s) => typeof v === "string" && s.test(v),
text: (v, s) => typeof v === "string" && v.length <= s,
set: (v, s) => s.includes(v),
number: (v, s) => Number.isInteger(v) && v >= s[0] && v <= s[1],
};
// 1) Allowlist: the decision derives from the declaration. An endpoint with no
// declaration, a field with no declaration, and a type not in the table -- all rejected.
export function allowlist(endpoint, record, d = DECLARATION, type = TYPE) {
const fields = d[endpoint];
if (!fields) return "reject";
for (const [name, v] of Object.entries(record)) {
const f = fields[name];
if (!f || !type[f[0]] || !type[f[0]](v, f[1])) return "reject";
}
return Object.keys(fields).every((f) => f in record) ? "accept" : "reject";
}
// 2) Denylist: named shapes are rejected; the endpoint's declaration is never read.
const FORBIDDEN_FIELD = ["permission", "role", "amount"], MAX_LENGTH = 200;
export function denylist(endpoint, record) {
for (const [name, v] of Object.entries(record)) {
if (FORBIDDEN_FIELD.includes(name)) return "reject";
const s = typeof v === "string" ? v : String(v ?? "");
if (s.length === 0 || s.length > MAX_LENGTH) return "reject";
}
return "accept";
}
// 3) Scattered defense: hand-written check per endpoint; written for three, two left empty.
export function handWritten(endpoint, record) {
if (endpoint === "POST /subscriber") return /^A-\d{6}$/.test(record.subscriberNo ?? "") ? "accept" : "reject";
if (endpoint === "GET /invoice") return /^A-\d{6}$/.test(record.subscriberNo ?? "") &&
Number.isInteger(record.page) && record.page >= 1 ? "accept" : "reject";
if (endpoint === "POST /reading") return /^S-\d{7}$/.test(record.meterNo ?? "") ? "accept" : "reject";
return "accept";
}
```
All three forms are probed with the same fifteen probes. The fourth column keeps the
cost: which endpoint stops when called with a valid record.
```js
// coverage.mjs — each point is probed with a value that breaks its own declaration
import { DECLARATION, points, probe, validRecord } from "./points.mjs";
import { allowlist, denylist, handWritten } from "./validator.mjs";
const P = points(), EP = Object.keys(DECLARATION);
console.log(`${EP.length} entry points, ${P.length} field points, ${P.length} probes\n`);
console.log(`${"defense form".padEnd(24)}${"covered".padStart(9)}${"uncovered".padStart(11)}${"legit endpoints stopped".padStart(25)}`);
for (const [name, f] of [["allowlist", allowlist], ["denylist", denylist],
["hand-written check", handWritten]]) {
const uncovered = P.filter(([e, a]) => f(e, probe(e, a)) === "accept");
const stopped = EP.filter((e) => f(e, validRecord(e)) !== "accept").length;
console.log(name.padEnd(24) + String(P.length - uncovered.length).padStart(9) +
String(uncovered.length).padStart(11) + String(stopped).padStart(25));
console.log(" uncovered per endpoint: " +
EP.map((e) => `${e.split(" ")[1]} ${uncovered.filter(([x]) => x === e).length}`).join(" "));
}
```
```
5 entry points, 15 field points, 15 probes
defense form covered uncovered legit endpoints stopped
allowlist 15 0 0
uncovered per endpoint: /subscriber 0 /invoice 0 /reading 0 /work-order 0 /report 0
denylist 1 14 0
uncovered per endpoint: /subscriber 3 /invoice 3 /reading 3 /work-order 2 /report 3
hand-written check 3 12 0
uncovered per endpoint: /subscriber 2 /invoice 2 /reading 2 /work-order 3 /report 3
```
The denylist closes **one** of the fifteen points. Which point it closes is also
incidental: the work order description's upper bound is two hundred characters, and the
list's general length rule is also two hundred, so the probe gets caught there. On the
remaining fourteen points the list knows nothing — it does not recognize identifier
format, set membership, or number range, because these are specific to the point and are
not written into the list. **The denylist's cost turned out to be zero; its narrowness is
exactly why its coverage is low.**
The hand-written check closes three points, and all three are from the same class: the
subscriber and meter identifier. Whoever wrote the check saw the first field at every
endpoint and did not see the second or third. Two endpoints have no check at all; the six
points there are uncovered from the start.
The allowlist's zero does not come from validation being written more carefully. The
declaration already carried the point's constraint; validation just reads that
declaration. **Coverage is the result of the defense deriving from the same place as the
declaration.**
#### Outside the Inventory
Fifteen out of fifteen is a number that depends on IO1: the inventory was assumed
complete. Once that assumption is lifted, coverage leaks from somewhere else. The list of
endpoints registered in the code lives in the routing table; the list of declarations is
separate, and the two are not required to be equal.
**IO3 (assumption):** the routing table has been extracted from the code and contains
every registered endpoint. The declaration set, on the other hand, was written by hand.
```js
// inventory-gap.mjs — the difference between endpoints registered in the code and declared endpoints
import { DECLARATION } from "./points.mjs";
import { allowlist } from "./validator.mjs";
// Routing table (model): the endpoints actually registered in the code and the fields the handler reads.
const ROUTES = {
"POST /subscriber": ["subscriberNo", "name", "tariff"], "GET /invoice": ["subscriberNo", "period", "page"],
"POST /reading": ["meterNo", "value", "source"], "POST /work-order": ["subscriberNo", "type", "description"],
"GET /report": ["period", "region", "format"],
"POST /reading/bulk": ["fileNo", "row", "source"], // no declaration
"GET /subscriber/history": ["subscriberNo", "start", "end"], // no declaration
};
const undeclared = Object.keys(ROUTES).filter((e) => !DECLARATION[e]);
const pointCount = (l) => l.reduce((t, e) => t + ROUTES[e].length, 0);
const PROBE = "x".repeat(5000); // a length no field declaration will accept
const record = (e, broken) => Object.fromEntries(ROUTES[e].map((f) => [f, f === broken ? PROBE : "value"]));
console.log(`routing table ${Object.keys(ROUTES).length} endpoints / ${pointCount(Object.keys(ROUTES))} points, ` +
`declaration ${Object.keys(DECLARATION).length} endpoints / ${pointCount(Object.keys(DECLARATION))} points`);
console.log(`undeclared endpoints: ${undeclared.join(", ")} -> ${pointCount(undeclared)} points\n`);
// Two policies for an undeclared endpoint.
const passThrough = (e, r) => (DECLARATION[e] ? allowlist(e, r) : "accept");
console.log(`${"undeclared endpoint policy".padEnd(29)}${"uncovered points".padStart(17)}${"legit endpoints stopped".padStart(25)}`);
for (const [name, f] of [["default reject", allowlist], ["default pass", passThrough]]) {
const out = undeclared.flatMap((e) => ROUTES[e].map((f2) => [e, f2]))
.filter(([e, f2]) => f(e, record(e, f2)) === "accept").length;
const stopped = undeclared.filter((e) => f(e, record(e, null)) !== "accept").length;
console.log(name.padEnd(29) + String(out).padStart(17) + String(stopped).padStart(25));
}
```
```
routing table 7 endpoints / 21 points, declaration 5 endpoints / 15 points
undeclared endpoints: POST /reading/bulk, GET /subscriber/history -> 6 points
undeclared endpoint policy uncovered points legit endpoints stopped
default reject 0 2
default pass 6 0
```
Six of the twenty-one points have no declaration. The central pattern **cannot close
these six points on its own**; the only thing it can decide is what to do with an
endpoint that has no declaration. Default reject closes all six points and, in exchange,
stops two endpoints — one is the bulk reading upload endpoint, the other is subscriber
history. Default pass stops no legitimate work and leaves the six points in the open.
The difference between the two rows lies in silence. Under the top policy, a missing
declaration is seen on the first call; under the bottom one there is no sign at all.
**Because coverage itself is measurable, this difference can also be written down:**
comparing the two lists is a single subtraction, and it can be repeated at every change.
#### When a New Point Is Added
The real test of coverage begins when the inventory grows. A new endpoint is opened for
meter-change declarations: first with three fields using only known types, then with a
fourth field of a new type — a list carrying multiple values.
**IO4 (assumption):** the new endpoint's declaration is hand-written and is assumed to be
written correctly. What is measured is not the declaration's correctness but the
defense's automatic behavior once the declaration exists.
```js
// new-point.mjs — what happens to coverage when a new endpoint and a new field type are added
import { DECLARATION, points, probe, validRecord } from "./points.mjs";
import { allowlist, denylist, handWritten, TYPE } from "./validator.mjs";
const NEW_ENDPOINT = "POST /meter/change";
const A = { ...DECLARATION, [NEW_ENDPOINT]: { meterNo: ["code", /^S-\d{7}$/], newNo: ["code", /^S-\d{7}$/],
reason: ["set", ["fault", "expiry", "request"]] } }; // known types only
const B = { ...A, [NEW_ENDPOINT]: { ...A[NEW_ENDPOINT], equipment: ["list", ["seal", "cover"]] } }; // new type
// The single central change that recognizes the type: one row in the type table.
const TYPE2 = { ...TYPE, list: (v, s) => Array.isArray(v) && v.every((x) => s.includes(x)) };
const allowlistV2 = (endpoint, record, d) => allowlist(endpoint, record, d, TYPE2);
const measure = (d, f) => {
const P = points(d);
const uncovered = P.filter(([e, a]) => f(e, probe(e, a, d), d) === "accept").length;
const stopped = Object.keys(d).filter((e) => f(e, validRecord(e, d), d) !== "accept").length;
return [P.length, uncovered, stopped];
};
console.log(`${"stage".padEnd(34)}${"points".padStart(6)}${"allow".padStart(6)}${"deny".padStart(5)}${"hand".padStart(6)}${"legit endpoints stopped".padStart(25)}`);
for (const [name, d, f] of [["current inventory", DECLARATION, allowlist],
["new endpoint, known types", A, allowlist], ["new field type added", B, allowlist],
["single type-recognizing change", B, allowlistV2]]) {
const [n, al, st] = measure(d, f);
console.log(name.padEnd(34) + String(n).padStart(6) + String(al).padStart(6) +
String(measure(d, denylist)[1]).padStart(5) + String(measure(d, handWritten)[1]).padStart(6) +
String(st).padStart(25));
}
console.log(`\nvalidation code written for the new endpoint: allowlist 0 lines (3 lines of declaration), ` +
`hand-written ${Object.keys(B[NEW_ENDPOINT]).length} checks`);
```
```
stage points allow deny hand legit endpoints stopped
current inventory 15 0 14 12 0
new endpoint, known types 18 0 17 15 0
new field type added 19 0 18 16 1
single type-recognizing change 19 0 18 16 0
validation code written for the new endpoint: allowlist 0 lines (3 lines of declaration), hand-written 4 checks
```
The two middle columns rise at every stage: as the inventory grows from fifteen to
nineteen, the points the denylist leaves uncovered go from fourteen to eighteen, and the
hand-written check's go from twelve to sixteen. **Every new point is directly a candidate
opening in these two forms** — because the defense sees the point only if someone writes
it in by hand.
The allowlist column stays at zero through all three stages. When the new endpoint was
added, not a single line of validation code was written; only the declaration was
written, and validation read that declaration. Coverage stayed fixed.
The third row is this lesson's only nonzero cost cell. Adding a new **type** is a
different situation: a declaration with no counterpart in the type table is rejected, and
so the endpoint's valid record is rejected too. Coverage stays at nineteen, but one
endpoint stops. This is the error on the safe side, and it is noisy — it is seen
immediately because a legitimate request is rejected. The fourth row shows the fix: one
line in the type table, nothing else touched.
**The difference between adding a point and adding a type is permanent.** Adding a point
is a frequent, scattered task — every endpoint, every field, every release. Adding a type
is rare and happens in a single central place. The pattern that keeps coverage fixed is
the pattern that ties the frequent task to no code at all.
#### Summary
- The unit of the inventory is not the endpoint but the point: five entry points carry
fifteen field points. Each point was probed with a value that breaks its own
declaration; none of the probes are malicious, all of them are boundary values.
- On the same fifteen probes, the allowlist closed 15 points, the denylist closed 1, and
the hand-written check per endpoint closed 3. The number of uncovered points was 0,
14, and 12, respectively.
- The one point the denylist closed is incidental: its general length rule happened to
coincide with one field's upper bound. The three points the hand-written check closed
are all from a single class, and two endpoints have no check at all.
- The routing table has 21 points, the declaration has 15; six points are uncovered by the
inventory. The central pattern closes them only through the policy it applies to an
undeclared endpoint: default reject closes 6 points and stops 2 endpoints, default pass
leaves 6 points in the open and gives no sign at all.
- When the inventory grew from fifteen to nineteen, the points the allowlist left uncovered
stayed at 0, the denylist's rose to 18, and the hand-written check's rose to 16. The
validation code written for the new endpoint is 0 lines under the allowlist.
- Adding a new field **type** is different: a type with no counterpart in the table is
rejected, one endpoint stops, and this is seen. The fix is a single line in the type
table. Adding a point is frequent and touches no code; adding a type is rare and happens
in one central place.
#### Next Step
A value that passes validation conforms to the contract; it carries whatever the point's
declaration says. What happens to this value when it **leaves** the system is a separate
question. The subscriber name is written into a page body, into an attribute's quotes,
into a script variable, and into an address component; all four take the same value, and
all four are subject to a different language's rules. The next lesson counts output
points and measures a single question: how many cases break when the context is chosen
wrong, and how many output points a single encoder can close on its own.
### Output Encoding
Source: https://academia.sh/en/courses/secure-coding/output-encoding
Measuring the coverage of context-aware escaping: six legitimate subscriber notes are placed into four output contexts with four separate encoders, the placements that cannot be read back across sixteen pairings are counted, in a fourteen-exit-point inventory the form that chooses the encoder at the call site is compared with the form that derives it from the context, and how many values the encoder order breaks in two nested contexts is measured.
The previous lesson counted entry points and showed that validation derived from a
declaration closes fifteen points on its own. A value that passes validation now
conforms to the contract: the subscriber note does not exceed two hundred characters, the
tariff is in the set, the period is formatted. Validation says nothing about what happens
to this value when it **leaves** the system.
This lesson counts exit points. The same subscriber note is written into a page body,
into an attribute's quotes, into a script variable, and into an address component. All
four take the same value; all four read that value by a different rule. What is measured
is **whether the context is chosen right, and how many exit points have to make that
choice on their own**.
#### Four Output Contexts
A context has three parts: the **container** that wraps the value, the container's own
**encoder**, and the container's own **reader**. The measure is a single question — after
the value is placed in the container and read back by the container's rule, does the same
value come out?
**IO5 (assumption):** the read-back represents the rule of the party that parses the
container. A placement counts as broken when the read-back does not give the value back
exactly; how harmful the breakage is is not measured.
```js
// context.mjs — four output contexts: the container, its own encoder, and its own reader
// Measure: after the value is placed in the container and read back by the container's rule, is it the same?
export const VALUES = ["cracked meter cover", 'subscriber "urgent" note dropped', "shelf 3 & floor 2",
"reading < 5 m3", "first line\nsecond line", "percent 5 increase %"];
const body = (s) => s.replace(/&/g, "&").replace(//g, ">");
export const ENCODER = {
body,
attribute: (s) => body(s).replace(/"/g, """).replace(/\n/g, "
"),
script: (s) => JSON.stringify(s).slice(1, -1),
address: (s) => encodeURIComponent(s),
};
const decode = (s) => s.replace(/"/g, '"').replace(/
/g, "\n")
.replace(/</g, "<").replace(/>/g, ">").replace(/&/g, "&");
export const CONTEXT = {
body: [(x) => `${x}`, (k) => {
const i = k.indexOf("<", 6);
return k.startsWith("") && k.slice(i) === "" ? decode(k.slice(6, i)) : null;
}],
attribute: [(x) => ``,
(k) => { const m = /note="([^"]*)"/.exec(k); return m ? decode(m[1]) : null; }],
script: [(x) => `const note = "${x}";`, (k) => {
try { return JSON.parse(k.slice(k.indexOf('"'), k.lastIndexOf('"') + 1)); } catch { return null; }
}],
address: [(x) => `/report?note=${x}&format=summary`, (k) => {
try {
const a = k.split("?")[1].split("&").find((p) => p.startsWith("note="));
return a === undefined ? null : decodeURIComponent(a.slice(5));
} catch { return null; }
}],
};
// Single placement: value is encoded, placed in the container, read back.
export const readsBack = (context, encoderName, value) => {
const [container, read] = CONTEXT[context];
return read(container(ENCODER[encoderName](value))) === value;
};
```
All six values are legitimate subscriber and field notes: a quote, an ampersand, a
less-than sign, a line break, a percent sign. None of them is attack input; all of them
are texts a field worker could write.
#### When the Context Is Chosen Wrong
Four contexts and four encoders give sixteen pairings. Four of them are the correct
pairing; the remaining twelve are the context chosen wrong.
```js
// choice.mjs — four contexts x four encoders: how many cases break when the context is chosen wrong
import { VALUES, ENCODER, CONTEXT, readsBack } from "./context.mjs";
const C = Object.keys(CONTEXT), E = Object.keys(ENCODER);
console.log(`${VALUES.length} values, ${C.length} contexts, ${E.length} encoders, ` +
`${C.length * E.length} pairings, ${VALUES.length * C.length * E.length} placements\n`);
console.log("number of broken values (row: context, column: chosen encoder)");
console.log("context".padEnd(12) + E.map((e) => e.padStart(11)).join(""));
let right = 0, wrong = 0;
for (const c of C) {
const row = E.map((e) => VALUES.filter((v) => !readsBack(c, e, v)).length);
E.forEach((e, i) => (e === c ? (right += row[i]) : (wrong += row[i])));
console.log(c.padEnd(12) + row.map((n) => String(n).padStart(11)).join(""));
}
console.log(`\nbroken in the ${C.length} pairings with the right context: ${right}`);
console.log(`broken in the ${C.length * E.length - C.length} pairings with the wrong context: ${wrong}`);
```
```
6 values, 4 contexts, 4 encoders, 16 pairings, 96 placements
number of broken values (row: context, column: chosen encoder)
context body attribute script address
body 0 0 3 6
attribute 1 0 2 6
script 4 4 0 6
address 3 5 4 0
broken in the 4 pairings with the right context: 0
broken in the 12 pairings with the wrong context: 44
```
The diagonal is zero: every context returns all six values with its own encoder. Of the
seventy-two placements off the diagonal, forty-four break.
The last column is the most instructive spot in this table. The address encoder breaks
**all six of the six values**, no matter which context it is placed in. This is not
under-escaping but over-escaping: spaces and punctuation turn into percent-sign
sequences, and the note read on the page is no longer the note that was written. The
wrong context produces error in both directions at once; the assumption that a strict
encoder is harmless does not hold up under measurement.
The script row is the second point. The body encoder breaks four values in the script
context, because the quote and the line break are not special under the body rule — the
three characters special there are different ones. It is not enough for the encoder to be
"safer"; **it must be the same rule as the container's rule**.
#### Exit Point Inventory
The context choice is not made in one place; it is made separately at every exit point.
The exit points of the regional measurement network are counted: the subscriber portal,
invoicing, the field application, and the report generator.
**IO6 (assumption):** the real context of every exit point is known and correctly
recorded in the inventory. What is measured is whether the encoder chosen at the call
site matches that context.
```js
// exit-points.mjs — exit point inventory (fictional model) and two writing forms
import { VALUES, ENCODER, CONTEXT } from "./context.mjs";
const CODEC = { ...ENCODER, raw: (s) => s };
const roundTrips = (c, e, v) => { const [container, read] = CONTEXT[c]; return read(container(CODEC[e](v))) === v; };
// [service, exit point, real context, encoder chosen at the call site]
const EXIT = [
["subscriber portal", "profile heading", "body", "body"],
["subscriber portal", "note list", "body", "body"],
["subscriber portal", "form preview", "attribute", "body"],
["subscriber portal", "search link", "address", "raw"],
["subscriber portal", "status script", "script", "body"],
["invoice", "statement line", "body", "body"],
["invoice", "print heading", "attribute", "attribute"],
["invoice", "download link", "address", "address"],
["field application", "work order card", "body", "body"],
["field application", "map label", "attribute", "raw"],
["field application", "server call", "address", "address"],
["report generator", "summary body", "body", "body"],
["report generator", "link generator", "address", "body"],
["report generator", "embedded data script", "script", "script"],
];
// New exit points: no one chose an encoder at the call site.
const NEW_EXITS = [["field application", "photo heading", "attribute", "raw"],
["report generator", "external link", "address", "raw"], ["subscriber portal", "notification script", "script", "raw"]];
const measure = (list, choice) => {
let brokenPoints = 0, brokenPlacements = 0;
for (const [, , context, byHand] of list) {
const n = VALUES.filter((v) => !roundTrips(context, choice === "central" ? context : byHand, v)).length;
if (n) brokenPoints++;
brokenPlacements += n;
}
return [brokenPoints, brokenPlacements];
};
console.log(`${"stage".padEnd(28)}${"points".padStart(6)}${"placements".padStart(12)}` +
`${"scattered points".padStart(18)}${"scattered placements".padStart(22)}${"central".padStart(9)}`);
for (const [name, l] of [["current inventory", EXIT], ["three new points added", [...EXIT, ...NEW_EXITS]]]) {
const [dp, dpl] = measure(l, "scattered"), [, cpl] = measure(l, "central");
console.log(name.padEnd(28) + String(l.length).padStart(6) + String(l.length * VALUES.length).padStart(12) +
String(dp).padStart(18) + String(dpl).padStart(22) + String(cpl).padStart(9));
}
const count = (c) => EXIT.filter((x) => x[2] === c).length;
console.log("\ncontext distribution: " + Object.keys(CONTEXT).map((c) => `${c} ${count(c)}`).join(" "));
console.log("call-site choice: right " + EXIT.filter((x) => x[2] === x[3]).length +
", wrong " + EXIT.filter((x) => x[3] !== "raw" && x[2] !== x[3]).length +
", none at all " + EXIT.filter((x) => x[3] === "raw").length);
```
```
stage points placements scattered points scattered placements central
current inventory 14 84 5 11 0
three new points added 17 102 8 16 0
context distribution: body 5 attribute 3 script 2 address 4
call-site choice: right 9, wrong 3, none at all 2
```
Five of the fourteen exit points write broken output: three have the wrong encoder
chosen, two have no encoder at all. Under the central form, breakage is zero, because the
encoder is not chosen at the call site — it derives from the point's context declaration.
The difference between the two is not a difference of knowledge but a difference in
**where the decision is made**: nine call sites chose right, five did not; under the
central form there is nothing to choose.
The second row shows how coverage behaves at a new point. When three new exit points are
added, the number of broken points under the scattered form rises from five to eight —
**every new exit point is directly a candidate for breakage**, because it has no encoder
unless someone writes one. Under the central form, the same three points stay at zero.
#### Nested Context
Not every point in the inventory writes into a single context. When a piece of script is
placed inside an attribute's quotes, the value sits inside two containers and is read by
two rules.
```js
// nested.mjs — a second context embedded inside a first: script context inside an attribute
import { VALUES, ENCODER } from "./context.mjs";
const container = (x) => ``;
const decode = (s) => s.replace(/"/g, '"').replace(/
/g, "\n")
.replace(/</g, "<").replace(/>/g, ">").replace(/&/g, "&");
const read = (k) => { // first the outer container, then the inner container is read by its own rule
const m = /op="([^"]*)"/.exec(k);
if (!m) return null;
const code = decode(m[1]);
try { return JSON.parse(code.slice(code.indexOf('"'), code.lastIndexOf('"') + 1)); } catch { return null; }
};
console.log(`${VALUES.length} values, two nested contexts: script inside an attribute\n`);
console.log(`${"encoder choice".padEnd(32)}${"not read back".padStart(16)}`);
for (const [name, f] of [["outer context only (attribute)", ENCODER.attribute],
["inner context only (script)", ENCODER.script],
["outer then inner", (d) => ENCODER.script(ENCODER.attribute(d))],
["inner then outer", (d) => ENCODER.attribute(ENCODER.script(d))]])
console.log(name.padEnd(32) + String(VALUES.filter((d) => read(container(f(d))) !== d).length).padStart(16));
```
```
6 values, two nested contexts: script inside an attribute
encoder choice not read back
outer context only (attribute) 2
inner context only (script) 1
outer then inner 2
inner then outer 0
```
A single encoder is not enough in either direction: the outer context's breaks two
values, the inner context's breaks one value. When the two encoders are used together,
**order** decides the result. Going from outside in, two values stay broken; going from
inside out, all six are read back correctly.
The coverage consequence of this is direct. A declaration that holds **a single context**
per exit point cannot express a nested point; those points stay uncovered by the central
pattern, and the decision returns to the call site. A declaration that wants to keep
coverage fixed must carry not a context but a **context sequence**.
The context distribution line says something separate: of the fourteen points, five are
in the body context, four in address, three in attribute, two in script. All four
contexts are in use, and none of them dominates. It is impossible for a codebase working
with a single encoder to give the right result across this distribution; the number of
off-diagonal cells comes directly from this distribution.
#### The Second Place the Pattern Falls Short
Even when the central pattern chooses the right encoder at every point, it rests on one
assumption: the value coming from storage is raw. When that assumption is lifted, the
pattern breaks for a reason outside its control.
```js
// storage.mjs — what the central pattern does when the value enters storage already encoded
import { VALUES, ENCODER, CONTEXT } from "./context.mjs";
const C = Object.keys(CONTEXT);
const place = (c, v) => { const [container, read] = CONTEXT[c]; return read(container(ENCODER[c](v))); };
console.log(`${VALUES.length} values x ${C.length} contexts; the central pattern chooses the right encoder at every point\n`);
console.log(`${"storage decision".padEnd(28)}${"broken".padStart(9)} per context`);
for (const [name, store] of [["raw value stored", (v) => v],
["encoded once on the way in", ENCODER.body]]) {
const row = C.map((c) => VALUES.filter((v) => place(c, store(v)) !== v).length);
console.log(name.padEnd(28) + String(row.reduce((a, n) => a + n, 0)).padStart(9) +
" " + C.map((c, i) => `${c} ${row[i]}`).join(" "));
}
```
```
6 values x 4 contexts; the central pattern chooses the right encoder at every point
storage decision broken per context
raw value stored 0 body 0 attribute 0 script 0 address 0
encoded once on the way in 8 body 2 attribute 2 script 2 address 2
```
When the value is encoded once on the way in and stored that way, all four contexts each
break two values. The encoder choice is right, the container is right, the reader is
right; what breaks is the transformation the value goes through on the way into storage.
And the two notes carrying the ampersand and the less-than sign arrive at the reader as
escape sequences, because they get encoded a second time on the way out.
The coverage consequence of this breakage is heavier than the first one. A wrong choice
at the call site breaks **one** point; a wrong storage decision breaks **every** point
that reads that value — all fourteen of them in this inventory at once. The place for
encoding is the output, and the reason is not a principle but a count: at output the
decision is made per context, at input a single decision is imposed on every context.
#### Summary
- Four contexts and four encoders gave sixteen pairings. None of the 24 placements in the
correct pairings broke; 44 of the 72 placements in the twelve pairings with the wrong
context could not be read back.
- The wrong context breaks in both directions at once: the address encoder broke all six
of the six values in the other three contexts — not under-escaping but over-escaping.
The body encoder broke four values in the script context.
- Of the fourteen exit points, 9 had the right encoder chosen, 3 had the wrong one, and 2
had none chosen at all; under the scattered form, 5 points and 11 placements are
broken. When the encoder is derived from the context declaration, breakage is 0.
- When three new exit points were added, the number of broken points under the scattered
form rose from 5 to 8 and stayed at 0 under the central form. A new exit point is
broken from the start unless someone writes its encoder.
- When the value is encoded once on the way in and stored that way, 8 placements broke
even though the central pattern worked correctly. A wrong choice at the call site
breaks a single point; a wrong storage decision breaks all 14 points that read that
value at once.
- In the nested context, a single encoder was not enough: the outer context's broke 2
values, the inner context's broke 1. Order decides the outcome — inside out gives 0,
outside in gives 2. A declaration that holds a single context per point cannot cover
these points.
#### Next Step
Output encoding targets the container the value will be **read from**. Placing the value
into a query text somewhere inside the system is a separate problem: there, the container
is not a document but a query language, and the party reading that language has to
separate the data from the command within the same text. The next lesson counts query
points and measures a single question: how many query values pass through an interface
that carries them as a separate operand, how many query values are appended into the
text itself, and which of these two forms closes the point on its own when a new
query is added.
### Injection Defense
Source: https://academia.sh/en/courses/secure-coding/injection-defense
Measuring coverage at query points: sixteen query points are run with the same set of values and the number of points where the value reaches the query text is counted, the spot the parameterized query interface closes structurally is separated from why the identifier spot cannot be closed, and the coverage of the strict layer versus the flexible layer is compared when four new queries are added.
The previous lesson targeted the container the value would be read into: body, attribute,
script, address. In all four, the container was a document and the reader was a viewer.
Placing a value into a **query text** is a different problem, because there the reader has
to separate data from command, and the only thing it has to make that separation with is the
text itself.
This lesson counts query points. The Server Security and Going to Production course measured
the outcome difference two setups produced; what is measured here is not the outcome but
**coverage**: how many queries keep the value outside the text, how many fold it into the
text, and which point is covered on its own when a new query is written.
#### The Value That Reaches the Query Text
The measure is a single question: once the query text is built, does the value itself appear
in that text. This question does not require a database engine; the text is a string, and the
value is either in that string or it is not.
**IO7 (assumption):** the query text is a model; there is no engine running it. A point counts
as covered if none of its values appear in the text. How an engine would parse the text is not
measured.
```js
// interface.mjs — two ways to build a query. The query text is a model; there is no engine
// running it. The thing measured is a single question: did the value reach the query text?
// Parameterized query interface: accepts only a tagged template; does not accept a string.
export function query(parts, ...values) {
if (!Array.isArray(parts?.raw)) throw new TypeError("query text cannot be given as a string");
let text = parts[0];
const bound = [];
values.forEach((d, i) => {
if (d && d.identifier !== undefined) {
if (d.identifier === null) throw new RangeError("identifier not in allowlist");
text += d.identifier; // not a spot, a name: cannot be passed as bound
} else { text += "?"; bound.push(d); }
text += parts[i + 1];
});
return { text, bound, tracked: true };
}
// Manual concatenation: everything is folded into the text, no bound values remain.
export function manual(parts, ...values) {
let text = parts[0];
values.forEach((d, i) => {
text += String(d && d.identifier !== undefined ? d.raw : d) + parts[i + 1];
});
return { text, bound: [], tracked: false };
}
// Identifier spot: allowlist for field and table names.
const FIELD = { reading_time: "reading_time", period: "period", amount: "amount" };
export const name = (x) => ({ identifier: FIELD[x] ?? null, raw: x });
// Leak checker: how many of the given values appear in the query text.
export const leak = (s, values) =>
values.filter((d) => s.text.includes(String(d))).length;
```
The two forms are written almost identically — both take a tagged template, both build the
same query. The one place they diverge is where the value goes: one puts a placeholder in the
text and writes the value into a separate array, the other folds the value into the text
itself.
#### Query Point Inventory
The query points of the regional measurement network are counted: the subscriber, invoicing,
reading, work order, and report services. Each point writes one query shape with one
interface.
```js
// coverage.mjs — query point inventory (fictional model) and the coverage of the two forms
import { query, manual, name, leak } from "./interface.mjs";
// Four query shapes; every shape takes the same values, the only difference is which interface writes it.
const SHAPE = {
search: (b, v) => b`SELECT no FROM subscriber WHERE region = ${v.region} AND tariff = ${v.tariff}`,
fetch: (b, v) => b`SELECT * FROM invoice WHERE subscriber = ${v.subscriber} AND period = ${v.period}`,
sort: (b, v) => b`SELECT * FROM reading ORDER BY ${name(v.field)} LIMIT ${v.limit}`,
count: (b, v) => b`SELECT count(*) FROM work_order WHERE type = ${v.type}`,
};
const V = { region: "north", tariff: "residential", subscriber: "A-100432", period: "2026-03",
field: "reading_time", limit: 50, type: "disconnection" };
const VALUES_OF = { search: ["north", "residential"], fetch: ["A-100432", "2026-03"], sort: [50], count: ["disconnection"] };
// [service, query point, shape, interface]
const POINT = [
["subscriber", "regionSearch", "search", query], ["subscriber", "tariffSearch", "search", query],
["subscriber", "historySearch", "search", manual], ["invoice", "periodFetch", "fetch", query],
["invoice", "debtFetch", "fetch", query], ["invoice", "archiveFetch", "fetch", manual],
["reading", "latestReading", "sort", query], ["reading", "meterHistory", "sort", query],
["reading", "aggregateReport", "sort", manual], ["reading", "outlierReading", "search", query],
["work-order", "openCount", "count", query], ["work-order", "regionCount", "count", query],
["work-order", "historyCount", "count", manual], ["report", "summaryFetch", "fetch", query],
["report", "dumpSort", "sort", manual], ["report", "monthlyCount", "count", query],
];
const measure = (list) => {
let reached = 0, identifierSpots = 0, untracked = 0;
for (const [, , shape, b] of list) {
const s = SHAPE[shape](b, V);
reached += leak(s, VALUES_OF[shape]);
if (shape === "sort") { identifierSpots++; if (!s.tracked) untracked++; }
}
return [reached, identifierSpots, untracked];
};
const P = POINT.filter((n) => n[3] === query), M = POINT.filter((n) => n[3] === manual);
console.log(`${POINT.length} query points, ${Object.keys(SHAPE).length} query shapes\n`);
console.log(`${"form".padEnd(25)}${"points".padStart(8)}${"value spots".padStart(13)}` +
`${"reached text".padStart(14)}${"identifier spots".padStart(18)}${"untracked".padStart(11)}`);
for (const [label, l] of [["parameterized interface", P], ["manual concatenation", M]]) {
const spots = l.reduce((t, n) => t + VALUES_OF[n[2]].length, 0);
const [u, t, d] = measure(l);
console.log(label.padEnd(25) + String(l.length).padStart(8) + String(spots).padStart(13) +
String(u).padStart(14) + String(t).padStart(18) + String(d).padStart(11));
}
console.log("\nsample text (parameterized): " + SHAPE.fetch(query, V).text);
console.log("sample text (manual) : " + SHAPE.fetch(manual, V).text);
```
```
16 query points, 4 query shapes
form points value spots reached text identifier spots untracked
parameterized interface 11 17 0 2 0
manual concatenation 5 7 7 2 2
sample text (parameterized): SELECT * FROM invoice WHERE subscriber = ? AND period = ?
sample text (manual) : SELECT * FROM invoice WHERE subscriber = A-100432 AND period = 2026-03
```
Eleven of the sixteen points go through the parameterized interface, and across those eleven,
**none** of the seventeen value spots reach the text. Five points use manual concatenation,
and there all seven of the seven value spots are inside the text. The number of points left
exposed is five.
These five have something in common: three are archive and history queries, two are reports.
The main flow is protected, the edge flows are not — because the decision was made per point,
and the edge flows were written by a different person, or at a different time.
The same measure applies to command points as well: does the value pass to the subprocess as
a separate argument, or does it get folded into a single command string. The question is the
same question; the measure is the same measure — did the value reach the text.
How the number is obtained is itself part of coverage. The leak checker scans the text here,
but the `tracked` flag the interface returns gives the same information without scanning:
every query result carries which form built it. Coverage being countable depends on the
defense itself producing the count — a scanner looking from the outside sees the query text,
not how that text was built.
#### The Spot That Cannot Be Closed
The one column in the table that the parameterized interface does not zero out is
`identifier spots`. Two points put a field name into the query, and a field name cannot be
passed through a placeholder: a placeholder represents a **value**, not the structure of the
query. The sort field, the table name, and the direction keyword have to be written into the
text itself.
The parameterized interface cannot provide coverage here; the only thing it provides is
making this spot **visible**. Because the code is marked with a `name()` call, identifier
spots become countable, and the only defense there is the pattern from the first lesson: the
allowlist. The block below puts three decisions side by side.
```js
// identifier.mjs — three decisions at the identifier spot: manual, allowlist, placeholder attempt
import { query, manual, name } from "./interface.mjs";
const INPUT = ["reading_time", "period", "amount", "subscriber_no", "reading_time desc", "internal_note"];
const tail = (f) => { try { return f().text.split("ORDER BY ")[1]; } catch (e) { return e.constructor.name; } };
console.log(`${INPUT.length} sort field inputs, one identifier spot\n`);
console.log(`${"input".padEnd(20)}${"manual".padStart(20)}${"allowlist".padStart(14)}${"placeholder".padStart(13)}`);
let raw = 0, accepted = 0, broken = 0;
for (const g of INPUT) {
const m = tail(() => manual`SELECT * FROM reading ORDER BY ${name(g)}`);
const q = tail(() => query`SELECT * FROM reading ORDER BY ${name(g)}`);
const p = tail(() => query`SELECT * FROM reading ORDER BY ${g}`);
if (m === g) raw++;
if (q === g) accepted++;
if (p !== g) broken++;
console.log(g.padEnd(20) + m.padStart(20) + q.padStart(14) + p.padStart(13));
}
console.log(`\nmanual: ${raw}/${INPUT.length} inputs entered the text as-is`);
console.log(`allowlist: ${accepted} accepted, ${INPUT.length - accepted} rejected`);
console.log(`placeholder pass-through: ${broken}/${INPUT.length} queries had their structure broken`);
```
```
6 sort field inputs, one identifier spot
input manual allowlist placeholder
reading_time reading_time reading_time ?
period period period ?
amount amount amount ?
subscriber_no subscriber_no RangeError ?
reading_time desc reading_time desc RangeError ?
internal_note internal_note RangeError ?
manual: 6/6 inputs entered the text as-is
allowlist: 3 accepted, 3 rejected
placeholder pass-through: 6/6 queries had their structure broken
```
The last column is the proof of why this spot cannot be closed: when the input is passed as a
value, the text becomes `ORDER BY ?` in all six of the six queries, and the sort field is
erased from the query. A placeholder here is not a defense, it is a bug.
The middle column gives the only solution. Three of the six inputs are on the allowlist and
pass through as-is; three are rejected — one is a field not in the schema, one is a field with
a direction word appended, one is a name not on the list. In manual concatenation, all six
enter the query. **Coverage at the identifier spot comes from the length of the allowlist, not
from the parameterized interface**, and a legitimate field that is not on the list gets
rejected too — that is the cost of this spot.
#### When a New Query Is Added
The test of coverage begins again when the inventory grows. Four new query points open, and
they are written in two separate access layers: one offers only the parameterized interface,
the other also accepts the query text as a string.
**IO8 (assumption):** in the flexible layer, which form the person writing the new query will
choose is not known; what is measured is the worst case the layer **allows**. In the strict
layer, no such case exists, because there is no choice.
```js
// new-query.mjs — when a new query is added, which form covers the point on its own
import { query, manual, name, leak } from "./interface.mjs";
const attempt = (f) => { try { return "passed -> " + f().text; } catch (e) { return e.constructor.name + ": " + e.message; } };
console.log("query given as a string : " + attempt(() => query("SELECT no FROM subscriber WHERE region = 'north'")));
console.log("identifier outside allowlist: " + attempt(() => query`SELECT * FROM reading ORDER BY ${name("internal_note")}`));
console.log("same identifier, manual : " + attempt(() => manual`SELECT * FROM reading ORDER BY ${name("internal_note")}`));
// Four new query points; same shapes, only the interface changes.
const NEW = [
["missingReading", (b, v) => b`SELECT no FROM reading WHERE source = ${v.source}`, ["field"]],
["debtorSearch", (b, v) => b`SELECT subscriber FROM invoice WHERE amount > ${v.amount}`, [1200]],
["workOrderFilter", (b, v) => b`SELECT * FROM work_order WHERE type = ${v.type} AND region = ${v.region}`, ["connection", "south"]],
["dailyDump", (b, v) => b`SELECT * FROM reading ORDER BY ${name(v.field)} LIMIT ${v.limit}`, [100]],
];
const V = { source: "field", amount: 1200, type: "connection", region: "south", field: "period", limit: 100 };
const spots = NEW.reduce((t, [, , d]) => t + d.length, 0);
console.log(`\n${NEW.length} new query points, ${spots} value spots\n`);
console.log(`${"layer form".padEnd(39)}${"reached text".padStart(14)}${"untracked points".padStart(18)}`);
for (const [label, b] of [["strict layer (parameterized only)", query], ["flexible layer (also accepts strings)", manual]]) {
const reached = NEW.reduce((t, [, f, d]) => t + leak(f(b, V), d), 0);
const untracked = NEW.filter(([, f]) => !f(b, V).tracked).length;
console.log(label.padEnd(39) + String(reached).padStart(14) + String(untracked).padStart(18));
}
```
```
query given as a string : TypeError: query text cannot be given as a string
identifier outside allowlist: RangeError: identifier not in allowlist
same identifier, manual : passed -> SELECT * FROM reading ORDER BY internal_note
4 new query points, 5 value spots
layer form reached text untracked points
strict layer (parameterized only) 0 0
flexible layer (also accepts strings) 5 4
```
The first three lines show the interface's own limits. The query text cannot be given as a
string; if it is, the call stops at the first line. A field name that is not on the allowlist
does not get through either. That same field name enters the query in manual concatenation and
gives no sign of it.
The table below gives the real number. In the strict layer, none of the five value spots
across the four new queries reach the text, and the reason is not diligence, it is **that
there is no other way**. In the flexible layer, all five of the five value spots can reach the
text; the layer does not prevent this, it only allows it.
The difference here is not a difference in habit. In the strict layer, coverage is fixed
**without writing any code**; in the flexible layer, every new query depends on a person
choosing the right form, and across the sixteen-point inventory, that person chose something
else five times.
#### Summary
- The measure is a single question: once the query text is built, does the value appear in
that text. This question does not require a database engine; the query text is a string.
- 11 of the sixteen query points go through the parameterized interface, and there 0 of the
17 value spots reach the text; 5 points use manual concatenation, and there 7 of the 7 value
spots are inside the text. The number of points left exposed is 5, and all five are edge
flows.
- There is one spot the parameterized interface cannot close: the identifier spot. A field
name cannot be passed through a placeholder, because a placeholder represents a value, not
structure. There are 2 such points in the inventory. When the six sort inputs were passed as
values, the structure of 6 queries broke; the allowlist accepted 3 of them and rejected 3;
in manual concatenation, all 6 entered the query.
- The interface does not accept the query text as a string, and it rejects an identifier that
is outside the allowlist; both stop at the moment of the call. In manual concatenation, the
same identifier enters the query and gives no sign of it.
- Across four new queries, the strict layer leaked 0 value spots, the flexible layer leaked 5.
In the strict layer, coverage is fixed without writing code; in the flexible layer, every
new query depends on a decision.
#### Next Step
The parameterized interface keeps the value outside the text; at the identifier spot, the
allowlist takes over instead. Both treat the text as **fixed**: the skeleton of the query is
what the developer wrote. There is one place where this assumption does not hold.
Notification texts, report headers, and invoice description lines are produced from
templates, and some of these templates are built at runtime — some take only values, some
take user data into their body. The next lesson counts templates and measures a single
question: how many templates take user data only as a value, and how many turn that data into
part of an expression evaluated on the server.
### Template and Expression Injection
Source: https://academia.sh/en/courses/secure-coding/template-and-expression-injection
Measuring server-side evaluation risk as a surface: the number of names resolvable in the evaluation environment is counted with probes generated from the environment itself, a twelve-template inventory separates how many take user data only as a value versus as an expression, and the total evaluation surface of templates whose body comes from outside is compared across three setups.
The previous lesson assumed the query text was fixed: the developer writes the skeleton,
and the value comes from outside. Notification texts, report headers, and field labels do
not carry that assumption. Their body is a template, and some of these templates are read
at runtime from a configuration record or from a user field.
This lesson counts templates. The quantity measured is not whether the text gets corrupted
but the **evaluation surface**: how many names can be resolved while the template is
processed. If a template takes user data only as a **value**, the surface is limited to the
declared fields; if it takes the data as an **expression**, the surface is the entire
environment.
#### Two Engines, One Environment
The model contains two engines. The **value engine** substitutes only the declared names
and never rescans the result. The **expression engine** resolves a dotted path in the
environment; it has no notion of declaration. The environment is the set of objects
accessible while the template is processed.
**IO9 (assumption):** the environment and the engines are a model. The surface is the
number of leaf names resolvable in the environment by a dotted path; how sensitive the
value a name carries is, is not measured.
```js
// engine.mjs — two template engines and the evaluation environment (fictional model)
// Value engine: substitutes only the declared names; the result is never rescanned.
export const valueEngine = (template, env, declared) =>
template.replace(/\{([\w.]+)\}/g, (t, path) =>
declared.includes(path) ? String(path.split(".").reduce((o, k) => o?.[k], env)) : t);
// Expression engine: resolves a dotted path in the environment; it has no notion of declaration.
export const expressionEngine = (template, env) =>
template.replace(/\{([\w.]+)\}/g, (t, path) => {
const v = path.split(".").reduce((o, k) => o?.[k], env);
return v === undefined || typeof v === "object" ? t : String(v);
});
// Accessible name: the list of leaf names resolvable in the environment by a dotted path.
export const accessible = (o, prefix = "") =>
o === null || typeof o !== "object" ? [prefix]
: Object.entries(o).flatMap(([k, v]) => accessible(v, prefix ? `${prefix}.${k}` : k));
export const WIDE = {
subscriber: { name: "A. Yilmaz", number: "A-100432", tariff: "residential" },
invoice: { period: "2026-03", amount: 412, dueDate: "2026-04-15" },
settings: { region: "north", version: "3", logLevel: "info", serverName: "measurement-1" },
connection: { storeName: "measurement", roleName: "report_reader", timeout: 30 },
system: { workingDirectory: "/srv/measurement", processId: 4120 },
};
export const NARROW = { subscriber: WIDE.subscriber, invoice: WIDE.invoice };
export const DECLARED = ["subscriber.name", "subscriber.number", "invoice.period", "invoice.amount", "invoice.dueDate"];
```
The wide environment represents an application handing its template processor everything
within reach: alongside subscriber and invoice data, settings, connection information, and
process information. The narrow environment carries only the two objects the notification
requires.
The existence of the expression engine is not a flaw but a request. Letting the
administrator edit the notification text, letting the region name enter the report header,
and generating the field label from a template are genuine requirements of the work. The
evaluation surface is the numeric counterpart of that requirement: making the text editable
from outside also makes the environment that processes it queryable from outside. What must
be measured is the size of that trade-off.
#### Evaluation Surface
When the surface is counted, the probe set is not written by hand; it is **generated from
the environment itself**. A single placement is tried for every accessible name, and how
many resolve is counted. This way, the measurement updates itself whenever the environment
changes.
```js
// surface.mjs — evaluation surface: how many names can be resolved in the environment
import { valueEngine, expressionEngine, accessible, WIDE, NARROW, DECLARED } from "./engine.mjs";
// The probe set is generated from the environment itself: a single placement per accessible name.
const probes = (env) => accessible(env).map((path) => [path, `{${path}}`]);
console.log(`declared names: ${DECLARED.length}\n`);
console.log(`${"environment / engine".padEnd(23)}${"accessible names".padStart(19)}${"resolved".padStart(10)}${"undeclared".padStart(12)}`);
for (const [name, env, engine] of [["wide / expression", WIDE, "expression"], ["wide / value", WIDE, "value"],
["narrow / expression", NARROW, "expression"], ["narrow / value", NARROW, "value"]]) {
const s = probes(env);
const apply = (x) => (engine === "expression" ? expressionEngine(x, env) : valueEngine(x, env, DECLARED));
const resolved = s.filter(([, x]) => apply(x) !== x);
const outside = resolved.filter(([path]) => !DECLARED.includes(path)).length;
console.log(name.padEnd(23) + String(s.length).padStart(19) + String(resolved.length).padStart(10) +
String(outside).padStart(12));
}
const outsideWide = accessible(WIDE).filter((y) => !DECLARED.includes(y));
console.log(`\nnames left undeclared in the wide environment: ${outsideWide.length}`);
console.log(`top-level name set: wide ${Object.keys(WIDE).join(" ")} / narrow ${Object.keys(NARROW).join(" ")}`);
```
```
declared names: 5
environment / engine accessible names resolved undeclared
wide / expression 15 15 10
wide / value 15 5 0
narrow / expression 6 6 1
narrow / value 6 5 0
names left undeclared in the wide environment: 10
top-level name set: wide subscriber invoice settings connection system / narrow subscriber invoice
```
The four rows separate two independent defenses. **Narrowing the environment** brings the
accessible names down from fifteen to six; the expression engine still resolves all of
them, but there are no names left to resolve. **Fixing the engine to value mode** brings
the resolved names down to the declared set regardless of the environment: even in the
wide environment, the number of names resolved outside the declared set is zero.
The last row gives the combined meaning of the two. In the narrow environment, the value
engine resolves five names, the expression engine six — the single name in between is
`subscriber.tariff`, a field that does not appear in the declared set. The surface is not
zeroed out; **it narrows and stays countable**.
#### Template Inventory
The surface is measured per template, and one thing alone determines it: where the
template's **body** comes from. If the body is fixed, the developer wrote the names to be
resolved, and no name comes from outside. If the body is read from a configuration record
or a user field, every resolvable name is that template's surface.
**IO10 (assumption):** a template whose body is fixed is counted as having zero surface.
What is measured is not who wrote the template but **where it is read from**.
```js
// inventory.mjs — template inventory (fictional model) and total evaluation surface
import { accessible, WIDE, NARROW, DECLARED } from "./engine.mjs";
const ENV = { wide: WIDE, narrow: NARROW };
// [service, template, body source, engine, environment]
const TEMPLATE = [
["notification", "invoiceIssued", "fixed", "value", "narrow"],
["notification", "dueDateApproaching", "fixed", "value", "narrow"],
["notification", "readingMissing", "fixed", "value", "narrow"],
["invoice", "statementHeader", "fixed", "value", "narrow"],
["invoice", "descriptionLine", "fixed", "value", "narrow"],
["work-order", "taskText", "fixed", "value", "narrow"],
["work-order", "closingNote", "fixed", "value", "narrow"],
["notification", "campaignText", "configuration", "expression", "wide"],
["report", "header", "configuration", "expression", "wide"],
["report", "footer", "configuration", "value", "narrow"],
["report", "summaryHeading", "user", "expression", "wide"],
["field", "labelText", "user", "expression", "wide"],
];
const NEW = [["report", "regionHeading", "configuration"], ["notification", "reminder", "configuration"],
["field", "formHeading", "configuration"]];
// Surface: if the body is fixed, the set of names to be resolved is already inside the template.
const surface = ([, , source, engine, e]) =>
source === "fixed" ? 0 : engine === "expression" ? accessible(ENV[e]).length : DECLARED.length;
const total = (l) => l.reduce((t, s) => t + surface(s), 0);
const count = (l, i, v) => l.filter((s) => s[i] === v).length;
console.log(`${TEMPLATE.length} templates, all take user data`);
console.log(` engine: value ${count(TEMPLATE, 3, "value")}, expression ${count(TEMPLATE, 3, "expression")}`);
console.log(` body source: fixed ${count(TEMPLATE, 2, "fixed")}, configuration ` +
`${count(TEMPLATE, 2, "configuration")}, user ${count(TEMPLATE, 2, "user")}\n`);
const SETUP = [
["current state", (s) => s],
["pattern: value engine + narrow environment", ([a, b, k]) => [a, b, k, "value", "narrow"]],
["pattern: body fixed too", ([a, b]) => [a, b, "fixed", "value", "narrow"]],
];
console.log(`${"setup".padEnd(44)}${"templates".padStart(11)}${"external body".padStart(15)}${"total surface".padStart(15)}`);
for (const [name, d] of SETUP) {
const l = TEMPLATE.map(d), y = NEW.map(([a, b, k]) => d([a, b, k, "expression", "wide"]));
console.log(name.padEnd(44) + String(l.length).padStart(11) +
String(l.length - count(l, 2, "fixed")).padStart(15) + String(total(l)).padStart(15));
console.log(` after three new templates: ${l.length + y.length} templates, surface ${total([...l, ...y])}`);
}
```
```
12 templates, all take user data
engine: value 8, expression 4
body source: fixed 7, configuration 3, user 2
setup templates external body total surface
current state 12 5 65
after three new templates: 15 templates, surface 110
pattern: value engine + narrow environment 12 5 25
after three new templates: 15 templates, surface 40
pattern: body fixed too 12 0 0
after three new templates: 15 templates, surface 0
```
All twelve templates take user data; that alone carries no weight. The split is that eight
take the data only as a value, four as an expression. And the real split is in the body
source: seven templates have a fixed body, three are read from configuration, two directly
from a user field.
The total surface is sixty-five. All of that number comes from five templates; the
remaining seven contribute nothing at all. **The surface grows not with the number of
templates but with the number of templates whose body comes from outside.**
Two of the five templates are a separate class. The report summary heading and the field
label text take their bodies not from a configuration record but directly from a user
field: the heading the user writes is processed as the template's body. The number of
people who can edit a body coming from configuration is limited, and that record can pass
through an approval flow; there is no such limit for a body coming from a user field. The
surface of the two is the same — fifteen names each — but the number of people who can
reach that surface is not.
The second setup gives the pattern's payoff: when all templates are moved to the value
engine and the narrow environment, the surface drops from sixty-five to twenty-five. It
does not drop to zero, because the body is still read from outside and the five declared
names can be resolved in every one of them. In the third setup, the body is fixed too, and
the surface is zeroed out — but in this setup the administrator can no longer edit the
notification text. That is the cost, and it needs to be written down.
The sub-rows give the third column. When three new templates are added, the surface in the
current state rises from sixty-five to one hundred ten: **each new template adds fifteen
names**. Under the pattern, the same three templates raise the surface from twenty-five to
forty — five names each. In the fixed-body pattern, three new templates do not change the
surface at all.
Comparing the three setups says one more thing. In the second setup, while the surface
drops to twenty-five, none of the templates lose their function: the notification text can
still be edited by the administrator, only the set of names the edited text can resolve
narrows. In the third setup, the twenty-five units gained are paid for with a lost
capability. **The only way to zero out the surface is to stop the text from being editable
from outside**, and that is not a security decision but a product decision.
#### Body Review
For templates whose body cannot be fixed, a third defense remains: passing the body itself
through an allowlist. This means reading the body, extracting the names that appear in it,
and rejecting the template if any name is not in the list. Its gain and its cost are read
from the same number.
```js
// body-review.mjs — passing an externally sourced template body through an allowlist
import { accessible, WIDE, DECLARED } from "./engine.mjs";
// Template bodies read from configuration (fictional): all legitimate texts written by the administrator.
const BODY = [
"Dear {subscriber.name}, your invoice for the {invoice.period} period is ready.",
"Due date {invoice.dueDate}.",
"{invoice.amount} units were charged for subscription {subscriber.number}.",
"Monthly statement for region {settings.region}.",
"Generated via server {settings.serverName}, version {settings.version}.",
];
const names = (s) => [...s.matchAll(/\{([\w.]+)\}/g)].map((m) => m[1]);
const passes = (s, allowlist) => names(s).every((a) => allowlist.includes(a));
const ALLOWLIST = [["declared name set", DECLARED],
["declared + settings.region", [...DECLARED, "settings.region"]],
["no body review", accessible(WIDE)]];
console.log(`${BODY.length} template bodies, ${new Set(BODY.flatMap(names)).size} distinct names used\n`);
console.log(`${"allowlist".padEnd(28)}${"length".padStart(8)}${"passing bodies".padStart(16)}${"blocked legitimate bodies".padStart(27)}${"surface".padStart(9)}`);
for (const [name, allowlist] of ALLOWLIST) {
const passing = BODY.filter((s) => passes(s, allowlist)).length;
console.log(name.padEnd(28) + String(allowlist.length).padStart(8) + String(passing).padStart(16) +
String(BODY.length - passing).padStart(27) + String(allowlist.length).padStart(9));
}
```
```
5 template bodies, 8 distinct names used
allowlist length passing bodies blocked legitimate bodies surface
declared name set 5 3 2 5
declared + settings.region 6 4 1 6
no body review 15 5 0 15
```
The three rows give a single curve: every time the allowlist grows longer, one legitimate
body is rescued and the surface grows by one. With a list of five names, three bodies pass,
and two legitimate bodies are blocked. When the region name is added to the list, four
pass, and the surface rises to six. When the review is removed, all five pass, and the
surface becomes fifteen.
One case remains outside the review's coverage. In the two templates whose body is read
directly from a user field, a text carrying curly braces can be legitimate — a unit of
measure can be written in a report heading, or a field code in a field label. There, the
allowlist rejects the legitimate heading and the cost becomes visible; the decision to
read the body from a user field is a decision the allowlist cannot resolve.
This is the allowlist from the first lesson applied to the template body, and it carries
the same property: a legitimate name not in the list is rejected. The difference is what
is measured — there, the uncovered points were counted; here, it is **the number of
resolvable names**. The length of the allowlist is directly the surface itself; that is
why every request for a new name is a surface decision and needs to be written down
somewhere.
#### Summary
- The quantity measured is the evaluation surface: the number of names resolvable in the
environment while the template is processed. The probe set is not written by hand; it is
generated from the environment itself.
- In the wide environment, the expression engine resolved 15 of 15 names, and 10 of those
were outside the declared set. Narrowing the environment brought the accessible names
down to 6; fixing the engine to value mode brought the names resolved outside the
declared set to 0 even in the wide environment. The two defenses are independent.
- All twelve templates take user data; 8 only as a value, 4 as an expression. The
body-source split is decisive: 7 fixed, 3 from configuration, 2 from a user field.
- The total surface is 65, and all of it comes from the 5 templates whose body comes from
outside. The value engine and the narrow environment bring the surface down to 25; fixing
the body too brings it down to 0. The cost of the third option is that the notification
text can no longer be edited by the administrator.
- For a template whose body cannot be fixed, the third defense is passing the body through
an allowlist: with a list of 5 names, 3 of the 5 bodies passed and 2 legitimate bodies
were blocked; when the list grew to 6, 4 passed and the surface became 6. The length of
the allowlist is directly the surface itself.
- With three new templates, the surface rose from 65 to 110 in the current state, from 25
to 40 under the pattern, and stayed at 0 in the fixed-body setup. The surface grows not
with the number of templates but with the number of templates whose body comes from
outside.
#### Next Step
The template body was text, and its coming from outside grew the surface. The same
question is asked for inputs that are not text. The field application uploads a photo and
a measurement file at work-order closure; the subscriber portal accepts a file for a
meter-reading printout. This input has no contract — it is a byte sequence, and everything
said about it is the client's claim. The next lesson counts the upload flow as a layer of
checks: how many of the type, size, name, and storage-location checks are present, which
check alone is not enough, and how many points exposing the storage location opens up.
### File Upload Security
Source: https://academia.sh/en/courses/secure-coding/file-upload-security
Counting the upload flow as a layer of checks: ten upload items pass through five check layers alone and layer by layer, measuring which check alone is not enough and how many unexpected items pass when one is missing, counting the points that a directly addressable storage location opens, and comparing the manual check count against the upload point count as it grows.
The previous lesson measured how much a template body coming from outside grows the
surface. There, what came in was at least text, and the names inside it could be counted.
In the upload flow, what comes in is a byte sequence, and everything said about it — its
type, its name, its size — is the client's claim.
This lesson counts the upload flow as a **layer of checks**. What is measured is not
whether a single file is malicious, but how many layers it passes through: which check
alone is not enough, how many items pass through when one is missing, and how many
points the choice of storage location opens.
#### Upload Items
The field application uploads a measurement photo and a printout document at work-order
closure. The item set is fictional; each item's **head bytes are real**, and the rest is a
modeled size.
**IO11 (assumption):** each item's expected decision is manually labeled and correct. The
counts are relative to this label set; they say nothing about an upload form outside the
set.
```js
// item.mjs — upload items from the field application (fictional model)
// Each item's head bytes are real; the rest of the item is a modeled size.
export const SIGNATURE = { "image-png": [0x89, 0x50, 0x4e, 0x47], "image-jpeg": [0xff, 0xd8, 0xff],
"document-pdf": [0x25, 0x50, 0x44, 0x46], "compressed": [0x50, 0x4b, 0x03, 0x04] };
const headBytes = (t) => Uint8Array.from(SIGNATURE[t] ?? [0x6f, 0x6b, 0x0a]);
export const findType = (b) =>
Object.entries(SIGNATURE).find(([, s]) => s.every((x, i) => b[i] === x))?.[0] ?? "unknown";
// [name (given by the client), declared type, actual content type, size, expected decision]
export const ITEM = [
["measurement-north.png", "image-png", "image-png", 120_000, "accept"],
["measurement-south.jpg", "image-jpeg", "image-jpeg", 240_000, "accept"],
["printout-2026-03.pdf", "document-pdf", "document-pdf", 380_000, "accept"],
["meter.png", "image-png", "image-jpeg", 90_000, "reject"],
["archive.png", "image-png", "compressed", 150_000, "reject"],
["panorama.png", "image-png", "image-png", 12_000_000, "reject"],
["note.txt", "text-plain", "unknown", 2_000, "reject"],
["field/measurement.png", "image-png", "image-png", 110_000, "reject"],
["measurement-north.png", "image-png", "image-png", 130_000, "reject"],
["cover.jpeg", "image-png", "image-jpeg", 95_000, "reject"],
].map(([name, declared, actual, size, expected], i) =>
({ no: i + 1, name, declared, size, expected, data: headBytes(actual) }));
export const ALLOWED = ["image-png", "image-jpeg", "document-pdf"];
export const EXTENSION = { png: "image-png", jpg: "image-jpeg", jpeg: "image-jpeg", pdf: "document-pdf" };
export const MAX_SIZE = 5_000_000;
```
Seven items were labeled unexpected, and none carries malicious content. One photo is
named with the wrong extension, one compressed file's name carries an image extension, one
panorama exceeds the limit, one name contains a path separator, and one name collides with a
file already uploaded. These are the ordinary accidents of field use; the measurement counts
them too.
#### Five Check Layers
Each layer asks a single question and looks at a single thing. The declared type looks at
the type the client reports, the extension looks at the end of the file name, the signature
looks at the actual bytes, the size looks at the limit, and the name looks at how the
storage name is determined.
```js
// checks.mjs — five check layers. Each layer asks a single question.
import { ITEM, ALLOWED, EXTENSION, MAX_SIZE, findType } from "./item.mjs";
export const LAYER = {
declared: (k) => ALLOWED.includes(k.declared),
extension: (k) => EXTENSION[k.name.split(".").pop().toLowerCase()] === k.declared,
signature: (k) => findType(k.data) === k.declared,
size: (k) => k.size <= MAX_SIZE,
name: (k, seen) => !k.name.includes("/") && !seen.has(k.name),
};
// Applies a check set to all items.
export function run(layerNames) {
const seen = new Set(), acceptedItems = [];
let unexpectedPassed = 0, legitimateBlocked = 0;
for (const k of ITEM) {
const passed = layerNames.every((a) => LAYER[a](k, seen));
if (passed) { acceptedItems.push(k); seen.add(k.name); }
if (passed && k.expected === "reject") unexpectedPassed++;
if (!passed && k.expected === "accept") legitimateBlocked++;
}
return { accepted: acceptedItems.length, acceptedItems, unexpectedPassed, legitimateBlocked };
}
```
```js
// coverage.mjs — each layer alone, then layer by layer
import { ITEM } from "./item.mjs";
import { LAYER, run } from "./checks.mjs";
const LAYERS = Object.keys(LAYER);
const unexpected = ITEM.filter((k) => k.expected === "reject").length;
console.log(`${ITEM.length} upload items: ${ITEM.length - unexpected} expected, ${unexpected} unexpected\n`);
console.log(`${"layer alone".padEnd(13)}${"accepted".padStart(10)}${"unexpected passed".padStart(19)}${"legitimate blocked".padStart(20)}`);
for (const a of LAYERS) {
const r = run([a]);
console.log(a.padEnd(13) + String(r.accepted).padStart(10) + String(r.unexpectedPassed).padStart(19) +
String(r.legitimateBlocked).padStart(20));
}
console.log(`\n${"layer by layer".padEnd(48)}${"accepted".padStart(10)}${"unexpected passed".padStart(19)}${"legitimate blocked".padStart(20)}`);
for (let i = 1; i <= LAYERS.length; i++) {
const set = LAYERS.slice(0, i), r = run(set);
console.log(set.join(" + ").padEnd(48) + String(r.accepted).padStart(10) +
String(r.unexpectedPassed).padStart(19) + String(r.legitimateBlocked).padStart(20));
}
```
```
10 upload items: 3 expected, 7 unexpected
layer alone accepted unexpected passed legitimate blocked
declared 9 6 0
extension 8 5 0
signature 6 3 0
size 9 6 0
name 8 5 0
layer by layer accepted unexpected passed legitimate blocked
declared 9 6 0
declared + extension 8 5 0
declared + extension + signature 6 3 0
declared + extension + signature + size 5 2 0
declared + extension + signature + size + name 3 0 0
```
The top table shows that no layer alone is enough. The strongest layer is the signature,
and even it stops only four of the seven unexpected items and lets three through. The
declared type and the size each let six items through. This is the number behind the
sentence **"type checking alone is not enough"**: the declared type alone does not look at
the content even once, in six of the items it lets through.
The bottom table shows how the layers complete each other. With all five layers, the
unexpected items passing through drop to zero, and the legitimate-blocked column stays at
zero throughout — in this set, the layers do not block legitimate uploads. The cost coming
out at zero is because the set is narrow, and IO11 says so.
#### The Missing Layer and the Storage Location
Which layer actually carries the load is seen by removing it. The same block also measures
the storage location: where the accepted items are written and whether that location is
directly addressable.
**IO12 (assumption):** every accepted item that is directly addressable is counted as a
point. What is measured is not the outcome the point opens, but its **count**.
```js
// storage.mjs — the cost of a missing layer and the points the storage location opens
import { ITEM } from "./item.mjs";
import { LAYER, run } from "./checks.mjs";
const LAYERS = Object.keys(LAYER), FULL = run(LAYERS);
console.log(`${"missing layer".padEnd(15)}${"accepted".padStart(10)}${"unexpected passed".padStart(19)}`);
for (const a of LAYERS) {
const r = run(LAYERS.filter((x) => x !== a));
console.log(a.padEnd(15) + String(r.accepted).padStart(10) + String(r.unexpectedPassed).padStart(19));
}
console.log(`${"none missing".padEnd(15)}${String(FULL.accepted).padStart(10)}${String(FULL.unexpectedPassed).padStart(19)}`);
// Storage location: same accepted set, three storage forms. The storage name is derived from the item.
const generatedName = (k) => `y-${String(k.no).padStart(4, "0")}.bin`;
const STORAGE = [["served directory + client name", true, (k) => k.name],
["served directory + generated name", true, generatedName],
["unserved store + intermediary endpoint", false, generatedName]];
console.log(`\n${"storage form".padEnd(40)}${"addressable".padStart(13)}${"with client name".padStart(18)}${"authorization point".padStart(21)}`);
for (const [name, served, nameFn] of STORAGE) {
const addressable = served ? FULL.acceptedItems.length : 0;
const withClientName = served ? FULL.acceptedItems.filter((k) => nameFn(k) === k.name).length : 0;
console.log(name.padEnd(40) + String(addressable).padStart(13) +
String(withClientName).padStart(18) + String(served ? 0 : 1).padStart(21));
}
```
```
missing layer accepted unexpected passed
declared 3 0
extension 3 0
signature 5 2
size 4 1
name 5 2
none missing 3 0
storage form addressable with client name authorization point
served directory + client name 3 3 0
served directory + generated name 3 0 0
unserved store + intermediary endpoint 0 0 1
```
The top table separates the layers by weight. When the declared type and extension are
removed, the unexpected items passing through are still zero: whatever these two layers
stop, the signature also stops. When the signature, name, and size are removed, two, two,
and one item pass through, respectively. **Three of the five layers carry the load, two are
redundant.** Being redundant does not mean they are unnecessary — they are cheap and reject
earlier — but they are not what provides the coverage.
The signature being load-bearing does not mean it is sufficient. The signature looks only at
the first few bytes and says nothing about the rest of the file; the only thing it measures
is whether the declared type is consistent with the content. This is the boundary of that
measurement's scope, and it needs to be written down: the number that hits zero in the table
is **the inconsistency between the declared type and the content**, not that the file is
safe.
The middle table measures the storage decision. All three items are accepted; the only thing
that changes is where they are written. When written to a served directory, all three open
three directly addressable points. If the name comes from the client, all three of those
three addresses are addresses the uploader knows. The server-generated name brings this
count down to zero but does not change the point count. In the unserved store and
intermediary endpoint setup, the addressable point is zero, but in exchange one
authorization check point is born — the file is now handed out through an endpoint, and at
that endpoint, who can obtain what has to be asked.
The storage decision being independent of the acceptance decision is this measurement's
main finding. Even if all five layers work correctly and the unexpected items passing
through are zero, the storage location still opens three points. The two decisions are made
in two separate places, and one does not close the other; the coverage of the upload flow is
known only when both are counted together. A review that counts the check layers but not
the storage location has left out half the count.
#### Upload Point Inventory
Everything measured so far was on a single upload point. The regional measurement network
has five upload points, and each has its layers written separately.
```js
// points.mjs — upload point inventory: which layer is manually written at which point
import { ITEM } from "./item.mjs";
import { LAYER, run } from "./checks.mjs";
const LAYERS = Object.keys(LAYER);
// [service, upload point, layers manually written at that point]
const POINT = [
["field", "work-order attachment", ["declared", "extension", "signature", "size", "name"]],
["field", "photo", ["declared", "extension", "signature"]],
["subscriber portal", "document", ["declared", "size"]],
["subscriber portal", "reading printout", ["declared", "extension", "size", "name"]],
["invoice", "dispute attachment", ["declared"]],
];
console.log(`${POINT.length} upload points, ${LAYERS.length} layers, ${ITEM.length} items\n`);
console.log(`${"upload point".padEnd(23)}${"layers written".padStart(16)}${"missing".padStart(9)}${"unexpected passed".padStart(19)}`);
let manual = 0, passed = 0, full = 0;
for (const [, n, k] of POINT) {
const r = run(k);
manual += k.length; passed += r.unexpectedPassed; full += k.length === LAYERS.length ? 1 : 0;
console.log(n.padEnd(23) + String(k.length).padStart(16) + String(LAYERS.length - k.length).padStart(9) +
String(r.unexpectedPassed).padStart(19));
}
console.log(`\n${"setup".padEnd(26)}${"checks written".padStart(16)}${"fully covered points".padStart(22)}${"unexpected passed".padStart(19)}`);
console.log("manual per-point writing".padEnd(26) + String(manual).padStart(16) +
String(full).padStart(22) + String(passed).padStart(19));
const central = run(LAYERS).unexpectedPassed * POINT.length;
console.log("central upload pattern".padEnd(26) + String(LAYERS.length).padStart(16) +
String(POINT.length).padStart(22) + String(central).padStart(19));
console.log(`\nchecks to write when a new upload point is added: manual ${LAYERS.length}, central pattern 0`);
```
```
5 upload points, 5 layers, 10 items
upload point layers written missing unexpected passed
work-order attachment 5 0 0
photo 3 2 3
document 2 3 5
reading printout 4 1 2
dispute attachment 1 4 6
setup checks written fully covered points unexpected passed
manual per-point writing 15 1 16
central upload pattern 5 5 0
checks to write when a new upload point is added: manual 5, central pattern 0
```
Only one of the five points carries all five layers. The remaining four have a combined ten
missing layers, and the same set of ten items passing through these points produces sixteen
unexpected acceptances. The weakest point, the dispute attachment, carries a single layer and
accepts six of the seven unexpected items.
The difference between the two setups also shows up in the number of checks written. In
manual per-point writing, fifteen checks are written and one point is fully covered; in the
central pattern, five checks are written in a single place and all five of the five points
are covered. **The same coverage is achieved with three times less code**, and the number of
checks to write when a new upload point is added stays at zero.
#### Summary
- Seven of the ten upload items were labeled unexpected, and none carries malicious content;
there is a wrong extension, a wrong declared type, a limit overrun, a name with a path
separator, and a name collision.
- No layer alone was enough: the strongest, the signature, stopped 4 of 7 unexpected items
and let 3 through, while the declared type and size each let 6 items through. The five
layers together brought the unexpected items passing through down to 0, and no legitimate
item was blocked at any stage.
- The layers do not carry equal weight: with the declared type and extension removed, the
items passing through stay at 0; with the signature and name removed, 2 pass each; with
the size removed, 1 passes. Three layers carry the load, two are redundant.
- The storage location is a point independent of the acceptance decision: in the served
directory, 3 accepted items open 3 addressable points. The server-generated name brings
the guessable address down to 0; the unserved store brings the addressable point down to 0
and adds 1 authorization check point.
- Only 1 of the five upload points carries all five layers; the remaining four have 10
missing layers, and the same item set produces 16 unexpected acceptances. Manual
per-point writing has 15 checks written, the central pattern covers all five points with 5
checks, and the checks to write for a new point are 0.
#### Next Step
In the upload flow, what came from outside was a byte sequence, and the server only stored
it. In some flows the server goes further: it makes a request itself to an address that
comes from outside. The report generator fetches a document from the link the subscriber
portal provides, and the field application reaches an external endpoint for a map tile. The
party making those requests is the server, and the server's place on the network is
different from the client's. The next lesson passes outbound request targets through a set
of constraints and measures a single question: how many targets can reach the internal
network, and at what point does applying the constraint bring the coverage down.
### Server-Side Request Forgery
Source: https://academia.sh/en/courses/secure-coding/server-side-request-forgery
Passing outbound request targets through a set of constraints: twelve targets are passed through four constraint forms and how many can reach the internal network is counted with real address arithmetic, how many requests a redirect chain delivers to an internal address when the constraint is applied only at the first step is measured, and the point that repeating name resolution between the check and the connection opens is shown.
The previous lesson counted the bytes the server received in the upload flow; there, the
server only stored them. In some flows the server goes further and **itself** makes a
request to an address that comes from outside: the report generator fetches a document from
the link the subscriber portal provides, and the field application gets a map tile from an
external endpoint.
In the earlier measured form of request forgery, the request was sent by the user's browser
and carried the user's identity. Here, the server itself sends the request, and the
privilege it carries is not identity but **its place on the network**: the server is inside
the internal network, the client is not. This lesson measures that difference in terms of
target count.
#### Address Arithmetic
Whether a target reaches the internal network is not determined by the name, it is
determined by the address. The address is a thirty-two-bit number; whether it falls into a
block is an AND operation with a mask. The block below writes this arithmetic and builds a
fictional name resolver — the network is never touched.
**IO13 (assumption):** name resolution is a model; no real query is made, and the names in
the table resolve to ranges reserved for documentation. What is measured is not the accuracy
of the resolution, but **which value the constraint is applied to**.
```js
// address.mjs — address arithmetic and a fictional name resolver. No network access, all model.
export const parse = (a) => {
const p = a.split(".");
if (p.length !== 4) return null;
const s = p.map((x) => (/^\d{1,3}$/.test(x) ? Number(x) : -1));
return s.every((n) => n >= 0 && n <= 255)
? (((s[0] << 24) >>> 0) + (s[1] << 16) + (s[2] << 8) + s[3]) >>> 0 : null;
};
export const block = (c) => {
const [base, length] = c.split("/");
const m = length === "0" ? 0 : (0xffffffff << (32 - Number(length))) >>> 0;
return [(parse(base) & m) >>> 0, m, c];
};
export const inRange = (addr, [base, m]) => ((addr & m) >>> 0) === base;
// Address blocks reserved for local and private use.
export const PRIVATE = ["0.0.0.0/8", "10.0.0.0/8", "100.64.0.0/10", "127.0.0.0/8",
"169.254.0.0/16", "172.16.0.0/12", "192.168.0.0/16"].map(block);
export const isPrivate = (addr) => addr !== null && PRIVATE.some((b) => inRange(addr, b));
// Fictional name resolver: name -> address. No real resolution is performed.
export const RESOLVE = {
"document.ext-1.example": "203.0.113.24", "tile.ext-2.example": "198.51.100.9",
"archive.ext-1.example": "203.0.113.40", "archive-2.ext-1.example": "203.0.113.41",
"gateway.ext-3.example": "192.168.30.5",
"report.internal": "10.12.4.7", "measurement-store.internal": "10.12.4.20", "local.internal": "127.0.0.1",
"edge.internal": "169.254.10.5", "meter.internal": "172.20.8.3",
};
export const resolve = (a) => parse(RESOLVE[a] ?? a);
// Outbound request targets (fictional): the addresses fetched by the report generator and the field application.
export const TARGET = [
["document.ext-1.example", "declared"], ["tile.ext-2.example", "declared"],
["archive.ext-1.example", "user"], ["gateway.ext-3.example", "user"],
["report.internal", "user"], ["measurement-store.internal", "user"], ["local.internal", "user"],
["edge.internal", "user"], ["meter.internal", "user"], ["203.0.113.99", "user"],
["10.12.4.9", "user"], ["127.0.0.1", "user"],
];
export const DECLARED = ["document.ext-1.example", "tile.ext-2.example", "archive.ext-1.example"];
```
Three of the targets are declared external hosts; nine come from user data, and one is both.
This split is
the starting point of the coverage: the set the constraint has to close is nine items, three
items are already inside the contract.
#### Four Constraint Forms
The same target set is passed through four constraints. With no constraint, with a denylist
that looks at the name, with a denylist that looks at the resolved address, and with an
allowlist that counts declared hosts.
```js
// constraint.mjs — the same target set passed through four constraint forms
import { TARGET, DECLARED, PRIVATE, resolve, isPrivate, block, inRange, parse } from "./address.mjs";
const CONSTRAINT = [
["no constraint", () => true],
["name-based denylist", (h) => !h.endsWith(".internal")],
["address-based denylist", (h) => !isPrivate(resolve(h))],
["allowlist (declared host)", (h) => DECLARED.includes(h)],
];
console.log(`${TARGET.length} outbound request targets, ${PRIVATE.length} private address blocks`);
console.log(`declared external hosts: ${DECLARED.length}, targets from user data: ` +
`${TARGET.filter(([, k]) => k === "user").length}\n`);
console.log(`${"constraint form".padEnd(27)}${"passed".padStart(8)}${"reaching internal network".padStart(27)}${"declared blocked".padStart(18)}`);
for (const [name, f] of CONSTRAINT) {
const passed = TARGET.filter(([h]) => f(h));
const internal = passed.filter(([h]) => isPrivate(resolve(h))).length;
const blocked = DECLARED.filter((h) => !f(h)).length;
console.log(name.padEnd(27) + String(passed.length).padStart(8) + String(internal).padStart(27) +
String(blocked).padStart(18));
}
// The address arithmetic really runs: which target falls into which block.
console.log("\ntargets falling into a private block:");
for (const [h] of TARGET) {
const a = resolve(h);
const b = PRIVATE.find((x) => a !== null && inRange(a, x));
if (b) console.log(` ${h.padEnd(28)} -> ${String(a >>> 0).padStart(10)} block ${b[2]}`);
}
console.log(`\n10.12.4.7 -> ${parse("10.12.4.7")}, 10.0.0.0/8 base ${block("10.0.0.0/8")[0]}, ` +
`mask ${block("10.0.0.0/8")[1] >>> 0}`);
```
```
12 outbound request targets, 7 private address blocks
declared external hosts: 3, targets from user data: 10
constraint form passed reaching internal network declared blocked
no constraint 12 8 0
name-based denylist 7 3 0
address-based denylist 4 0 0
allowlist (declared host) 3 0 0
targets falling into a private block:
gateway.ext-3.example -> 3232243205 block 192.168.0.0/16
report.internal -> 168559623 block 10.0.0.0/8
measurement-store.internal -> 168559636 block 10.0.0.0/8
local.internal -> 2130706433 block 127.0.0.0/8
edge.internal -> 2851998213 block 169.254.0.0/16
meter.internal -> 2886993923 block 172.16.0.0/12
10.12.4.9 -> 168559625 block 10.0.0.0/8
127.0.0.1 -> 2130706433 block 127.0.0.0/8
10.12.4.7 -> 168559623, 10.0.0.0/8 base 167772160, mask 4278190080
```
With no constraint, eight of the twelve targets reach the internal network. The name-based
denylist brings this down to three and **still lets three targets through** — because it
looks at the name, not the address. One of the ones that passes carries an externally
looking name and resolves to an internal address; two are already written as numbers as
internal addresses and are not caught by the denylist because their names do not end in
`.internal`.
The address-based denylist zeroes out the target reaching the internal network. The
sub-listing shows how: each target is converted to a thirty-two-bit number, and which of the
seven blocks it falls into is computed. Eight of the eight targets fall into a block, and
none of them is found by looking at the name.
The allowlist also gives zero, but by a different route: it rejects nine of the ten
user-sourced targets. The coverage of the two constraints is the same, the form of the
coverage is not. A denylist stays open when a block that is not listed shows up; an
allowlist also stops a legitimate new external host when one is added. The cost of both
appears as zero in this table because the three declared hosts pass through both.
#### The Point Where the Constraint Is Applied
**When** the constraint is applied determines the coverage as much as which value it is
applied to. Two situations separate this: whether the request follows a redirect chain, and
whether name resolution is repeated between the check and the connection.
**IO14 (assumption):** the chains and the second resolution are a model. What is measured is
not the probability of an event happening, but **whether the constraint sees that step**.
```js
// chain.mjs — the point where the constraint is applied: first step, every step, and repeating resolution
import { TARGET, RESOLVE, resolve, isPrivate, parse } from "./address.mjs";
// Redirect chains (fictional): the sequence of steps the request follows.
const CHAIN = [
["document.ext-1.example", ["document.ext-1.example"]],
["tile.ext-2.example", ["tile.ext-2.example", "203.0.113.60"]],
["archive.ext-1.example", ["archive.ext-1.example", "archive-2.ext-1.example", "10.12.4.20"]],
["gateway.ext-3.example", ["gateway.ext-3.example"]],
["document.ext-1.example", ["document.ext-1.example", "127.0.0.1"]],
];
const steps = CHAIN.reduce((t, [, z]) => t + z.length, 0);
console.log(`${CHAIN.length} redirect chains, ${steps} steps\n`);
console.log(`${"point where the constraint is applied".padEnd(39)}${"completed".padStart(11)}${"reaching internal address".padStart(27)}`);
for (const [name, select] of [["first step only", (z) => [z[0]]], ["every step", (z) => z]]) {
const completed = CHAIN.filter(([, z]) => select(z).every((h) => !isPrivate(resolve(h))));
const internal = completed.filter(([, z]) => isPrivate(resolve(z[z.length - 1]))).length;
console.log(name.padEnd(39) + String(completed.length).padStart(11) + String(internal).padStart(27));
}
// Repeating the resolution: the address at check time and the address at connect time may differ.
const SECOND = { ...RESOLVE, "document.ext-1.example": "10.12.4.31" };
const resolveSecond = (a) => parse(SECOND[a] ?? a);
const passed = TARGET.filter(([h]) => !isPrivate(resolve(h)));
console.log(`\ntargets passing the address-based constraint: ${passed.length}`);
console.log(`${"connection form".padEnd(47)}${"reaching internal address".padStart(27)}`);
console.log("check and connect resolved separately".padEnd(47) +
String(passed.filter(([h]) => isPrivate(resolveSecond(h))).length).padStart(27));
console.log("single resolution, connect to checked address".padEnd(47) +
String(passed.filter(([h]) => isPrivate(resolve(h))).length).padStart(27));
```
```
5 redirect chains, 9 steps
point where the constraint is applied completed reaching internal address
first step only 4 2
every step 2 0
targets passing the address-based constraint: 4
connection form reaching internal address
check and connect resolved separately 1
single resolution, connect to checked address 0
```
The top table measures the location of the constraint. When the same constraint is applied
only to the first step, four chains complete and two end at an internal address; when
applied to every step, two chains complete and none reaches an internal address. The
constraint is the same constraint — the only thing that changes is how many of the nine
steps are seen. **A request's target is not a single value, it is a sequence.**
The bottom table gives the second point. One of the four targets that passes the constraint
falls into an internal address when it is re-resolved at connect time. The check had worked
correctly; the checked value is no longer the value that is connected to. The second row
shows the fix: resolution is done once, the constraint is applied to that address, and the
connection is made **not to the name, but to the checked address**. In that case the target
reaching an internal address is zero.
The shared lesson of the two rows is about the definition of coverage. A constraint covers
for as long as the value it is applied to and the value that is used are the same; the
moment the two separate, the constraint stays valid but has no coverage. This is the
question to ask when a new outbound request point is added: which value and which step does
this point apply the constraint to.
#### Outbound Request Point Inventory
Which value and which step the constraint is applied to has been measured. The remaining
question is how many points it is applied at. The regional measurement network has eight
outbound request points, and half of them go through a central outbound request client.
```js
// point.mjs — outbound request point inventory and the coverage of the central outbound request client
import { resolve, isPrivate } from "./address.mjs";
// [service, point, targets, target source, client]
const POINT = [
["report", "fetchDocument", ["document.ext-1.example", "archive.ext-1.example"], "user", "central"],
["report", "fetchArchive", ["archive.ext-1.example", "10.12.4.9"], "user", "direct"],
["field", "fetchTile", ["tile.ext-2.example"], "declared", "central"],
["field", "preview", ["gateway.ext-3.example", "local.internal"], "user", "direct"],
["subscriber portal", "fetchLogo", ["document.ext-1.example", "edge.internal"], "user", "direct"],
["invoice", "fetchRate", ["tile.ext-2.example"], "declared", "central"],
["notification", "webhook", ["203.0.113.99", "report.internal"], "user", "direct"],
["measurement", "fetchCollector", ["meter.internal"], "declared", "central"],
];
const targetCount = POINT.reduce((t, n) => t + n[2].length, 0);
// Whether a target passes in a setup: the constraint is either not applied at all or applied to the address.
const passes = (n, h, setup) => {
const constrained = setup === "all" || (setup === "exception" ? n[3] === "user" : n[4] === "central");
return constrained ? !isPrivate(resolve(h)) : true;
};
console.log(`${POINT.length} outbound request points, ${targetCount} targets, ` +
`${POINT.filter((n) => n[4] === "central").length} points go through the central client\n`);
console.log(`${"setup".padEnd(34)}${"targets passed".padStart(16)}${"reaching internal network".padStart(27)}${"legitimate internal call blocked".padStart(34)}`);
for (const [name, setup] of [["current: direct calls exist", "current"],
["all through central client", "all"], ["central + declared internal call", "exception"]]) {
let passed = 0, internal = 0, blocked = 0;
for (const n of POINT) for (const h of n[2]) {
const g = passes(n, h, setup);
if (g) { passed++; if (isPrivate(resolve(h))) internal++; }
else if (n[3] === "declared" && isPrivate(resolve(h))) blocked++;
}
console.log(name.padEnd(34) + String(passed).padStart(16) + String(internal).padStart(27) + String(blocked).padStart(34));
}
console.log(`\nwhen a new outbound request point is added: constraint to write in central client 0, ` +
`direct call ${1} constraint`);
```
```
8 outbound request points, 13 targets, 4 points go through the central client
setup targets passed reaching internal network legitimate internal call blocked
current: direct calls exist 12 5 1
all through central client 7 0 1
central + declared internal call 8 1 0
when a new outbound request point is added: constraint to write in central client 0, direct call 1 constraint
```
The first row gives the current state: five of the thirteen targets reach the internal
network, and all five come from the four direct calls that do not go through the central
client. The constraint exists in the code base and is written correctly; **it is just only
invoked at four points.**
The second row shows all points being brought into the central client. The reach into the
internal network drops to zero, but the last column stays at one: the call to the
measurement collector is a legitimate internal call, and the constraint stops it too. For a
point whose target does not come from user data, the constraint is in the wrong place.
The third row measures the fix: when the constraint is applied only to points whose target
comes from user data, the legitimate call blocked drops to zero and the reach into the
internal network rises to one — that one is the declared internal service call. **The
measure of coverage is not the number of targets reaching the internal network, but the
number of targets that come from user data and reach the internal network**; if this
distinction is not written down, the constraint either stops legitimate work or gets removed.
#### Summary
- Whether an outbound request target reaches the internal network is determined not by the
name but by thirty-two-bit address and mask arithmetic. Seven private blocks caught eight
targets.
- 8 of the twelve targets reach the internal network with no constraint. The name-based
denylist brings this to 3, and of the three it lets through, one is an externally looking
name and two are internal addresses written as numbers. The address-based denylist and the
allowlist give 0.
- The two zeros are not the same: the denylist stays open when a block that is not listed
shows up, the allowlist rejects nine of the ten targets coming from user data and also
blocks a new legitimate external host.
- When the constraint is applied only to the first step, 4 of 5 chains completed and 2 ended
at an internal address; when applied to every step, 2 chains completed and reaching an
internal address was 0. A request's target is not a single value, it is a sequence.
- 4 of the eight outbound request points go through the central client; because of the
remaining four, 5 of 13 targets reach the internal network. When all are brought into the
central client this becomes 0, but 1 legitimate internal call is blocked; when the
constraint is applied only to user-sourced targets, the legitimate call blocked is 0.
- When the name is re-resolved between the check and the connection, 1 of the 4 targets
passing the constraint fell into an internal address. When a single resolution is done and
the connection is made to the checked address, this number is 0. The constraint loses its
coverage the moment the value it is applied to and the value that is used separate.
#### Next Step
The six lessons so far drew the boundary of data coming from outside and counted the
coverage at each boundary: the entry point, the output context, the query location, the
template body, the upload layer, and the outbound request target. Their shared assumption
was that what comes in is a **value** — a string, a number, a byte sequence. In some flows,
what comes in is not a value but an object definition: the data itself writes its fields,
its type, and how it is built. The next lesson measures deserialization in terms of a
trusted type set and closes the topic: how many types are accepted, how many of them an
allowlist closes off, and how many points choosing a data format removes.
### Insecure Deserialization
Source: https://academia.sh/en/courses/secure-coding/insecure-deserialization
Measuring deserialization in terms of the trusted type set: type-tagged nodes across six messages are counted at every depth, the type and node count accepted by an explicit type registry is compared with an allowlist, an untagged data format is measured for how far it drives the structure-building point to zero and the mapping point it introduces in exchange, and how the accepted set changes when three new types are added to the codebase is derived.
The previous six lessons always treated the thing coming from outside as a **value**: a
string, a number, a byte sequence, an address. Each time, the defense was built knowing
the rule of where that value would go. In some flows, what arrives is not a value: next
to its fields, the data itself also carries its **type**, and the receiving side reads
that type and builds the corresponding structure.
This lesson counts that construction moment. The unit of measure is the **trusted type
set**: how many types are accepted, how many the allowlist closes off, and how many points
are eliminated by a data format that does not read the type from within the data.
#### Type-Tagged Format
The model keeps a registry: the types the application recognizes and the function that
builds each type. In the incoming message, the `$type` field says which registry entry
gets called. Some of the types in the registry carry only data; some open a resource or
start a job while being built.
**IO15 (assumption):** the side-effect marker is a declaration; no type in this lesson
actually runs anything. What is measured is not whether an effect occurs, but **how many
types are reachable**.
```js
// format.mjs — a type-tagged deserialization format (fictional model)
// sideEffect: a type that opens a resource or starts a job while being built. Here it
// is only marked; none of them actually run anything.
export const REGISTRY = {
Subscriber: { sideEffect: false, build: (a) => ({ type: "Subscriber", id: a.id }) },
Reading: { sideEffect: false, build: (a) => ({ type: "Reading", value: a.value }) },
Invoice: { sideEffect: false, build: (a) => ({ type: "Invoice", amount: a.amount }) },
WorkOrder: { sideEffect: false, build: (a) => ({ type: "WorkOrder", id: a.id }) },
FileSource: { sideEffect: true, build: (a) => ({ type: "FileSource", path: a.path }) },
Scheduler: { sideEffect: true, build: (a) => ({ type: "Scheduler", every: a.every }) },
ExternalCall: { sideEffect: true, build: (a) => ({ type: "ExternalCall", target: a.target }) },
Logger: { sideEffect: true, build: (a) => ({ type: "Logger", channel: a.channel }) },
Cache: { sideEffect: true, build: (a) => ({ type: "Cache", size: a.size }) },
};
export const ALLOWLIST = ["Subscriber", "Reading", "Invoice", "WorkOrder"];
// Incoming messages (fictional): the formats the endpoint really exchanges with the system.
export const MESSAGE = [
["reading upload", { $type: "Reading", value: 412 }, "expected"],
["subscriber record", { $type: "Subscriber", id: "A-100432", extra: { $type: "WorkOrder", id: 12 } }, "expected"],
["invoice summary", { $type: "Invoice", amount: 512, items: [{ $type: "Reading", value: 5 }] }, "expected"],
["history import", { $type: "Reading", value: 7, source: { $type: "FileSource", path: "d1" } }, "unexpected"],
["batch job", { $type: "WorkOrder", id: 3,
job: { $type: "Scheduler", every: 60, step: { $type: "ExternalCall", target: "h" } } }, "unexpected"],
["settings migration", { $type: "Cache", size: 100 }, "unexpected"],
];
// Collects every type tag in a message, at whatever depth it appears.
export const tags = (o) =>
o === null || typeof o !== "object" ? []
: (Array.isArray(o) ? [] : o.$type ? [o.$type] : []).concat(Object.values(o).flatMap(tags));
```
The `tags` function walks a message recursively and collects every `$type` value it
encounters, no matter how deep. A message is not a single decision: a node buried inside
another node's field carries its own type tag, and that tag is invisible unless something
walks the whole tree.
#### The Accepted Type Set
```js
// coverage.mjs — three resolution formats: explicit registry, allowlist, untagged data format
import { REGISTRY, ALLOWLIST, MESSAGE, tags } from "./format.mjs";
const FORMAT = [
["explicit registry", Object.keys(REGISTRY)],
["allowlist", ALLOWLIST],
["untagged data format", []],
];
const allTags = MESSAGE.flatMap(([, b]) => tags(b));
console.log(`${MESSAGE.length} messages, ${allTags.length} nodes carrying a type tag, ` +
`${new Set(allTags).size} distinct types\n`);
console.log(`${"resolution format".padEnd(24)}${"accepted types".padStart(14)}${"nodes built".padStart(13)}` +
`${"side-effectful builds".padStart(23)}${"accepted msgs".padStart(14)}`);
for (const [name, accepted] of FORMAT) {
let built = 0, sideEffectful = 0, acceptedMessages = 0;
for (const [, body] of MESSAGE) {
const t = tags(body);
const passes = t.every((x) => accepted.includes(x));
if (accepted.length === 0) { acceptedMessages++; continue; } // no structure is ever built
if (passes) { acceptedMessages++; built += t.length; sideEffectful += t.filter((x) => REGISTRY[x].sideEffect).length; }
}
console.log(name.padEnd(24) + String(accepted.length).padStart(14) + String(built).padStart(13) +
String(sideEffectful).padStart(23) + String(acceptedMessages).padStart(14));
}
const reachable = (k) => k.filter((x) => REGISTRY[x].sideEffect).length;
console.log(`\nthe registry has ${Object.keys(REGISTRY).length} types, ${reachable(Object.keys(REGISTRY))} of them side-effectful`);
console.log(`the allowlist accepts ${ALLOWLIST.length} types, ${reachable(ALLOWLIST)} of them side-effectful`);
console.log(`structure-building points in the untagged format: 0, manual mapping points: ` +
`${MESSAGE.filter(([, , b]) => b === "expected").length}`);
```
```
6 messages, 11 nodes carrying a type tag, 8 distinct types
resolution format accepted types nodes built side-effectful builds accepted msgs
explicit registry 9 11 4 6
allowlist 4 5 0 3
untagged data format 0 0 0 6
the registry has 9 types, 5 of them side-effectful
the allowlist accepts 4 types, 0 of them side-effectful
structure-building points in the untagged format: 0, manual mapping points: 3
```
The explicit registry accepts all six of the six messages and builds all eleven of the
eleven nodes; four are side-effectful types. The accepted type count here is not a
decision but a **consequence**: whatever is in the registry gets accepted.
The allowlist brings the accepted type count down to four, the built node count down to
five, and zeroes out side-effectful builds. Three messages are rejected, and all three are
the messages tagged as unexpected. The reason for rejection is not the message's top-level
type but **a node at depth**: the root of the batch job message is on the allowlist, its
two child nodes are not.
The third row is the actual point of the measurement. In the untagged data format, the
built node count is zero, and that is not a defense succeeding — **there is nothing to
build**. If the type decision does not come from within the data, there is no
attacker-controllable type choice either. In exchange stands the number in the last row:
the application must map the incoming plain data into its own types by hand, in three
separate places.
The three formats also live in different places in the codebase. The explicit registry is
a convenience designed among internal services, and it produces the measured count when it
stays at the outer boundary. The allowlist accepts that the type decision comes from
within the data and narrows its set. The untagged data format moves the decision out of
the data and leaves no type set to measure. The choice is not a matter of taste but of
which boundary is being held: at the outer boundary, the first format's measure is the size
of the codebase.
#### The Depth at Which the Check Is Applied
The allowlist gives not the whole of coverage but only as much as where it is applied. A
message is a tree, and a check can be applied to the root alone.
```js
// depth.mjs — the depth at which the allowlist is applied
import { REGISTRY, ALLOWLIST, MESSAGE, tags } from "./format.mjs";
const root = (b) => (b && b.$type ? [b.$type] : []);
console.log(`${"where the check is applied".padEnd(28)}${"accepted msgs".padStart(14)}` +
`${"nodes built".padStart(14)}${"side-effectful builds".padStart(23)}`);
for (const [name, sel] of [["root node only", root], ["every node", tags]]) {
let messages = 0, built = 0, sideEffectful = 0;
for (const [, b] of MESSAGE) {
if (!sel(b).every((x) => ALLOWLIST.includes(x))) continue;
messages++;
const t = tags(b);
built += t.length;
sideEffectful += t.filter((x) => REGISTRY[x].sideEffect).length;
}
console.log(name.padEnd(28) + String(messages).padStart(14) + String(built).padStart(14) +
String(sideEffectful).padStart(23));
}
const deep = MESSAGE.filter(([, b]) => tags(b).length > 1);
console.log(`\n${deep.length} of ${MESSAGE.length} messages carry a type tag on more than one node`);
console.log(`deepest message: ${tags(MESSAGE[4][1]).join(" -> ")}`);
```
```
where the check is applied accepted msgs nodes built side-effectful builds
root node only 5 10 3
every node 3 5 0
4 of 6 messages carry a type tag on more than one node
deepest message: WorkOrder -> Scheduler -> ExternalCall
```
The same allowlist, when applied only to the root node, accepts five messages and builds
ten nodes; three of those are side-effectful types. When applied to every node, the
accepted message count drops to three, the built node count to five, and side-effectful
builds to zero. **It is the same list; the only thing that changes is how many nodes are
seen.**
The last line shows why: four of the six messages carry a type tag on more than one node,
and the deepest message goes three levels down. A root-node check approves the first level
of these messages and passes the rest through unseen. It is the same shape as the redirect
chain in the previous lesson: coverage disappears once a step is placed between the value
that is checked and the value that is used.
#### When a New Type Is Added
The real test of coverage begins when the codebase changes after the measurement is taken.
Three new types are being added, and none of them is written with deserialization in mind.
**IO16 (assumption):** adding a type to the registry requires no announcement; a type
enters the registry the moment it is defined. What is measured is the effect of that
addition on the accepted set.
```js
// new-type.mjs — how the accepted set changes when a new type is added to the codebase
import { REGISTRY, ALLOWLIST } from "./format.mjs";
// Three new types enter the codebase; two of them start a job while being built.
const NEW = { ReportJob: { sideEffect: true }, Notification: { sideEffect: false }, NetworkClient: { sideEffect: true } };
const REGISTRY2 = { ...REGISTRY, ...NEW };
const sideEffectful = (registry, set) => set.filter((x) => registry[x].sideEffect).length;
console.log(`${"stage".padEnd(24)}${"types in registry".padStart(18)}${"explicit accept".padStart(18)}` +
`${"allowlist accept".padStart(19)}${"types closed off".padStart(18)}${"reachable side-effectful".padStart(25)}`);
for (const [name, r] of [["current codebase", REGISTRY], ["three new types added", REGISTRY2]]) {
const all = Object.keys(r);
console.log(name.padEnd(24) + String(all.length).padStart(18) + String(all.length).padStart(18) +
String(ALLOWLIST.length).padStart(19) + String(all.length - ALLOWLIST.length).padStart(18) +
`${sideEffectful(r, all)} / ${sideEffectful(r, ALLOWLIST)}`.padStart(25));
}
console.log(`\ncode written for the new types: 0 lines in the explicit registry (a type is accepted the moment it is defined),`);
console.log(`1 line in the allowlist (for the legitimate type), 1 mapping point in the untagged format`);
```
```
stage types in registry explicit accept allowlist accept types closed off reachable side-effectful
current codebase 9 9 4 5 5 / 0
three new types added 12 12 4 8 7 / 0
code written for the new types: 0 lines in the explicit registry (a type is accepted the moment it is defined),
1 line in the allowlist (for the legitimate type), 1 mapping point in the untagged format
```
The explicit-accept column grows together with the codebase: from nine to twelve. Reachable
side-effectful types rise from five to seven, and nobody had to write anything for that
increase — three types were defined, and the accepted set expanded on its own. **In an
explicit registry, coverage moves in the opposite direction from codebase growth.**
The allowlist column is four in both rows. Types closed off rise from five to eight: the
list did not change, yet its coverage increased, because coverage lies not in the list's
length but in **the set it leaves uncovered**. Side-effectful access is zero in both rows.
The untagged format never appears in this table, because there is no such quantity as an
accepted type set there. Its cost is in the bottom line: every new type's mapping is
written by hand somewhere. The three formats do their work in three separate places — one
does nothing and accepts everything, one keeps a list, one removes the type from the data
entirely.
Three mapping points is today's number, and it too grows. The untagged format's cost is not
a fixed one-time cost but one repeated per message format: when a fourth legitimate message
format is added, a fourth mapping is written. The difference is that when this work is
**forgotten, the message does not get resolved** — the gap is not a hole but a broken flow,
and it is caught on the first call. On the allowlist, a forgotten line does not silently
narrow or widen coverage either; coverage stays fixed unless it is added to the list.
#### Summary
- The unit of measure is the trusted type set. The type decision in a message is not
singular: six messages contain 11 tag-carrying nodes, and the rejection decision often
comes from a node at depth.
- The explicit registry accepted 9 of 9 types, built 11 nodes, and 4 were side-effectful
types. The allowlist brought the accepted type count down to 4, the built node count down
to 5, zeroed out side-effectful builds, and rejected 3 messages.
- In the untagged data format, the structure-building point is 0, and that is not a
defense succeeding but the decision being removed from the data. Its cost is 3 manual
mapping points.
- When 3 new types were added to the codebase, explicit acceptance went from 9 to 12,
reachable side-effectful types went from 5 to 7, and no line was written for it. The
allowlist's acceptance stayed at 4, and the types it closed off went from 5 to 8.
- The same allowlist, applied only to the root node, accepted 5 messages, built 10 nodes,
and had 3 side-effectful builds; applied to every node, these numbers become 3, 5, and 0.
4 of the six messages carry a tag on more than one node.
- Coverage lies not in the list's length but in the set it leaves uncovered: while the list
stays fixed, its coverage grows as the codebase grows.
#### Next Step
Across seven lessons, a single boundary was drawn: where data coming from outside enters,
where it exits, which text it joins, which template it reaches, in which bytes it is
stored, which address it goes to, and which type it builds. In each one, coverage was
counted and the uncovered point was written down. The shared assumption behind all these
measurements was that the problem lay **in the data itself**.
Even when the data is correct, a question remains. A subscriber ID that fully matches the
contract, an invoice period with a flawless format, and a type that passes the allowlist —
while all of it is valid, nobody ever asked who is making the request and whether that
person has the right to this record. The next topic, **Identity, Session and Authorization
Errors**, takes up this same question with the same measure: the information an
authentication implementation leaks, the point at which a session fails to renew, and how
many endpoints perform object-level checks.
## Identity, Session and Authorization Errors
Measuring coverage at the points where identity is established and where it is decided who can do what: the user existence leaked by response-time differences and distinct error messages, the session identifier not being renewed at login and not being invalidated at logout, the number of endpoints that perform only role checks, the paths where step order can be skipped, and how many times the same control is passed by concurrent requests.
### Authentication Implementation Errors
Source: https://academia.sh/en/courses/secure-coding/authentication-implementation-errors
Measuring the implementation error in terms of leaked information: at eight authentication points, a distinct error message, work skipped when no record exists, and a comparison that exits on the first mismatch are counted as three channels, and five points leak; the response-time difference is shown through a band check rather than by writing the raw duration; a countermeasure that closes only one channel closes just one of the five; a central validator that closes all three channels automatically covers the ninth point added; and when a fourth channel is included in the count, the same pattern's number of uncovered points rises to two.
The previous topic drew the boundary of data coming from outside and wrote the same three
columns for every point set: how many points there are, how many the pattern covers, and how
many points are left uncovered. All of those measurements rested on a single assumption — that
the identity of the person behind the request is already known. Data can conform to the rules
and still have come from the wrong person.
This topic removes that assumption. Which flow authentication is done with is not addressed
here; the types of flows and how credentials are stored are the subject of the Authentication
and Authorization course. What is measured here is the **implementation**: does the response
the server gives to someone who does not know the correct credential at all say something even
without knowing the correct information?
#### Leaked Information Can Be Counted
An authentication point knows two things separately: whether the requested record exists, and
whether the submitted credential is correct. The response should only state the second. The
first being distinguishable from the outside is called **user enumeration**: an attacker learns
which subscriber numbers are registered without knowing any password at all. This information
does not grant access by itself; it narrows the target set for every subsequent attempt.
Attempting enumeration against a service **is both illegal and unprofessional without written
authorization and a defined scope**. The inventory and measurement below run on the fictional
regional measurement network's own code; there is no real service, no real account, and no real
password.
```js
// identity/point.mjs — the subscriber portal's authentication points (model): no real
// service, no real account and no real password. No randomness, the amount of work is fixed.
import { createHash } from "node:crypto";
// IE1: a point can leak user existence through three separate channels — is the error
// message distinct, is the work done different when no record exists, does the comparison
// exit on the first mismatch. The fourth field `lock` is a channel not counted in the
// inventory; it is added to the count in the last section of the lesson.
export const POINT = [
{ name: "portal-login", message: "single", work: "fixed", compare: "fixed", lock: "distinct" },
{ name: "portal-reset", message: "distinct", work: "early-return", compare: "fixed", lock: "single" },
{ name: "portal-email-change", message: "distinct", work: "fixed", compare: "fixed", lock: "single" },
{ name: "field-app-login", message: "single", work: "fixed", compare: "fixed", lock: "single" },
{ name: "meter-reading-key", message: "single", work: "early-return", compare: "early-exit", lock: "single" },
{ name: "invoice-query", message: "distinct", work: "early-return", compare: "early-exit", lock: "single" },
{ name: "second-factor-code", message: "single", work: "fixed", compare: "fixed", lock: "distinct" },
{ name: "report-endpoint-key", message: "single", work: "fixed", compare: "early-exit", lock: "single" },
];
// IE2: the unit of work is a fixed-step hash loop; the raw duration is not written, the step
// count is.
export const SEARCH = 2000, VERIFY = 20000;
const loop = (seed, step) => {
let s = seed;
for (let i = 0; i < step; i++) s = createHash("sha256").update(s).digest("hex");
return s;
};
const RECORD = new Set(["a-4471", "a-5120", "a-6033"]);
// Flawed implementation: when no record exists, the verification work is never done, and the
// message diverges too.
export function flawed(subscriber, submitted) {
loop(subscriber, SEARCH);
if (!RECORD.has(subscriber)) return { message: "subscriber not found", step: SEARCH };
loop(submitted, VERIFY);
return { message: "incorrect password", step: SEARCH + VERIFY };
}
// Patterned implementation: the same work is done even when no record exists, the message is
// single.
export function patterned(subscriber, submitted) {
loop(subscriber, SEARCH);
const found = RECORD.has(subscriber);
loop(found ? submitted : "placeholder-comparison-seed", VERIFY);
return { message: "incorrect subscriber number or password", step: SEARCH + VERIFY };
}
```
```js
// identity/coverage.mjs — how many points there are, how many the pattern covers, how many
// points are left uncovered.
import { POINT, flawed, patterned } from "./point.mjs";
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
// IE3: a point is considered to leak user existence if at least one of the three channels is
// open.
const CHANNELS = ["message", "work", "compare"];
const channel = (n, closed = [], list = CHANNELS) => list.filter((k) =>
!closed.includes(k) && n[k] !== "single" && n[k] !== "fixed");
const A = [-24, 9, 14, 15, 10, 12];
console.log(`1. ${POINT.length} authentication points and their open channels`);
write(A, "point", "message", "no record", "compare", "channels", "leaking");
for (const n of POINT)
write(A, n.name, n.message, n.work, n.compare, channel(n).length, channel(n).length ? "YES" : "no");
const leaking = (set, closed, list) => set.filter((n) => channel(n, closed, list).length).length;
console.log(`leaking point: ${leaking(POINT, [])} / ${POINT.length}`);
// IE4: duration is not written raw. The expected ratio is derived from the step count; the
// measured value only passes through a band check and yes/no is written.
const median = (d) => d.slice().sort((a, b) => a - b)[d.length >> 1];
const measure = (f, subscriber) => {
const d = [];
for (let i = 0; i < 15; i++) {
const t = process.hrtime.bigint();
f(subscriber, "submitted-value");
d.push(Number(process.hrtime.bigint() - t));
}
return median(d);
};
const B = [-16, 11, 12, 16, 13, 9, 17];
console.log("\n2. response-time difference (the raw duration is not written; the expected ratio comes from the step count)");
write(B, "implementation", "no record", "has record", "expected ratio", "band", "in band", "existence leaks");
for (const [name, f] of [["flawed", flawed], ["patterned", patterned]]) {
const none = f("a-0000", "x").step, has = f("a-4471", "x").step;
const expected = has / none, low = expected / 2, high = expected * 2;
const measured = measure(f, "a-4471") / measure(f, "a-0000");
write(B, name, none, has, expected.toFixed(2), `${low.toFixed(2)}..${high.toFixed(2)}`,
measured >= low && measured <= high ? "yes" : "NO", expected > 1.5 ? "YES" : "no");
}
// 3. coverage: how many points a countermeasure closes on its own
const NEW_POINT = { name: "field-device-pairing", message: "distinct", work: "early-return", compare: "early-exit" };
const extended = [...POINT, NEW_POINT];
const C = [-44, 7, 11, 12];
console.log("\n3. how many points a countermeasure closes");
write(C, "countermeasure", "points", "leaking", "uncovered %");
for (const [name, k, closed] of [
["today's distributed state", POINT, []],
["constant-time comparison only", POINT, ["compare"]],
["single error message only", POINT, ["message"]],
["central validator (three channels)", POINT, ["message", "work", "compare"]],
["new point added, distributed", extended, []],
["new point added, central pattern", extended, ["message", "work", "compare"]],
]) write(C, name, k.length, leaking(k, closed), ((leaking(k, closed) / k.length) * 100).toFixed(1));
// 4. if the inventory itself is incomplete: the lock response is included in the count as a
// fourth channel
const FOUR = [...CHANNELS, "lock"];
console.log("\n4. once a fourth channel is included in the count");
write(C, "count", "points", "leaking", "uncovered %");
for (const [name, list, closed] of [
["three channels, central validator", CHANNELS, CHANNELS],
["four channels, central validator", FOUR, CHANNELS],
["four channels, lock response also uniform", FOUR, FOUR],
]) write(C, name, POINT.length, leaking(POINT, closed, list),
((leaking(POINT, closed, list) / POINT.length) * 100).toFixed(1));
```
```
1. 8 authentication points and their open channels
point message no record compare channels leaking
portal-login single fixed fixed 0 no
portal-reset distinct early-return fixed 2 YES
portal-email-change distinct fixed fixed 1 YES
field-app-login single fixed fixed 0 no
meter-reading-key single early-return early-exit 2 YES
invoice-query distinct early-return early-exit 3 YES
second-factor-code single fixed fixed 0 no
report-endpoint-key single fixed early-exit 1 YES
leaking point: 5 / 8
2. response-time difference (the raw duration is not written; the expected ratio comes from the step count)
implementation no record has record expected ratio band in band existence leaks
flawed 2000 22000 11.00 5.50..22.00 yes YES
patterned 22000 22000 1.00 0.50..2.00 yes no
3. how many points a countermeasure closes
countermeasure points leaking uncovered %
today's distributed state 8 5 62.5
constant-time comparison only 8 4 50.0
single error message only 8 4 50.0
central validator (three channels) 8 0 0.0
new point added, distributed 9 6 66.7
new point added, central pattern 9 0 0.0
4. once a fourth channel is included in the count
count points leaking uncovered %
three channels, central validator 8 0 0.0
four channels, central validator 8 2 25.0
four channels, lock response also uniform 8 0 0.0
```
The numbers belong to the **measurement** class; their inputs are IE1–IE4.
#### Five Points and Three Channels
The first table opens the inventory. **Five** of the eight points leak user existence, and how
many channels the leak comes from varies from point to point.
`portal-email-change` leaks only through the message: when no record exists it returns a
different text, while the amount of work stays the same. `report-endpoint-key` is the opposite;
its message is single and it does the same work when no record exists, but the key comparison
exits as soon as it reaches the first mismatched byte. `invoice-query` keeps all three channels
open at once.
This distribution shows why the inventory is derived point by point. The sentence "error
messages are single" is a coverage claim, and it has a counterpart in the table: five points
whose message column reads `single`. But two of those five points continue to leak through
another channel. The channel count does not say how open a point is; it says **how many
separate places need a fix**.
It is worth noting that not every point in the inventory is a login screen. `meter-reading-key`
and `report-endpoint-key` are endpoints no human sees; the credential is a key, authentication
still happens, and the same three channels are still open. If a point is counted as an
authentication point only when a "user login" is found, the inventory comes out incomplete. The
criterion is a single sentence: if a submitted value at a point is compared against a known
record, that point belongs in the inventory.
#### Duration Is Not Written, the Band Is Checked
The second table is this lesson's measurement pattern. The response-time difference is a real
leak channel, but a raw millisecond is not a measurement: it depends on the machine, the load,
and the workload at that moment, it gives two different numbers across two runs of the same
code, and whether the two numbers are comparable at all is unknown.
The measurement is set up as follows. The expected value **is derived from the input**: the
flawed implementation does 2,000 steps when no record exists and 22,000 steps when a record
exists; the expected ratio is 11.00. The band lies between half and twice that value. The
measured duration ratio is reported only by whether it falls within this band — the number
itself is not written to the output, only `yes` is. In the patterned implementation, both paths
take 22,000 steps, the expected ratio is 1.00, and the measured value falls in the 0.50–2.00
band.
The last column makes the call. If the expected ratio is close to 1, there is no information to
be drawn from the time difference; if it is 11, there is. This distinction does not change from
run to run, because it rests on the step count, not the raw duration. The measured value's only
job is to confirm that the model actually behaves this way.
**Constant-time comparison** closes only one part of this channel: the early exit inside the
comparison itself. The verification work not being done at all when no record is found comes
before that point, and constant-time comparison does not touch it. The 11.00 ratio in the
second table stays at 11.00 with constant-time comparison.
#### What a Single-Channel Countermeasure Closes
The third table counts coverage. In today's distributed state, five of the eight points leak;
the uncovered rate is **62.5%**.
If only constant-time comparison is applied, the number of leaking points drops from 5 to
**4**. The single point it closes is `report-endpoint-key`, because its only open channel was
the comparison. Applying only a single error message gives the same result: 5 to 4. The two
countermeasures close different points, but both have the same coverage. A control that
measures a leak class through only one of the three channels leaves four-fifths of the job
open.
The **central validator**, which closes all three channels together, brings the number of
leaking points to zero. The pattern's content is a single sentence: the same work is done
whether or not the record is found, the same message is returned, and the comparison is
constant-time. This pattern is not a list of rules but a single function; the points call it.
The difference is where the rule is written. In distributed defense, the rule "do not let error
messages diverge" has to be applied in eight separate places, and whether it was applied
correctly can only be known by reading all eight places one by one. In the central pattern, the
same rule is applied in one place; there is one place to read and one place to test. This is
also what the coverage count means: 8 out of 8 coverage is not eight correct implementations,
it is **one** correct implementation and eight calls.
The last two rows answer this course's third question. When a ninth point —
`field-device-pairing` — is added to the inventory, the number of leaking points in the
distributed state rises from 5 to 6 and the uncovered rate climbs to 66.7%. In the central
pattern, the same point is covered with no extra work, and the number of leaking points stays
at 0. The difference is not in the pattern's strength but in **whether the new point has to
call the defense**: in distributed defense every new point is a candidate for a hole, in the
central pattern it is only a call.
#### The Inventory Itself Can Be Incomplete
The last row of the third table reads zero, and this number is misleading. Zero does not say
"no point leaks"; it says **no point leaks through the three channels being counted**. The
coverage measure depends on the set of channels being counted.
The fourth table adds a fourth channel to the inventory: the response reporting that the
account has been locked after successive failed attempts. This response says that a lockable
account **exists**; a subscriber number that is not registered is never locked. In the
three-channel count, the central validator scored zero; in the four-channel count, the same
validator leaks **at 2 points** — an uncovered rate of 25%. The lock decision is made not in
the validator itself but in the counter layer that comes after it; because the pattern does not
call that layer, it does not touch it.
The last row shows how the pattern is extended: once the lock response is also pulled into a
uniform shape, the count resets to zero. The lesson here is not the pattern's inadequacy but
**the limit of the measure**. The coverage percentage does not state a defense's strength; it
states its position relative to the number of channels counted in the inventory. When a new
channel is discovered, every past coverage count has to be recalculated; the pattern's value is
that this recalculation can be done in a single place.
#### Summary
- An authentication implementation can leak information even without the correct credential
ever being known; the leaked information is user existence, and it is measured by point
count.
- Three channels were counted in an eight-point inventory — a distinct error message, work
skipped when no record exists, and a comparison that exits on the first mismatch — and five
points leaked through at least one channel.
- The response-time difference is written not as a raw duration but as a band check: the
expected ratio is derived from the step count, and the measured value is reported only by
whether it falls inside the band.
- Constant-time comparison closes only one of the three channels; four of the five leaking
points stay open. A single error message alone also leaves four open.
- A central validator that closes all three channels at once gives zero leaking points and
covers a ninth point added with no extra work; in the distributed state, the same point
raises the uncovered count to 6.
- The coverage count depends on the set of channels being counted: when the lock response is
added as a fourth channel, the same central validator's uncovered point count rises from 0
to 2.
#### Next Step
This lesson measured the moment identity **is established**. Once authentication succeeds, a
new object appears: the session identifier generated so identity does not have to be asked for
again on subsequent requests. When that identifier is generated, whether it changes when
authentication succeeds, and whether it is actually invalidated server-side when the user logs
out are separate decisions — and each of these decisions is made separately within a flow. The
next lesson derives the flow set: how many flows generate a session identifier, how many of
them fail to renew the old one when they do, and how many sessions continue to live on after
logout when there is no server-side invalidation.
### Session Management Errors
Source: https://academia.sh/en/courses/secure-coding/session-management-errors
Counting session decisions over a flow set: four of seven flows raise the session's authorization level, and two of those are fixation points because they do not renew the identifier; two of three invalidation flows delete no record server-side, so eight sessions continue to live on; a central session pattern brings this number down to two; when an eighth flow is added, the number of flows left uncovered in the distributed setup rises to five; and it is measured that the transfer flow, which never calls the pattern at all, is covered only by a pattern tied to the authorization version.
The previous lesson measured the moment identity is established and counted that five of eight
points leak user existence. Once authentication succeeds, a new object appears: the session
identifier generated so identity does not have to be asked for again on subsequent requests.
This identifier is generated once and afterward is changed, upgraded, or released by many
flows.
What a session is and how it is stored was established in the Authentication and Authorization
course. What is measured here is **where the decisions are made**. Every session-related
decision is made inside a flow; as the number of flows grows, the number of decisions grows,
and when each decision is written separately, some of them are written wrong. This lesson's
question is: how many flows touch the session identifier, how many touch it correctly, and how
many are left uncovered.
#### Two Rules, Seven Flows
The flows that touch the session identifier fall into two types. An **upgrade flow** raises the
session's authorization level: an anonymous session becomes logged in, a logged-in session
completes its second factor, a staff session gains administrator rights. The rule for this type
is single — when the level changes, the session identifier **is renewed**. When it is not
renewed, an identifier known before the level changed stays valid after the level changes too;
this is called **session fixation**.
An **invalidation flow**, in turn, takes authorization back: logout, a password change,
transferring a subscription to another person. The rule for this type is server-side — what has
to be deleted is the record in the ledger. Deleting the cookie on the client side does nothing
to the ledger; a client that holds on to the identifier can keep sending it.
The flow set and session ledger below are a **model**: there is no real session store, no real
cookie, and no real subscriber.
```js
// session/model.mjs — the subscriber portal's session flows and server-side session ledger
// (model): no real session store, no real cookie and no real subscriber.
// IE5: if a flow raises the session's authorization level, the session identifier must be
// renewed (expected: "renew"); if it takes authorization back, the affected sessions must be
// deleted server-side (expected: "single" for logout, "all" for the others). The `now` field
// is today's behavior.
export const FLOW = [
{ name: "portal-login", type: "upgrade", now: "renew" },
{ name: "second-factor-approval", type: "upgrade", now: "in-place" },
{ name: "role-elevation", type: "upgrade", now: "in-place" },
{ name: "field-app-pairing", type: "upgrade", now: "renew" },
{ name: "logout", type: "invalidate", now: "single" },
{ name: "password-change", type: "invalidate", now: "none" },
{ name: "subscription-transfer", type: "invalidate", now: "none" },
];
export const expected = (a) =>
a.type === "upgrade" ? "renew" : a.name === "logout" ? "single" : "all";
// IE6: the session ledger is server-side; an identifier resolves only while its record exists
// in the ledger. Deleting the cookie on the client side does not touch the ledger. When
// `versioned` is on, the record carries the subscriber's authorization version and becomes
// invalid when the version changes, without being deleted.
export function openLedger({ versioned = false } = {}) {
const records = new Map(), version = new Map();
let counter = 0;
const versionOf = (subscriber) => version.get(subscriber) ?? 1;
return {
open(subscriber, level) {
const k = `s-${++counter}`;
records.set(k, { subscriber, level, version: versionOf(subscriber) });
return k;
},
resolve(k) {
const v = records.get(k);
if (!v) return null;
return versioned && v.version !== versionOf(v.subscriber) ? null : v;
},
updateInPlace(k, level) { const v = records.get(k); if (v) v.level = level; },
remove: (k) => records.delete(k),
removeAll(subscriber) {
let n = 0;
for (const [k, v] of records) if (v.subscriber === subscriber) { records.delete(k); n++; }
return n;
},
bumpVersion(subscriber) { version.set(subscriber, versionOf(subscriber) + 1); },
live(subscriber) {
return [...records.keys()].filter((k) => this.resolve(k)?.subscriber === subscriber).length;
},
};
}
```
```js
// session/measurement.mjs — how many flows generate a session identifier, how many fail to
// renew it, and how many sessions continue to live on when there is no invalidation.
import { FLOW, expected, openLedger } from "./model.mjs";
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
// IE7: a fixation point is a flow where, after the upgrade, the old identifier still resolves
// to the upgraded level. The loop below actually runs this on the ledger.
const upgrades = FLOW.filter((a) => a.type === "upgrade");
const A = [-24, 10, 24, 20];
console.log(`1. ${upgrades.length} upgrade flows: what the old identifier resolves to after login`);
write(A, "flow", "now", "old identifier's level", "fixation point");
let fixation = 0;
for (const a of upgrades) {
const d = openLedger();
const old = d.open("a-4471", "anonymous");
if (a.now === "renew") { d.remove(old); d.open("a-4471", "logged-in"); }
else d.updateInPlace(old, "logged-in");
const level = d.resolve(old)?.level ?? "not-resolved";
if (level === "logged-in") fixation++;
write(A, a.name, a.now, level, level === "logged-in" ? "YES" : "no");
}
console.log(`session-identifier-generating flow: ${upgrades.length}, not renewing: ${fixation}`);
// IE8: every subscriber has three live sessions opened from three separate endpoints; what is
// measured is the number of sessions remaining in the ledger after an invalidation flow.
const apply = (policy, d, k) =>
policy === "single" ? d.remove(k) : policy === "all" ? d.removeAll("a-4471") : 0;
const living = (policy) => {
const d = openLedger();
const identifiers = [d.open("a-4471", "logged-in"), d.open("a-4471", "logged-in"), d.open("a-4471", "logged-in")];
apply(policy, d, identifiers[0]);
return d.live("a-4471");
};
const invalidations = FLOW.filter((a) => a.type === "invalidate");
const B = [-22, 12, 10, 12, 10];
console.log("\n2. invalidation flows (start: 3 live sessions per flow)");
write(B, "flow", "now", "living", "expected", "living");
let nowTotal = 0, patternTotal = 0;
for (const a of invalidations) {
const s = living(a.now), b = living(expected(a));
nowTotal += s; patternTotal += b;
write(B, a.name, a.now, s, expected(a), b);
}
write(B, "total", "", nowTotal, "", patternTotal);
// 3. coverage: how many flows already do what the pattern expects
const covered = (set) => set.filter((a) => a.now === expected(a)).length;
const NEW_FLOW = { name: "field-device-swap", type: "upgrade", now: "in-place" };
const extended = [...FLOW, NEW_FLOW];
const C = [-34, 6, 10, 10, 12];
console.log("\n3. flow set coverage");
write(C, "set", "flows", "covered", "uncovered", "uncovered %");
for (const [name, k, central] of [
["today's distributed state", FLOW, false],
["central session pattern", FLOW, true],
["new flow added, distributed", extended, false],
["new flow added, central pattern", extended, true],
]) {
const cov = central ? k.length : covered(k);
write(C, name, k.length, cov, k.length - cov, (((k.length - cov) / k.length) * 100).toFixed(1));
}
// 4. the flow the pattern cannot reach: `subscription-transfer` changes the subscription
// record from outside the portal and may not call the session pattern. A version-bound
// pattern needs no call.
const transfer = (versioned, notifies) => {
const d = openLedger({ versioned });
d.open("a-4471", "logged-in"); d.open("a-4471", "logged-in"); d.open("a-4471", "logged-in");
if (notifies) d.removeAll("a-4471");
d.bumpVersion("a-4471"); // the subscription record changes in every case
return d.live("a-4471");
};
const D = [-24, 16, 16, 16];
console.log("\n4. subscription-transfer flow (3 live sessions): what happens if the pattern is not called");
write(D, "setup", "notified event", "living session", "covered flow");
for (const [name, versioned, notifies] of [
["call-bound pattern", false, true],
["call-bound pattern", false, false],
["version-bound pattern", true, false],
]) write(D, name, notifies ? "yes" : "no", transfer(versioned, notifies),
`${versioned || notifies ? invalidations.length : invalidations.length - 1} / ${invalidations.length}`);
```
```
1. 4 upgrade flows: what the old identifier resolves to after login
flow now old identifier's level fixation point
portal-login renew not-resolved no
second-factor-approval in-place logged-in YES
role-elevation in-place logged-in YES
field-app-pairing renew not-resolved no
session-identifier-generating flow: 4, not renewing: 2
2. invalidation flows (start: 3 live sessions per flow)
flow now living expected living
logout single 2 single 2
password-change none 3 all 0
subscription-transfer none 3 all 0
total 8 2
3. flow set coverage
set flows covered uncovered uncovered %
today's distributed state 7 3 4 57.1
central session pattern 7 7 0 0.0
new flow added, distributed 8 3 5 62.5
new flow added, central pattern 8 8 0 0.0
4. subscription-transfer flow (3 live sessions): what happens if the pattern is not called
setup notified event living session covered flow
call-bound pattern yes 0 3 / 3
call-bound pattern no 3 2 / 3
version-bound pattern no 0 3 / 3
```
The numbers belong to the **measurement** class; their inputs are IE5–IE8.
#### Four Flows Generate, Two Do not Renew
The first table counts the upgrade side. Four flows change the session's authorization level;
two renew the identifier, two upgrade the record **in place**. In the two flows that upgrade in
place, the old identifier continues to resolve to the `logged-in` level; in the two that renew,
it is `not-resolved`.
Which two are the fixation points is instructive. `portal-login` is written correctly — this is
the flow everyone looks at, and the rule is best known there. The two written wrong are
`second-factor-approval` and `role-elevation`: both are flows that come **after** login and are
not mistaken for login itself. When the rule is memorized as "the identifier is renewed at
login," flows that raise the level after login fall outside it.
This is also where the number's meaning lies. The sentence "we are protected against session
fixation" is true about one flow and half-true about four flows. Its measurable form is this:
**session-identifier-generating flow: 4, not renewing: 2.**
It is also easy to see why in-place upgrading was written this way. Updating the record in
place does less work: no new identifier is generated, no new value is sent to the client,
requests in flight are not affected. Renewing, on the other hand, requires copying the record,
deleting the old one, and having the client receive the new value. Both fixation points having
chosen this cheap path is not carelessness — it is the rule never having been asked at that flow
at all.
#### Eight Sessions Continue to Live On
The second table counts the invalidation side. Every flow runs on a subscriber who has three
live sessions; what is measured is the number of records left in the ledger after the flow.
The `logout` flow deletes its own session, and two sessions remain. This is correct: logging
out from one endpoint does not close sessions on the other endpoints. `password-change` and
`subscription-transfer`, however, delete no record in the ledger at all; all three continue to
live on. A total of eight sessions are still alive after the three flows where authorization is
supposed to have been taken back.
Why the two flows are in this state has to do with where the work is done. The password-change
flow's visible job is updating the credential; touching the session ledger does not look like
part of that job. Subscription transfer, in turn, is a flow that belongs to the party that
manages the subscription, not the portal, and it may not even know the session ledger exists.
In both cases, what is missing is not a check — it is **a call**.
How the number is measured also matters. Every flow runs on a subscriber who has three live
sessions; the three sessions are the same person's separate endpoints. As the number of
sessions per subscriber grows, the number of sessions a non-invalidating flow leaves behind
grows directly with it. This is why the unit of the measure is not "flow" but **session living
on after the flow**: two missing flows are an item that grows as it is multiplied by the
session count.
The expected column runs the same flows with the correct policy: `single` for `logout`, `all`
for the other two. The total drops from eight to **two**, and the two remaining sessions are
the sessions at the logout flow's other endpoints — sessions that are not supposed to be
closed. The measurement shows one more thing here: the correct number is not zero. If "close
all" were correct for every flow, the user would also be dropped from the other endpoints when
logging out from one.
#### The Flow the Pattern Covers
The third table counts coverage. **Three** of the seven flows do what the pattern expects; four
are left uncovered — the uncovered rate is 57.1%.
The central session pattern is a single function and keeps two things in one place: which event
corresponds to which policy, and how that policy is applied to the ledger. Flows do not make
the decision themselves; they report the event. In this setup the number of covered flows is 7
and the uncovered count is zero — but this zero does not come from the pattern being clever, it
comes from the decision having been made **once**.
The last two rows answer the new-flow question. The eighth flow — `field-device-swap`, a field
technician replacing their terminal — raises the distributed setup's uncovered count from 4 to
5 and the rate climbs to 62.5%. In the central pattern, the same flow reports an event and the
uncovered count stays at zero. In a distributed setup, every new flow requires remembering both
rules all over again; in the pattern, only writing the event's name correctly.
#### The Flow That Does Not Call the Pattern
The fourth table exposes the assumption the third table's zero rests on. The central pattern is
a **call-bound** pattern: it runs when a flow reports the event, and does not run when it does
not. This assumption holds for six of the seven flows without issue, because six of them are
the portal's own code.
`subscription-transfer` is different. The party that changes the subscription record is not the
portal; the transfer is a separate flow that updates the subscription record directly. The
table separates two cases: when the flow reports the event, the three sessions drop to zero;
when it does not, **all three continue to live on** and the covered-flow count drops from 3 to
2. The central pattern stays right where it is, written correctly, and working; the only thing
it does not cover is the flow that does not call it.
The last row changes the pattern's shape. If the session record carries the subscriber's
**authorization version**, resolution compares the version in the record against the
subscriber's current version on every request. When the transfer flow changes the subscription
record, the version increases regardless; even though it never calls the session pattern, all
three sessions become invalid — living session: 0, covered flow: 3/3. The difference is where
the defense is attached: a call-bound pattern waits for the flow to remember; a data-bound
pattern uses a change the flow was already making as its criterion. The latter's coverage stays
constant as the flow set grows.
#### Summary
- Session decisions reduce to two rules: a flow that raises the authorization level renews the
identifier, and a flow that takes authorization back deletes the record server-side. Both are
decided separately per flow.
- Four of seven flows generate a session identifier; two are fixation points because they do
not renew the identifier, and the two written wrong are flows that come after login.
- Two of three invalidation flows delete no record in the ledger at all; eight sessions
continue to live on after all three, and the correct policy brings this number down to two.
- The correct number is not zero: the two sessions at the other endpoints living on after the
logout flow is the expected outcome; the policy is set per flow.
- Three of seven flows do what the pattern expects, four are left uncovered (57.1%); when an
eighth flow is added, the uncovered count stays at zero in the central pattern while it rises
to five in the distributed setup.
- A call-bound pattern does not cover a flow that does not call it: when the transfer flow does
not report the event, all three sessions live on. When the session record carries the
subscriber's authorization version, the same flow invalidates all three sessions without
making any call at all.
#### Next Step
These two lessons secured the identity behind the request: who it is was established without
leaking, and the carrier of that identity was renewed at the right moments and deleted at the
right moments. Knowing the identity correctly, however, is only half the decision. When a known
identity sends a request to an endpoint, there is a second question that has to be asked, and
it is asked per endpoint: can this identity touch **this object**? The next lesson derives the
authorization matrix and gives this topic's most important number: how many endpoints perform
only a role check, how many also check the ownership of the requested object, and whether the
central pattern covers a newly added endpoint.
### Broken Access Control
Source: https://academia.sh/en/courses/secure-coding/broken-access-control
Deriving, from the authorization matrix, the number of endpoints with no object-level control: five of fourteen endpoints perform only a role check, fourteen of the seventy-two requests that pass the role check reach someone else's object, a central gate built on default-deny brings this number down to zero while stopping ten legitimate requests, and a fifteenth endpoint added without declaring a criterion raises the distributed check's wrongly-passed count to seventeen while opening no new path at the central gate.
The previous two lessons secured the identity behind the request: who it is was established
without leaking, and the carrier of that identity was renewed at the right moments and deleted
at the right moments. Knowing the identity correctly is only half the decision. When a known
identity sends a request to an endpoint, a second question is asked, and it is asked per
endpoint.
The question has two separate forms, and mixing them up produces the flaw. The first is "can
this role perform this action?", the second is "can this principal touch **this object**?". An
endpoint that asks only the first opens up every object belonging to everyone in its role. This
is **broken object level authorization**: it is done with a valid session, the right role, and a
single path parameter changed. Trying this against a service **is both illegal and
unprofessional without written authorization and a defined scope**; the measurement below runs
on a matrix we wrote ourselves.
#### Matrix, Principal and Object
The measurement's input is an authorization matrix: for each endpoint, which roles can call it
and how the requested object is tied to the request. The second field is written twice — the
criterion the endpoint **is supposed to carry** and the criterion the code **actually applies**
today. Writing the two separately is what lets the gap turn into a number.
The criterion is written in three forms. `owner` means the object directly belongs to the
requester; `assigned` means a work order has been given to that staff member; `branch` means the
object is tied to the same branch as the requester. All three test a **relationship** between
the principal and the object — not an attribute the principal alone carries, the way a role is.
The administrator role is reduced, in all three criteria, to the branch boundary: the
administrator's privilege is not access to every object, but access to their own branch's
objects.
The matrix, principal set, and object set below are a **model**: there is no real service, no
real endpoint, and no real subscriber.
```js
// authorization/matrix.mjs — the subscriber portal's authorization matrix (model): no real
// service, no real endpoint and no real subscriber. Roles, objects and endpoints are written
// as data.
// IE9: every endpoint has two separate checks — the role check authorizes the action type, the
// criterion check authorizes the requested object's relationship with the requester. `correct`
// is the criterion the endpoint is supposed to carry; `reported` is the criterion the code
// actually applies today (null: no object check).
export const ENDPOINT = [
{ path: "GET /invoices/{id}", type: "invoice", role: ["subscriber", "admin"], correct: "owner", reported: "owner" },
{ path: "GET /invoices/{id}/items", type: "invoice", role: ["subscriber", "admin"], correct: "owner", reported: null },
{ path: "POST /invoices/{id}/dispute", type: "invoice", role: ["subscriber"], correct: "owner", reported: null },
{ path: "DELETE /invoices/{id}", type: "invoice", role: ["admin"], correct: "branch", reported: "branch" },
{ path: "GET /readings/{id}", type: "reading", role: ["subscriber", "staff", "admin"], correct: "owner", reported: "owner" },
{ path: "GET /readings/{id}/history", type: "reading", role: ["subscriber", "staff"], correct: "owner", reported: null },
{ path: "POST /readings/{id}/correction", type: "reading", role: ["staff", "admin"], correct: "branch", reported: "branch" },
{ path: "GET /subscribers/{id}/profile", type: "subscriber", role: ["subscriber", "admin"], correct: "owner", reported: "owner" },
{ path: "PUT /subscribers/{id}/contact", type: "subscriber", role: ["subscriber"], correct: "owner", reported: "owner" },
{ path: "POST /subscribers/{id}/plan", type: "subscriber", role: ["admin"], correct: "branch", reported: "branch" },
{ path: "GET /work-orders/{id}", type: "work-order", role: ["staff", "admin"], correct: "assigned", reported: "assigned" },
{ path: "POST /work-orders/{id}/close", type: "work-order", role: ["staff", "admin"], correct: "assigned", reported: "assigned" },
{ path: "POST /work-orders/{id}/note", type: "work-order", role: ["staff"], correct: "assigned", reported: null },
{ path: "GET /reports/{id}", type: "report", role: ["admin"], correct: "branch", reported: null },
];
// IE10: the principal and object sets are fictional; every object has a branch, and some have
// an owner or an assignee.
export const PRINCIPAL = [
{ code: "a-4471", role: "subscriber", branch: "S-1" },
{ code: "a-5120", role: "subscriber", branch: "S-2" },
{ code: "g-31", role: "staff", branch: "S-1" },
{ code: "g-47", role: "staff", branch: "S-2" },
{ code: "y-08", role: "admin", branch: "S-1" },
];
export const OBJECT = [
{ code: "f-1", type: "invoice", owner: "a-4471", branch: "S-1" },
{ code: "f-2", type: "invoice", owner: "a-5120", branch: "S-2" },
{ code: "o-1", type: "reading", owner: "a-4471", branch: "S-1" },
{ code: "o-2", type: "reading", owner: "a-5120", branch: "S-2" },
{ code: "b-1", type: "subscriber", owner: "a-4471", branch: "S-1" },
{ code: "b-2", type: "subscriber", owner: "a-5120", branch: "S-2" },
{ code: "i-1", type: "work-order", assigned: "g-31", branch: "S-1" },
{ code: "i-2", type: "work-order", assigned: "g-47", branch: "S-2" },
{ code: "r-1", type: "report", branch: "S-1" },
{ code: "r-2", type: "report", branch: "S-2" },
];
// IE11: the criterion tests the relationship between the principal and the object. An admin
// reaches their own branch's objects; ownership and assignment are direct code equality.
export const criterion = (name, principal, object) =>
name === "branch" ? object.branch === principal.branch
: principal.role === "admin" ? object.branch === principal.branch
: name === "owner" ? object.owner === principal.code
: object.assigned === principal.code;
export const roleAllows = (endpoint, principal) => endpoint.role.includes(principal.role);
```
```js
// authorization/coverage.mjs — how many endpoints perform an object-level check, how many
// requests from an unchecked endpoint reach someone else's object, how many the central gate
// closes, and how many legitimate requests it stops.
import { ENDPOINT, PRINCIPAL, OBJECT, criterion, roleAllows } from "./matrix.mjs";
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
// IE12: the request set is every (principal, endpoint, object) triple that passes the role
// check. Three setups run on the same set: the correct decision, distributed checking, and the
// central gate (default-deny when there is no criterion).
const requests = (set) => {
const l = [];
for (const endpoint of set) for (const p of PRINCIPAL) for (const o of OBJECT)
if (o.type === endpoint.type && roleAllows(endpoint, p)) l.push({ endpoint, p, o });
return l;
};
const decide = (r, setup) => {
if (setup === "correct") return criterion(r.endpoint.correct, r.p, r.o);
if (r.endpoint.reported) return criterion(r.endpoint.reported, r.p, r.o);
return setup === "distributed"; // default-deny at the central gate
};
const count = (set, setup, condition) => requests(set).filter((r) => condition(
decide(r, setup), decide(r, "correct"))).length;
const A = [-30, 6, 9, 12, 16];
console.log(`1. ${ENDPOINT.length} endpoints: which ones perform object-level control`);
write(A, "endpoint", "roles", "correct", "reported", "wrongly passed");
for (const endpoint of ENDPOINT) {
const y = requests([endpoint]).filter((r) => decide(r, "distributed") && !decide(r, "correct")).length;
write(A, endpoint.path, endpoint.role.length, endpoint.correct, endpoint.reported ?? "-", y || "-");
}
const noCriterion = ENDPOINT.filter((e) => !e.reported);
console.log(`endpoints with no object-level control: ${noCriterion.length} / ${ENDPOINT.length}`);
const B = [-40, 9, 9];
console.log("\n2. the set of requests that pass the role check");
write(B, "measure", "requests", "share %");
const T = requests(ENDPOINT).length;
const row = (name, n) => write(B, name, n, ((n / T) * 100).toFixed(1));
row("requests that pass the role check", T);
row("should pass (correct decision)", count(ENDPOINT, "correct", (k) => k));
row("passing under distributed checking", count(ENDPOINT, "distributed", (k) => k));
row("distributed: wrongly passed request", count(ENDPOINT, "distributed", (k, d) => k && !d));
row("central gate: wrongly passed", count(ENDPOINT, "central", (k, d) => k && !d));
row("central gate: legitimate request stopped", count(ENDPOINT, "central", (k, d) => !k && d));
// 3. when a new endpoint is added: an endpoint added without declaring a criterion
const NEW_ENDPOINT = { path: "GET /invoice/{id}/payment-plan", type: "invoice", role: ["subscriber", "admin"],
correct: "owner", reported: null };
const extended = [...ENDPOINT, NEW_ENDPOINT];
const C = [-31, 9, 14, 16, 20];
console.log("\n3. when a new endpoint is added without declaring a criterion");
write(C, "setup", "endpoints", "no criterion", "wrongly passed", "legitimate stopped");
for (const [name, set, setup] of [
["today's distributed check", ENDPOINT, "distributed"],
["central gate", ENDPOINT, "central"],
["new endpoint, distributed check", extended, "distributed"],
["new endpoint, central gate", extended, "central"],
]) write(C, name, set.length, set.filter((e) => !e.reported).length,
count(set, setup, (k, d) => k && !d), count(set, setup, (k, d) => !k && d));
```
```
1. 14 endpoints: which ones perform object-level control
endpoint roles correct reported wrongly passed
GET /invoices/{id} 2 owner owner -
GET /invoices/{id}/items 2 owner - 3
POST /invoices/{id}/dispute 1 owner - 2
DELETE /invoices/{id} 1 branch branch -
GET /readings/{id} 3 owner owner -
GET /readings/{id}/history 2 owner - 6
POST /readings/{id}/correction 2 branch branch -
GET /subscribers/{id}/profile 2 owner owner -
PUT /subscribers/{id}/contact 1 owner owner -
POST /subscribers/{id}/plan 1 branch branch -
GET /work-orders/{id} 2 assigned assigned -
POST /work-orders/{id}/close 2 assigned assigned -
POST /work-orders/{id}/note 1 assigned - 2
GET /reports/{id} 1 branch - 1
endpoints with no object-level control: 5 / 14
2. the set of requests that pass the role check
measure requests share %
requests that pass the role check 72 100.0
should pass (correct decision) 32 44.4
passing under distributed checking 46 63.9
distributed: wrongly passed request 14 19.4
central gate: wrongly passed 0 0.0
central gate: legitimate request stopped 10 13.9
3. when a new endpoint is added without declaring a criterion
setup endpoints no criterion wrongly passed legitimate stopped
today's distributed check 14 5 14 0
central gate 14 5 0 10
new endpoint, distributed check 15 6 17 0
new endpoint, central gate 15 6 0 13
```
The numbers belong to the **measurement** class; their inputs are IE9–IE12.
#### Five Endpoints, Fourteen Requests
The first table gives the inventory: **five** of fourteen endpoints do not perform object-level
control. These are the endpoints carrying a dash in the `reported` column, and next to each one
is written how many requests wrongly pass.
Four of the five endpoints are a **sub-resource** or a **sub-action**: invoice items, reading
history, work-order note, and invoice dispute. The criterion written for the parent resource is
assumed to already apply to the sub-path, and the question is never asked there again. The
fifth, `GET /reports/{id}`, is a single-role endpoint; the single role was mistaken for the
whole of the authorization check.
The wrongly-passed request counts are not equal across endpoints. `GET /readings/{id}/history`
leaks six requests, `GET /reports/{id}` only one. The difference comes from the role count:
reading history is open to two roles, and those two roles have four principals in total, while
the report endpoint is open to one role and one principal. The risk an endpoint carries is the
product of its missing criterion and **the width of its role set**. The sentence "five endpoints
are missing" does not show this difference; the request count does.
The ranking also gives a fix order. If all five endpoints cannot be fixed the same day, the
endpoint leaking six requests comes before the one leaking two; fixing two endpoints closes 64%
of the leak. It is worth noting that this ranking cannot be seen from a code review: all five
endpoints look the same in the code, missing the same single line. What produces the difference
is not the code itself, but the intersection of the endpoint's role set with the object set.
#### Fourteen of Seventy-Two Requests
The second table opens up the request set. The number of requests that pass the role check is
**72**; **32** of those are requests that should pass (44.4%). The gap between them shows what
the role check alone leaves behind: one out of every two requests that pass the role check
should be rejected under the correct decision.
The number of requests passing under distributed checking is 46. This number is 14 more than
32, and those 14 requests are the ones that **reach someone else's object** — 19.4% of the
request set. All of them can be made with a valid session, the right role, and a single path
parameter changed.
What these 14 requests have in common is that they produce no error on the server at all. The
response is 200, the record genuinely exists, the query runs correctly. The only measurable
trace is the request set itself: the same principal has gone to the same endpoint, one after
another, with identifiers that do not belong to it. The audit log can see this pattern but
cannot make the decision; the criterion is what has to make the decision.
Being able to generate the request set this way is the measurement's real gain. The set was not
written by hand; it was **derived** from the principal, endpoint, and object lists. The same
derivation is also used as a test set: for every endpoint, a "call with a principal that is not
the owner and expect a rejection" request is generated on its own, and the number of tests grows
as the number of endpoints grows. Because the criterion is data, it can both be audited and be
tested; if it were buried inside an `if` line, neither would be possible.
#### The Central Gate and Its Cost
The central gate combines two things: the criterion is applied in a single place, and an
endpoint that does not report a criterion **is denied by default**. The second table's last two
rows give this setup's two columns. Wrongly passed requests are **0**; legitimate requests
stopped are **10** — 13.9% of the request set.
These ten requests are a cost, and it has to be written down. Default-deny closes all five
endpoints that are missing a criterion entirely; legitimate use of those endpoints stops too. A
subscriber requesting invoice items for their own invoice also gets a 403. But the difference
between the two kinds of error lies in observability: a stopped legitimate request produces an
error log **on day one** and gets fixed; a wrongly-passed request produces no log at all and may
go unnoticed for years. Default-deny turns a silent flaw into a noisy one.
The third table measures how this holds up over time. A fifteenth endpoint added without
declaring a criterion — `GET /invoice/{id}/payment-plan` — raises the distributed check's
wrongly-passed count from 14 to **17**. At the central gate, the same endpoint leaves the
wrongly-passed count at 0 and raises the stopped legitimate requests from 10 to 13. Coverage
stays constant; what changes is the visible cost.
The difference is not in the pattern's power, it is in **which way the default points**. In
distributed checking, forgetting to write the criterion results in an open endpoint; at the
central gate, it results in a non-working endpoint. Both are flaws, but one sits silently in
production while the other reports immediately. The central gate's measurable contribution is
not the coverage percentage — it is that **it changes what forgetting results in**.
#### Summary
- The authorization question has two forms: the role authorizes the action type, and the
criterion authorizes the requested object's relationship with the requester. When only the
first is asked, everyone in the role set reaches every object.
- Five of fourteen endpoints do not perform object-level control; four are a sub-resource or a
sub-action, one is a single-role endpoint.
- 32 of the 72 requests that pass the role check should pass; 46 requests pass under
distributed checking, and the 14 in between (19.4%) reach someone else's object.
- The risk an endpoint carries is the product of its missing criterion and the width of its
role set: the same gap leaks 6 requests at one endpoint and 1 at another.
- A central gate built on default-deny brings wrongly passed requests down to 0 and stops 10
legitimate requests; when a fifteenth endpoint is added, wrongly passed stays at 0, while it
rises to 17 under distributed checking.
- The central gate's measurable contribution is that it turns the result of forgetting the
criterion from a silent gap into an observable error.
#### Next Step
This lesson asked the authorization question over a single request: can this principal touch
this object, at this endpoint? All three numbers rest on the single-request assumption. Yet a
system's rules live not only in single requests but also in **sequences of requests**. A
discount is applied once, a refund is taken once, a quota is consumed once per period. None of
these rules are written at a single endpoint; they are written in the order of steps. The next
lesson models flows as a sequence of steps and asks: how many paths exist where steps can be
skipped or repeated, how many of them does a state machine close, and how many are left
uncovered.
### Business Logic Flaws
Source: https://academia.sh/en/courses/secure-coding/business-logic-flaws
Modeling the business logic flaw as a flow of steps: in a five-step dispute-and-refund flow, five hundred and one of the six hundred and twenty-five four-step paths yield a rule-breaking result under per-step authorization, three hundred sixty-nine make a payment, and one hundred thirteen exceed the quota; a state machine brings these numbers down to zero and the paying path down to one, and when a sixth step with an undeclared transition is added, the rule-breaking path count under per-step setup rises to one thousand one hundred seventy-two while the state machine stays at zero, and as the sequence length grows the share of rule-breaking paths rises from sixty percent to ninety percent.
The previous lesson asked the authorization question over a single request and gave three
numbers: how many endpoints perform object-level control, how many requests wrongly pass, how
many the central gate closes. That whole measurement rested on the single-request assumption. A
request being correct on its own, though, does not mean a rule has been preserved.
Part of a system's rules live not in single requests but in **sequences of requests**. A refund
is paid once, a discount is applied once per period, an approval comes before a payment. None of
these sentences can be written inside a single endpoint; they are all about the **order** and
**repetition** of steps. A **business logic flaw** is born where these rules are not checked:
each step is authorized one by one, while the sequence itself is not.
#### The Rule Lives in the Order of Steps
For the measurement, a flow is reduced to a set of steps. Two properties are written for each
step: its effect (payment, discount) and which prior states it is allowed to follow in the state
machine. If the second field is left empty, the step cannot run in any state — this is the flow
equivalent of the previous lesson's default-deny.
Two setups are compared. **Per-step authorization** authorizes each step on its own and never
asks about order; this is the behavior of a service that performs authorization per endpoint.
The **state machine** tests the current state before every step and does not run a step that is
not permitted.
The flow below is a **model**: there is no real payment, no real amount, and no real currency;
the amounts are fictional units.
```js
// flow/model.mjs — the subscriber portal's invoice dispute and refund flow (model): no real
// payment, no real amount and no real currency. The amounts are fictional units.
// IE13: every step carries two things — its effect (payment, discount) and which prior states
// it can follow in the state machine. If `allowed` is empty, the step cannot run in any state.
export const EFFECT = {
"apply": { allowed: ["none"], next: "applied" },
"review": { allowed: ["applied"], next: "reviewed" },
"approve": { allowed: ["reviewed"], next: "approved" },
"refund-pay": { allowed: ["approved"], next: "closed", payment: 100 },
"apply-discount": { allowed: ["applied", "reviewed"], next: null, discount: 1 },
};
export const STEP = Object.keys(EFFECT);
export const QUOTA = 1; // discount quota: once per period
// IE14: two setups run the same sequence of steps. In the "per-step" setup, every step carries
// its own authorization and order is never asked; in the "state-machine" setup, a step that is
// not permitted is skipped.
export function run(sequence, setup, effect = EFFECT) {
const d = { state: "none", payment: 0, discount: 0 };
for (const step of sequence) {
const e = effect[step];
if (setup === "state-machine") {
if (!e.allowed.includes(d.state)) continue;
if (e.discount && d.discount >= QUOTA) continue;
}
d.payment += e.payment ?? 0;
d.discount += e.discount ?? 0;
if (e.next) d.state = e.next;
}
return d;
}
// IE15: the path set is every fixed-length sequence generated from the given step set.
export function paths(steps, length) {
let l = [[]];
for (let i = 0; i < length; i++) l = l.flatMap((d) => steps.map((a) => [...d, a]));
return l;
}
```
```js
// flow/measurement.mjs — how many paths exist where steps can be skipped or repeated, how many
// the state machine closes, and how many are left uncovered.
import { EFFECT, STEP, QUOTA, run, paths } from "./model.mjs";
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const CODE = { b: "apply", i: "review", o: "approve", d: "refund-pay", n: "apply-discount" };
const A = [-10, 10, 13, 12, 13];
console.log("1. four-step example sequences (b=apply i=review o=approve d=refund-pay n=discount)");
write(A, "sequence", "PS payment", "PS discount", "SM payment", "SM discount");
for (const short of ["biod", "bdio", "bidd", "bddd", "bnnn"]) {
const sequence = [...short].map((c) => CODE[c]);
const a = run(sequence, "per-step"), m = run(sequence, "state-machine");
write(A, [...short].join(" "), a.payment, a.discount, m.payment, m.discount);
}
// IE16: a path is rule-breaking if the per-step setup gives a more generous result than the
// state machine. The count below actually runs every path.
const LENGTH = 4;
const measure = (steps, effect, length = LENGTH) => {
const path = paths(steps, length);
const s = { path: path.length, paying: [0, 0], over: [0, 0], quota: [0, 0],
outside: [0, 0], same: 0 };
for (const d of path) {
const k = [run(d, "per-step", effect), run(d, "state-machine", effect)];
for (let i = 0; i < 2; i++) {
if (k[i].payment > 0) s.paying[i]++;
if (k[i].payment > 100) s.over[i]++;
if (k[i].discount > QUOTA) s.quota[i]++;
if (k[i].payment > k[1].payment || k[i].discount > k[1].discount) s.outside[i]++;
}
if (k[0].payment === k[1].payment && k[0].discount === k[1].discount) s.same++;
}
return s;
};
const t = measure(STEP, EFFECT);
const B = [-34, 16, 16];
console.log(`\n2. all length-${LENGTH} sequences (${STEP.length} steps, ${t.path} paths)`);
write(B, "measure", "per-step", "state-machine");
for (const [name, field] of [["paying path", "paying"], ["paying more than 100 units", "over"],
["exceeding the discount quota", "quota"], ["yielding a rule-breaking result", "outside"]])
write(B, name, t[field][0], t[field][1]);
console.log(`paths where the two setups give the same result: ${t.same} / ${t.path}` +
` (${((t.same / t.path) * 100).toFixed(1)}%)`);
// 3. an undeclared new step: because its transition is not defined, the state machine does
// not run it
const EXTENDED = { ...EFFECT, "extra-payment": { allowed: [], next: null, payment: 100 } };
const g = measure(Object.keys(EXTENDED), EXTENDED);
const C = [-28, 7, 8, 14, 15];
console.log("\n3. when a new step without a declared transition (extra-payment) is added");
write(C, "setup", "steps", "paths", "paying", "rule-breaking");
for (const [name, s, i, n] of [["per-step", t, 0, 5], ["state-machine", t, 1, 5],
["per-step, new step", g, 0, 6], ["state-machine, new step", g, 1, 6]])
write(C, name, n, s.path, s.paying[i], s.outside[i]);
// 4. as the sequence length grows, the path set and the rule-breaking path count
const D = [-9, 9, 18, 20, 18];
console.log(`\n4. as the sequence length grows (${STEP.length} steps)`);
write(D, "length", "paths", "PS rule-breaking", "PS rule-breaking %", "SM rule-breaking");
for (let u = 2; u <= 6; u++) {
const s = measure(STEP, EFFECT, u);
write(D, u, s.path, s.outside[0], ((s.outside[0] / s.path) * 100).toFixed(1), s.outside[1]);
}
```
```
1. four-step example sequences (b=apply i=review o=approve d=refund-pay n=discount)
sequence PS payment PS discount SM payment SM discount
b i o d 100 0 100 0
b d i o 100 0 0 0
b i d d 200 0 0 0
b d d d 300 0 0 0
b n n n 0 3 0 1
2. all length-4 sequences (5 steps, 625 paths)
measure per-step state-machine
paying path 369 1
paying more than 100 units 113 0
exceeding the discount quota 113 0
yielding a rule-breaking result 501 0
paths where the two setups give the same result: 124 / 625 (19.8%)
3. when a new step without a declared transition (extra-payment) is added
setup steps paths paying rule-breaking
per-step 5 625 369 501
state-machine 5 625 1 0
per-step, new step 6 1296 1040 1172
state-machine, new step 6 1296 1 0
4. as the sequence length grows (5 steps)
length paths PS rule-breaking PS rule-breaking % SM rule-breaking
2 25 15 60.0 0
3 125 90 72.0 0
4 625 501 80.2 0
5 3125 2686 86.0 0
6 15625 14075 90.1 0
```
The numbers belong to the **measurement** class; their inputs are IE13–IE16.
#### Five Sequences, Three Kinds of Flaw
The first table runs five example sequences through both setups. The first row is the expected
flow: apply, review, approve, pay. Both setups pay 100 units.
The second row is **step skipping**: the payment comes before the approval. Under per-step
authorization, 100 units are paid, because the payment step itself is authorized and whether the
steps before it were completed is never asked. Under the state machine, the same sequence pays
0.
The third and fourth rows are **step repetition**: the same payment step passes two and three
times, giving results of 200 and 300 units. In both sequences, the payment step is authorized on
its own each time; what is missing is the knowledge that the step can run **only once**.
The fifth row is a **quota overrun**. The discount step is applied three times, and under
per-step authorization the discount rises to 3. Under the state machine it stays at 1, because
the quota is also state information: once the discount has been applied, a second application is
not permitted.
What the three kinds of flaw have in common is that none of them breaks the authorization check.
Every step was called with the right role, on the right object, with the right criterion. What
is broken is the flow itself.
This also shows why the previous lesson's measurement is not enough. The authorization matrix
found a missing criterion in five of fourteen endpoints; here, even if every criterion is in
place, 501 paths still yield a rule-breaking result. The two measurements do not substitute for
one another: one checks **who can touch which object**, the other checks **in which order** they
can touch it. Saying "authorization check complete" for an endpoint says nothing about that
endpoint being called in the middle of a sequence.
#### Five Hundred and One of Six Hundred and Twenty-Five Paths
The second table counts the entire path set. The number of four-step sequences generated from
five steps is **625**, and every one of them is actually run through both setups.
Generating the path set at a fixed length is a deliberate choice. A real client can call
whatever step it wants, as many times as it wants; the four-step window is a countable slice of
that freedom. When the slice is enlarged, the result does not change, only the numbers grow — the
fourth table shows this. The measurement's validity does not rest on the window's width, but on
the same window being used in both setups.
Under per-step authorization, **369 paths** make a payment; under the state machine, **1**. The
gap between them is 368 paths, and all of them are payment without approval. **113 paths** pay
more than 100 units — these are the sequences where the payment step passes two or three times.
The same number comes up for the discount quota too: 113 paths exceed the quota.
The total is in the last row: **501 paths** give a more generous result than the state machine —
80.2% of the path set. Only 124 paths give the same result under both setups, that is, 19.8%.
This ratio says that in a flow that does not check order, the paths that "behave correctly" are
a minority: the rule-breaking result is not the exception, it is the majority.
This number being so large is not surprising despite the smallness of the step set. The path
count is the step count raised to the power of the length; five steps and length four give 625,
six steps and length four give 1,296. Every time a new step is added to the flow, the number of
sequences that need checking grows exponentially. That is why the rule cannot be written per
sequence; it has to be written **per state**.
The fourth table measures this growth. As the sequence length goes from 2 to 6, the path count
goes from 25 to 15,625, and the rule-breaking path count under the per-step setup goes from 15
to 14,075. The real information is not in the last column but in the ratio column: the share of
rule-breaking paths rises from 60.0% to **90.1%**. As the sequence lengthens, the share of
correctly-behaving paths falls, because the correct order is unique while the number of wrong
orders multiplies with length. The state-machine column is zero in all five rows — the
transition table is independent of sequence length.
This also says why protecting order through testing is not enough. A test set that tests all
four-step sequences contains 625 cases and rises to 3,125 at five steps; a five-row transition
table, however, does not change. What keeps the rule-breaking path count at zero is not the
number of tests, but where the decision sits.
#### What the State Machine Closes, and What It Costs
The state machine closes 501 of 625 paths and brings the rule-breaking path count to zero. The
closing mechanism is not a checklist: every step's permitted prior states are written down, and
every transition not on that list is rejected. A five-row table closes 501 rule-breaking paths
at once.
The third table answers the new-step question. When a sixth step with an undeclared transition —
`extra-payment` — is added, under the per-step setup the path count rises to 1,296, the paying
path count to 1,040, and the rule-breaking path count to **1,172**. Under the state machine, the
same step opens no path at all: the paying path count stays at 1, the rule-breaking count at 0.
Because the `allowed` list is empty, the step never runs after any state.
The cost is here too, and it has to be written down. An undeclared step **never runs at all**;
the new feature stays dead until it is added to the transition table. The same cost shows up in
legitimate flexibility too: the discount step is only permitted in the `applied` and `reviewed`
states, so a legitimate piece of work that wants to apply a discount after approval is not
accepted. Satisfying that request means adding a row to the table, and that row is a deliberate
decision — one that can be seen in code review, tested, and reverted.
The size of the cost is also in the table. Under the state machine, the paying path count is 1;
under per-step authorization, it is 369. All 368 of those paths have been closed, and the user is
left with exactly one order to follow to complete the flow. If a flow has legitimate flexibility
— a document being addable before review too, a discount being applicable at two different
points — that flexibility has to show up as a row in the transition table. Flexibility that is
not in the table is not a product decision, it is a closed path.
One detail matters in implementation. In the model above, a step that is not permitted is
**silently skipped**; in a real service, a rejection has to be returned instead of skipping it.
Silent skipping leads the caller to believe the step ran, and that puts a consistency bug in
place of the flaw that was closed. The state machine's value is in making the decision;
reporting that decision is a separate responsibility.
#### Summary
- A business logic flaw is born not in a single request's authorization, but in the order and
repetition of the step sequence; each step can be authorized individually while the sequence
itself is not.
- Three kinds of flaw were measured: step skipping (payment without approval), step repetition
(paying 200 and 300 units), and quota overrun (the discount applied three times).
- Of the 625 four-step paths generated from five steps, 369 make a payment, 113 pay more than
100 units, 113 exceed the quota, and 501 (80.2%) yield a rule-breaking result.
- The state machine brings the rule-breaking path count to zero and the paying path count to
one; the closing mechanism is a five-row transition table, not a check written per sequence.
- When a sixth step with an undeclared transition is added, the rule-breaking path count under
the per-step setup rises from 501 to 1,172 while the state machine stays at 0; as the
sequence length goes from 2 to 6, the share of rule-breaking paths rises from 60.0% to 90.1%
while the state-machine column stays at zero.
- A step that is not permitted is not silently skipped, it is rejected; silent skipping puts a
consistency bug in place of the flaw.
#### Next Step
This lesson's entire measurement rests on a single assumption: the steps run **one after
another**. When the state machine reaches the payment step, it reads that the state is
`approved`, makes its decision, and sets the state to `closed`. It is assumed that nothing else
happens in the time between the read and the write. This assumption falls apart when two
requests arrive at the same time: both can read the state as `approved`, both can pass the
check, and both can pay. The next lesson counts how many times the same check passes in a
concurrent-request model, and measures how many of them a lock and a single-writer queue close.
### Race Conditions
Source: https://academia.sh/en/courses/secure-coding/race-conditions
Measuring how many times the same check passes under concurrent requests: for a single-use quota, sixteen concurrent requests pass sixteen times under the unprotected setup, a lock and a single-writer queue bring this down to the quota, and a retry-less atomic compare-and-update, with a quota of three, lets only one request through and rejects two legitimate requests; four of seven checkpoints are unprotected and produce twenty-seven excess passes under eight concurrent requests, a single-writer queue brings this to zero, and when an eighth checkpoint is added the excess passes rise to thirty-four under the distributed setup.
The previous lesson's entire measurement rested on a single assumption: the steps run one
after another. When the state machine reached the payment step, it read the state, made its
decision, and updated the state. It was assumed that nothing else happened in the time between
the read and the write.
This assumption falls apart when two requests arrive at the same time. Both can read the state
as `approved`, both can pass the check, and both can write. The rule is written in the code, the
check runs, and the result is still wrong. This is a **race condition**: the flaw is not in the
check's existence, but in the gap between the check and the effect.
#### Stepping Between the Three Steps
Every request that changes a shared value is three steps: read, check, write. Another request
stepping in between this trio means the second request reads the value the first has not yet
written. Both requests decide by looking at the old value, and both decisions come out
positive.
The measurement below is a **model**: there is no real server, no real client, and no real
network. The timing is not real; requests are interleaved step by step and run in a scheduler we
wrote ourselves, which is why the output is the same on every run. Attempting to bypass a rule
by sending concurrent requests to a service **is both illegal and unprofessional without
written authorization and a defined scope**; what is measured here is the behavior of our own
model.
```js
// race/model.mjs — concurrent-request model (model): no real server, no real client and no
// real network. The timing is not real, it is produced by step-by-step interleaving; that is
// why the output is the same on every run.
// IE17: every request is three steps — read the shared state, decide, write. The race
// condition is born when another request steps in between these three.
export function* request(name, state, setup) {
const local = state.remaining; // 1. read
yield "read";
const passed = local > 0; // 2. check
yield "check";
if (!passed) return { name, result: "rejected" };
if (setup === "queue") return { name, result: "queued" };
if (setup === "atomic" && state.remaining !== local) return { name, result: "collision" };
state.remaining = local - 1; // 3. write
return { name, result: "passed" };
}
// IE18: four setups run the same set of requests. "unprotected" interleaves every request
// step by step; "lock" serializes the read-check-write trio; "atomic" retests the value right
// before writing; "queue" leaves the write to a single-writer queue, and the queue rechecks.
export function run(concurrent, setup, quota = 1) {
const state = { remaining: quota }, results = [];
const generators = Array.from({ length: concurrent },
(_, i) => request(`r-${i + 1}`, state, setup));
if (setup === "lock") {
for (const g of generators) { let r; while (!(r = g.next()).done); results.push(r.value); }
} else {
const open = generators.map((g) => ({ g, finished: false }));
while (open.some((a) => !a.finished))
for (const a of open) {
if (a.finished) continue;
const r = a.g.next();
if (r.done) { a.finished = true; results.push(r.value); }
}
}
if (setup === "queue")
for (const s of results)
if (s.result === "queued")
s.result = state.remaining > 0 ? (state.remaining--, "passed") : "rejected";
const passed = results.filter((s) => s.result === "passed").length;
return { passed, excess: Math.max(0, passed - quota), remaining: state.remaining };
}
// IE19: a checkpoint is any place that reads, checks, and writes a shared value.
export const CHECKPOINT = [
{ name: "use-discount-code", quota: 1, setup: "unprotected" },
{ name: "refund-pay", quota: 1, setup: "unprotected" },
{ name: "decrement-quota", quota: 3, setup: "lock" },
{ name: "approve-subscriber-transfer", quota: 1, setup: "lock" },
{ name: "assign-work-order", quota: 1, setup: "unprotected" },
{ name: "lock-invoice", quota: 1, setup: "atomic" },
{ name: "record-meter-reading", quota: 2, setup: "unprotected" },
];
```
```js
// race/measurement.mjs — how many times the same check passes, and how many of them a lock
// and a single-writer queue close.
import { run, CHECKPOINT } from "./model.mjs";
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const SETUP = ["unprotected", "lock", "atomic", "queue"];
const A = [-11, 13, 8, 9, 9];
console.log("1. a single-use discount code (quota 1): how many requests pass the check");
write(A, "concurrent", ...SETUP);
for (const n of [1, 2, 4, 8, 16]) write(A, n, ...SETUP.map((k) => run(n, k).passed));
const B = [-11, 13, 8, 9, 9];
console.log("\n2. the same measurement with quota 3 (at most 3 requests should pass)");
write(B, "concurrent", ...SETUP);
for (const n of [1, 2, 4, 8, 16]) write(B, n, ...SETUP.map((k) => run(n, k, 3).passed));
// IE20: an excess-passing request is the number of requests that pass above the quota; this
// is what is measured.
const CONCURRENT = 8;
const C = [-30, 7, 13, 9, 9, 11];
console.log(`\n3. ${CHECKPOINT.length} checkpoints, ${CONCURRENT} concurrent requests at each`);
write(C, "checkpoint", "quota", "setup", "passed", "excess", "queue");
let excessTotal = 0, queueTotal = 0;
for (const n of CHECKPOINT) {
const s = run(CONCURRENT, n.setup, n.quota), q = run(CONCURRENT, "queue", n.quota);
excessTotal += s.excess; queueTotal += q.excess;
write(C, n.name, n.quota, n.setup, s.passed, s.excess, q.excess);
}
write(C, "total", "", "", "", excessTotal, queueTotal);
// 4. coverage: when a new checkpoint is added
const NEW = { name: "create-payment-plan", quota: 1, setup: "unprotected" };
const extended = [...CHECKPOINT, NEW];
const excess = (set, central) => set.reduce((t, n) =>
t + run(CONCURRENT, central ? "queue" : n.setup, n.quota).excess, 0);
const D = [-42, 13, 12, 14, 13];
console.log("\n4. coverage: unprotected checkpoints and excess-passing requests");
write(D, "set", "checkpoints", "unprotected", "excess passes", "uncovered %");
for (const [name, k, central] of [
["today's distributed state", CHECKPOINT, false],
["single-writer queue at every checkpoint", CHECKPOINT, true],
["new checkpoint added, distributed", extended, false],
["new checkpoint added, central pattern", extended, true],
]) {
const open = central ? 0 : k.filter((n) => n.setup === "unprotected").length;
write(D, name, k.length, open, excess(k, central), ((open / k.length) * 100).toFixed(1));
}
```
```
1. a single-use discount code (quota 1): how many requests pass the check
concurrent unprotected lock atomic queue
1 1 1 1 1
2 2 1 1 1
4 4 1 1 1
8 8 1 1 1
16 16 1 1 1
2. the same measurement with quota 3 (at most 3 requests should pass)
concurrent unprotected lock atomic queue
1 1 1 1 1
2 2 2 1 2
4 4 3 1 3
8 8 3 1 3
16 16 3 1 3
3. 7 checkpoints, 8 concurrent requests at each
checkpoint quota setup passed excess queue
use-discount-code 1 unprotected 8 7 0
refund-pay 1 unprotected 8 7 0
decrement-quota 3 lock 3 0 0
approve-subscriber-transfer 1 lock 1 0 0
assign-work-order 1 unprotected 8 7 0
lock-invoice 1 atomic 1 0 0
record-meter-reading 2 unprotected 8 6 0
total 27 0
4. coverage: unprotected checkpoints and excess-passing requests
set checkpoints unprotected excess passes uncovered %
today's distributed state 7 4 27 57.1
single-writer queue at every checkpoint 7 0 0 0.0
new checkpoint added, distributed 8 5 34 62.5
new checkpoint added, central pattern 8 0 0 0.0
```
The numbers belong to the **measurement** class; their inputs are IE17–IE20.
#### The Check Passes as Often as There Is Concurrency
The first table measures a single-use discount code: quota 1, meaning at most one request
should pass. Under the unprotected setup, the number of passing requests equals the number of
concurrent requests — 2 requests pass 2 times, 16 requests pass 16 times. The check genuinely
ran on every request and gave a positive result on every request, because all of them read
before writing.
This does not mean the check is **ineffective**; it means it does not scale. A test run with a
single request can never see this flaw: in the first row, all four setups give 1. The flaw only
becomes visible once concurrency is a variable.
The lock and the single-writer queue both give 1 at every concurrency level. The two reach the
same result by two different routes: the lock serializes the read-check-write trio, the queue
leaves reading free and gathers only the writing into a single writer, rechecking at the moment
of writing. The queue's distinguishing feature is that slow reads do not hold a lock.
The lock has a cost too, and that cost is invisible in this measurement. Because the lock
serializes the critical section, it increases the number of waiting requests as concurrency
grows; if a read inside the critical section is slow, every request waits for that read in turn.
The `passed` column in the table does not measure this waiting time, it only measures
correctness. The two columns have to be kept separate: on the correctness column the lock and
the queue give the same result, on the latency column they do not.
#### The Cost of Defense: Collision
The second table repeats the same measurement with quota 3 and exposes the cost of a defense.
The unprotected setup again lets through as many as there is concurrency; the lock and the queue
stop at 3. The atomic compare-and-update, however, gives **1**.
The reason is the implementation in the model: if the value has changed by the time of writing,
the request is counted as a collision and rejected, and it is **not retried**. With a quota of
3, three requests should pass, but only one does; two legitimate requests are lost. The
compare-and-update blocks excess passes, but when used without a retry it also stops part of the
legitimate work.
This is an example of this curriculum's rule: a defense whose cost is not written down is not
counted as measured. The correct form of the compare-and-update is to retry a colliding request
up to a set limit; the limit is itself a decision, and it determines how many requests will be
dropped under load.
The difference between the first table's quota-1 result and the second table's quota-3 result is
itself a warning. At quota 1, the atomic setup gives the correct result and the flaw is
invisible; the flaw only surfaces once the quota rises above 1. A defense behaving correctly in
a single configuration does not mean it behaves correctly in every configuration. The
measurement set has to include at least two quota values.
#### Twenty-Seven Excess Passes at Seven Checkpoints
The third table carries the measurement from a single piece of code to the codebase. There are
seven checkpoints; each of them reads, checks, and writes a shared value. Four are unprotected,
two are locked, one is atomic.
Eight concurrent requests run at each checkpoint, and the **excess-passing** request count is
recorded: requests that pass above the quota. The four unprotected checkpoints produce 7, 7, 7,
and 6 excess passes respectively; the total is **27**. The three protected checkpoints produce
0.
The difference between the numbers comes from the quota. Because the `record-meter-reading`
checkpoint has a quota of 2, six of the eight requests are excess; at checkpoints with a quota
of 1, it is seven. The excess-pass count is therefore measured not by the number of checkpoints,
but by **the gap between the quota and the concurrency**. The sentence "four of our checkpoints
are unprotected" does not give you the number 27.
The last column shows what the same checkpoints give when run through a single-writer queue: 0
at all seven of the seven checkpoints. The closing happens in a single place; a separate lock
was not designed for each checkpoint.
Which three checkpoints are protected is also meaningful. `decrement-quota` and
`approve-subscriber-transfer` are locked, because both are checkpoints that change money or
subscription state and were thought through carefully. The unprotected four are checkpoints
that decrement a counter, mark a record, or make an assignment; none of them look "critical."
The place where race conditions are found is not the checkpoint deemed critical, but the
checkpoint that **reads and writes a shared value**. The inventory's criterion should be exactly
this: not criticality, but shared writing.
#### Coverage and the Eighth Checkpoint
The fourth table records coverage. In today's distributed state, four of the seven checkpoints
are unprotected — the uncovered share is **57.1%** — and the total excess passes are 27. When
every write goes through the single-writer queue, the unprotected checkpoint count is 0 and the
excess passes are 0.
If an eighth checkpoint — `create-payment-plan` — is added unprotected under the distributed
setup, the excess passes rise from 27 to **34**, and the uncovered share rises to 62.5%. Under the
central pattern, the same checkpoint produces no excess passes at all; its write path already
goes through the queue.
The difference here, too, is not in the strength of the pattern, but in the write being done
through **a single path**. Under the distributed setup, the question "who else is writing this
value" has to be asked again for every new checkpoint; with the queue, the question has been
asked once. Concurrency is the hardest class of flaw to see in code review: the code is correct
when read on its own, and is only wrong when two copies of it run at the same time.
This is why the measurement itself is a form of testing. Because the scheduler above does not
use real timing, its result is the same on every run; the same interleaving can be applied to
every checkpoint, and the excess-pass count can be recorded as a number. A test left to real
timing, on the other hand, sometimes passes and sometimes fails, and passing proves nothing. The
existence of a race condition is counted as demonstrated if it can be shown in an interleaving we
wrote ourselves; if it cannot, what is known is not its absence, but that it did not show up in
that interleaving.
#### Summary
- A race condition is born not in the absence of a check, but in the gap between the check and
the effect; a second request that steps in between the read-check-write trio reads the old
value.
- For a single-use quota, the unprotected setup produces as many passes as there is
concurrency: 16 concurrent requests pass 16 times. A test run with a single request can never
see this flaw.
- The lock and the single-writer queue protect the quota at every level; the queue's
distinguishing feature is that reads do not hold a lock.
- A retry-less atomic compare-and-update, with a quota of 3, lets only 1 request through; while
it closes excess passes, it also rejects two legitimate requests. A defense whose cost is not
written down has not been measured.
- Four of seven checkpoints are unprotected; at eight concurrent requests they produce a total
of 27 excess passes. The excess-pass count is measured not by the number of checkpoints, but
by the gap between the quota and the concurrency.
- The single-writer queue brings excess passes down to 0; when an eighth unprotected
checkpoint is added, excess passes under the distributed setup rise to 34 while the central
pattern stays at 0.
#### Next Step
This topic measured a single question across five lessons: at how many points in the code is
who-can-do-what checked, and how many points are left uncovered. At authentication points,
leaked information was counted; in session flows, identity that is not renewed and sessions that
keep living were counted; in the authorization matrix, endpoints without object-level control
and wrongly-passing requests were counted; in flow order, rule-breaking paths were counted; and
in this lesson, the excess passes produced by concurrent requests were counted. The same shape
came out in all five measurements: the distributed defense left points uncovered in every set,
the central pattern gave zero on the same set, and every time a new item was added to a set,
what the distributed side left uncovered grew, while the pattern's did not change.
One thing all these checks rest on, though, was never addressed. Authentication compared a
value, the session identifier was generated from somewhere, the authorization criterion trusted
a signature or a record, the queue operated with an identity. How these values are generated,
how predictable they are, and where in the codebase they sit was never asked — **keys and
secrets** still sit somewhere inside the code. The next topic, **Secrets in Code**,
takes up this question with the same coverage measure: how many secrets there are, how many are
outside the codebase, and how many are left behind in the repository history.
## Secrets in Code
The coverage measure for the key, generator, and dependency items the code relies on: how many of the cryptography call points pass through the central layer, the concrete consequence of reusing the same starting value and of having no integrity tag, how much a weak generator gives away in advance once the seed is known, how many commits a secret stays exposed in a fictional repository's history and what history cleanup does not rescue, the portion of the leaked items in error messages and log lines that the central masking layer closes versus leaves exposed, and the transitive depth in the dependency graph along with the number of packages a single component affects.
### Cryptographic Misuse
Source: https://academia.sh/en/courses/secure-coding/cryptographic-misuse
The coverage of cryptography call points and the concrete consequence of misuse: how seven of fourteen call points stay uncovered by the central layer, how reusing the same key and the same starting value makes forty positions readable, how the format check in encryption without an integrity tag misses four of twenty-four changes, and how a new call point is covered automatically once the primitive call is confined to a single file.
The previous lesson audited who could do what in code: it counted the endpoints in the
authorization matrix and measured the case where a concurrent request passed the same check
twice. All of those checks rested on one thing. A session carried a signature, a key had
generated the field application's upload token, the local store had been locked with an
encryption call. This topic takes up those items, and the first question is not where they are
stored but **how they are called**.
The Identity, Access and Cryptography course measured the properties of algorithms. Here those
properties are the input; what is measured is the **call point**. How many points in a codebase
call cryptography, how many of them pass through a central layer, and what the wrong setting at
the uncovered points concretely costs.
#### Cryptography Call Points
**SC1 — the fictional regional measurement network's codebase has fourteen cryptography call
points.** Some of them call the four functions of a central crypto layer; the rest make the
primitive call directly and choose settings such as mode, starting value, and salt on their own.
The inventory below is a model; the source names are fictional.
```js
// crypto/coverage.mjs — inventory of the cryptography call points in the fictional
// measurement network's codebase (model). source: the component making the call; layer:
// whether it goes through the central crypto layer; flaw: the wrong setting chosen in a
// direct call.
const points = [
['portal/session', 'encryption', false, 'no-tag mode'],
['portal/registration', 'password hash', true, null],
['portal/invoice-link', 'signature', true, null],
['ingestion/unpacking', 'encryption', true, null],
['ingestion/backup-path', 'encryption', false, 'fixed starting value'],
['ingestion/sync', 'hash', false, 'custom hash chain'],
['field/upload-token', 'signature', false, null],
['field/local-cache', 'encryption', false, null],
['report/export', 'encryption', true, null],
['report/filename', 'hash', true, null],
['report/archive', 'encryption', false, 'no-tag mode'],
['work-order/signature', 'signature', true, null],
['work-order/legacy-endpoint', 'password hash', false, 'unsalted hash'],
['notification/callback', 'signature', true, null],
];
const countWhere = (set, predicate) => set.filter(predicate).length;
const flawTally = (set) => {
const g = {};
for (const [, , , k] of set.filter(([, , layer]) => !layer)) if (k) g[k] = (g[k] ?? 0) + 1;
return Object.entries(g).sort((a, b) => b[1] - a[1]);
};
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(''));
const D = [-26, 10, 10, 12, 12];
write(D, 'operation type', 'points', 'in layer', 'uncovered', 'flawed');
for (const type of ['encryption', 'signature', 'hash', 'password hash']) {
const t = points.filter(([, i]) => i === type);
write(D, type, t.length, countWhere(t, ([, , k]) => k), countWhere(t, ([, , k]) => !k),
countWhere(t, ([, , , k]) => k));
}
write(D, 'total', points.length, countWhere(points, ([, , k]) => k),
countWhere(points, ([, , k]) => !k), countWhere(points, ([, , , k]) => k));
console.log('\ndefect distribution of the uncovered points:');
for (const [name, n] of flawTally(points)) console.log(` ${name.padEnd(24)}${n}`);
// What happens when a new point is added: three new call points, two separate arrangements.
const added = [
['field/offline-queue', 'encryption'],
['portal/export', 'signature'],
['ingestion/measurement-archive', 'encryption'],
];
console.log('\nthree new call points are being added:');
const E = [-40, 8, 11, 19];
write(E, 'arrangement', 'points', 'in layer', 'new gap candidates');
// 1) Primitive call free: a new point is not obliged to use the layer.
write(E, 'layer exists, direct calls allowed', points.length + added.length,
countWhere(points, ([, , k]) => k), added.length);
// 2) Primitive call gathered into one file: a point stepping outside the layer does not compile.
write(E, 'primitive calls confined to one file', points.length + added.length,
countWhere(points, ([, , k]) => k) + added.length, 0);
console.log(`\nin the second arrangement, points that must be closed first: `
+ `${countWhere(points, ([, , k]) => !k)}`);
```
```
operation type points in layer uncovered flawed
encryption 6 2 4 3
signature 4 3 1 0
hash 2 1 1 1
password hash 2 1 1 1
total 14 7 7 5
defect distribution of the uncovered points:
no-tag mode 2
fixed starting value 1
custom hash chain 1
unsalted hash 1
three new call points are being added:
arrangement points in layer new gap candidates
layer exists, direct calls allowed 17 7 3
primitive calls confined to one file 17 10 0
in the second arrangement, points that must be closed first: 7
```
The total row gives this lesson's three columns: there are fourteen points, seven of them are
covered by the central layer, and seven are left uncovered. The fourth column says that being left
uncovered does not immediately mean flawed — five of the seven points are flawed, two call it with
the correct setting. Those two points are correct today because the person who wrote them chose
the setting knowingly; no structure guarantees they will still be correct tomorrow. The measure
of coverage is **not the number of points that are currently flawed, but the number of points
that could become flawed**, and that number is seven.
The defect distribution shows four distinct classes, and two of them come from the same root:
choosing a mode without an integrity tag and leaving the starting value fixed. The third is
writing a custom hash chain, the fourth is hashing a password without a salt. The two
measurements below count the concrete counterpart first of the third class, then of the first
two.
#### The Measure of a Custom Mix
The ingestion service uses a hand-written mixing function to distinguish repeated meter readings:
the code of each character is summed and a thirty-two-bit value is produced. The function works,
it passes the tests, and it never throws an error.
```js
// crypto/chain.mjs — measuring the "custom hash chain" flaw from the inventory: the
// ingestion service uses a hand-written mixing function to distinguish repeated readings
// (model).
import { createHash } from 'node:crypto';
const RECORDS = 20000;
const keys = Array.from({ length: RECORDS }, (_, i) =>
`MTR-${4000 + (i % 2500)}|2026-03-${String(1 + (i % 28)).padStart(2, '0')}T06:00Z|${41000 + i}`);
// Hand-written mix: sum of character codes, 32-bit output.
const customMix = (s) => {
let h = 0;
for (let i = 0; i < s.length; i++) h = (h + s.charCodeAt(i)) | 0;
return h >>> 0;
};
const builtinHash = (s) => createHash('sha256').update(s).digest().readUInt32BE(0);
const measure = (f) => {
const seen = new Set();
let dropped = 0;
for (const a of keys) {
const k = f(a);
if (seen.has(k)) dropped += 1; else seen.add(k);
}
return [seen.size, dropped];
};
const expected = RECORDS - 2 ** 32 * (1 - (1 - 1 / 2 ** 32) ** RECORDS);
console.log(`${RECORDS} fictional reading keys, all distinct, output width 32 bits:`);
console.log(` theoretical expected silent drop at 32 bits: ${expected.toFixed(2)}`);
for (const [name, f] of [['builtin hash', builtinHash], ['hand-written mix', customMix]]) {
const [distinct, dropped] = measure(f);
console.log(` ${name.padEnd(22)} distinct value ${String(distinct).padStart(5)}, `
+ `silent drops ${dropped}`);
}
```
```
20000 fictional reading keys, all distinct, output width 32 bits:
theoretical expected silent drop at 32 bits: 0.05
builtin hash distinct value 20000, silent drops 0
hand-written mix distinct value 67, silent drops 19933
```
At a thirty-two-bit output width, a collision among twenty thousand distinct keys is
theoretically expected at 0.05; the first thirty-two bits of the builtin hash hit this
expectation and produce twenty thousand distinct values. The hand-written mix produces only
**sixty-seven** distinct values in the same set, and nineteen thousand nine hundred thirty-three
readings are silently dropped as "already seen." The reason is plain: the summation ignores both
the order of the characters and the length of the record, so the thirty-two-bit width of the
output space is never actually used. This is the concrete consequence of writing your own
algorithm — what you write runs, no one sees an error, and there is no property that was never
considered.
#### Same Key, Same Starting Value
**SC2 — in an encryption that works in stream mode, if the same key and the same starting value
are used twice, the byte difference between the two ciphertexts equals the byte difference
between the two plaintexts.** This is not an algorithm flaw; it is a violation of the
construction's condition of use. The measurement is done on our own hand-written model: the
stream generator below is not a real cipher, and it stands in for an approach that writes its own
cipher in production code.
```js
// crypto/model.mjs — comparing our own simple stream model with built-in authenticated
// encryption at the same scale. The model is not a real cipher; it exists only to count
// the information that leaks when the starting value is reused and when there is no
// integrity tag. Two meter readings from the fictional regional measurement network are used.
import { createHash, createCipheriv, createDecipheriv } from 'node:crypto';
const KEY = createHash('sha256').update('fictional-field-key').digest();
const READING_A = Buffer.from('SAY-4417|2026-03-01T06:00Z|41982|sube-01');
const READING_B = Buffer.from('SAY-4418|2026-03-01T06:00Z|41982|sube-02');
const FORMAT = /^SAY-\d{4}\|\d{4}-\d{2}-\d{2}T\d{2}:\d{2}Z\|\d+\|sube-\d{2}$/;
// Our own stream generator: derives a byte sequence from the key and the starting value.
function stream(sv, length) {
const chunks = [];
for (let block = 0; chunks.length * 32 < length; block++) {
chunks.push(createHash('sha256').update(KEY).update(sv).update(String(block)).digest());
}
return Buffer.concat(chunks).subarray(0, length);
}
const modelEncrypt = (plain, sv) => Buffer.from(plain.map((b, i) => b ^ stream(sv, plain.length)[i]));
const modelDecrypt = (cipher, sv) => Buffer.from(cipher.map((b, i) => b ^ stream(sv, cipher.length)[i]));
// Built-in construction: authenticated encryption (AES-256-GCM); the tag travels with the packet.
const aeEncrypt = (plain, sv) => {
const s = createCipheriv('aes-256-gcm', KEY, sv);
return { body: Buffer.concat([s.update(plain), s.final()]), tag: s.getAuthTag() };
};
const aeDecrypt = ({ body, tag }, sv) => {
const c = createDecipheriv('aes-256-gcm', KEY, sv);
c.setAuthTag(tag);
return Buffer.concat([c.update(body), c.final()]);
};
const SV1 = Buffer.alloc(12, 7), SV2 = Buffer.alloc(12, 9);
const equalCount = (x, y) => [...x].filter((b, i) => b === y[i]).length;
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(''));
// 1) Same key, same starting value: comparing two ciphertexts.
console.log(`SC2 — same key, two readings, length ${READING_A.length} bytes,`
+ ` equal positions in plaintext ${equalCount(READING_A, READING_B)}:`);
const D = [-32, 14, 14, 14];
write(D, 'arrangement', 'equal cipher', 'diff readable', 'leaked bytes');
for (const [name, c1, c2] of [
['model, same starting value', modelEncrypt(READING_A, SV1), modelEncrypt(READING_B, SV1)],
['model, separate starting value', modelEncrypt(READING_A, SV1), modelEncrypt(READING_B, SV2)],
['tagged, separate starting value', aeEncrypt(READING_A, SV1).body, aeEncrypt(READING_B, SV2).body],
]) {
const readable = [...c1].every((b, i) => (b ^ c2[i]) === (READING_A[i] ^ READING_B[i]));
write(D, name, equalCount(c1, c2), readable ? 'yes' : 'no', readable ? READING_A.length : 0);
}
// 2) Encryption without an integrity tag: who catches the tampered packet.
const TRIALS = 24;
console.log(`\nSC3 — a single bit is flipped in each of ${TRIALS} packets:`);
const E = [-32, 14, 14, 14];
write(E, 'arrangement', 'format passed', 'caught', 'changed bytes');
let passed = 0, changed = 0, aePassed = 0;
for (let k = 0; k < TRIALS; k++) {
const sv = Buffer.alloc(12, k + 16);
const flipped = Buffer.from(modelEncrypt(READING_A, sv));
flipped[k] ^= 1 << (k % 8);
const decoded = modelDecrypt(flipped, sv);
changed += [...decoded].filter((b, i) => b !== READING_A[i]).length;
if (FORMAT.test(decoded.toString('latin1'))) passed += 1;
const packet = aeEncrypt(READING_A, sv);
packet.body[k] ^= 1 << (k % 8);
try { aeDecrypt(packet, sv); aePassed += 1; } catch { /* tag rejected it */ }
}
write(E, 'model + format check', passed, TRIALS - passed, changed);
write(E, 'authenticated encryption', aePassed, TRIALS - aePassed, 0);
```
```
SC2 — same key, two readings, length 40 bytes, equal positions in plaintext 38:
arrangement equal cipher diff readable leaked bytes
model, same starting value 38 yes 40
model, separate starting value 0 no 0
tagged, separate starting value 0 no 0
SC3 — a single bit is flipped in each of 24 packets:
arrangement format passed caught changed bytes
model + format check 4 20 24
authenticated encryption 0 24 0
```
The first row of the first table gives the leak as a number. The two meter readings are
forty-byte records and are equal to each other at thirty-eight positions in plaintext — both
carry the same date, the same reading value, and the same format, and differ only in the meter
number and the branch code. When encrypted with the same starting value, the ciphertexts also
come out equal at **exactly thirty-eight positions**. Anyone looking at the ciphertext reads
where the two records are the same and where they differ, without ever knowing the key.
The "diff readable" column says something sharper: across all forty positions, the byte
difference between the two ciphertexts equals the byte difference between the two plaintexts.
Encryption hides only the record's raw form, not its **value**; the entire relationship between
the two records passes through to the ciphertext unchanged. In a set of readings, this means it
is possible to count which records are identical and when a branch's readings stopped. The second
and third rows give zero on the same measure: when the starting value is separate for each
encryption, the equal-position count is zero and there is no readable difference. The flaw is not
in the encryption; it is in leaving a single parameter fixed.
#### Encryption That Does Not Notice Tampering
**SC3 — encryption without an integrity tag has no item that will refuse to decrypt; every
ciphertext decodes into some plaintext.** The second table gives this sentence's coverage
counterpart. A single bit was flipped in each of twenty-four packets; each flip changed **exactly
one byte** in the decoded plaintext — twenty-four bytes total, one per packet. In stream mode, a
change at one position of the ciphertext lands at the same position of the plaintext; it neither
spreads nor gets noticed.
At this point, the only defense is the receiving side's format check, and the table counts its
coverage: it catches twenty of the twenty-four changes and **misses four**. The four it misses
are the ones that land on positions that still match the plaintext's format pattern — the digits
of the meter number or of the reading value. The result is a packet that still looks valid but
carries a different value; on the billing side, the counterpart of this is a silently wrong
consumption figure. A defense can cover most of the points and still, exactly the points it does
not cover can be the ones that matter.
The last row is the other end of the same measure: with authenticated encryption, twenty-four of
the twenty-four changes were rejected and the format check had no work left to do. The difference
here is not a difference in strength; it is a **difference in coverage** — the integrity check
covers the whole packet, the format check covers only the fields that fall into the pattern.
#### When a New Point Is Added
The third table asks the course's third question. Three new call points are added to the
codebase: the field application's offline queue, the portal's export flow, the ingestion
service's measurement archive. In the first arrangement, the central layer exists, but the
primitive call is free. The point count rises from fourteen to seventeen, the number of points in
the layer **stays at seven**, and the three new points become direct exposure candidates. Because
the layer binds no one, a new point is not obliged to use it; coverage does not stay fixed, it
drops with every addition.
In the second arrangement, the primitive calls are gathered into a single file, and nowhere
outside that file can call the encryption, signature, or hash primitive. The three new points
must pass through the layer; the number of points in the layer rises to ten, and the new exposure
candidate count is zero. What keeps coverage fixed is not that the layer exists, but that **it is
impossible to step outside the layer**. The cost is written on the last line: before moving to
this arrangement, all seven of the seven uncovered points must be migrated, because the
single-file rule only works when it has no exception. A debt of seven points replaces a decision
that would otherwise be repeated at each of the three new points.
#### Summary
- The fictional measurement network's codebase has fourteen cryptography call points; seven pass
through the central layer, seven are left uncovered, and five of the uncovered ones call it with a
flawed setting. The measure of coverage is not the number of points that are flawed, but the
number of points that could become flawed.
- The hand-written summation mix produced sixty-seven distinct values across twenty thousand
distinct keys, and 19,933 readings dropped silently; the first thirty-two bits of the builtin
hash matched the theoretical expectation and gave zero drops.
- When the same key and the same starting value are used, the ciphertexts of two forty-byte
records come out equal at thirty-eight positions, and the difference is directly readable
across all forty positions; with a separate starting value, these numbers are zero.
- In encryption without an integrity tag, flipping a single bit changes exactly one byte in the
plaintext; the format check catches twenty of the twenty-four changes and misses four.
Authenticated encryption rejects all twenty-four of the same twenty-four.
- While the primitive call is free, three new points add three exposure candidates and coverage
drops; once the primitive call is confined to a single file, coverage stays fixed, and the cost
is migrating seven points first.
#### Next Step
Every measurement in this lesson took one thing as given: that the source generating the key and
the starting value is unpredictable. Yet that source is also a call point in the codebase, and it
does not appear as a separate row in the fourteen-point inventory. The calls that generate the
field application's upload token, the portal's password reset link, and the invoice filename
share the same question: can a third party precompute the generated value? The next lesson
answers this question by running two generators at the same scale — how many values come out
precomputed once the seed is known, and how many of the token-generating points call the weak
generator.
### Randomness
Source: https://academia.sh/en/courses/secure-coding/randomness
A same-scale comparison of the weak generator and the cryptographically secure generator: when the seed is known, all ten thousand values can be precomputed; out of ten thousand four-digit codes, only 903 come out distinct; mapping a byte to a digit by remainder produces a 1.04 bias; and six of the nine token-generation points call the weak generator.
Every measurement in the previous lesson took one thing as given: that the source producing the
key and the starting value is unpredictable. That source is itself a call point in the codebase,
and it does not appear as a separate line in the fourteen-point cryptography inventory. The field
application's upload token, the portal's password-reset link, and the four-digit verification
code all share the same question: can a third party precompute the value that gets generated?
The question is not one of quality. A generator is either deterministic or it is not, and there
is no middle ground. The measurements below run two generators on the same draw set: one is a
hand-written linear congruential generator, the other is the cryptographically secure generator
inside `node:crypto`.
#### Two Generators, One Measure
**SC4 — the weak generator is deterministic; its state advances from the seed by a single
arithmetic step, so a party that knows the seed can precompute every value it will produce.**
The cryptographically secure generator has no such seed item to guess; its output is fed from
the operating system's entropy pool.
**SC5 — the number of distinct values in a draw set gives the generator's real output width**;
if the number falls below the theoretical expectation, the generator is not using its nominal
width. **SC6 — mapping a random byte to a range by remainder does not give a uniform
distribution if the range width does not evenly divide the number of values a byte can take.**
```js
// random/generator.mjs — compares the weak generator with the cryptographically secure
// generator on the same scale. The weak generator is our own hand-written linear congruential
// generator; the seed is visible. Because the cryptographic generator's output changes from run
// to run, the values read from it are not printed directly; they go through a band check derived
// from the input.
import { randomBytes, randomInt } from 'node:crypto';
const SEED = 20260301;
const DRAW = 10000;
const makeGenerator = (seed) => {
let state = seed;
return () => (state = (state * 1103515245 + 12345) % 2 ** 31);
};
// 1) How many values can be precomputed once the seed is known.
const a = makeGenerator(SEED), b = makeGenerator(SEED);
let same = 0;
for (let i = 0; i < DRAW; i++) if (a() === b()) same += 1;
const g1 = randomBytes(4).readUInt32BE(0), g2 = randomBytes(4).readUInt32BE(0);
console.log(`SC4 — precomputed value when seed ${SEED} is known: ${same}/${DRAW}`);
console.log(` the cryptographic generator has no same-seed item; are two calls equal: `
+ `${g1 === g2 ? 'yes' : 'no'}`);
console.log(` candidate seeds per day for a generator seeded from a second-resolution clock: ${24 * 3600}`);
// 2) Four-digit verification code: weak generator's low digits vs. the secure generator.
const code = makeGenerator(SEED);
const weak = new Set();
for (let i = 0; i < DRAW; i++) weak.add(code() % 10000);
const secure = new Set();
for (let i = 0; i < DRAW; i++) secure.add(randomInt(0, 10000));
const expected = Math.round(10000 * (1 - (1 - 1 / 10000) ** DRAW));
console.log(`\nSC5 — ${DRAW} four-digit codes, theoretical expected distinct value: ${expected}`);
console.log(` weak generator, distinct code : ${weak.size}`);
console.log(` secure generator, distinct code in band: `
+ `${Math.abs(secure.size - expected) < 120 ? 'yes' : 'no'} (${expected} +/- 120)`);
// 3) Bias of deriving a digit from a single byte: exact count first, then the run.
const ways = Array.from({ length: 10 }, (_, r) =>
Array.from({ length: 256 }, (_, x) => x).filter((x) => x % 10 === r).length);
const RATIO = ways[0] / ways[9];
const DRAW2 = 1000000;
const biased = Array(10).fill(0), rejectionSampled = Array(10).fill(0);
let rejected = 0;
for (const x of randomBytes(DRAW2)) {
biased[x % 10] += 1; // direct modulo: 0-5 come out too high
if (x < 250) rejectionSampled[x % 10] += 1; else rejected += 1; // rejection sampling
}
const ratioOf = (s) => (s.slice(0, 6).reduce((p, q) => p + q, 0) / 6)
/ (s.slice(6).reduce((p, q) => p + q, 0) / 4);
const inBand = (o, target) => (Math.abs(o - target) < 0.008 ? 'yes' : 'no');
const expectedRejected = (DRAW2 * 6) / 256;
console.log(`\nSC6 — remainder of 256 byte values divided by 10: 0-5 has ${ways[0]} ways, `
+ `6-9 has ${ways[9]} ways, ratio ${RATIO.toFixed(4)}`);
const label = (name) => ` ${name.padEnd(46)}: `;
console.log(label(`${DRAW2} bytes, direct modulo, ratio in band`)
+ `${inBand(ratioOf(biased), RATIO)} (${RATIO.toFixed(4)} +/- 0.008)`);
console.log(label('same bytes, rejection sampling, ratio in band')
+ `${inBand(ratioOf(rejectionSampled), 1)} (1.0000 +/- 0.008)`);
console.log(label('rejected byte band')
+ `${Math.abs(rejected - expectedRejected) < 600 ? 'yes' : 'no'} (${expectedRejected} +/- 600)`);
```
```
SC4 — precomputed value when seed 20260301 is known: 10000/10000
the cryptographic generator has no same-seed item; are two calls equal: no
candidate seeds per day for a generator seeded from a second-resolution clock: 86400
SC5 — 10000 four-digit codes, theoretical expected distinct value: 6321
weak generator, distinct code : 903
secure generator, distinct code in band: yes (6321 +/- 120)
SC6 — remainder of 256 byte values divided by 10: 0-5 has 26 ways, 6-9 has 25 ways, ratio 1.0400
1000000 bytes, direct modulo, ratio in band : yes (1.0400 +/- 0.008)
same bytes, rejection sampling, ratio in band : yes (1.0000 +/- 0.008)
rejected byte band : yes (23437.5 +/- 600)
```
#### What the Seed Gives Away
The first measurement is not a ratio but an equality: two generators starting from the same seed
produce **all ten thousand** of the ten thousand draws in the same order. The number here is
unimportant; what matters is that it is the full count. The weak generator is not a probability
distribution but a function — given a seed, its output sequence is fixed. The same measurement
cannot be made on the cryptographic generator, because there is no seed item to compare.
The third line explains why the seed is a separate item. If the generator is seeded from a
clock — from a second-resolution value at application startup — the number of candidate seeds in
a one-day window is **86,400**. This is a ceiling independent of how long the output looks: a
token that appears to be a hundred and twenty-eight bits can be carrying a guessing space of only
sixteen and a half bits. Length is a visible quantity; the seed space is not, which is why
looking at a token's length during code review is misleading.
#### How Many Values a Code Has
Predictability is not a yes-or-no property but a number, and that number comes from three
separate places: the seed space, the output space the generator actually traverses, and the way
the output is mapped onto the desired range. The three are independent of each other, and the
smallest one sets the ceiling. The first measurement counted the first of these; the second and
third measurements count the remaining two.
The second measurement says something even when the seed is not known at all. When ten thousand
four-digit codes are drawn, 6,321 distinct values are theoretically expected; when ten thousand
balls are thrown into ten thousand boxes, some boxes get more than one ball. The secure generator
hits this expectation. The weak generator produces only **903** distinct codes in the same
draw — exactly one-seventh of what is expected.
The reason lies in how the code is produced: the four-digit code is taken from the **low digits**
of the generator's output, and in this generator the low digits repeat on a much shorter cycle
than the high ones. Nominally there is a space of ten thousand values; in reality only around
nine hundred values are in circulation. The security consequence of this number is direct: for a
party trying to guess the verification code, the space is not ten thousand but nine hundred, and
reaching it does not require knowing the seed — only collecting codes.
#### The Bias of Deriving a Digit
The third measurement shows that even the secure generator can be called incorrectly. A random
byte takes a value between 0 and 255; if code that wants a base-ten digit takes the remainder of
a division, the remainders of the 256 values are not evenly distributed. The remainders zero
through five get **26** byte values, six through nine get **25**. The ratio is 1.04, and this
number is read from arithmetic, not from measurement; the run only confirms it. The ratio
measured over one million bytes comes out inside the band.
When the same bytes are run through rejection sampling — discarding any byte greater than or
equal to two hundred fifty and drawing a new one — the ratio drops to one. The cost is on the
third line: **23,437.5** out of one million bytes get rejected, that is, one in every
forty-three. An extra two-and-a-half percent of draws removes a four percent skew in the first
digit of a four-digit code. What the measurement says here is this: calling the secure generator
is not enough; the line that **maps** its output onto the range is also part of the defense, and
that line is usually written outside the central pattern.
#### Token-Generating Points
It is time to return to the inventory measure. In the fictional measurement network there are
nine points that generate a token, an identifier, or a filename. The table below places each
point's nominal width, generator choice, and the **effective** width bounded by the seed space
side by side.
```js
// random/token.mjs — inventory of the points that generate a token in the fictional
// measurement network (model). [point, carries, generator, nominal bits]. The weak
// generator's guessing space is bounded by the seed space: a generator seeded from a
// second-resolution clock has 86400 candidate seeds per day.
const points = [
['portal/session-id', 'identity', 'secure', 128],
['portal/password-reset', 'authorization', 'weak', 128],
['portal/verification-code', 'authorization', 'weak', 13.3],
['portal/invoice-link', 'authorization', 'secure', 128],
['field/upload-name', 'authorization', 'weak', 64],
['field/work-order-id', 'identity', 'weak', 32],
['ingestion/package-id', 'none', 'weak', 32],
['report/filename', 'authorization', 'weak', 64],
['tracing/request-id', 'none', 'secure', 64],
];
const SEED_SPACE = Math.log2(24 * 3600);
const effective = ([, , generator, bits]) => (generator === 'weak' ? Math.min(bits, SEED_SPACE) : bits);
const count = (rule) => points.filter(rule).length;
const write = (widths, ...values) => console.log(values.map((v, i) =>
(widths[i] < 0 ? String(v).padEnd(-widths[i]) : String(v).padStart(widths[i]))).join(''));
const D = [-26, 14, 10, 13, 15];
write(D, 'point', 'carries', 'generator', 'nominal bits', 'effective bits');
for (const n of points) write(D, n[0], n[1], n[2], n[3], effective(n).toFixed(1));
const weak = points.filter(([, , g]) => g === 'weak');
const consequential = weak.filter(([, c]) => c !== 'none');
console.log(`\npoints ${points.length}, weak generator ${weak.length}, `
+ `carrying identity or authorization ${consequential.length}, not carrying ${weak.length - consequential.length}`);
console.log(`total nominal bits ${points.reduce((a, n) => a + n[3], 0).toFixed(1)}, `
+ `total effective bits ${points.reduce((a, n) => a + effective(n), 0).toFixed(1)}`);
console.log(`largest loss: ${weak.map((n) => [n[0], n[3] - effective(n)])
.sort((x, y) => y[1] - x[1])[0].map((v) => (typeof v === 'number' ? v.toFixed(1) : v)).join(' ')} bits`);
// Two new token-generating points are added: one where the pattern is optional, one where it is mandatory.
const newPoints = ['portal/share-link', 'field/offline-queue-id'];
const E = [-46, 8, 12, 12, 19];
console.log('\ntwo new token-generating points are added:');
write(E, 'arrangement', 'points', 'in pattern', 'uncovered', 'new gap candidate');
write(E, 'pattern exists, direct call allowed', points.length + newPoints.length,
count(([, , g]) => g === 'secure'), count(([, , g]) => g === 'weak') + newPoints.length, newPoints.length);
write(E, 'single identity-generating function exposed', points.length + newPoints.length,
points.length + newPoints.length, 0, 0);
console.log(`points that must be moved first in the second arrangement: ${weak.length}`);
```
```
point carries generator nominal bits effective bits
portal/session-id identity secure 128 128.0
portal/password-reset authorization weak 128 16.4
portal/verification-code authorization weak 13.3 13.3
portal/invoice-link authorization secure 128 128.0
field/upload-name authorization weak 64 16.4
field/work-order-id identity weak 32 16.4
ingestion/package-id none weak 32 16.4
report/filename authorization weak 64 16.4
tracing/request-id none secure 64 64.0
points 9, weak generator 6, carrying identity or authorization 5, not carrying 1
total nominal bits 653.3, total effective bits 415.3
largest loss: portal/password-reset 111.6 bits
two new token-generating points are added:
arrangement points in pattern uncovered new gap candidate
pattern exists, direct call allowed 11 3 8 2
single identity-generating function exposed 11 11 0 0
points that must be moved first in the second arrangement: 6
```
**Six** of the nine points call the weak generator, and **five** of those carry identity or
authorization. The sixth — the ingestion service's package id — is the one point whose
predictability carries no consequence; no door opens with that identifier. This distinction
matters when counting coverage: the number of uncovered points is six, the number of points
with a security consequence is five, and the work of carrying the pattern starts from those five.
The effective-bits column shows in a single line why looking at nominal width is misleading. The
password-reset link produces a string of a hundred and twenty-eight bits; once the seed space is
taken into account, only **16.4 bits** remain, producing a loss of **111.6 bits** at a single
point. Of the 653.3 nominal bits in total, 415.3 survive. The verification code is an interesting
exception: because its nominal width is already 13.3 bits, changing the generator gains it no
bits. That point's defense is not in the generator but in the rule that limits the number of
attempts — and that rule is outside the scope of this lesson.
This inventory intersects with the previous lesson's fourteen-point cryptography inventory at one
place. There, two points had been flagged with the "fixed starting value" flaw; a fixed starting
value means **never calling** the generator at all, and it reduces that item's guessing space to
zero. The same generator shows up in both inventories at once: in one place it produces a token,
in the other it produces an encryption parameter. Carrying the two patterns separately means
closing the same call twice; tying both inventories to a single "generates an unpredictable
value" function closes it once, and covers every new point added after that in both at the same
time.
#### When a New Point Is Added
The last table adds two new token-generating points: the portal's share link and the field
application's offline queue id. In the first arrangement there is a pattern that calls the secure
generator, but it is not mandatory. The number of points rises from nine to eleven, the number of
points in the pattern **stays at three**, the number of uncovered points rises to eight, and the two new
points become direct gap candidates. Whoever writes a new point can choose their own generator
even if they know the pattern exists; coverage drops with every addition.
In the second arrangement, a single identity-generating function is exposed, and the generator
call appears nowhere else. All eleven of the eleven points pass through the pattern, and the
new-gap-candidate count is zero. The cost is on the last line again: moving to this arrangement
requires carrying over all six of the six points that used the weak generator. What keeps
coverage constant is not the quality of the pattern but the fact that it is the **only path**.
#### Summary
- Two weak generators starting from the same seed produced all ten thousand of the ten thousand
draws in the same order; the weak generator is a function, not a distribution.
- A generator seeded from a second-resolution clock has 86,400 candidate seeds per day; this sets
a guessing ceiling independent of the output's length.
- The theoretical expectation for ten thousand four-digit codes is 6,321 distinct values, but the
weak generator produced only 903 distinct codes; reading from the low digits narrowed the
nominal space tenfold.
- Mapping a byte to a digit by remainder leaves 26 ways for 0-5 and 25 ways for 6-9, producing a
1.04 bias; rejection sampling brings this down to one, at a cost of one in forty-three bytes.
- Six of the nine token-generating points call the weak generator, and five carry identity or
authorization; the password-reset link alone accounts for a loss of 111.6 bits.
- When the pattern is not mandatory, two new points leave coverage at three and add two gap
candidates; when the generator call is closed into a single function, eleven of the eleven
points are covered.
#### Next Step
These two lessons measured how the key and the token get generated, and both rested on the same
assumption: that the generated value stays only in the process that produced it. But a portion of
those values was never generated at all — it was written as a fixed literal in a configuration
line, and from there it entered version control. The next lesson counts this item over a
fictional repository history: how many commits a secret stayed in, how many days it stood exposed
from the moment it was added until the moment it was rotated, how much of that history cleanup
rescues, and how many false positives a rule that scans for the pattern produces.
### Hardcoded Secrets
Source: https://academia.sh/en/courses/secure-coding/hardcoded-secrets
The number of commits a secret stays exposed in a fictional repository history, and what pattern scanning covers: five secrets appear in 124 commits in the working tree and 223 commits in history, history cleanup fails to reach eighteen of nineteen copies, snapshot scanning sees only one of the five secrets, and the same rule produces a 76.8x increase in findings with a 0.69 false-positive rate over history.
The previous two lessons measured how a key and a token are generated, and both rested on the same
assumption: that the generated value stays only in the process that produced it. This assumption
never holds for a subset of values. Those values were not generated; they were typed by hand into a
configuration line, and from there entered version control. The moment they enter, they no longer
sit in a file — they sit in a **history**, and history has a measure of its own.
The Secrets Management lesson measured the rotation window of a value in production and how long it
stays valid after it leaks. What is measured here is not storage practice but **coverage**: how many
commits a secret has stayed in, how many commits deletion closes off, how many points a scanning
rule sees, and how many false positives are paid to see them. The repository history below is a
model; the commit counts, file names, and values are fictional and none of them carries a real
secret format.
#### A Secret in Repository History
**SC7 — a value enters repository history the moment it is committed; deleting it from the working
tree is not deleting it from history.** Deletion is a new commit, and the old commits stay in place.
The moment a secret is truly closed off is the moment the value is invalidated — rotated.
```js
// repo/history.mjs — a fictional repository history (model): 60 commits, five fictional secrets. Secret
// values below are generated with our own generator; no real secret format, real key pattern, or
// real service name is used. entered: the commit the value was committed in; deleted: the commit it
// was removed from the working tree in; rotated: the commit the value was invalidated in (null = never).
export const COMMITS = 60;
let state = 4703;
export const random = () => (state = (state * 1103515245 + 12345) % 2 ** 31) / 2 ** 31;
const ABC = 'abcdefghijkmnpqrstuvwxyzACDEFGHJKLMNPQRTUVWXYZ2346789';
export const generate = (n) => Array.from({ length: n }, () => ABC[Math.floor(random() * ABC.length)]).join('');
export const secrets = [
{ name: 'field signing key', entered: 7, deleted: 19, rotated: null },
{ name: 'report export password', entered: 12, deleted: null, rotated: null },
{ name: 'portal session salt', entered: 3, deleted: 41, rotated: 44 },
{ name: 'ingestion master key', entered: 22, deleted: 26, rotated: null },
{ name: 'work-order callback key', entered: 33, deleted: 55, rotated: 51 },
];
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(''));
const none = (x) => (x === null ? '-' : x);
// SC7: a secret enters history the moment it is committed; deleting it from the working tree
// is not deleting it from history. Closure happens only through rotation.
const D = [-23, 9, 9, 9, 6, 9, 6];
write(D, 'secret', 'entered', 'deleted', 'rotated', 'tree', 'history', 'open');
let tree = 0, history = 0, open = 0;
for (const s of secrets) {
const a = (s.deleted ?? COMMITS) - s.entered;
const h = COMMITS - s.entered;
const c = (s.rotated ?? COMMITS) - s.entered;
tree += a; history += h; open += c;
write(D, s.name, s.entered, none(s.deleted), none(s.rotated), a, h, c);
}
write(D, 'total', '', '', '', tree, history, open);
console.log(`secrets deleted from the tree but not rotated: `
+ `${secrets.filter((s) => s.deleted !== null && s.rotated === null).length}/${secrets.length}`);
// SC8: history cleanup changes only the copy it reaches; rotation devalues every
// copy. The copy counts are fictional.
console.log('\ncopies the value reaches once committed:');
const holders = [
['central repository', 1, true],
['developer clone', 9, false],
['mirror and backup', 2, false],
['build workspace and log', 4, false],
['packaged build output', 3, false],
];
const E = [-23, 8, 17, 10];
write(E, 'holder', 'copies', 'cleanup reaches', 'rotation');
for (const [name, n, reaches] of holders) write(E, name, n, reaches ? 'yes' : 'no', 'yes');
const total = holders.reduce((a, [, n]) => a + n, 0);
const reached = holders.filter(([, , e]) => e).reduce((a, [, n]) => a + n, 0);
write(E, 'total', total, `${reached}`, `${total}`);
console.log(`copies uncovered by history cleanup: ${total - reached}/${total}`);
```
```
secret entered deleted rotated tree history open
field signing key 7 19 - 12 53 53
report export password 12 - - 48 48 48
portal session salt 3 41 44 38 57 41
ingestion master key 22 26 - 4 38 38
work-order callback key 33 55 51 22 27 18
total 124 223 198
secrets deleted from the tree but not rotated: 2/5
copies the value reaches once committed:
holder copies cleanup reaches rotation
central repository 1 yes yes
developer clone 9 no yes
mirror and backup 2 no yes
build workspace and log 4 no yes
packaged build output 3 no yes
total 19 1 19
copies uncovered by history cleanup: 18/19
```
#### What Deletion Does not Close
The three number columns correspond to three separate questions, and all three come out different
for the same five secrets. The five secrets together appear in the working tree for a total of
**124 commits**; the same five secrets are reachable in repository history for **223 commits**; the
time they are actually left open is **198 commits**. The gap between the middle column and the left
one is the size of the illusion that deletion produces: a span of ninety-nine commits looks closed
off as "cleaned up," but the content still sits in history.
Reading the rows one by one sharpens the measure further. The ingestion service's master key stayed
in the working tree for only **four commits** — it was noticed and deleted four commits after it
entered. This looks like a quick fix. The history column shows **38** for the same value, and the
open column also shows 38, because the value was never rotated. Four commits of visibility
corresponds to thirty-eight commits of exposure. The same pattern shows up in the field signing key
too: twelve commits in the tree, fifty-three commits open.
The last row shows the arrangement done in the right order. The work-order callback key was rotated
at commit 51 and deleted at commit 55; the time it stayed open is **18 commits**, the shortest of the
five secrets. This is the only row where the time in the tree (22 commits) is **longer** than the
time left open. Rotating first and cleaning up after is the only row that reverses the measure.
The second table counts why. A committed value reaches nineteen copies once it is pushed: one
central repository, nine developer clones, two mirrors and backups, four build workspaces and logs,
three packaged build outputs. Rewriting history reaches **one** of these; the remaining **eighteen**
stay untouched. Each clone has to be updated with a separate commit, build logs are mostly
immutable, and packaged output has already been distributed. The rotation column, on the other hand,
says `yes` for nineteen of nineteen — because rotation does not delete the copy, it makes the copy
**worthless**. Once a value has spread, the only way out is rotation; rewriting history does not
replace that, at best it stops a new copy from being produced.
#### Same Rule, Two Separate Scans
**SC9 — a scanning rule only covers the points in the set it scans; a snapshot and a history are two
separate sets.** The rule below is given and is not up for debate: a value longer than twenty
characters with a character entropy above 3.6 is flagged. What is measured is not the quality of
the rule, but what the same rule covers across two separate sets.
```js
// repo/scan.mjs — pattern scanning over the same fictional repository history (model, runs on its own).
// The rule is given: a value with length >= 20 and character entropy >= 3.6 is flagged.
// Values are generated with our own generator; none of them carries a real secret format.
const COMMITS = 60;
let state = 4703;
const random = () => (state = (state * 1103515245 + 12345) % 2 ** 31) / 2 ** 31;
const ABC = 'abcdefghijkmnpqrstuvwxyzACDEFGHJKLMNPQRTUVWXYZ2346789';
const generate = (n) => Array.from({ length: n }, () => ABC[Math.floor(random() * ABC.length)]).join('');
// [file, value, is secret, entered commit, deleted commit]
const items = [
['field/config.mjs', generate(32), true, 7, 19],
['report/export.mjs', generate(28), true, 12, null],
['portal/session.mjs', generate(30), true, 3, 41],
['ingestion/source.mjs', generate(14), true, 22, 26],
['work-order/endpoint.mjs', generate(26), true, 33, 55],
['tracing/sample.mjs', generate(28), false, 0, null],
['portal/icon.mjs', generate(40), false, 0, null],
['build/version.mjs', generate(24), false, 5, null],
['catalog/id.mjs', generate(22), false, 0, null],
['field/test.mjs', generate(21), false, 9, 38],
['penalty/config.mjs', './data/temp/penalty', false, 0, null],
['notification/sample.mjs', 'CHANGE-ME', false, 0, null],
['membership/tier.mjs', 'subscriber-std-plan', false, 0, null],
];
const entropy = (d) => {
const count = {};
for (const c of d) count[c] = (count[c] ?? 0) + 1;
return -Object.values(count).reduce((a, n) => a + (n / d.length) * Math.log2(n / d.length), 0);
};
const RULE = (d) => d.length >= 20 && entropy(d) >= 3.6;
const exists = (k, commit) => commit >= k[3] && (k[4] === null || commit < k[4]);
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(''));
const D = [-22, 9, 7, 8, 8, 16, 6];
write(D, 'scan', 'scanned', 'found', 'caught', 'missed', 'false positive', 'rate');
const measure = (name, pairs, dedup) => {
const found = dedup ? new Set(pairs.filter(([k]) => RULE(k[1])).map(([k]) => k[1])).size
: pairs.filter(([k]) => RULE(k[1])).length;
const caught = dedup ? new Set(pairs.filter(([k]) => RULE(k[1]) && k[2]).map(([k]) => k[1])).size
: pairs.filter(([k]) => RULE(k[1]) && k[2]).length;
const secretTotal = dedup ? new Set(pairs.filter(([k]) => k[2]).map(([k]) => k[1])).size
: pairs.filter(([k]) => k[2]).length;
write(D, name, pairs.length, found, caught, secretTotal - caught, found - caught,
found ? ((found - caught) / found).toFixed(2) : '-');
};
const snapshot = items.filter((k) => exists(k, COMMITS - 1)).map((k) => [k]);
const history = [];
for (let i = 0; i < COMMITS; i++) for (const k of items) if (exists(k, i)) history.push([k, i]);
measure('snapshot (last commit)', snapshot, false);
measure('full history', history, false);
measure('full history + dedup', history, true);
const secretHistory = history.filter(([k]) => k[2]).length;
console.log(`\nsecret items still present at the last commit: ${snapshot.filter(([k]) => k[2]).length}/5;`
+ ` secret item-commit pairs seen in history: ${secretHistory}`);
const missed = items.filter((k) => k[2] && !RULE(k[1]));
console.log(`secret item the rule misses: ${missed.map((k) => `${k[0]} (length ${k[1].length})`).join(', ')}`);
console.log(`history scan's finding multiplier over the snapshot: `
+ `${(history.filter(([k]) => RULE(k[1])).length / snapshot.filter(([k]) => RULE(k[1])).length).toFixed(1)}x`);
```
```
scan scanned found caught missed false positive rate
snapshot (last commit) 8 5 1 0 4 0.80
full history 568 384 120 4 264 0.69
full history + dedup 568 9 4 1 5 0.56
secret items still present at the last commit: 1/5; secret item-commit pairs seen in history: 124
secret item the rule misses: ingestion/source.mjs (length 14)
history scan's finding multiplier over the snapshot: 76.8x
```
#### The Cost of the False Positive
The first row gives coverage in its clearest form. The working tree at the last commit holds only
eight items, and **only one** of them is secret. The snapshot scan finds that one, misses nothing,
and looks perfect. Yet four of the five secrets have already been deleted, and the scan never sees
them at all. The coverage ratio on this row is 1/5, and the scan does not report that; what it
reports is a finding it caught.
The line beneath the table writes this same 1/5 from another angle: the number of item-commit pairs
in which a secret item appears in history is **124**. That is the same number as the total of the
first table's "tree" column, which shows that the two measurements are reading the same model. The
snapshot scan sees only the one at the last commit out of these 124 pairs.
The second row applies the same rule to the full history. The scanned item-commit pairs rise to 568
and findings to 384, and all five of the five secrets become visible. The finding count has grown by
**76.8 times** over the snapshot. The false-positive count is 264 and its rate is 0.69 — meaning two
out of every three findings is a non-secret value. This rate does not come from the rule being bad;
it comes from the same harmless value being counted sixty times across sixty commits.
The third row removes this multiplier. Once findings are deduplicated by content, the 384 findings
drop to **nine**: four secret, five false positive, rate 0.56. The number of items to review shrinks
by two orders of magnitude, and coverage does not change at all. What should be measured in a
history scan is not the finding count but the count of **distinct values**; otherwise the scan
measures its own noise.
The last line writes the rule's limit: the ingestion service's master key is fourteen characters
long, stays under the threshold, and does not show up in any scan. Four of the five secrets are
covered; one is left out. This is a gap the scan cannot show with its own measure; only someone who
knows what it missed from somewhere else can count it.
#### When a New Secret Is Added
The third column carries this lesson's sharpest distinction. Pattern scanning is a **filter, not a
guarantee**: a secret added tomorrow is covered only if it clears the threshold. One of the five
secrets is under the threshold today, so the chance that a new value gets covered is not something
measured but something hoped for. Lowering the threshold increases coverage and grows the
false-positive count; the false-positive rate is already 0.56, and every increase multiplies the
review workload.
The pattern in which the secret never enters the source tree at all solves this problem from a
different direction. If the value is read from a secrets store at run time, no value ever exists in
the source at all; coverage no longer rests on what the scan finds, but on there being no value in
the source to begin with. When a new secret is added, coverage continues on its own, because there
is no place to write the value into the code. In this arrangement, scanning does not disappear — it
stays to find the point that falls **outside** the pattern — but it is no longer the only defense.
When the coverage of the two defenses is measured on the same five secrets, one gives 4/5, the other
5/5, and the one-row gap between them corresponds to a 38-commit exposure across a sixty-commit
history.
#### Summary
- Five secrets appear in the working tree for 124 commits and in repository history for 223 commits;
the time they are actually left open is 198 commits. Deletion makes a ninety-nine-commit span look
closed off.
- The ingestion service's key stayed in the tree for four commits but, since it was never rotated,
was left open for 38 commits; as long as the time in the tree is shorter than the time left open,
deletion gives no real measure.
- The only row that rotated first and cleaned up after produced the shortest exposure: 18 commits.
- A committed value reaches nineteen copies; history cleanup reaches one, leaving eighteen uncovered
its reach. Rotation makes all nineteen worthless at once.
- Snapshot scanning sees only one of the five secrets; the same rule finds all five across the full
history, but findings grow 76.8-fold and the false-positive rate reaches 0.69. Content
deduplication brings the findings down to nine without changing coverage.
- Pattern scanning covers a new secret only if it clears the threshold (4/5); the pattern that never
writes the value to the source covers the new point on its own.
#### Next Step
This lesson counted the trace a secret leaves in the source tree and in repository history. That
trace closes once the value is pulled out of the source and moved into a store — but the value still
sits in the process at run time, and the process talks. The response that comes back when a request
errors, the log line written when an operation runs, the stack trace that lands in a file after a
crash: all of these are points where values inside the process leak out. The next lesson counts
these points as items — how many responses carry a stack trace, how many log lines carry a
credential or personal data, how many of them a central masking layer closes off, and how many items
are left exposed.
### Error Messages and Log Leakage
Source: https://academia.sh/en/courses/secure-coding/error-messages-and-log-leakage
Counting the leakage in an error response and a log line as items: eight endpoints carrying forty item-response pairs in direct mode, the central error handler covering six endpoints and leaving two outside, a top-level name list closing three of eleven log items, the same list run at every level closing eight, format-based masking closing nine, and two items in free text and a query string closing under no list.
The previous lesson counted the trace a secret leaves in the source tree and in repository
history. That trace closes once the value is pulled out of the source and moved into a store — but
the value still sits in the process at run time, and the process talks. The response that comes
back when a request errors, the log line written when an operation runs, the stack trace that lands
in a file after a crash: all of these are points where values inside the process leak out.
What these points have in common is that none of them was written as a security decision. The line
that puts the stack trace into the response is there to debug; the line that writes the request to
the log with all its fields is there to find a problem. The measure, then, is not "which line is
wrong" but the question of **how many items leak out** and how many of them a central layer closes.
#### Items in the Error Response
**SC10 — an error response is the sum of the items that carry internal application information;
each item is counted separately.** A stack trace, an internal file path, an internal component
name, the error class's name, and the echo of the raw request are five separate items, and a single
response can carry all of them at once. **SC11 — a central error handler only covers the error that
reaches it**; an endpoint that catches its own error and writes its own response falls outside its
coverage.
```js
// log/response.mjs — a real local server is started, requests are sent to eight endpoints, and
// the internal-information items carried in the returned response body are counted. Bodies are not
// printed (they contain an absolute path); only item counts are written. The two endpoints marked
// "own" catch their own error and write their own response, so they never reach the central
// handler. Fictional measurement-network endpoints.
import { createServer } from 'node:http';
import { randomUUID } from 'node:crypto';
// [path, function that throws the error, whether it writes its own response]
const ENDPOINTS = [
['/subscriber/search?name=fictional-subscriber-118', () => { throw new Error('subscriber-directory/search.mjs: directory response empty'); }, false],
['/invoice/account?no=118', () => { throw new RangeError('invoice-account/period.mjs: period out of range'); }, false],
['/measurement/get?meter=MTR-4417', () => { throw new TypeError('measurement-ingestion/packet.mjs: field expected'); }, true],
['/report/generate?type=monthly', () => { throw new Error('report-generator/template.mjs: template not found'); }, false],
['/work-order/assign?team=3', () => { throw new Error('work-order/assignment.mjs: team capacity exceeded'); }, false],
['/field/upload?name=measurement.csv', () => { throw new Error('field-upload/storage.mjs: storage response invalid'); }, true],
['/portal/login?user=fictional-subscriber-118', () => { throw new Error('portal-identity/verify.mjs: comparison failed'); }, false],
['/notification/send?channel=sms', () => { throw new Error('notification-queue/send.mjs: queue full'); }, false],
];
let mode = 'direct';
const server = createServer((req, res) => {
const endpoint = ENDPOINTS.find(([u]) => u === req.url);
try {
endpoint[1]();
} catch (err) {
const open = mode === 'direct' || (mode === 'central' && endpoint[2]);
const body = open
? { error: err.message, type: err.constructor.name, trace: err.stack, request: req.url }
: { error: 'operation could not complete', id: randomUUID() };
res.writeHead(500, { 'content-type': 'application/json' });
res.end(JSON.stringify(body));
}
});
server.listen(8794, '127.0.0.1');
const MARKERS = [
['stack trace', (g) => /\bat\s+\S+\s*\(/.test(g)],
['internal file path', (g) => /\.mjs/.test(g)],
['internal component name', (g) => /(subscriber-directory|invoice-account|measurement-ingestion|report-generator|work-order|field-upload|portal-identity|notification-queue)/.test(g)],
['error class', (g) => /"type":"(Error|RangeError|TypeError)"/.test(g)],
['request echo', (g) => /"request":/.test(g)],
['correlation id', (g) => /"id":/.test(g)],
];
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(''));
const D = [-26, 13, 21, 26, 13, 14, 16];
write(D, 'mode', ...MARKERS.map(([a]) => a));
for (const m of ['direct', 'central', 'own response disabled']) {
mode = m === 'own response disabled' ? 'closed' : m;
const counter = MARKERS.map(() => 0);
for (const [path] of ENDPOINTS) {
const body = await (await fetch(`http://127.0.0.1:8794${path}`)).text();
MARKERS.forEach(([, rule], i) => { if (rule(body)) counter[i] += 1; });
}
write(D, m, ...counter);
}
server.close();
const item = MARKERS.length - 1;
const own = ENDPOINTS.filter((u) => u[2]).length;
console.log(`\nendpoints ${ENDPOINTS.length}, items measured per response ${item}, `
+ `item-response pairs in direct mode ${ENDPOINTS.length * item}`);
console.log(`central handler covers ${ENDPOINTS.length - own} endpoints, ${own} endpoints stay uncovered; `
+ `uncovered item-response pairs ${own * item}`);
```
```
mode stack trace internal file path internal component name error class request echo correlation id
direct 8 8 8 8 8 0
central 2 2 2 2 2 6
own response disabled 0 0 0 0 0 8
endpoints 8, items measured per response 5, item-response pairs in direct mode 40
central handler covers 6 endpoints, 2 endpoints stay uncovered; uncovered item-response pairs 10
```
#### The Endpoint That Bypasses the Center
The first row is the measure's starting point: all eight of the eight endpoints carry all five of
the five items, which comes to **40 item-response pairs**. This is the state where no one made a
decision — the error object was written straight into the response as it is, and the stack trace
tells the server's file layout, the error class tells its internal logic, and the raw request echo
tells that the sent value comes back.
The second row gives the central handler's coverage, and it is the clearest example of the course's
rule. **Six** of the eight endpoints are covered, the item count drops to zero, and a correlation id
takes its place. The remaining **two endpoints** — the measurement-ingestion endpoint and the
field-upload endpoint — catch their own error and write their own response, so they never reach the
central handler at all. On these two endpoints all five of the five items still stand: the number of
uncovered pairs is **10**. Coverage has dropped from 40 to 10, not to zero.
The distinction here matters: the central handler is not flawed, it works flawlessly on the six
endpoints it covers. What lowers coverage is not the handler's quality but **the fact that it can be
bypassed**. An endpoint that writes its own error response mostly looks like a good habit in review
— handling the error where it happens. The third row writes the rule: once local response writing
is disabled, all eight of the eight endpoints drop down to the correlation id and the items left
outside become zero.
The correlation-id column also carries this arrangement's cost. The five items pulled out of the
response do not disappear; they get tied to an id and carried **inward**. In a troubleshooting
search, that id leads to the internal record, and the stack trace is found there. What the measure
says is that the leak is not closed, only relocated: the response is open to eight clients, the
internal record to a specific set of readers. The second measurement counts exactly this relocated
pile itself, because a log is kept longer than a response and copied to more places.
#### Items in the Log Line
The response goes outward, the log stays inward — but the log too gets written somewhere, moved
somewhere, and mostly kept longer than the response. **SC12 — a masking layer works by looking at
the field name; a value sitting where there is no name is not closed off by any name list.**
```js
// log/masking.mjs — the masking layer's coverage over fictional log records (model).
// Values are fictional; none of them is a real secret format or a real person's data.
const T = 'fictionaltoken7f2a9c41d8b3e6';
const IMZA = 'fictionalsignature4b7d2e9a1c8f5306';
const P = 'fictional-password-9c1x', AD = 'A. Smith', ADRES = 'Example St. 12/4';
const TEL = '05000000000', KN = '10000000146';
// [record, list of sensitive values that appear in that record, leak type]
const records = [
[{ event: 'login', user: 'fictional-subscriber-118', token: T }, [T], 'top-level field'],
[{ event: 'registration', subscriber: { subscriberName: AD, address: ADRES } }, [AD, ADRES], 'nested field'],
[{ event: 'error', message: `validation failed: ${P}` }, [P], 'free text'],
[{ event: 'request', target: `/portal/login?token=${T}` }, [T], 'query string'],
[{ event: 'batch', records: [{ phone: TEL }, { nationalId: KN }] }, [TEL, KN], 'array item'],
[{ event: 'sync', parent: T }, [T], 'name not in list'],
[{ event: 'refresh', context: { session: { signature: IMZA } } }, [IMZA], 'third level'],
[{ event: 'notification', subscriberName: AD, phone: TEL }, [AD, TEL], 'top-level field'],
];
const HARMLESS = ['a1b2c3d4e5f60718293a4b5c6d7e8f90', '9f3c2a1b8d7e6540c9b2a1f0e3d4c5b6'];
const harmlessRecords = [{ event: 'tracing', requestId: HARMLESS[0] }, { event: 'build', version: HARMLESS[1] }];
const NAMES = ['token', 'password', 'signature', 'nationalId', 'phone', 'subscriberName', 'address'];
const FORMAT = (d) => typeof d === 'string' && /^[A-Za-z0-9]{20,}$/.test(d);
function mask(value, { deep, format }, level = 0) {
if (Array.isArray(value)) return deep ? value.map((x) => mask(x, { deep, format }, level + 1)) : value;
if (value && typeof value === 'object') {
if (!deep && level > 0) return value;
return Object.fromEntries(Object.entries(value).map(([k, v]) =>
[k, NAMES.includes(k) ? '***' : mask(v, { deep, format }, level + 1)]));
}
return format && FORMAT(value) ? '***' : value;
}
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(''));
const ITEMS = records.reduce((a, [, d]) => a + d.length, 0);
const D = [-34, 8, 10, 12, 16];
write(D, 'layer', 'items', 'closed', 'uncovered', 'false positive');
const result = {};
for (const [name, settings] of [
['no layer', null],
['name list, top level only', { deep: false, format: false }],
['name list, every level', { deep: true, format: false }],
['name list + format', { deep: true, format: true }],
]) {
let remaining = 0;
const remainingByType = {};
for (const [record, sensitive, type] of records) {
const text = JSON.stringify(settings ? mask(record, settings) : record);
const k = sensitive.filter((d) => text.includes(d)).length;
remaining += k;
if (k) remainingByType[type] = (remainingByType[type] ?? 0) + k;
}
const alarm = settings ? harmlessRecords.filter((k) =>
!JSON.stringify(mask(k, settings)).includes(HARMLESS[harmlessRecords.indexOf(k)])).length : 0;
result[name] = remainingByType;
write(D, name, ITEMS, ITEMS - remaining, remaining, alarm);
}
console.log('\ntype of uncovered items:');
for (const [name, type] of Object.entries(result)) {
if (name === 'no layer') continue;
console.log(` ${name.padEnd(30)}${Object.entries(type).map(([t, n]) => `${t} ${n}`).join(', ') || '-'}`);
}
```
```
layer items closed uncovered false positive
no layer 11 0 11 0
name list, top level only 11 3 8 0
name list, every level 11 8 3 0
name list + format 11 9 2 2
type of uncovered items:
name list, top level only nested field 2, free text 1, query string 1, array item 2, name not in list 1, third level 1
name list, every level free text 1, query string 1, name not in list 1
name list + format free text 1, query string 1
```
#### The Value With No Name
Eight log records hold eleven sensitive items, and these come from two separate classes: part of
them are secret-natured values — token, password, signature — and part of them are personal data.
The two classes have different reasons for closure. A secret leak hands over an authority and
closes with rotation; a personal-data leak does not hand over anything transferable but cannot be
undone, because there is no rotating counterpart for it. The same masking layer covers both at once,
so the item count can be done without separating them; but **which class** the uncovered item
belongs to is what decides prioritization.
The second row is most setups' starting point — there is a name list and it only looks at top-level
fields. It closes **three** of the eleven items, eight stay uncovered. The types of those left uncovered
are in the table below: nested field two, array item two, third level one, free text one, query
string one, name not in list one.
The third row shows how much a single change buys: when the same name list also runs at every level
and inside arrays, the closed-item count rises from three to **eight**. It is a structural fix,
requires no new rule, and on its own gains more than half of the coverage. Three items remain, and
all three remain for the same reason: the place where the value sits has no **field name**.
The fourth row adds format-based masking — values longer than twenty characters made up only of
letters and digits get masked. The closed-item count rises to nine; the one item gained is the
"name not in list" row, because even though the value's name is not known, its format is recognized.
The cost is written on the same row: **two false positives**. The tracing id and the build version
stamp also fit the same format and get masked, and these are exactly the fields read for
troubleshooting. Here, masking buys a diagnostic loss in exchange for a security gain.
The last two items do not close under any arrangement. One is a password embedded inside a free-text
error message, the other is a token in a link's query portion. Neither has a field name; one is part
of a sentence, the other part of a string. A masking layer can only find these by parsing the
string's content, and at that point it is no longer a masking layer but a text scan — a defense
whose coverage is uncertain and whose false-alarm rate is unmeasured.
#### When a New Point Is Added
The third column gives two separate answers across the two measurements. On the error-response
side, the pattern **covers on its own**: once local response writing is disabled, a ninth endpoint
added tomorrow also has to pass through the central handler, and this requires no decision. Coverage
is complete regardless of the endpoint count.
On the log side, the pattern does not cover on its own. A name list running at every level covers a
new **record**, but does not cover a new **field name**: a sensitive field written to the log
tomorrow keeps showing up if its name is not in the list. In this lesson's measure, the counterpart
to that is the "name not in list" row, and format-based masking only catches it if the format is
recognized. This is where the two defenses differ: one closes a **path**, the other updates a
**list**. The path-closing defense covers the new point on its own; the list-updating defense waits
to be remembered once more at every new point.
#### Summary
- Each of the eight endpoints carries five internal-information items; when the error object is
written straight into the response, the item-response pairs leaking out come to 40.
- The central error handler covers six endpoints, the two endpoints that write their own response
stay uncovered and ten pairs remain standing; once local response writing is disabled, the count
drops to zero.
- Eight log records hold eleven sensitive items; a name list that only looks at the top level closes
three, the same list running at every level and inside arrays closes eight.
- Format-based masking raises coverage to nine and produces two false positives; the two values it
masks are the tracing id and the version stamp read for troubleshooting.
- The value embedded in free text and the value in the link's query string close under no
arrangement, because neither has a field name.
- A new endpoint is covered on its own once the central handler is mandatory; a new log field is not
covered until the name list is updated.
#### Next Step
This lesson counted the points where the process talks outward, and in every one of them what was
measured was the behavior of **the code we wrote ourselves**: which line puts the stack trace into
the response, which field gets masked. Yet most of the code that serves a request is not lines the
team wrote. The same process loads dozens of components, each one pulls in other components, and
none of their names appear anywhere we wrote. The next lesson treats this set as a graph: how deep
the dependency chain runs, how many packages a single component affects, and why a flaw in a
component that is never called is prioritized differently.
### Dependency-Related Risks
Source: https://academia.sh/en/courses/secure-coding/dependency-related-risks
The dependency graph's coverage measure: eight names appear in the manifest file while thirty-four packages exist in the process, the transitive depth rising to six, a single component affecting twenty packages, three of eight direct dependencies never being called, and four of six open findings arriving with the same severity being in reachable components and two being in an unreached component.
The previous lesson counted the points where the process talks outward, and in every one of them
what was measured was the behavior of the code we wrote ourselves: which line puts the stack trace
into the response, which field gets masked. Yet most of the code that serves a request is not
lines the team wrote. When the portal service's process comes up, dozens of components get loaded;
part of them appear by name in a manifest file, the rest come along only because they are some
other component's dependency.
The Dependency Risk lesson treated this set as a configuration decision: how many days of exposure
the upgrade policy accumulates, how many versions it breaks. Here, policy is never discussed. What
is measured is the graph's **own structure** — how many components there are, how far apart they
sit, how many are affected when one falls, and how many stay loaded in the process without ever
being called.
#### The Dependency Graph
**SC13 — the number of components named in the manifest file does not give the number of
components loaded in the process**; the latter is the transitive closure and a separate quantity.
**SC14 — the area a component affects is measured by the number of packages transitively depending
on it, and is independent of depth.** **SC15 — being loaded is not being called**; a component
unreached from the code path still sits in the graph, still takes up space in the process, and
never runs.
```js
// dependency/graph.mjs — the dependency graph of the fictional measurement-network portal
// service (model). Eight components appear by name in the manifest file; the rest come in
// transitively. Edges are generated with our own generator, the seed is visible. Package
// names are fictional (p01, p02, ...).
const SEED = 91237;
let state = SEED;
const random = () => (state = (state * 1103515245 + 12345) % 2 ** 31) / 2 ** 31;
const N = 44, DIRECT = 8;
const name = (i) => `p${String(i).padStart(2, '0')}`;
const edges = new Map(Array.from({ length: N }, (_, i) => [i, []]));
edges.set(-1, Array.from({ length: DIRECT }, (_, i) => i)); // -1 = our own code
for (let i = 0; i < N; i++) {
const count = i < DIRECT ? 1 + Math.floor(random() * 4) : Math.floor(random() * 3);
for (let k = 0; k < count; k++) {
const target = DIRECT + Math.floor(random() * (N - DIRECT));
if (target !== i && !edges.get(i).includes(target)) edges.get(i).push(target);
}
}
// Depth: shortest distance from our own code. A package that is never reached is not in the graph.
const depth = new Map();
let frontier = [-1];
for (let level = 0; frontier.length; level++) {
const next = [];
for (const u of frontier) for (const v of edges.get(u) ?? []) {
if (!depth.has(v)) { depth.set(v, level + 1); next.push(v); }
}
frontier = next;
}
const packages = [...depth.keys()].sort((a, b) => a - b);
// The packages a single component affects: the number of packages transitively depending on it.
const reverseReach = (target) => {
const seen = new Set();
let changed = true;
while (changed) {
changed = false;
for (const u of packages) {
if (seen.has(u)) continue;
if ((edges.get(u) ?? []).some((v) => v === target || seen.has(v))) { seen.add(u); changed = true; }
}
}
return seen.size;
};
// Our code path actually calls five of the eight direct components; three are loaded but never called.
const CALLED = [0, 1, 3, 4, 6];
const reached = new Set();
frontier = [...CALLED];
while (frontier.length) {
const node = frontier.pop();
if (reached.has(node)) continue;
reached.add(node);
for (const v of edges.get(node) ?? []) frontier.push(v);
}
const write = (g, ...s) => console.log(s.map((v, i) =>
(g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(''));
const distribution = {};
for (const p of packages) distribution[depth.get(p)] = (distribution[depth.get(p)] ?? 0) + 1;
console.log(`seed ${SEED}: ${packages.length} packages in the graph, ${DIRECT} of them appear `
+ `by name in the manifest file, ${packages.length - DIRECT} of them are transitive`);
console.log(`distribution by depth: ${Object.entries(distribution).map(([k, v]) => `${k}: ${v}`).join(', ')}`);
console.log(`largest depth ${Math.max(...depth.values())}, `
+ `edge count ${packages.reduce((a, p) => a + (edges.get(p) ?? []).length, 0)}`);
console.log(`packages reached from our code path ${reached.size}, loaded but unreached `
+ `${packages.length - reached.size}`);
console.log(`depth no one chose: ${packages.filter((p) => depth.get(p) >= 3).length} packages at `
+ `three and beyond, ${packages.filter((p) => depth.get(p) >= 5).length} packages at five and beyond`);
const impact = packages.map((p) => [p, reverseReach(p)]).sort((a, b) => b[1] - a[1]);
console.log('\nthe five components touching the most packages:');
const D = [-10, 10, 12, 18];
write(D, 'package', 'depth', 'affected', 'reachable');
for (const [p, n] of impact.slice(0, 5)) {
write(D, name(p), depth.get(p), n, reached.has(p) ? 'yes' : 'no');
}
// Six fictional open findings: nothing is given except which package each is in.
const OPEN_FINDINGS = [17, 29, 12, 4, 27, 7]; // fictional findings; only the package name is given
console.log('\nsix open findings, all with the same severity:');
const E = [-10, 10, 12, 18, 8];
write(E, 'package', 'depth', 'affected', 'reachable', 'rank');
const sorted = OPEN_FINDINGS
.map((p) => ({ p, e: reached.has(p), n: reverseReach(p) }))
.sort((a, b) => (b.e - a.e) || (b.n - a.n));
sorted.forEach((s, i) => write(E, name(s.p), depth.get(s.p), s.n, s.e ? 'yes' : 'no', i + 1));
console.log(`${sorted.filter((s) => s.e).length} in reachable packages, ${sorted.filter((s) => !s.e).length} `
+ `in unreached packages; both came in with the same severity`);
```
```
seed 91237: 34 packages in the graph, 8 of them appear by name in the manifest file, 26 of them are transitive
distribution by depth: 1: 8, 2: 15, 3: 6, 4: 2, 5: 2, 6: 1
largest depth 6, edge count 44
packages reached from our code path 28, loaded but unreached 6
depth no one chose: 11 packages at three and beyond, 3 packages at five and beyond
the five components touching the most packages:
package depth affected reachable
p17 3 20 yes
p24 3 20 yes
p36 6 19 yes
p10 5 18 yes
p13 2 18 yes
six open findings, all with the same severity:
package depth affected reachable rank
p17 3 20 yes 1
p29 4 18 yes 2
p12 2 6 yes 3
p04 1 0 yes 4
p27 2 3 no 5
p07 1 0 no 6
4 in reachable packages, 2 in unreached packages; both came in with the same severity
```
#### The Depth No One Chose
The first line puts two numbers side by side: **eight** names appear in the manifest file,
**thirty-four** packages sit in the process. No one chose the twenty-six components in between;
none of them was the subject of a decision, none of their names appear in a file the team wrote.
For coverage purposes this number is decisive: every audit done on the codebase — review, rule,
inspection — covers the eight names and never touches the twenty-six.
The depth distribution details the distance. There are eight packages at the first level, fifteen
at the second, six at the third; five more across the fourth, fifth, and sixth levels. **Eleven
packages sit at depth three and beyond**, three at depth five and beyond. Reaching a component at
the sixth level means following five separate components' dependency lists; none of those lists is
ours, and each one can change independently.
It also needs to be said that depth is not on its own a measure of risk. Depth only tells **who
made the decision**: the first level is our decision, the second level is our direct dependency's
decision, the sixth level is the composite of five decisions. The column that measures risk is the
next one.
#### The Area a Single Component Touches
The second table looks in the reverse direction: how many packages' dependency chain a component
appears in. The top two components affect **twenty** of the thirty-four packages; when a flaw is
found in one of them, twenty packages become relevant at once. The third row shows that this
measure is independent of depth: the component at the sixth level affects nineteen packages,
meaning the graph's most distant component is also one of its most-touching ones. Distance and
impact do not track each other, and the two must be measured separately.
The graph's edge count says where this pile-up comes from: there are **forty-four** edges among
the thirty-four packages, an average of 1.3 outgoing edges per package. The number is small, but
the graph is not a tree — the same component is reached by more than one path. In a tree, the
number of packages a leaf affects is zero, the number the root affects is the whole of it, and
there is a clean drop-off in between. In a shared graph, even a node in the middle can touch twenty
packages; the pile-up in the impact column is the direct result of this sharing, and it is born
even with a small number of edges.
The direct dependencies at the first level have an affected count of zero; this is not a flaw, it
is the definition of the measure. Nothing depends on them, only our own code does. A problem in a
direct dependency concerns a single component; a problem in a deep shared component concerns
twenty — and both arrive in the same record format, with the same severity. The record itself does
not carry this difference; only the graph gives it.
#### The Flaw in a Component That Is Never Called
The last measurement adds a second filter to the graph. Five of the eight direct dependencies are
actually called by our code path, three are loaded but never called. Together with the transitive
closure of those three, **six packages** sit in the process and never run at all; the number of
reached packages is twenty-eight.
When the six open findings arrive with the same severity, the table produces a ranking, and two
columns decide that ranking: first reachability, then the number of packages affected. The
first-ranked finding is both in a reachable component and affects twenty packages. The
fourth-ranked finding is also in a reachable component, but its affected count is zero — a
first-level dependency. The two findings ranked fifth and sixth are in components that are never
called; one affects three packages, the other none.
It needs to be written out plainly why a flaw in an unreached component gets a different priority,
because this distinction is often misread. A flaw in an unreached component **is not ignored**:
the component is loaded in the process, a code change tomorrow could call it, and the reachability
measurement is an estimate made against our own graph, not a certain result. The distinction is not
in the decision to close it but in the order of closing it. All six of the six findings get closed;
the first three get closed first, because leaving them open concerns a code path running today.
The number that matters for coverage is this: of the six findings that arrive with the same
severity, **four** are in reachable components, **two** are in an unreached component. A setup
that does not make this distinction treats all six findings as equal and spends a third of its
review time on code that never runs. A setup that makes the distinction has to answer two questions
at once: which component is called, and when was this measurement last refreshed. The second
question is outside this course's scope.
#### Summary
- Eight names appear in the manifest file while thirty-four packages sit in the process;
twenty-six components were never the subject of a decision and no audit in the codebase touches
them.
- The depth distribution gives eight packages at the first level, fifteen at the second; eleven
packages sit at depth three and beyond, three at depth five and beyond. Depth measures not risk
but who made the decision.
- A single component affects twenty of the thirty-four packages, and the component at the sixth
level affects nineteen; distance and impact do not track each other.
- Three of the eight direct dependencies are loaded but never called; together with their
transitive closure, six packages sit in the process and never run.
- Of six open findings arriving with the same severity, four are in reachable components and two
are in an unreached component; the distinction decides the order of closing, not the decision to
close.
#### Course Wrap-Up
| Lesson | How Many Points | What the Pattern Covers | Uncovered |
|---|---|---|---|
| Input Validation Patterns | 15 field points | allowlist 15 | 0 points (denylist 14) |
| Output Encoding | 14 output points | context declaration 14 | 0 points (scattered 5) |
| Injection Defense | 16 query points | parameterized 11 | 5 edge-case flows |
| Template and Expression Injection | 12 templates, 65 names | restricted environment 40 names | 25 names, 5 templates |
| File Upload Security | 5 upload points | central 5 points | 4 points, 10 layers |
| Server-Side Request Forgery | 12 outbound targets | address restriction 8 | 0 targets (name list 3) |
| Insecure Deserialization | 9 types, 11 nodes | allowlist 5 types | 0 side-effectful (open 4) |
| Authentication Implementation Errors | 8 points, 3 channels | central validator 8 | 0 points (distributed 5) |
| Session Management Errors | 7 flows | 3 flows | 4 flows |
| Broken Access Control | 14 endpoints | 9 endpoints | 5 endpoints |
| Business Logic Flaws | 625 paths | state machine 625 | 0 paths (per step 501) |
| Race Conditions | 7 checkpoints | 3 points | 4 points, 27 excess passes |
| Cryptographic Misuse | 14 call points | 7 points | 7 points, 5 flawed |
| Randomness | 9 token points | 3 points | 6 points, 5 consequential |
| Hardcoded Secrets | 5 secrets, 60 commits | pattern scan 4 | 1 secret, 198 open commits |
| Error Messages and Log Leakage | 8 endpoints, 11 items | 6 endpoints, 9 items | 2 endpoints, 2 items |
| Dependency-Related Risks | 34 packages | 8 direct names | 26 transitive, 6 unreached |
The table asks the same question seventeen times and answers it seventeen times with a different
unit: field point, output point, query point, template, upload point, outbound target, flow,
endpoint, path, checkpoint, call point, secret, log item, package. The units being different does not
mean the measure changes — the third column says the same thing on every row: not the number of
points that are vulnerable today, but the number that **could become vulnerable tomorrow**.
The middle column should not be read on its own either. On some rows, the number the pattern covers
equals the first column — the allowlist covers fifteen of fifteen field points, the state machine
covers 625 of 625 paths. On other rows it does not, and the gap shows a class of point the pattern
does not cover: the parameterized interface covers eleven of sixteen query points, because five
edge-case flows never call it at all. In the first case, the defense closed off a path; in the
second, it offered an option.
This is also where the course's rule is read from. Secure coding is not a list of rules, it is how
many points a pattern closes **on its own**. None of the rows with a zero in the table's third
column had a large rule count; in every one of them, the defense had been placed somewhere a point
could not step outside of — a type chart, a context declaration, a parameterized interface, a state
machine, a single-writer queue, a primitive call sealed inside a single file. On the rows that
are not zero, the defense had been written correctly, but it had to be revisited at every new point.
**A defense that has to be manually repeated at every new point is a defense that will eventually be
forgotten**; the table's uncovered column is today's count of that forgetting.
There are two questions this course does not ask. The first is **who counts all this coverage on
every change**: the seventeen measurements here ran on hand-built inventories, and the inventory
itself goes stale as the code changes. The second is that all of the measurements stay inside the
codebase. This last lesson's graph showed the limit of that: twenty-six components come from lines
we did not write, and a code review never reaches them. The same is true of a build's output, a
deployed image, and every layer that goes into that image; none of it was counted in this course.
**M25/K03 Security Automation and the Supply Chain** takes up both of these questions together: who
counts coverage on every change, and how the layer outside the code gets measured.