Insecure Deserialization

The previous six lessons always treated the thing coming from outside as a value: a string, a number, a byte sequence, an address. Each time, the defense was built knowing the rule of where that value would go. In some flows, what arrives is not a value: next to its fields, the data itself also carries its type, and the receiving side reads that type and builds the corresponding structure.

This lesson counts that construction moment. The unit of measure is the trusted type set: how many types are accepted, how many the allowlist closes off, and how many points are eliminated by a data format that does not read the type from within the data.

Type-Tagged Format

The model keeps a registry: the types the application recognizes and the function that builds each type. In the incoming message, the $type field says which registry entry gets called. Some of the types in the registry carry only data; some open a resource or start a job while being built.

IO15 (assumption): the side-effect marker is a declaration; no type in this lesson actually runs anything. What is measured is not whether an effect occurs, but how many types are reachable.

// format.mjs — a type-tagged deserialization format (fictional model)
// sideEffect: a type that opens a resource or starts a job while being built. Here it
// is only marked; none of them actually run anything.
export const REGISTRY = {
  Subscriber:    { sideEffect: false, build: (a) => ({ type: "Subscriber", id: a.id }) },
  Reading:       { sideEffect: false, build: (a) => ({ type: "Reading", value: a.value }) },
  Invoice:       { sideEffect: false, build: (a) => ({ type: "Invoice", amount: a.amount }) },
  WorkOrder:     { sideEffect: false, build: (a) => ({ type: "WorkOrder", id: a.id }) },
  FileSource:    { sideEffect: true,  build: (a) => ({ type: "FileSource", path: a.path }) },
  Scheduler:     { sideEffect: true,  build: (a) => ({ type: "Scheduler", every: a.every }) },
  ExternalCall:  { sideEffect: true,  build: (a) => ({ type: "ExternalCall", target: a.target }) },
  Logger:        { sideEffect: true,  build: (a) => ({ type: "Logger", channel: a.channel }) },
  Cache:         { sideEffect: true,  build: (a) => ({ type: "Cache", size: a.size }) },
};
export const ALLOWLIST = ["Subscriber", "Reading", "Invoice", "WorkOrder"];

// Incoming messages (fictional): the formats the endpoint really exchanges with the system.
export const MESSAGE = [
  ["reading upload", { $type: "Reading", value: 412 }, "expected"],
  ["subscriber record", { $type: "Subscriber", id: "A-100432", extra: { $type: "WorkOrder", id: 12 } }, "expected"],
  ["invoice summary", { $type: "Invoice", amount: 512, items: [{ $type: "Reading", value: 5 }] }, "expected"],
  ["history import", { $type: "Reading", value: 7, source: { $type: "FileSource", path: "d1" } }, "unexpected"],
  ["batch job", { $type: "WorkOrder", id: 3,
    job: { $type: "Scheduler", every: 60, step: { $type: "ExternalCall", target: "h" } } }, "unexpected"],
  ["settings migration", { $type: "Cache", size: 100 }, "unexpected"],
];

// Collects every type tag in a message, at whatever depth it appears.
export const tags = (o) =>
  o === null || typeof o !== "object" ? []
    : (Array.isArray(o) ? [] : o.$type ? [o.$type] : []).concat(Object.values(o).flatMap(tags));

The tags function walks a message recursively and collects every $type value it encounters, no matter how deep. A message is not a single decision: a node buried inside another node’s field carries its own type tag, and that tag is invisible unless something walks the whole tree.

The Accepted Type Set

// coverage.mjs — three resolution formats: explicit registry, allowlist, untagged data format
import { REGISTRY, ALLOWLIST, MESSAGE, tags } from "./format.mjs";
const FORMAT = [
  ["explicit registry", Object.keys(REGISTRY)],
  ["allowlist", ALLOWLIST],
  ["untagged data format", []],
];
const allTags = MESSAGE.flatMap(([, b]) => tags(b));
console.log(`${MESSAGE.length} messages, ${allTags.length} nodes carrying a type tag, ` +
  `${new Set(allTags).size} distinct types\n`);
console.log(`${"resolution format".padEnd(24)}${"accepted types".padStart(14)}${"nodes built".padStart(13)}` +
  `${"side-effectful builds".padStart(23)}${"accepted msgs".padStart(14)}`);
for (const [name, accepted] of FORMAT) {
  let built = 0, sideEffectful = 0, acceptedMessages = 0;
  for (const [, body] of MESSAGE) {
    const t = tags(body);
    const passes = t.every((x) => accepted.includes(x));
    if (accepted.length === 0) { acceptedMessages++; continue; }        // no structure is ever built
    if (passes) { acceptedMessages++; built += t.length; sideEffectful += t.filter((x) => REGISTRY[x].sideEffect).length; }
  }
  console.log(name.padEnd(24) + String(accepted.length).padStart(14) + String(built).padStart(13) +
    String(sideEffectful).padStart(23) + String(acceptedMessages).padStart(14));
}
const reachable = (k) => k.filter((x) => REGISTRY[x].sideEffect).length;
console.log(`\nthe registry has ${Object.keys(REGISTRY).length} types, ${reachable(Object.keys(REGISTRY))} of them side-effectful`);
console.log(`the allowlist accepts ${ALLOWLIST.length} types, ${reachable(ALLOWLIST)} of them side-effectful`);
console.log(`structure-building points in the untagged format: 0, manual mapping points: ` +
  `${MESSAGE.filter(([, , b]) => b === "expected").length}`);

6 messages, 11 nodes carrying a type tag, 8 distinct types

resolution format       accepted types  nodes built  side-effectful builds accepted msgs
explicit registry                    9           11                      4             6
allowlist                            4            5                      0             3
untagged data format                 0            0                      0             6

the registry has 9 types, 5 of them side-effectful
the allowlist accepts 4 types, 0 of them side-effectful
structure-building points in the untagged format: 0, manual mapping points: 3

The explicit registry accepts all six of the six messages and builds all eleven of the eleven nodes; four are side-effectful types. The accepted type count here is not a decision but a consequence: whatever is in the registry gets accepted.

The allowlist brings the accepted type count down to four, the built node count down to five, and zeroes out side-effectful builds. Three messages are rejected, and all three are the messages tagged as unexpected. The reason for rejection is not the message’s top-level type but a node at depth: the root of the batch job message is on the allowlist, its two child nodes are not.

The third row is the actual point of the measurement. In the untagged data format, the built node count is zero, and that is not a defense succeeding — there is nothing to build. If the type decision does not come from within the data, there is no attacker-controllable type choice either. In exchange stands the number in the last row: the application must map the incoming plain data into its own types by hand, in three separate places.

The three formats also live in different places in the codebase. The explicit registry is a convenience designed among internal services, and it produces the measured count when it stays at the outer boundary. The allowlist accepts that the type decision comes from within the data and narrows its set. The untagged data format moves the decision out of the data and leaves no type set to measure. The choice is not a matter of taste but of which boundary is being held: at the outer boundary, the first format’s measure is the size of the codebase.

The Depth at Which the Check Is Applied

The allowlist gives not the whole of coverage but only as much as where it is applied. A message is a tree, and a check can be applied to the root alone.

// depth.mjs — the depth at which the allowlist is applied
import { REGISTRY, ALLOWLIST, MESSAGE, tags } from "./format.mjs";

const root = (b) => (b && b.$type ? [b.$type] : []);
console.log(`${"where the check is applied".padEnd(28)}${"accepted msgs".padStart(14)}` +
  `${"nodes built".padStart(14)}${"side-effectful builds".padStart(23)}`);
for (const [name, sel] of [["root node only", root], ["every node", tags]]) {
  let messages = 0, built = 0, sideEffectful = 0;
  for (const [, b] of MESSAGE) {
    if (!sel(b).every((x) => ALLOWLIST.includes(x))) continue;
    messages++;
    const t = tags(b);
    built += t.length;
    sideEffectful += t.filter((x) => REGISTRY[x].sideEffect).length;
  }
  console.log(name.padEnd(28) + String(messages).padStart(14) + String(built).padStart(14) +
    String(sideEffectful).padStart(23));
}
const deep = MESSAGE.filter(([, b]) => tags(b).length > 1);
console.log(`\n${deep.length} of ${MESSAGE.length} messages carry a type tag on more than one node`);
console.log(`deepest message: ${tags(MESSAGE[4][1]).join(" -> ")}`);

where the check is applied   accepted msgs   nodes built  side-effectful builds
root node only                           5            10                      3
every node                               3             5                      0

4 of 6 messages carry a type tag on more than one node
deepest message: WorkOrder -> Scheduler -> ExternalCall

The same allowlist, when applied only to the root node, accepts five messages and builds ten nodes; three of those are side-effectful types. When applied to every node, the accepted message count drops to three, the built node count to five, and side-effectful builds to zero. It is the same list; the only thing that changes is how many nodes are seen.

The last line shows why: four of the six messages carry a type tag on more than one node, and the deepest message goes three levels down. A root-node check approves the first level of these messages and passes the rest through unseen. It is the same shape as the redirect chain in the previous lesson: coverage disappears once a step is placed between the value that is checked and the value that is used.

When a New Type Is Added

The real test of coverage begins when the codebase changes after the measurement is taken. Three new types are being added, and none of them is written with deserialization in mind.

IO16 (assumption): adding a type to the registry requires no announcement; a type enters the registry the moment it is defined. What is measured is the effect of that addition on the accepted set.

// new-type.mjs — how the accepted set changes when a new type is added to the codebase
import { REGISTRY, ALLOWLIST } from "./format.mjs";

// Three new types enter the codebase; two of them start a job while being built.
const NEW = { ReportJob: { sideEffect: true }, Notification: { sideEffect: false }, NetworkClient: { sideEffect: true } };
const REGISTRY2 = { ...REGISTRY, ...NEW };

const sideEffectful = (registry, set) => set.filter((x) => registry[x].sideEffect).length;
console.log(`${"stage".padEnd(24)}${"types in registry".padStart(18)}${"explicit accept".padStart(18)}` +
  `${"allowlist accept".padStart(19)}${"types closed off".padStart(18)}${"reachable side-effectful".padStart(25)}`);
for (const [name, r] of [["current codebase", REGISTRY], ["three new types added", REGISTRY2]]) {
  const all = Object.keys(r);
  console.log(name.padEnd(24) + String(all.length).padStart(18) + String(all.length).padStart(18) +
    String(ALLOWLIST.length).padStart(19) + String(all.length - ALLOWLIST.length).padStart(18) +
    `${sideEffectful(r, all)} / ${sideEffectful(r, ALLOWLIST)}`.padStart(25));
}
console.log(`\ncode written for the new types: 0 lines in the explicit registry (a type is accepted the moment it is defined),`);
console.log(`1 line in the allowlist (for the legitimate type), 1 mapping point in the untagged format`);

stage                    types in registry   explicit accept   allowlist accept  types closed off reachable side-effectful
current codebase                         9                 9                  4                 5                    5 / 0
three new types added                   12                12                  4                 8                    7 / 0

code written for the new types: 0 lines in the explicit registry (a type is accepted the moment it is defined),
1 line in the allowlist (for the legitimate type), 1 mapping point in the untagged format

The explicit-accept column grows together with the codebase: from nine to twelve. Reachable side-effectful types rise from five to seven, and nobody had to write anything for that increase — three types were defined, and the accepted set expanded on its own. In an explicit registry, coverage moves in the opposite direction from codebase growth.

The allowlist column is four in both rows. Types closed off rise from five to eight: the list did not change, yet its coverage increased, because coverage lies not in the list’s length but in the set it leaves uncovered. Side-effectful access is zero in both rows.

The untagged format never appears in this table, because there is no such quantity as an accepted type set there. Its cost is in the bottom line: every new type’s mapping is written by hand somewhere. The three formats do their work in three separate places — one does nothing and accepts everything, one keeps a list, one removes the type from the data entirely.

Three mapping points is today’s number, and it too grows. The untagged format’s cost is not a fixed one-time cost but one repeated per message format: when a fourth legitimate message format is added, a fourth mapping is written. The difference is that when this work is forgotten, the message does not get resolved — the gap is not a hole but a broken flow, and it is caught on the first call. On the allowlist, a forgotten line does not silently narrow or widen coverage either; coverage stays fixed unless it is added to the list.

Summary

• The unit of measure is the trusted type set. The type decision in a message is not singular: six messages contain 11 tag-carrying nodes, and the rejection decision often comes from a node at depth.
• The explicit registry accepted 9 of 9 types, built 11 nodes, and 4 were side-effectful types. The allowlist brought the accepted type count down to 4, the built node count down to 5, zeroed out side-effectful builds, and rejected 3 messages.
• In the untagged data format, the structure-building point is 0, and that is not a defense succeeding but the decision being removed from the data. Its cost is 3 manual mapping points.
• When 3 new types were added to the codebase, explicit acceptance went from 9 to 12, reachable side-effectful types went from 5 to 7, and no line was written for it. The allowlist’s acceptance stayed at 4, and the types it closed off went from 5 to 8.
• The same allowlist, applied only to the root node, accepted 5 messages, built 10 nodes, and had 3 side-effectful builds; applied to every node, these numbers become 3, 5, and 0. 4 of the six messages carry a tag on more than one node.
• Coverage lies not in the list’s length but in the set it leaves uncovered: while the list stays fixed, its coverage grows as the codebase grows.

Next Step

Across seven lessons, a single boundary was drawn: where data coming from outside enters, where it exits, which text it joins, which template it reaches, in which bytes it is stored, which address it goes to, and which type it builds. In each one, coverage was counted and the uncovered point was written down. The shared assumption behind all these measurements was that the problem lay in the data itself.

Even when the data is correct, a question remains. A subscriber ID that fully matches the contract, an invoice period with a flawless format, and a type that passes the allowlist — while all of it is valid, nobody ever asked who is making the request and whether that person has the right to this record. The next topic, Identity, Session and Authorization Errors, takes up this same question with the same measure: the information an authentication implementation leaks, the point at which a session fails to renew, and how many endpoints perform object-level checks.