Race Conditions

The previous lesson’s entire measurement rested on a single assumption: the steps run one after another. When the state machine reached the payment step, it read the state, made its decision, and updated the state. It was assumed that nothing else happened in the time between the read and the write.

This assumption falls apart when two requests arrive at the same time. Both can read the state as approved, both can pass the check, and both can write. The rule is written in the code, the check runs, and the result is still wrong. This is a race condition: the flaw is not in the check’s existence, but in the gap between the check and the effect.

Stepping Between the Three Steps

Every request that changes a shared value is three steps: read, check, write. Another request stepping in between this trio means the second request reads the value the first has not yet written. Both requests decide by looking at the old value, and both decisions come out positive.

The measurement below is a model: there is no real server, no real client, and no real network. The timing is not real; requests are interleaved step by step and run in a scheduler we wrote ourselves, which is why the output is the same on every run. Attempting to bypass a rule by sending concurrent requests to a service is both illegal and unprofessional without written authorization and a defined scope; what is measured here is the behavior of our own model.

// race/model.mjs — concurrent-request model (model): no real server, no real client and no
// real network. The timing is not real, it is produced by step-by-step interleaving; that is
// why the output is the same on every run.

// IE17: every request is three steps — read the shared state, decide, write. The race
// condition is born when another request steps in between these three.
export function* request(name, state, setup) {
  const local = state.remaining;                 // 1. read
  yield "read";
  const passed = local > 0;                      // 2. check
  yield "check";
  if (!passed) return { name, result: "rejected" };
  if (setup === "queue") return { name, result: "queued" };
  if (setup === "atomic" && state.remaining !== local) return { name, result: "collision" };
  state.remaining = local - 1;                   // 3. write
  return { name, result: "passed" };
}

// IE18: four setups run the same set of requests. "unprotected" interleaves every request
// step by step; "lock" serializes the read-check-write trio; "atomic" retests the value right
// before writing; "queue" leaves the write to a single-writer queue, and the queue rechecks.
export function run(concurrent, setup, quota = 1) {
  const state = { remaining: quota }, results = [];
  const generators = Array.from({ length: concurrent },
    (_, i) => request(`r-${i + 1}`, state, setup));
  if (setup === "lock") {
    for (const g of generators) { let r; while (!(r = g.next()).done); results.push(r.value); }
  } else {
    const open = generators.map((g) => ({ g, finished: false }));
    while (open.some((a) => !a.finished))
      for (const a of open) {
        if (a.finished) continue;
        const r = a.g.next();
        if (r.done) { a.finished = true; results.push(r.value); }
      }
  }
  if (setup === "queue")
    for (const s of results)
      if (s.result === "queued")
        s.result = state.remaining > 0 ? (state.remaining--, "passed") : "rejected";
  const passed = results.filter((s) => s.result === "passed").length;
  return { passed, excess: Math.max(0, passed - quota), remaining: state.remaining };
}

// IE19: a checkpoint is any place that reads, checks, and writes a shared value.
export const CHECKPOINT = [
  { name: "use-discount-code",           quota: 1, setup: "unprotected" },
  { name: "refund-pay",                  quota: 1, setup: "unprotected" },
  { name: "decrement-quota",             quota: 3, setup: "lock" },
  { name: "approve-subscriber-transfer", quota: 1, setup: "lock" },
  { name: "assign-work-order",           quota: 1, setup: "unprotected" },
  { name: "lock-invoice",                quota: 1, setup: "atomic" },
  { name: "record-meter-reading",        quota: 2, setup: "unprotected" },
];

// race/measurement.mjs — how many times the same check passes, and how many of them a lock
// and a single-writer queue close.
import { run, CHECKPOINT } from "./model.mjs";
const write = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const SETUP = ["unprotected", "lock", "atomic", "queue"];

const A = [-11, 13, 8, 9, 9];
console.log("1. a single-use discount code (quota 1): how many requests pass the check");
write(A, "concurrent", ...SETUP);
for (const n of [1, 2, 4, 8, 16]) write(A, n, ...SETUP.map((k) => run(n, k).passed));

const B = [-11, 13, 8, 9, 9];
console.log("\n2. the same measurement with quota 3 (at most 3 requests should pass)");
write(B, "concurrent", ...SETUP);
for (const n of [1, 2, 4, 8, 16]) write(B, n, ...SETUP.map((k) => run(n, k, 3).passed));

// IE20: an excess-passing request is the number of requests that pass above the quota; this
// is what is measured.
const CONCURRENT = 8;
const C = [-30, 7, 13, 9, 9, 11];
console.log(`\n3. ${CHECKPOINT.length} checkpoints, ${CONCURRENT} concurrent requests at each`);
write(C, "checkpoint", "quota", "setup", "passed", "excess", "queue");
let excessTotal = 0, queueTotal = 0;
for (const n of CHECKPOINT) {
  const s = run(CONCURRENT, n.setup, n.quota), q = run(CONCURRENT, "queue", n.quota);
  excessTotal += s.excess; queueTotal += q.excess;
  write(C, n.name, n.quota, n.setup, s.passed, s.excess, q.excess);
}
write(C, "total", "", "", "", excessTotal, queueTotal);

// 4. coverage: when a new checkpoint is added
const NEW = { name: "create-payment-plan", quota: 1, setup: "unprotected" };
const extended = [...CHECKPOINT, NEW];
const excess = (set, central) => set.reduce((t, n) =>
  t + run(CONCURRENT, central ? "queue" : n.setup, n.quota).excess, 0);
const D = [-42, 13, 12, 14, 13];
console.log("\n4. coverage: unprotected checkpoints and excess-passing requests");
write(D, "set", "checkpoints", "unprotected", "excess passes", "uncovered %");
for (const [name, k, central] of [
  ["today's distributed state", CHECKPOINT, false],
  ["single-writer queue at every checkpoint", CHECKPOINT, true],
  ["new checkpoint added, distributed", extended, false],
  ["new checkpoint added, central pattern", extended, true],
]) {
  const open = central ? 0 : k.filter((n) => n.setup === "unprotected").length;
  write(D, name, k.length, open, excess(k, central), ((open / k.length) * 100).toFixed(1));
}

1. a single-use discount code (quota 1): how many requests pass the check
concurrent   unprotected    lock   atomic    queue
1                      1       1        1        1
2                      2       1        1        1
4                      4       1        1        1
8                      8       1        1        1
16                    16       1        1        1

2. the same measurement with quota 3 (at most 3 requests should pass)
concurrent   unprotected    lock   atomic    queue
1                      1       1        1        1
2                      2       2        1        2
4                      4       3        1        3
8                      8       3        1        3
16                    16       3        1        3

3. 7 checkpoints, 8 concurrent requests at each
checkpoint                      quota        setup   passed   excess      queue
use-discount-code                   1  unprotected        8        7          0
refund-pay                          1  unprotected        8        7          0
decrement-quota                     3         lock        3        0          0
approve-subscriber-transfer         1         lock        1        0          0
assign-work-order                   1  unprotected        8        7          0
lock-invoice                        1       atomic        1        0          0
record-meter-reading                2  unprotected        8        6          0
total                                                             27          0

4. coverage: unprotected checkpoints and excess-passing requests
set                                         checkpoints unprotected excess passes  uncovered %
today's distributed state                             7           4            27         57.1
single-writer queue at every checkpoint               7           0             0          0.0
new checkpoint added, distributed                     8           5            34         62.5
new checkpoint added, central pattern                 8           0             0          0.0

The numbers belong to the measurement class; their inputs are IE17–IE20.

The Check Passes as Often as There Is Concurrency

The first table measures a single-use discount code: quota 1, meaning at most one request should pass. Under the unprotected setup, the number of passing requests equals the number of concurrent requests — 2 requests pass 2 times, 16 requests pass 16 times. The check genuinely ran on every request and gave a positive result on every request, because all of them read before writing.

This does not mean the check is ineffective; it means it does not scale. A test run with a single request can never see this flaw: in the first row, all four setups give 1. The flaw only becomes visible once concurrency is a variable.

The lock and the single-writer queue both give 1 at every concurrency level. The two reach the same result by two different routes: the lock serializes the read-check-write trio, the queue leaves reading free and gathers only the writing into a single writer, rechecking at the moment of writing. The queue’s distinguishing feature is that slow reads do not hold a lock.

The lock has a cost too, and that cost is invisible in this measurement. Because the lock serializes the critical section, it increases the number of waiting requests as concurrency grows; if a read inside the critical section is slow, every request waits for that read in turn. The passed column in the table does not measure this waiting time, it only measures correctness. The two columns have to be kept separate: on the correctness column the lock and the queue give the same result, on the latency column they do not.

The Cost of Defense: Collision

The second table repeats the same measurement with quota 3 and exposes the cost of a defense. The unprotected setup again lets through as many as there is concurrency; the lock and the queue stop at 3. The atomic compare-and-update, however, gives 1.

The reason is the implementation in the model: if the value has changed by the time of writing, the request is counted as a collision and rejected, and it is not retried. With a quota of 3, three requests should pass, but only one does; two legitimate requests are lost. The compare-and-update blocks excess passes, but when used without a retry it also stops part of the legitimate work.

This is an example of this curriculum’s rule: a defense whose cost is not written down is not counted as measured. The correct form of the compare-and-update is to retry a colliding request up to a set limit; the limit is itself a decision, and it determines how many requests will be dropped under load.

The difference between the first table’s quota-1 result and the second table’s quota-3 result is itself a warning. At quota 1, the atomic setup gives the correct result and the flaw is invisible; the flaw only surfaces once the quota rises above 1. A defense behaving correctly in a single configuration does not mean it behaves correctly in every configuration. The measurement set has to include at least two quota values.

Twenty-Seven Excess Passes at Seven Checkpoints

The third table carries the measurement from a single piece of code to the codebase. There are seven checkpoints; each of them reads, checks, and writes a shared value. Four are unprotected, two are locked, one is atomic.

Eight concurrent requests run at each checkpoint, and the excess-passing request count is recorded: requests that pass above the quota. The four unprotected checkpoints produce 7, 7, 7, and 6 excess passes respectively; the total is 27. The three protected checkpoints produce 0.

The difference between the numbers comes from the quota. Because the record-meter-reading checkpoint has a quota of 2, six of the eight requests are excess; at checkpoints with a quota of 1, it is seven. The excess-pass count is therefore measured not by the number of checkpoints, but by the gap between the quota and the concurrency. The sentence “four of our checkpoints are unprotected” does not give you the number 27.

The last column shows what the same checkpoints give when run through a single-writer queue: 0 at all seven of the seven checkpoints. The closing happens in a single place; a separate lock was not designed for each checkpoint.

Which three checkpoints are protected is also meaningful. decrement-quota and approve-subscriber-transfer are locked, because both are checkpoints that change money or subscription state and were thought through carefully. The unprotected four are checkpoints that decrement a counter, mark a record, or make an assignment; none of them look “critical.” The place where race conditions are found is not the checkpoint deemed critical, but the checkpoint that reads and writes a shared value. The inventory’s criterion should be exactly this: not criticality, but shared writing.

Coverage and the Eighth Checkpoint

The fourth table records coverage. In today’s distributed state, four of the seven checkpoints are unprotected — the uncovered share is 57.1% — and the total excess passes are 27. When every write goes through the single-writer queue, the unprotected checkpoint count is 0 and the excess passes are 0.

If an eighth checkpoint — create-payment-plan — is added unprotected under the distributed setup, the excess passes rise from 27 to 34, and the uncovered share rises to 62.5%. Under the central pattern, the same checkpoint produces no excess passes at all; its write path already goes through the queue.

The difference here, too, is not in the strength of the pattern, but in the write being done through a single path. Under the distributed setup, the question “who else is writing this value” has to be asked again for every new checkpoint; with the queue, the question has been asked once. Concurrency is the hardest class of flaw to see in code review: the code is correct when read on its own, and is only wrong when two copies of it run at the same time.

This is why the measurement itself is a form of testing. Because the scheduler above does not use real timing, its result is the same on every run; the same interleaving can be applied to every checkpoint, and the excess-pass count can be recorded as a number. A test left to real timing, on the other hand, sometimes passes and sometimes fails, and passing proves nothing. The existence of a race condition is counted as demonstrated if it can be shown in an interleaving we wrote ourselves; if it cannot, what is known is not its absence, but that it did not show up in that interleaving.

Summary

• A race condition is born not in the absence of a check, but in the gap between the check and the effect; a second request that steps in between the read-check-write trio reads the old value.
• For a single-use quota, the unprotected setup produces as many passes as there is concurrency: 16 concurrent requests pass 16 times. A test run with a single request can never see this flaw.
• The lock and the single-writer queue protect the quota at every level; the queue’s distinguishing feature is that reads do not hold a lock.
• A retry-less atomic compare-and-update, with a quota of 3, lets only 1 request through; while it closes excess passes, it also rejects two legitimate requests. A defense whose cost is not written down has not been measured.
• Four of seven checkpoints are unprotected; at eight concurrent requests they produce a total of 27 excess passes. The excess-pass count is measured not by the number of checkpoints, but by the gap between the quota and the concurrency.
• The single-writer queue brings excess passes down to 0; when an eighth unprotected checkpoint is added, excess passes under the distributed setup rise to 34 while the central pattern stays at 0.

Next Step

This topic measured a single question across five lessons: at how many points in the code is who-can-do-what checked, and how many points are left uncovered. At authentication points, leaked information was counted; in session flows, identity that is not renewed and sessions that keep living were counted; in the authorization matrix, endpoints without object-level control and wrongly-passing requests were counted; in flow order, rule-breaking paths were counted; and in this lesson, the excess passes produced by concurrent requests were counted. The same shape came out in all five measurements: the distributed defense left points uncovered in every set, the central pattern gave zero on the same set, and every time a new item was added to a set, what the distributed side left uncovered grew, while the pattern’s did not change.

One thing all these checks rest on, though, was never addressed. Authentication compared a value, the session identifier was generated from somewhere, the authorization criterion trusted a signature or a record, the queue operated with an identity. How these values are generated, how predictable they are, and where in the codebase they sit was never asked — keys and secrets still sit somewhere inside the code. The next topic, Secrets in Code, takes up this question with the same coverage measure: how many secrets there are, how many are outside the codebase, and how many are left behind in the repository history.