Parallel Execution

All of the previous lesson’s tests ran one at a time, and the same number always stood behind the quarantine discussion: running time. The question here is no longer which test to write but whether running tests in sequence is a necessity or a choice.

It is a choice, and it has a cost. Splitting a team across workers shortens duration; that same move also makes test isolation mandatory, because every sharing that was harmless while running in sequence is now used at the same time. Test isolation should not be confused with the isolation level from the Advanced SQL course: there, the rule is about whether concurrent transactions can see each other’s data; here, it is about two tests not touching each other’s fixture. This lesson counts both sides: the time gained and the price paid for isolation.

Duration and Serial Share

Total duration is set not only by how the work is split but also by the part that cannot be split: team setup, compilation, and reporting each run once, as a single piece, on every run. This is the serial share established in the Horizontal and Vertical Scaling lesson of the Introduction to System Design course; its arithmetic is not repeated here, only applied to the end-to-end team.

PC1 — forty tests: twenty at three units, twelve at six units, eight at fifteen units; a serial share of twelve units. Durations are model units, not machine-dependent. The distribution starts from the longest test and places it on the least-loaded worker.

// scheduling.mjs — total duration of forty tests by worker count (model units)
const durations = [
  ...Array(20).fill(3), ...Array(12).fill(6), ...Array(8).fill(15),
];
const SERIAL = 12;

const distribute = (workerCount) => {
  const load = Array(workerCount).fill(0);
  for (const duration of [...durations].sort((a, b) => b - a)) {
    const least = load.indexOf(Math.min(...load));
    load[least] += duration;
  }
  return Math.max(...load);
};

const totalWork = durations.reduce((a, b) => a + b, 0);
const longest = Math.max(...durations);
const single = SERIAL + distribute(1);
const s = (n, g) => String(n).padStart(g);
console.log(`${'worker'.padEnd(6)}${s('parallel', 9)}${s('total', 8)}${s('speedup', 10)}${s('serial share', 14)}`);
for (const worker of [1, 2, 4, 8, 16, 40]) {
  const parallel = distribute(worker);
  const total = SERIAL + parallel;
  console.log(`${String(worker).padEnd(6)}${s(parallel, 9)}${s(total, 8)}`
    + `${s(`${(single / total).toFixed(2)}x`, 10)}${s(`%${((SERIAL / total) * 100).toFixed(1)}`, 14)}`);
}
console.log(`work ${totalWork} units, serial ${SERIAL} units, longest test ${longest} units, floor ${SERIAL + longest} units`);

worker parallel   total   speedup  serial share
1           252     264     1.00x          %4.5
2           126     138     1.91x          %8.7
4            63      75     3.52x         %16.0
8            33      45     5.87x         %26.7
16           18      30     8.80x         %40.0
40           15      27     9.78x         %44.4
work 252 units, serial 12 units, longest test 15 units, floor 27 units

The first two rows behave as expected: two workers cut duration almost in half. What follows does not. At eight workers, speedup stays at 5.87; at forty, at 9.78 — even given as many workers as tests, duration cannot drop below a tenth. Two limits are at work: the serial share holds a part that climbs to forty-four percent of the total, and no distribution can run shorter than the longest test. That is the twenty-seven-unit floor.

Two tasks follow from this, and both come before adding workers: shrinking the serial share and splitting the longest test. Adding more workers, without doing either of these first, does not repay its cost.

The Shared Fixture

The price paid is isolation. The run below distributes four tests across workers and breaks each test into steps, advancing them worker by worker in turn; the interleaving that a parallel run does to tests genuinely happens here. Each test sets up a member record, reads the open-loan count, writes a loan, verifies the limit was not exceeded, and tears down its fixture.

Three options are given as arguments: how many workers, whether the loan decision is made in two steps (read, then write) or in one step, and whether the fixture is shared.

// run.mjs — four tests run by worker count, service mode, and isolation mode
import { DatabaseSync } from 'node:sqlite';

const [workerCount = '1', service = 'two-step', mode = 'shared'] = process.argv.slice(2);
const WORKERS = Number(workerCount);
const TEST_COUNT = 4;
const LIMIT = 1;
const measurement = { step: 0, setupQuery: 0, collision: 0, dropped: [] };

const setupSchema = () => {
  const db = new DatabaseSync(':memory:');
  db.exec(`CREATE TABLE member (member_no TEXT PRIMARY KEY);
           CREATE TABLE loan (member_no TEXT, book_no TEXT UNIQUE)`);
  measurement.setupQuery += 2;
  return db;
};

const databases = mode === 'per-worker-schema'
  ? Array.from({ length: WORKERS }, setupSchema)
  : Array(WORKERS).fill(setupSchema());

const count = (db, memberNo) => db.prepare('SELECT COUNT(*) AS n FROM loan WHERE member_no = ?').all(memberNo)[0].n;

const makeTest = (no, worker) => {
  const db = databases[worker];
  const memberNo = mode === 'shared' ? 'U-17' : `U-${no}`;
  const bookNo = `K-${no}`;
  const local = {};
  const write = () => db.prepare('INSERT INTO loan VALUES (?, ?)').run(memberNo, bookNo);
  const steps = [
    () => {
      measurement.setupQuery += 1;
      try { db.prepare('INSERT INTO member VALUES (?)').run(memberNo); } catch { measurement.collision += 1; }
    },
    ...(service === 'two-step'
      ? [() => { local.n = count(db, memberNo); }, () => { if (local.n < LIMIT) write(); }]
      : [() => { if (count(db, memberNo) < LIMIT) write(); }]),
    () => {
      const n = count(db, memberNo);
      if (n > LIMIT) measurement.dropped.push(`test ${no}: open loans ${n}, limit ${LIMIT}`);
    },
    () => {
      db.prepare('DELETE FROM loan WHERE book_no = ?').run(bookNo);
      db.prepare('DELETE FROM member WHERE member_no = ?').run(memberNo);
    },
  ];
  return { steps, i: 0 };
};

const queues = Array.from({ length: WORKERS }, () => []);
for (let no = 1; no <= TEST_COUNT; no += 1) queues[(no - 1) % WORKERS].push(makeTest(no, (no - 1) % WORKERS));

while (queues.some((q) => q.length > 0)) {
  for (const queue of queues) {
    const t = queue[0];
    if (t === undefined) continue;
    t.steps[t.i]();
    measurement.step += 1;
    t.i += 1;
    if (t.i === t.steps.length) queue.shift();
  }
}

console.log(`worker ${WORKERS}  service ${service}  mode ${mode}`);
console.log(`step ${measurement.step}  setup query ${measurement.setupQuery}`
  + `  setup collision ${measurement.collision}  dropped test ${measurement.dropped.length}/${TEST_COUNT}`);
for (const line of measurement.dropped) console.log(line);

With one worker — that is, in sequence — the team is clean.

node run.mjs 1 two-step shared

worker 1  service two-step  mode shared
step 20  setup query 6  setup collision 0  dropped test 0/4

The Test That Drops Only in Parallel

Same code, same data, two workers.

node run.mjs 2 two-step shared

worker 2  service two-step  mode shared
step 20  setup query 6  setup collision 2  dropped test 4/4
test 1: open loans 2, limit 1
test 2: open loans 2, limit 1
test 3: open loans 2, limit 1
test 4: open loans 2, limit 1

Two separate events show up together. The first is a setup collision: two tests tried to set up the same member record at the same time, and one of the two hit the uniqueness constraint. This is a problem belonging to the test’s own fixture.

The second is a product defect, and its name is the check-then-act race: the loan decision is made in two steps — first the open-loan count is read, then the record is written. Another worker writing between those two steps invalidates the count that was read. Both tests read zero, both wrote, and the member’s limit of one became two. In a sequential run, this defect is never visible; the previous run’s zero drops is the proof of that. This is the class parallel execution catches.

The fix is to make the decision in a single, indivisible step.

node run.mjs 2 one-step shared

worker 2  service one-step  mode shared
step 16  setup query 6  setup collision 2  dropped test 0/4

The tests turned green, but the setup collision still stands at two — and that line is a second warning. The two tests that hit the collision could not set up their own member; their loan requests hit the limit, were silently rejected, and the assertion still passed. Even while green, they did not test what they meant to test. Sharing the fixture empties a test of meaning even when it does not drop the test.

Isolation’s Cost and Blind Spot

Isolation has two levels: giving each test its own record set, or setting up a separate schema for each worker. Both are run with the same defect present, the two-step service.

node run.mjs 2 two-step per-test-records

worker 2  service two-step  mode per-test-records
step 20  setup query 6  setup collision 0  dropped test 0/4

node run.mjs 2 two-step per-worker-schema

worker 2  service two-step  mode per-worker-schema
step 20  setup query 8  setup collision 0  dropped test 0/4

The cost shows up in two numbers. Per-test records bring the collision to zero and add nothing; per-worker schema raises the setup query count from six to eight, because each worker creates its own tables. That gap grows linearly as the worker count grows: schema setup is paid per worker, record sets are paid per test.

The class that escapes is right here too. Both runs are green, yet the defect stands in place — the service still decides in two steps. Isolation removed the race’s precondition, because it kept two tests from touching the same member at the same time. While it removes flakiness, isolation also makes invisible the defect class tied to the very sharing that produced the flakiness. Real users share records; as long as the test team does not, that race shows up only in production.

So the isolation decision is a balance to strike: the run order’s isolation must be complete, and the product’s concurrency handling must be tested separately. The run-dependent side of the cost is small here — each of the five configurations ran in under a millisecond in this run, because what runs is an in-process model. The run-independent numbers are in the measurements: twenty steps, six to eight setup queries, two colliding records, and a serial share climbing from 4.5 percent to 44.4 percent.

Summary

• Adding workers shortens duration only down to the serial share and the longest test: with forty tests, even forty workers gave only a 9.78x speedup, and the floor stayed at twenty-seven units.
• The shared fixture produced two setup collisions in parallel; even when a colliding test does not drop, it fails to test what it meant to test.
• The check-then-act race showed up only under parallel execution: zero drops in the sequential run, all four of four tests dropped with two workers.
• Reducing the decision to one step turned the tests green; isolation, by contrast, turned the run green while the same defect stayed in place.
• Isolation’s cost is measurable: per-test records did not raise the setup query count, per-worker schema raised it from six to eight, and that gap grows per worker.

Course Wrap-Up

This course’s fifteen lessons answered the same three questions. Every row in the table states what a test form sees, what it cannot see, and what it costs.

Lesson	Defect Class Caught	Missed	Cost
The Scope of Integration Testing	schema mismatch	recipient identity mismatch	4 queries per round, 9-line fixture
Test Environment Management	rejection of an unknown recipient	process-lifetime state	1 process, 6 steps, 1 file per run
Test Data Management	distribution-dependent report defect	content-dependent export defect	772 load queries, 78 kilobytes
Database Testing	data loss in the down migration	backfill value mismatch	13 migration steps, 21 queries, 1 file
Mocking External Services	request shape drift	the frozen response going stale	0 processes, 0 network calls on replay
API Testing	structural mismatch	semantic mismatch	2 processes, at least 1 request per test, 16 hand-maintained rules
Consumer-Driven Contract Testing	the loss of a field written into the contract	a field read by a branch that never ran	2 components, 100 requests, a 241-byte file
Schema Compatibility Checking	breaking change	a change that alters meaning without breaking shape	0 processes, 0 requests, a 9-row class table
Developing With Mock Servers	shape deviation	state deviation	2 processes, 10 requests, a 15-line fixture
The Test Pyramid	boundary comparison, at the unit level	status-code mapping, at the upper levels	242 queries and 24 requests for 24 cases
Browser Automation	the card status not updating	the mislabeled button	16,477 polls across 2,000 loads
Mobile Automation	version-dependent date parsing	24/30 of cell-specific defects	12 cell runs, a 30-cell matrix
Visual Verification	the button tone lightening	an inactive button producing the same pixels	800 pixels per image, 784 counted with exclusion
Flakiness Management	the missing warning on a rejected request	the class a quarantined test was protecting	40 tests × 200 runs, 8,000 test executions
Parallel Execution	the check-then-act race	the sharing defect isolation hides	setup queries from 6 to 8, serial share from 4.5% to 44.4%

The middle column is the course’s rule: a test layer’s value can only be told together with its limit. In none of the fifteen rows is there a “safer” option; every row has a class caught, a class missed, and a cost paid. The only way to defend a test form is to write out this triple, and the only justification for adding a form is a defect class with no counterpart in any other layer.

The course’s own limit is written by the same rule. The first topic asked whether test doubles matched reality, the second separated out the checkable part of a contract, the third ran the path the user sees as a whole. All three asked the same question: does the function work correctly? How the system behaves under load, what it gives up under attack, and how it degrades under failure were tested in no lesson here. The Non-Functional Testing course takes on these three questions.