--- title: 'Parallel Execution' source: 'https://academia.sh/en/courses/integration-testing/parallel-execution' course: 'Integration, Contract and End-to-End Testing' language: en updated: '2026-08-23T14:25:14+00:00' license: 'CC BY-SA 4.0' --- # Parallel Execution Measuring the time gained and the isolation lost by splitting an end-to-end team into workers; the ceiling the serial share puts on speedup, the collisions a shared fixture produces, the race defect visible only in parallel, and isolation's cost and blind spot alike. All of the previous lesson's tests ran one at a time, and the same number always stood behind the quarantine discussion: running time. The question here is no longer which test to write but whether running tests in sequence is a necessity or a choice. It is a choice, and it has a cost. Splitting a team across **workers** shortens duration; that same move also makes **test isolation** mandatory, because every sharing that was harmless while running in sequence is now used at the same time. Test isolation should not be confused with the isolation level from the Advanced SQL course: there, the rule is about whether concurrent transactions can see each other's data; here, it is about two tests not touching each other's fixture. This lesson counts both sides: the time gained and the price paid for isolation. ## Duration and Serial Share Total duration is set not only by how the work is split but also by the part that cannot be split: team setup, compilation, and reporting each run once, as a single piece, on every run. This is the **serial share** established in the Horizontal and Vertical Scaling lesson of the Introduction to System Design course; its arithmetic is not repeated here, only applied to the end-to-end team. **PC1 — forty tests: twenty at three units, twelve at six units, eight at fifteen units; a serial share of twelve units.** Durations are model units, not machine-dependent. The distribution starts from the longest test and places it on the least-loaded worker. ```js // scheduling.mjs — total duration of forty tests by worker count (model units) const durations = [ ...Array(20).fill(3), ...Array(12).fill(6), ...Array(8).fill(15), ]; const SERIAL = 12; const distribute = (workerCount) => { const load = Array(workerCount).fill(0); for (const duration of [...durations].sort((a, b) => b - a)) { const least = load.indexOf(Math.min(...load)); load[least] += duration; } return Math.max(...load); }; const totalWork = durations.reduce((a, b) => a + b, 0); const longest = Math.max(...durations); const single = SERIAL + distribute(1); const s = (n, g) => String(n).padStart(g); console.log(`${'worker'.padEnd(6)}${s('parallel', 9)}${s('total', 8)}${s('speedup', 10)}${s('serial share', 14)}`); for (const worker of [1, 2, 4, 8, 16, 40]) { const parallel = distribute(worker); const total = SERIAL + parallel; console.log(`${String(worker).padEnd(6)}${s(parallel, 9)}${s(total, 8)}` + `${s(`${(single / total).toFixed(2)}x`, 10)}${s(`%${((SERIAL / total) * 100).toFixed(1)}`, 14)}`); } console.log(`work ${totalWork} units, serial ${SERIAL} units, longest test ${longest} units, floor ${SERIAL + longest} units`); ``` ``` worker parallel total speedup serial share 1 252 264 1.00x %4.5 2 126 138 1.91x %8.7 4 63 75 3.52x %16.0 8 33 45 5.87x %26.7 16 18 30 8.80x %40.0 40 15 27 9.78x %44.4 work 252 units, serial 12 units, longest test 15 units, floor 27 units ``` The first two rows behave as expected: two workers cut duration almost in half. What follows does not. At eight workers, speedup stays at 5.87; at forty, at 9.78 — even given as many workers as tests, duration cannot drop below a tenth. Two limits are at work: the serial share holds a part that climbs to forty-four percent of the total, and no distribution can run shorter than the longest test. That is the twenty-seven-unit floor. Two tasks follow from this, and both come before adding workers: shrinking the serial share and splitting the longest test. Adding more workers, without doing either of these first, does not repay its cost. ## The Shared Fixture The price paid is isolation. The run below distributes four tests across workers and breaks each test into steps, advancing them worker by worker in turn; the interleaving that a parallel run does to tests genuinely happens here. Each test sets up a member record, reads the open-loan count, writes a loan, verifies the limit was not exceeded, and tears down its fixture. Three options are given as arguments: how many workers, whether the loan decision is made in **two steps** (read, then write) or in **one step**, and whether the fixture is shared. ```js // run.mjs — four tests run by worker count, service mode, and isolation mode import { DatabaseSync } from 'node:sqlite'; const [workerCount = '1', service = 'two-step', mode = 'shared'] = process.argv.slice(2); const WORKERS = Number(workerCount); const TEST_COUNT = 4; const LIMIT = 1; const measurement = { step: 0, setupQuery: 0, collision: 0, dropped: [] }; const setupSchema = () => { const db = new DatabaseSync(':memory:'); db.exec(`CREATE TABLE member (member_no TEXT PRIMARY KEY); CREATE TABLE loan (member_no TEXT, book_no TEXT UNIQUE)`); measurement.setupQuery += 2; return db; }; const databases = mode === 'per-worker-schema' ? Array.from({ length: WORKERS }, setupSchema) : Array(WORKERS).fill(setupSchema()); const count = (db, memberNo) => db.prepare('SELECT COUNT(*) AS n FROM loan WHERE member_no = ?').all(memberNo)[0].n; const makeTest = (no, worker) => { const db = databases[worker]; const memberNo = mode === 'shared' ? 'U-17' : `U-${no}`; const bookNo = `K-${no}`; const local = {}; const write = () => db.prepare('INSERT INTO loan VALUES (?, ?)').run(memberNo, bookNo); const steps = [ () => { measurement.setupQuery += 1; try { db.prepare('INSERT INTO member VALUES (?)').run(memberNo); } catch { measurement.collision += 1; } }, ...(service === 'two-step' ? [() => { local.n = count(db, memberNo); }, () => { if (local.n < LIMIT) write(); }] : [() => { if (count(db, memberNo) < LIMIT) write(); }]), () => { const n = count(db, memberNo); if (n > LIMIT) measurement.dropped.push(`test ${no}: open loans ${n}, limit ${LIMIT}`); }, () => { db.prepare('DELETE FROM loan WHERE book_no = ?').run(bookNo); db.prepare('DELETE FROM member WHERE member_no = ?').run(memberNo); }, ]; return { steps, i: 0 }; }; const queues = Array.from({ length: WORKERS }, () => []); for (let no = 1; no <= TEST_COUNT; no += 1) queues[(no - 1) % WORKERS].push(makeTest(no, (no - 1) % WORKERS)); while (queues.some((q) => q.length > 0)) { for (const queue of queues) { const t = queue[0]; if (t === undefined) continue; t.steps[t.i](); measurement.step += 1; t.i += 1; if (t.i === t.steps.length) queue.shift(); } } console.log(`worker ${WORKERS} service ${service} mode ${mode}`); console.log(`step ${measurement.step} setup query ${measurement.setupQuery}` + ` setup collision ${measurement.collision} dropped test ${measurement.dropped.length}/${TEST_COUNT}`); for (const line of measurement.dropped) console.log(line); ``` With one worker — that is, in sequence — the team is clean. ```bash node run.mjs 1 two-step shared ``` ``` worker 1 service two-step mode shared step 20 setup query 6 setup collision 0 dropped test 0/4 ``` ## The Test That Drops Only in Parallel Same code, same data, two workers. ```bash node run.mjs 2 two-step shared ``` ``` worker 2 service two-step mode shared step 20 setup query 6 setup collision 2 dropped test 4/4 test 1: open loans 2, limit 1 test 2: open loans 2, limit 1 test 3: open loans 2, limit 1 test 4: open loans 2, limit 1 ``` Two separate events show up together. The first is a **setup collision**: two tests tried to set up the same member record at the same time, and one of the two hit the uniqueness constraint. This is a problem belonging to the test's own fixture. The second is a product defect, and its name is the **check-then-act race**: the loan decision is made in two steps — first the open-loan count is read, then the record is written. Another worker writing between those two steps invalidates the count that was read. Both tests read zero, both wrote, and the member's limit of one became two. In a sequential run, this defect is **never** visible; the previous run's zero drops is the proof of that. This is the class parallel execution catches. The fix is to make the decision in a single, indivisible step. ```bash node run.mjs 2 one-step shared ``` ``` worker 2 service one-step mode shared step 16 setup query 6 setup collision 2 dropped test 0/4 ``` The tests turned green, but the setup collision still stands at two — and that line is a second warning. The two tests that hit the collision could not set up their own member; their loan requests hit the limit, were silently rejected, and the assertion still passed. Even while green, they did not test what they meant to test. Sharing the fixture empties a test of meaning even when it does not drop the test. ## Isolation's Cost and Blind Spot Isolation has two levels: giving each test its own record set, or setting up a separate schema for each worker. Both are run with the same defect present, the two-step service. ```bash node run.mjs 2 two-step per-test-records ``` ``` worker 2 service two-step mode per-test-records step 20 setup query 6 setup collision 0 dropped test 0/4 ``` ```bash node run.mjs 2 two-step per-worker-schema ``` ``` worker 2 service two-step mode per-worker-schema step 20 setup query 8 setup collision 0 dropped test 0/4 ``` The cost shows up in two numbers. Per-test records bring the collision to zero and add nothing; per-worker schema raises the setup query count from six to eight, because each worker creates its own tables. That gap grows linearly as the worker count grows: schema setup is paid per worker, record sets are paid per test. The class that escapes is right here too. Both runs are green, yet **the defect stands in place** — the service still decides in two steps. Isolation removed the race's precondition, because it kept two tests from touching the same member at the same time. While it removes flakiness, isolation also makes invisible the defect class tied to the very sharing that produced the flakiness. Real users share records; as long as the test team does not, that race shows up only in production. So the isolation decision is a balance to strike: the run order's isolation must be **complete**, and the product's concurrency handling must be tested **separately**. The run-dependent side of the cost is small here — each of the five configurations ran in under a millisecond in this run, because what runs is an in-process model. The run-independent numbers are in the measurements: twenty steps, six to eight setup queries, two colliding records, and a serial share climbing from 4.5 percent to 44.4 percent. ## Summary - Adding workers shortens duration only down to the serial share and the longest test: with forty tests, even forty workers gave only a 9.78x speedup, and the floor stayed at twenty-seven units. - The shared fixture produced two setup collisions in parallel; even when a colliding test does not drop, it fails to test what it meant to test. - The check-then-act race showed up only under parallel execution: zero drops in the sequential run, all four of four tests dropped with two workers. - Reducing the decision to one step turned the tests green; isolation, by contrast, turned the run green while the same defect stayed in place. - Isolation's cost is measurable: per-test records did not raise the setup query count, per-worker schema raised it from six to eight, and that gap grows per worker. ## Course Wrap-Up This course's fifteen lessons answered the same three questions. Every row in the table states what a test form sees, what it cannot see, and what it costs. | Lesson | Defect Class Caught | Missed | Cost | |---|---|---|---| | The Scope of Integration Testing | schema mismatch | recipient identity mismatch | 4 queries per round, 9-line fixture | | Test Environment Management | rejection of an unknown recipient | process-lifetime state | 1 process, 6 steps, 1 file per run | | Test Data Management | distribution-dependent report defect | content-dependent export defect | 772 load queries, 78 kilobytes | | Database Testing | data loss in the down migration | backfill value mismatch | 13 migration steps, 21 queries, 1 file | | Mocking External Services | request shape drift | the frozen response going stale | 0 processes, 0 network calls on replay | | API Testing | structural mismatch | semantic mismatch | 2 processes, at least 1 request per test, 16 hand-maintained rules | | Consumer-Driven Contract Testing | the loss of a field written into the contract | a field read by a branch that never ran | 2 components, 100 requests, a 241-byte file | | Schema Compatibility Checking | breaking change | a change that alters meaning without breaking shape | 0 processes, 0 requests, a 9-row class table | | Developing With Mock Servers | shape deviation | state deviation | 2 processes, 10 requests, a 15-line fixture | | The Test Pyramid | boundary comparison, at the unit level | status-code mapping, at the upper levels | 242 queries and 24 requests for 24 cases | | Browser Automation | the card status not updating | the mislabeled button | 16,477 polls across 2,000 loads | | Mobile Automation | version-dependent date parsing | 24/30 of cell-specific defects | 12 cell runs, a 30-cell matrix | | Visual Verification | the button tone lightening | an inactive button producing the same pixels | 800 pixels per image, 784 counted with exclusion | | Flakiness Management | the missing warning on a rejected request | the class a quarantined test was protecting | 40 tests × 200 runs, 8,000 test executions | | Parallel Execution | the check-then-act race | the sharing defect isolation hides | setup queries from 6 to 8, serial share from 4.5% to 44.4% | The middle column is the course's rule: a test layer's value can only be told together with its **limit**. In none of the fifteen rows is there a "safer" option; every row has a class caught, a class missed, and a cost paid. The only way to defend a test form is to write out this triple, and the only justification for adding a form is a defect class with **no counterpart in any other layer**. The course's own limit is written by the same rule. The first topic asked whether test doubles matched reality, the second separated out the checkable part of a contract, the third ran the path the user sees as a whole. All three asked the same question: **does the function work correctly?** How the system behaves under load, what it gives up under attack, and how it degrades under failure were tested in no lesson here. The Non-Functional Testing course takes on these three questions.