Delivery Models and Testing

Up to this point, testing has been treated as an activity where one person is alone with one function. In reality, when a test is written, who writes it, and how quickly its result reaches someone depends on the way of working. The same check suite carries different value when it is run once at the end of a release versus when it is run on every change.

This lesson makes that difference measurable. What gets measured is not testing itself, but the distance between a defect’s occurrence and its detection.

The Delivery Model Determines Where Testing Sits

In the waterfall model, work is divided into stages, and each stage hands a document to the next: analysis, design, development, testing, delivery. Testing here is a stage — performed at the end, on a product already finished beforehand. The model’s strength is that every stage leaves its own output in writing; its weakness is that the validation question gets deferred to the very end.

In the incremental and iterative model, the product grows in working pieces. Each iteration contains its own analysis, development, and testing. Testing turns from a stage into an activity distributed inside every iteration.

In continuous delivery, every change is considered a deliverable candidate; the only obstacle in front of delivery is the result of the automated check suite. Here testing stops being merely an activity and turns into a gate: a change whose suite does not pass does not move forward.

The V-model pairing mentioned in the second lesson holds in all three models — every decision level has a corresponding testing level. What changes is not the pairing but the time between its two ends.

The Feedback Point

A defect is born the moment the code is written. It is noticed at the first point where a suite runs against it. The distance between the two has two measures: delay, the time between the defect being born and word of it arriving; and candidate change count, the number of changes accumulated in that time that could be the defect’s source.

// feedback.mjs — the same check suite's feedback at different points
const POINTS = [
  { name: 'at commit', delayMin: 0.5, candidates: 1 },
  { name: 'before push', delayMin: 3, candidates: 4 },
  { name: 'integration', delayMin: 20, candidates: 12 },
  { name: 'nightly run', delayMin: 720, candidates: 40 },
  { name: 'before release', delayMin: 20160, candidates: 380 },
];

for (const p of POINTS) {
  console.log(`${p.name.padEnd(16)}: ${String(p.delayMin).padStart(7)} min delay, ${String(p.candidates).padStart(3)} candidate changes`);
}

const first = POINTS[0];
const last = POINTS[POINTS.length - 1];
console.log(`ratio: delay ${last.delayMin / first.delayMin}x, candidates ${last.candidates / first.candidates}x`);

at commit       :     0.5 min delay,   1 candidate changes
before push     :       3 min delay,   4 candidate changes
integration     :      20 min delay,  12 candidate changes
nightly run     :     720 min delay,  40 candidate changes
before release  :   20160 min delay, 380 candidate changes
ratio: delay 40320x, candidates 380x

The numbers in the table are assumptions representing a library team’s working tempo; they are not measured values but model inputs. The conclusion that comes out of the model, though, does not depend much on the numbers: as the feedback point shifts later, delay and candidate count grow together.

The effect of the two kinds of growth is different. Delay causes the person who wrote the defect to lose context; the rationale for a decision made two weeks ago is not remembered. Candidate count, on the other hand, grows the isolation work: with a single change, the problem is in it; with three hundred and eighty changes, the problem is somewhere among them, and finding it is a separate search problem.

This is where the waterfall model’s practical cost lies. When the testing stage is concentrated at a single point at the end of a release, every defect surfaces at the most expensive feedback point.

A Suite’s Duration Determines Its Placement

The feedback point cannot be chosen freely. If a check suite is to run on every commit, its duration must be under a second; as the suite’s duration grows, its point shifts later. Duration here is a placement criterion.

// wait.mjs — a blocking wait that simulates access to an external resource
export function wait(ms) {
  Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
}

// suite-duration.mjs — the duration of two check suites and which feedback point they belong to
import assert from 'node:assert/strict';
import { wait } from './wait.mjs';

const fee = (day) => Math.min(Math.max(day - 3, 0) * 2, 20);

const FAST = [
  () => assert.equal(fee(0), 0),
  () => assert.equal(fee(3), 0),
  () => assert.equal(fee(4), 2),
  () => assert.equal(fee(13), 20),
  () => assert.equal(fee(99), 20),
];

const SLOW = [
  () => { wait(400); assert.equal(fee(10), 14); },
  () => { wait(400); assert.equal(fee(20), 20); },
  () => { wait(400); assert.equal(fee(1), 0); },
];

const THRESHOLD_MS = 1000;
for (const [name, suite] of [['fast', FAST], ['slow', SLOW]]) {
  const start = performance.now();
  for (const check of suite) check();
  const duration = performance.now() - start;
  const location = duration < THRESHOLD_MS ? 'can run on every commit' : 'moves to before push';
  console.log(`${name} suite (${suite.length} checks): ${THRESHOLD_MS} ms threshold — ${location}`);
}

fast suite (5 checks): 1000 ms threshold — can run on every commit
slow suite (3 checks): 1000 ms threshold — moves to before push

The rule the two suites test is the same; where they differ is that every check in the slow suite contains a wait that simulates access to an external resource. The five-check suite is faster than the three-check suite, because it is not the check count but the external dependency that decides. Because this measurement uses a blocking wait, it gives the same verdict on every machine; only the raw duration is environment-dependent.

The rule that emerges is practical: a suite’s feedback point is determined not by its check count but by its slowest dependency. If you want to keep part of a suite fast, the external dependency has to be removed from it. This is exactly the subject of the next course.

The Practices the Model Brings

The delivery model determines not only timing but also which testing practices are necessary.

In a model where testing happens at a single point at the end of a release, a code freeze period is unavoidable: an interval is needed during which changes are stopped so defects can be found and fixed. When a change can be delivered at any moment, there is no such interval; instead, a quality gate is defined that every change must pass.

When the feedback point is late, defects are found in batches and prioritization becomes a separate work item; when it is early, defects are found one at a time and most are fixed instantly. This is why the discipline of keeping defect reports also shifts weight depending on the model.

The common point is this: no model makes testing cheaper. What the model determines is when the same testing happens and at what cost.

Summary

• In waterfall, testing is a stage; in the incremental model, an activity distributed inside every iteration; in continuous delivery, a gate in front of progress.
• The feedback point is measured by two quantities: delay and candidate change count; as the point shifts later, both grow together.
• Delay leads to a loss of context; a growing candidate count turns isolation into a separate search problem.
• A suite’s feedback point is determined by its slowest dependency, not its check count; in the example, the five-check suite came out faster than the three-check suite.
• The model does not make testing cheaper; it determines when the same testing happens and at what cost.

Next Step

This topic established what quality is, which questions it is tested against, and how the process positions that testing. What’s missing is testing’s own internal order: what does a check cover, how far does it reach, what information is it written with? The next topic classifies tests. The first question concerns scope — the same defect can be caught by a test that exercises a single function or by a test that brings up the entire system; the cost and the message of the two are not the same.