--- title: 'Unit Testing and Test-Driven Development' source: 'https://academia.sh/en/courses/unit-testing' language: en updated: '2026-08-23T14:25:20+00:00' license: 'CC BY-SA 4.0' lessons: 13 outcomes: - 'Writes fast and independent unit tests' - 'Uses the right kind of test double' - 'Applies the test-driven development cycle' --- # Unit Testing and Test-Driven Development Writing unit tests that are fast, independent, and readable in intent; the signal strength of the assertion chosen, injecting dependencies from outside, the kinds of test double, what a coverage measurement means, and the development cycle in which the test drives the code. ## Unit Testing in Practice The three parts of a unit test, the signal strength of an assertion, test names that describe intent, breaking order dependence, injecting dependencies from outside, the kinds of test double, what statement and branch coverage mean, and the cost of tests bound to an implementation. ### The Anatomy of a Unit Test Source: https://academia.sh/en/courses/unit-testing/anatomy-of-a-unit-test Converting a defect report's reproduction steps into a runnable test, the arrange-act-assert parts of a test body, and the expected value coming from a source independent of the implementation. The Quality and Testing Fundamentals course closed with a defect's reporting, prioritization, and the verification of its fix. That course established **what** a test is and **which** test to write: equivalence classes, boundary values, the test oracle, the defect report. What remained was a document — steps a person read and carried out by hand. This course takes up **how** those same steps are written. The "reproduction steps" field in a defect report is, in effect, the outline of a program: a specific starting state, a single operation, and an observed result. Once that triple is made runnable, no one has to check by hand that the defect has not come back. This lesson's question is: what exactly does that translation look like, and what parts make up a test's body? #### Reproduction Steps The same domain runs through the course: a small library module that applies library checkout rules. The rules are written down. Student members may borrow a book for twenty-eight days, regular members for fourteen. A late return is charged a daily rate, the first few days are free, and the total fee has a cap. Dates are represented as day numbers; calendar arithmetic is not this course's subject, and a plain day counter keeps the examples deterministic. ```js // library.mjs — version 1 export const RULES = { student: { loanDays: 28, maxLoans: 10 }, member: { loanDays: 14, maxLoans: 5 }, }; export const LATE_FEE = { graceDays: 3, dailyRate: 2, cap: 40 }; export function dueDate(checkoutDay, memberType) { const rule = RULES[memberType]; if (rule === undefined) throw new Error(`unknown member type: ${memberType}`); return checkoutDay + rule.loanDays; } export function lateFee(dueDay, returnDay) { const daysLate = returnDay - dueDay; if (daysLate <= LATE_FEE.graceDays) return 0; return (daysLate - LATE_FEE.graceDays) * LATE_FEE.dailyRate; } ``` The defect report from a library clerk says: a member who checks out a book on day one thousand and returns it forty days after the due date is charged a fee of seventy-four units, even though the written rule caps it at forty. The report's reproduction steps are three lines, and each line corresponds to a single program statement. ```js // reproduction.mjs — manual walkthrough of the defect report's steps import { dueDate, lateFee } from './library.mjs'; const due = dueDate(1000, 'member'); console.log(`due day : ${due}`); console.log(`return day : ${due + 40}`); console.log(`calculated fee : ${lateFee(due, due + 40)}`); console.log(`expected cap: 40`); ``` ``` due day : 1014 return day : 1054 calculated fee : 74 expected cap: 40 ``` This script reproduces the defect, but it is not a test. A person has to look at the result — an eye is needed to compare forty against seventy-four. What turns the script into a test is that the comparison itself enters the program. #### Arrange, Act, Assert A unit test's body is divided into three parts, and these parts come in the same order in every test. The **arrange** part sets up the starting state the operation will run against: inputs, objects, dependencies. The **act** part calls the operation under test **exactly once**. The **assert** part compares the observed result with the expected one and fails the test on a mismatch. ```js // late-fee.test.mjs — the defect report's reproduction steps import { test } from 'node:test'; import assert from 'node:assert/strict'; import { dueDate, lateFee } from './library.mjs'; test('late fee does not exceed the cap', () => { const due = dueDate(1000, 'member'); const fee = lateFee(due, due + 40); assert.equal(fee, 40); }); ``` The three parts are separated by blank lines; a separate comment line is not needed. The value of this layout is not aesthetic. The act part being a single line states, without ambiguity, **which** call the test is exercising. With two calls, a failing test does not say which one failed. A lengthening arrange part is a separate signal: it shows the unit under test depends on too many things. The fifth lesson measures that signal. Tests run with the Node runtime's built-in test runner. The runner's verbose output also includes duration, file paths, and process information, which vary with the environment. Throughout the course, only the result lines are filtered out, because those are what teach. ```sh node --test --test-reporter=tap late-fee.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` not ok 1 - late fee does not exceed the cap # tests 1 # pass 0 # fail 1 ``` The test failed, and this is the expected result: the defect has not been fixed yet. A failing run is proof that the test actually tests something. A test that has never failed cannot say whether it is green because it is correct or because it was written wrong. #### Where the Expected Value Comes From The number forty in the assert part came from the written rule. This detail carries the test's entire value. When the expected value is derived from the implementation itself, the test becomes a **tautology**: it confirms that what the code does is what the code does. ```js // tautological.test.mjs — a test that computes the expected value the same way as the implementation import { test } from 'node:test'; import assert from 'node:assert/strict'; import { LATE_FEE, lateFee } from './library.mjs'; test('late fee is calculated correctly', () => { const daysLate = 40; const expected = (daysLate - LATE_FEE.graceDays) * LATE_FEE.dailyRate; assert.equal(lateFee(1014, 1014 + daysLate), expected); }); ``` ```sh node --test --test-reporter=tap tautological.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - late fee is calculated correctly # tests 1 # pass 1 # fail 0 ``` This test came back green on the code the defect **stands** in. The formula producing the expected value is the same formula inside the function under test; both forget the cap. The test oracle concept introduced in the Quality and Testing Fundamentals course names exactly this point: the source of the expected result must be **independent** of the code under test. That source can be a written rule, a calculation done by hand, a known reference value, or a result produced some other way — but it cannot be the tested function's own logic. #### The Fix and Comparing Two Tests The fix is a single line: take the smaller of the raw fee and the cap. ```js // library.mjs — version 2: cap is applied export const RULES = { student: { loanDays: 28, maxLoans: 10 }, member: { loanDays: 14, maxLoans: 5 }, }; export const LATE_FEE = { graceDays: 3, dailyRate: 2, cap: 40 }; export function dueDate(checkoutDay, memberType) { const rule = RULES[memberType]; if (rule === undefined) throw new Error(`unknown member type: ${memberType}`); return checkoutDay + rule.loanDays; } export function lateFee(dueDay, returnDay) { const daysLate = returnDay - dueDay; if (daysLate <= LATE_FEE.graceDays) return 0; const raw = (daysLate - LATE_FEE.graceDays) * LATE_FEE.dailyRate; return Math.min(raw, LATE_FEE.cap); } ``` When the two test files are run together, both of their outcomes reverse. ```sh node --test --test-reporter=tap late-fee.test.mjs tautological.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - late fee does not exceed the cap not ok 2 - late fee is calculated correctly # tests 2 # pass 1 # fail 1 ``` The rule-bound test turned green on the correct behavior. The implementation-bound test, on the other hand, failed on **correct** code. If a test fails when the code is fixed, what it tests is not behavior. This is the plainest form of the flakiness that is the eighth lesson's subject; here it is enough just to name it. #### The Boundary of a Unit The word "unit" names a **boundary**, not a size. A unit test sets up a context in which nothing outside the code under test affects the result. In practice this means keeping three things out: out-of-process resources such as disk and network, the system clock and sources of randomness, and state left behind by other tests. This constraint is not arbitrary. Every resource kept out adds a share to the test's duration and stability. The test above runs in under a millisecond, and its result does not depend on the machine, the clock, or run order. If the same test had to write to a database, a failing run could be caused by the code, the configuration, the network, or leftover data — choosing among four possibilities is far more expensive than reading a single one. The `dueDate` call staying in the arrange part is an example of this boundary. The unit under test is the `lateFee` function; `dueDate` only produces input. Even though both are in the same file, the test's intent is single, and a failing run points to one function. #### Summary - A defect report's reproduction steps can be converted into a runnable test; what completes the conversion is that the comparison itself enters the program. - A test body consists of arrange, act, and assert parts; the act part being a single call removes ambiguity about the source of a failing run. - A failing run is proof that the test actually tests something; a test that has never failed cannot say why it is green. - The expected value must come from a source independent of the code under test; a test that repeats the implementation's formula also comes back green on code the defect stands in. - A unit is a boundary, not a size: out-of-process resources, the clock, randomness, and state left behind by other tests all stay outside that boundary. #### Next Step This lesson used a single assertion: `assert.equal`. That was enough for the equality of two numbers, because the function under test returned a number. When a loan operation returns an object, an error, or a list, an equality check either does not work or measures the wrong thing. The next lesson takes up assertion choice: which assertion catches which defect, how much signal an overly broad assertion loses, and how that loss is measured. ### Assertions Source: https://academia.sh/en/courses/unit-testing/assertions The distinction between equality and deep equality, the signal strength of an assertion measured by the number of corruptions it catches, partial matching cutting an unnecessary bond, and narrowing an error expectation with a matcher. The previous lesson used a single assertion: the equality of two numbers. That was enough because the function under test returned a number. Issuing a loan, though, does not return a number — it returns a record: member id, book id, checkout day, due day, and status. The same assertion behaves completely differently on this record, and how much of the test is actually tested changes with the shape of the assertion chosen. This lesson's question has two layers. The first is formal: which assertion is the right tool for which value type? The second is measurable: how many different corruptions of the tested code does an assertion catch? The second question makes the first exact, because "good assertion" stops being an intuitive judgment and becomes a countable quantity. #### Equality and Deep Equality The function under test is `issueLoan`, which produces a loan record. The member type determines the loan duration and the maximum number of open loans; the operation is rejected when the limit is exceeded. ```js // loan.mjs — the function that produces a loan record export const RULES = { student: { loanDays: 28, maxLoans: 10 }, member: { loanDays: 14, maxLoans: 5 }, }; export function issueLoan(member, bookId, day) { const rule = RULES[member.type]; if (rule === undefined) throw new Error(`unknown member type: ${member.type}`); if (member.openLoans >= rule.maxLoans) { throw new Error(`loan limit exceeded: ${rule.maxLoans}`); } return { memberId: member.id, bookId, checkoutDay: day, dueDay: day + rule.loanDays, status: 'checked-out', }; } ``` Writing the same expectation in two assertion forms makes the difference visible. ```js // equality.test.mjs — equality and deep equality on a function that returns an object import { test } from 'node:test'; import assert from 'node:assert/strict'; import { issueLoan } from './loan.mjs'; const MEMBER = { id: 'U-17', type: 'student', openLoans: 2 }; test('equality compares objects by identity', () => { const record = issueLoan(MEMBER, 'K-903', 1000); assert.equal(record, { memberId: 'U-17', bookId: 'K-903', checkoutDay: 1000, dueDay: 1028, status: 'checked-out', }); }); test('deep equality compares structure', () => { const record = issueLoan(MEMBER, 'K-903', 1000); assert.deepEqual(record, { memberId: 'U-17', bookId: 'K-903', checkoutDay: 1000, dueDay: 1028, status: 'checked-out', }); }); ``` ```sh node --test --test-reporter=tap equality.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` not ok 1 - equality compares objects by identity ok 2 - deep equality compares structure # tests 2 # pass 1 # fail 1 ``` A strict equality assertion compares objects by **identity**: two separate objects are not considered equal even if their fields match exactly. Deep equality, by contrast, walks the structure and compares the corresponding fields. The rule is plain: equality for numbers, strings, and booleans; deep equality for objects, arrays, and nested structures. #### An Assertion's Signal Strength An assertion's value shows up not when it passes but when it **fails**. Its measure is: how many different corruptions of the tested code does it catch? This question can be answered numerically. The script below produces five separately corrupted versions of the correct record and applies three assertion forms to each. ```js // assertion-strength.mjs — three assertion forms tried against five corrupted implementations import assert from 'node:assert/strict'; import { issueLoan } from './loan.mjs'; const MEMBER = { id: 'U-17', type: 'student', openLoans: 2 }; const correct = () => issueLoan(MEMBER, 'K-903', 1000); const CORRUPTIONS = { 'due-day-hardcoded': () => ({ ...correct(), dueDay: 1014 }), 'status-wrong': () => ({ ...correct(), status: 'reserved' }), 'checkout-day-shifted': () => ({ ...correct(), checkoutDay: 1001 }), 'book-id-dropped': () => { const r = correct(); delete r.bookId; return r; }, 'member-id-dropped': () => ({ ...correct(), memberId: null }), }; const EXPECTED = { memberId: 'U-17', bookId: 'K-903', checkoutDay: 1000, dueDay: 1028, status: 'checked-out', }; const ASSERTIONS = { 'ok(record)': (r) => assert.ok(r), 'equal(record.dueDay)': (r) => assert.equal(r.dueDay, 1028), 'deepEqual(record)': (r) => assert.deepEqual(r, EXPECTED), }; for (const [assertionName, assertion] of Object.entries(ASSERTIONS)) { const caught = []; for (const [corruptionName, corrupt] of Object.entries(CORRUPTIONS)) { try { assertion(corrupt()); } catch { caught.push(corruptionName); } } const count = `${caught.length}/${Object.keys(CORRUPTIONS).length}`; console.log(`${assertionName.padEnd(21)} ${count} ${caught.join(', ') || 'none'}`); } ``` ``` ok(record) 0/5 none equal(record.dueDay) 1/5 due-day-hardcoded deepEqual(record) 5/5 due-day-hardcoded, status-wrong, checkout-day-shifted, book-id-dropped, member-id-dropped ``` The first line shows the cost of an overly broad assertion. `assert.ok(record)` says only "an object was returned"; it stays green no matter what the record contains. Such a test takes up a slot in the run list, spends time, and reports no corruption at all. This is the mirror image of the false positive concept introduced in the Quality and Testing Fundamentals course: here the signal is not wrong, it is **absent**. The second line shows a narrow assertion's honest limit: when only one field is tested, only that field's corruption is caught. This is not a defect; if the test's name also names a single field, it is the expected behavior. The third line gives full coverage. But this measurement cuts only one way: as the number of caught corruptions grows, the test's risk of **flakiness** grows with it. The next section shows that cost. #### Partial Matching Suppose a transaction id used to track the operation is added to the record. None of the behavior rules change; the record just has a new field. ```js // loan.mjs — version 2: a transaction id was added to the record export const RULES = { student: { loanDays: 28, maxLoans: 10 }, member: { loanDays: 14, maxLoans: 5 }, }; let counter = 0; export function issueLoan(member, bookId, day) { const rule = RULES[member.type]; if (rule === undefined) throw new Error(`unknown member type: ${member.type}`); if (member.openLoans >= rule.maxLoans) { throw new Error(`loan limit exceeded: ${rule.maxLoans}`); } counter += 1; return { transactionId: `T-${String(counter).padStart(4, '0')}`, memberId: member.id, bookId, checkoutDay: day, dueDay: day + rule.loanDays, status: 'checked-out', }; } ``` ```sh node --test --test-reporter=tap equality.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` not ok 1 - equality compares objects by identity not ok 2 - deep equality compares structure # tests 2 # pass 0 # fail 2 ``` The deep equality assertion failed, because a full match means **full**: an unexpected field in the record is also a mismatch. Yet the rule under test — a student loan lasting twenty-eight days — did not change. The test failed because it was bound to a field it did not care about. The fix is not to weaken the assertion but to **narrow** it: the presence and value of the expected fields are tested, and the rest is left free. ```js // partial-match.test.mjs — partial matching tests the presence of the expected fields import { test } from 'node:test'; import assert from 'node:assert/strict'; import { issueLoan } from './loan.mjs'; function contains(actual, expected, path = 'record') { for (const [field, value] of Object.entries(expected)) { const subPath = `${path}.${field}`; assert.ok(field in actual, `${subPath} field missing`); if (value !== null && typeof value === 'object') contains(actual[field], value, subPath); else assert.equal(actual[field], value, `${subPath} mismatch`); } } const MEMBER = { id: 'U-17', type: 'student', openLoans: 2 }; test('a student loan is due twenty-eight days later', () => { const record = issueLoan(MEMBER, 'K-903', 1000); contains(record, { memberId: 'U-17', bookId: 'K-903', dueDay: 1028, status: 'checked-out' }); }); test('partial matching reports the wrong field', () => { const record = issueLoan(MEMBER, 'K-903', 1000); assert.throws(() => contains(record, { dueDay: 1014 }), /record.dueDay mismatch/); }); ``` ```sh node --test --test-reporter=tap partial-match.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student loan is due twenty-eight days later ok 2 - partial matching reports the wrong field # tests 2 # pass 2 # fail 0 ``` The `contains` helper walks the expected object's fields and reports a missing field and a mismatched value with separate messages. The second test tests this helper itself: given a wrong expectation, it must say which field mismatched, inside the message. Testing the helper may look unnecessary, but if an assertion helper passes silently, every test bound to it loses its signal. The dose of partial matching is a choice: left too loose, it is as weak as `assert.ok`; held too tight, it is as bound as full deep equality. The criterion is this: every field named in the assertion must be part of the rule named in the test's name. #### Expecting an Error The operation being rejected on a rule violation is also a behavior, and it needs to be tested. An error-expecting assertion's unmatched form, though, is a common source of false positives. ```js // error-expectation.test.mjs — an unmatched and a matched error expectation import { test } from 'node:test'; import assert from 'node:assert/strict'; import { issueLoan } from './loan.mjs'; const AT_LIMIT = { id: 'U-17', openLoans: 10 }; test('unmatched expectation: throws on limit exceeded', () => { assert.throws(() => issueLoan(AT_LIMIT, 'K-903', 1000)); }); test('matched expectation: throws on limit exceeded', () => { assert.throws(() => issueLoan(AT_LIMIT, 'K-903', 1000), /loan limit exceeded: 10/); }); ``` ```sh node --test --test-reporter=tap error-expectation.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - unmatched expectation: throws on limit exceeded not ok 2 - matched expectation: throws on limit exceeded # tests 2 # pass 1 # fail 1 ``` Same call, same fixture, two different outcomes. Looking at the error actually thrown explains the difference. ```js // thrown.mjs — the error the test actually catches import { issueLoan } from './loan.mjs'; const AT_LIMIT = { id: 'U-17', openLoans: 10 }; try { issueLoan(AT_LIMIT, 'K-903', 1000); } catch (error) { console.log(`${error.constructor.name}: ${error.message}`); } ``` ``` Error: unknown member type: undefined ``` The `type` field was forgotten on the member object in the fixture. The function never even looks at the loan limit; it throws a different error in an earlier check. The unmatched expectation stayed green because it saw "some error", and the rule it meant to test never ran. The matched expectation, by failing, reported that the fixture was broken. This is the most expensive mistake in assertion choice: a green test may never have run the rule it thought it was testing. An error expectation must always be bound to a criterion — an error class, a message pattern, or a field on the error object. #### Summary - Strict equality compares objects by identity; functions that return objects and arrays need deep equality or partial matching. - An assertion's signal strength is measured by how many corruptions of the tested code it catches; in the measurement, `assert.ok(record)` caught none of the five corruptions, and deep equality caught all five. - Full deep equality also binds to fields unrelated to the behavior; when a new field was added to the record, the test failed even though the rule had not changed. - Partial matching tests the expected fields and leaves the rest free; every field named in the assertion must be part of the rule named in the test's name. - An unmatched error expectation also accepts the wrong error, and can stay green without the rule it meant to test ever running. #### Next Step This lesson measured what assertions say, but test names stayed quietly in the background. Yet the first piece of information a failing run gives is not the assertion, it is the **name**: the sentence next to the `not ok 2` line. If that sentence says something like "record is correct", the reader has to go to the source; if it states the rule, most of the time it does not have to. The next lesson takes up test naming and turns a test suite's names into a readable list of rules. ### Test Naming Source: https://academia.sh/en/courses/unit-testing/test-naming Building a test name from its unit, condition, and expected-result parts, how bundling three rules into one test makes the name misleading, and how the names of passing tests form a readable list of rules. The first piece of information a failing run gives is not the assertion, it is the name. In the line the runner reports, the result comes first, then the test's name; the detailed error message sits below that. If the name states the rule, the reader usually knows what broke without going to the source. If the name says something like "record is correct", reading the error message and going from there to the source becomes mandatory. This lesson takes the name out of being a matter of style and ties it to something measurable: when the same three rules are written in two different forms, how much does the information a failing run gives change? #### A Name's Three Parts A good test name is a sentence and carries three parts: which **unit**, under which **condition**, gives which **result**. When all three are present, the name alone is a readable rule. In the sentence `a student loan lasts twenty-eight days`, the unit is the loan-duration calculation, the condition is the member type being a student, and the result is twenty-eight days. By contrast, the name `dueDate test` has neither a condition nor a result; the name `works correctly` does not even have a unit. Two application rules keep this shape. A name refers not to the steps **inside** the function under test, but to the behavior observed from outside: `Math.min is called` is an implementation detail, `does not exceed the cap` is a behavior. A name states a positive rule instead of a negation: `accepts a loan within the limit` instead of `does not throw`. #### Three Rules in One Test In the library version below, three rules have regressed at once: the student loan duration has dropped to fourteen days, the grace period has been zeroed out, and the late fee's cap has been removed. ```js // rules.mjs — regressed version: three rules broken at once export const RULES = { student: { loanDays: 14, maxLoans: 10 }, member: { loanDays: 14, maxLoans: 5 }, }; export const LATE_FEE = { graceDays: 0, dailyRate: 2, cap: 40 }; export function dueDate(checkoutDay, memberType) { return checkoutDay + RULES[memberType].loanDays; } export function lateFee(dueDay, returnDay) { const daysLate = returnDay - dueDay; if (daysLate <= LATE_FEE.graceDays) return 0; return (daysLate - LATE_FEE.graceDays) * LATE_FEE.dailyRate; } ``` The form that bundles three rules into one test can only say something generic in its name. ```js // combined.test.mjs — the same three rules verified in a single test import { test } from 'node:test'; import assert from 'node:assert/strict'; import { dueDate, lateFee } from './rules.mjs'; test('loan rules work correctly', () => { assert.equal(dueDate(1000, 'student'), 1028); assert.equal(lateFee(1000, 1003), 0); assert.equal(lateFee(1000, 1100), 40); }); ``` ```sh node --test --test-reporter=tap combined.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` not ok 1 - loan rules work correctly # tests 1 # pass 0 # fail 1 ``` Three rules are broken, the report is one line. The reason is not naming alone: when one assertion fails, the rest of the test never runs. The second and third assertions were never evaluated, so the state of those two rules is unknown. As fixes are made, the same test fails again each time, and each round surfaces the next rule. The same three rules, split into separate tests, produce a different report. ```js // separate.test.mjs — the same three rules, each in its own test import { test } from 'node:test'; import assert from 'node:assert/strict'; import { dueDate, lateFee } from './rules.mjs'; test('a student loan lasts twenty-eight days', () => { assert.equal(dueDate(1000, 'student'), 1028); }); test('a return within the grace period is free', () => { assert.equal(lateFee(1000, 1003), 0); }); test('late fee does not exceed the cap', () => { assert.equal(lateFee(1000, 1100), 40); }); ``` ```sh node --test --test-reporter=tap separate.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` not ok 1 - a student loan lasts twenty-eight days not ok 2 - a return within the grace period is free not ok 3 - late fee does not exceed the cap # tests 3 # pass 0 # fail 3 ``` Same code, same three rules, same three assertions. The difference is measurable: the first form reported that one rule was broken; the second form reported that three rules were broken, and which ones. The information obtained without looking at the source tripled. This is the companion half of the second lesson's signal-strength measurement. There, how many corruptions an assertion **catches** was measured; here, how much of a caught corruption **reaches** the report is measured. The rule that a test verifies one behavior is the joint result of these two measurements, not an aesthetic preference. #### The Sum of the Names Is a Specification Once the rules are fixed, the list of passing tests serves another purpose: it describes the library's behavior in readable form. ```js // rules.mjs — fixed version export const RULES = { student: { loanDays: 28, maxLoans: 10 }, member: { loanDays: 14, maxLoans: 5 }, }; export const LATE_FEE = { graceDays: 3, dailyRate: 2, cap: 40 }; export function dueDate(checkoutDay, memberType) { return checkoutDay + RULES[memberType].loanDays; } export function lateFee(dueDay, returnDay) { const daysLate = returnDay - dueDay; if (daysLate <= LATE_FEE.graceDays) return 0; const raw = (daysLate - LATE_FEE.graceDays) * LATE_FEE.dailyRate; return Math.min(raw, LATE_FEE.cap); } ``` Grouping tests under topic headings splits the load the name would otherwise carry: the unit and the context go to the group, the condition and the result stay with the test. ```js // specification.test.mjs — a test suite whose names read as a list of rules import { describe, test } from 'node:test'; import assert from 'node:assert/strict'; import { dueDate, lateFee } from './rules.mjs'; describe('loan duration', () => { test('twenty-eight days for a student member', () => { assert.equal(dueDate(1000, 'student'), 1028); }); test('fourteen days for a standard member', () => { assert.equal(dueDate(1000, 'member'), 1014); }); }); describe('late fee', () => { test('zero within the grace period', () => { assert.equal(lateFee(1000, 1003), 0); }); test('charged per day outside the grace period', () => { assert.equal(lateFee(1000, 1008), 10); }); test('does not exceed the cap', () => { assert.equal(lateFee(1000, 1100), 40); }); }); ``` ```sh node --test --test-reporter=tap specification.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - twenty-eight days for a student member ok 2 - fourteen days for a standard member ok 1 - loan duration ok 1 - zero within the grace period ok 2 - charged per day outside the grace period ok 3 - does not exceed the cap ok 2 - late fee # tests 5 # pass 5 # fail 0 ``` The indented lines give the individual rules, the unindented lines give the groups. Read from top to bottom, the output turns into a list of rules: the loan duration is twenty-eight days for a student member, fourteen for a standard member; the late fee is zero within the grace period, charged per day outside it, and does not exceed the cap. This list is not the written specification itself, but it can be compared against it. A rule present in the specification but missing from the list is the traceability gap named in the What Is Quality Assurance lesson in the Quality and Testing Fundamentals course. A line present in the list but missing from the specification is a warning in the opposite direction: either the specification was written incompletely, or the test has locked in a behavior nobody asked for. #### Alignment Between Name and Assertion There is a single consistency rule between name and assertion: every field named in the assertion must be part of the rule the name states. If a test named `twenty-eight days for a student member` also checks the record's status field, that test is testing two rules, and its name is hiding the second one. The reverse direction is equally binding: if the rule the name states does not appear in the assertion, the test is not living up to its name. The second lesson's unmatched error expectation was an example of this — its name spoke of the loan limit, but what it actually tested was a missing field. When an assertion grows without the name changing, this alignment breaks silently. The question to ask when adding a new assertion to a test is this: is this assertion within the existing name's scope, or is a new test needed? The second answer is usually the right one, and it costs three lines. #### Summary - A test name carries its unit, condition, and expected-result parts; when all three are present, the name alone is a readable rule. - A name refers to the behavior observed from outside, not to implementation steps, and it states a positive rule instead of a negation. - When one assertion fails, the test's remaining assertions never run; the form that bundled three rules into one test reported one failure, and the split form reported three. - Read from top to bottom, the names of passing tests give a list of rules; this list can be compared against the written specification. - Every field named in the assertion must be within the name's scope; when an assertion grows without the name changing, the test starts testing a rule its name does not state. #### Next Step Up to this point, every test looked only at its own input, and every run gave the same result. As tests multiply, this does not hold on its own: two tests sharing a common fixture see the state the other left behind, and run order starts to determine the result. The next lesson produces this bond with a real run — the same tests give different results in a different order — and builds the setup that keeps tests fast and independent of each other. ### Fast and Independent Tests Source: https://academia.sh/en/courses/unit-testing/fast-and-independent-tests Demonstrating the order dependence produced by module-level shared state with a fixed-seed permutation, breaking it with a fresh fixture, and measuring the share an out-of-process fixture adds to run time. Every test in the previous lesson was correct on its own, and stays correct as long as it stays on its own. As tests multiply, this condition does not hold by itself. If two tests use the same object, the same module variable, or the same file, the state the first one leaves behind mixes into the second one's input. The result is that run order turns into an input. Order dependence is an insidious defect, because most of the time it is invisible: as long as the runner runs the tests in the same order every time, the suite stays green. The dependence surfaces when a new test is added to the file or when tests are distributed in parallel, and at that moment it is not clear whether what broke is the code or the test. This lesson's first task is to turn that order into a variable and make the dependence visible. #### Turning Order into a Variable The way to search for order dependence is to change run order in a controlled way. This does not need randomness; a permutation derived from a fixed **seed** is enough, and it is more useful for being repeatable. The linear congruential generator introduced in the Test Stability lesson in the Frontend Quality course is a sufficient source for this job. ```js // runner.mjs — a small runner with a fixed-seed order shuffle export function generator(seed) { let state = (seed * 2654435761) % 2147483647; return () => { state = (state * 48271) % 2147483647; return state / 2147483647; }; } export function shuffle(array, seed) { const random = generator(seed); const copy = [...array]; for (let i = copy.length - 1; i > 0; i -= 1) { const j = Math.floor(random() * (i + 1)); [copy[i], copy[j]] = [copy[j], copy[i]]; } return copy; } export function run(tests, seed) { const order = shuffle(Object.keys(tests), seed); let passed = 0; const lines = []; for (const name of order) { try { tests[name](); passed += 1; lines.push(` ok ${name}`); } catch (error) { lines.push(` not ok ${name} (${error.message.split('\n')[0]})`); } } console.log(`seed ${seed}: ${passed} passed, ${order.length - passed} failed`); console.log(lines.join('\n')); } ``` The same seed always gives the same order; when a failing run is reported, sharing the seed is enough to reproduce the problem. #### The Dependence Produced by Shared State The catalog module below keeps loan records in a module-level map. The module loads once, and there is a single map for the life of the process. ```js // catalog.mjs — module-level shared loan record const records = new Map(); export function addLoan(memberId, bookId) { if (records.has(bookId)) throw new Error(`book is already checked out: ${bookId}`); records.set(bookId, memberId); } export function returnBook(bookId) { records.delete(bookId); } export function openLoanCount() { return records.size; } ``` Three tests exercise three separate rules of this module. Each one is correct read on its own. ```js // shared.mjs — run order determined by seed, over shared state import assert from 'node:assert/strict'; import { run } from './runner.mjs'; import { addLoan, returnBook, openLoanCount } from './catalog.mjs'; const TESTS = { 'a new loan appears in the catalog': () => { addLoan('U-17', 'K-903'); assert.equal(openLoanCount(), 1); }, 'a returned book drops from the catalog': () => { addLoan('U-17', 'K-101'); returnBook('K-101'); assert.equal(openLoanCount(), 0); }, 'the same book cannot be loaned out twice': () => { addLoan('U-42', 'K-903'); assert.throws(() => addLoan('U-58', 'K-903'), /already checked out/); }, }; run(TESTS, Number(process.argv[2] ?? 3)); ``` Running it in two separate processes with two different seeds is enough. A separate process is needed because the module-level map lives for the process's lifetime. ```sh node shared.mjs 3 node shared.mjs 4 ``` ``` seed 3: 1 passed, 2 failed ok a new loan appears in the catalog not ok a returned book drops from the catalog (Expected values to be strictly equal:) not ok the same book cannot be loaned out twice (book is already checked out: K-903) seed 4: 2 passed, 1 failed ok a returned book drops from the catalog ok a new loan appears in the catalog not ok the same book cannot be loaned out twice (book is already checked out: K-903) ``` Same code, same three tests, two different results. A test that passes at seed three fails at seed four. There is no answer, in this table, to the question of which result is "correct", because the tests are measuring not a rule but what ran before them. Two separate forms of contamination show up. The `a returned book drops from the catalog` test expects an absolute number: it assumes the catalog is empty. The `the same book cannot be loaned out twice` test, conversely, assumes the catalog does not already contain the book it is about to add. Both carry an unwritten assumption about the starting state, and that assumption is set by whichever tests ran before them. #### A Fresh Fixture The fix is not to make the tests insensitive to order, but to have every test set up its own starting state. For this, instead of a single shared map, a factory that produces a fresh catalog on every call is used. ```js // catalog.mjs — version 2: a fresh catalog is produced on every call export function createCatalog() { const records = new Map(); return { addLoan(memberId, bookId) { if (records.has(bookId)) throw new Error(`book is already checked out: ${bookId}`); records.set(bookId, memberId); }, returnBook(bookId) { records.delete(bookId); }, openLoanCount() { return records.size; }, }; } ``` This starting state a test sets up before it begins running is called a **fixture**. The test runner provides hooks to set up the fixture and, if needed, tear it down; the hook that runs before every test corresponds to setup, and the one that runs after every test corresponds to teardown. ```js // independent.test.mjs — a fresh fixture per test, registration order comes from the seed import { test, beforeEach } from 'node:test'; import assert from 'node:assert/strict'; import { shuffle } from './runner.mjs'; import { createCatalog } from './catalog.mjs'; let catalog; beforeEach(() => { catalog = createCatalog(); }); const TESTS = { 'a new loan appears in the catalog': () => { catalog.addLoan('U-17', 'K-903'); assert.equal(catalog.openLoanCount(), 1); }, 'a returned book drops from the catalog': () => { catalog.addLoan('U-17', 'K-101'); catalog.returnBook('K-101'); assert.equal(catalog.openLoanCount(), 0); }, 'the same book cannot be loaned out twice': () => { catalog.addLoan('U-42', 'K-903'); assert.throws(() => catalog.addLoan('U-58', 'K-903'), /already checked out/); }, }; for (const name of shuffle(Object.keys(TESTS), Number(process.env.SEED ?? 3))) { test(name, TESTS[name]); } ``` The tests now run with the built-in runner; the registration order comes from the seed, and the fixture is rebuilt before every test. ```sh SEED=3 node --test --test-reporter=tap independent.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' SEED=4 node --test --test-reporter=tap independent.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a new loan appears in the catalog ok 2 - a returned book drops from the catalog ok 3 - the same book cannot be loaned out twice # tests 3 # pass 3 # fail 0 ok 1 - a returned book drops from the catalog ok 2 - a new loan appears in the catalog ok 3 - the same book cannot be loaned out twice # tests 3 # pass 3 # fail 0 ``` The order changed, the result did not. This is the measure of independence: once run order stops being an input, what a test reports is only the rule it tests. The same principle has two practical consequences. Because no writable state remains shared between tests, the tests can be run **in parallel**. A failing test also fails when run on its own; reproducing it does not require running the whole suite. #### A Fixture's Share of the Duration Independence and speed look like separate virtues, but they draw on the same source. Every test setting up its own fixture requires the fixture to be **cheap**. Setting up a map in memory takes a few microseconds; setting up the same fixture on disk requires stepping outside the process. ```js // speed.mjs — comparing an in-memory fixture with a disk-backed fixture import { mkdtempSync, writeFileSync, readFileSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; const TEST_COUNT = 200; function inMemoryFixture() { let fileOps = 0; for (let i = 0; i < TEST_COUNT; i += 1) { const records = new Map([['K-903', 'U-17']]); if (records.size !== 1) throw new Error('fixture broken'); } return fileOps; } function diskBackedFixture() { let fileOps = 0; const root = mkdtempSync(join(tmpdir(), 'catalog-')); for (let i = 0; i < TEST_COUNT; i += 1) { const path = join(root, `catalog-${i}.json`); writeFileSync(path, JSON.stringify({ 'K-903': 'U-17' })); fileOps += 1; const records = new Map(Object.entries(JSON.parse(readFileSync(path, 'utf8')))); fileOps += 1; if (records.size !== 1) throw new Error('fixture broken'); } rmSync(root, { recursive: true, force: true }); return fileOps; } function measure(fn) { const start = performance.now(); const fileOps = fn(); return { duration: performance.now() - start, fileOps }; } const memory = measure(inMemoryFixture); const disk = measure(diskBackedFixture); console.log(`in-memory fixture : ${TEST_COUNT} tests, ${memory.fileOps} file operations`); console.log(`disk-backed fixture : ${TEST_COUNT} tests, ${disk.fileOps} file operations`); console.log(`disk-backed version took longer : ${disk.duration > memory.duration}`); console.log(`duration ratio is at least twentyfold : ${disk.duration / memory.duration >= 20}`); ``` ``` in-memory fixture : 200 tests, 0 file operations disk-backed fixture : 200 tests, 400 file operations disk-backed version took longer : true duration ratio is at least twentyfold : true ``` Absolute durations depend on the machine and the filesystem; that is why the output reports a comparison result instead of raw milliseconds. What does not change is the direction of the relationship: a suite of two hundred tests, at two file operations per test, produces four hundred out-of-process calls, and each of these calls carries a fixed baseline cost. The rule that follows from this is not a speed target but a boundary definition. The first lesson said that the "unit" boundary keeps out-of-process resources out; this measurement shows the cost of that constraint. It does not mean tests that work with disk, network, or a database will not be written — those tests are called **integration tests**, and they run in a separate layer, less often. #### Summary - Shared writable state turns run order into one of the test's inputs. - A permutation derived from a fixed seed surfaces order dependence in a repeatable way; the same three tests gave two different results at two seeds. - Contamination works in both directions: one test assumed the catalog was empty, another assumed it was not already occupied; the prior tests determined both assumptions. - Every test setting up its own fixture makes order independent of the result and makes the tests runnable in parallel. - Independence requires a cheap fixture: in a suite of two hundred tests, an out-of-process fixture added four hundred file operations and noticeably lengthened the run. #### Next Step In this lesson the fixture was in the test's hands: the catalog object was produced by a factory call. When the code under test creates its dependency internally, this option disappears — a clock, a source of randomness, or a store the test cannot change stops being one of the test's inputs. The next lesson writes the same business rule in two forms, compares the version that creates its dependency internally with the version it is supplied from outside, and measures the fixture a test sets up by its line count. ### Testable Design Source: https://academia.sh/en/courses/unit-testing/testable-design Comparing a version that creates its own dependency with a version that receives it from outside, measuring fixture cost in lines, and handing off time and randomness from outside. In the previous lesson the fixture was in the test's hands: the catalog object was produced by a factory call, and each test built its own copy. That option came from the code, not the test. If the tested code created the catalog inside itself, the test would have no lever left to change it. Testability is not a property of the test — it is a property of the **design**. The same business rule written in two different ways can both run correctly, but one leaves a testable lever and the other does not. This lesson's question is: where does that difference show up, and how much does it cost? #### Same Rule, Two Designs The business logic to test is this: a loan operation reads today's day number, produces a receipt number, checks that the book is not already on loan, and returns the loan record. The operation has three outside sources: the clock, the receipt-number generator, and the catalog. The first version creates all three inside itself. ```js // internal.mjs — version that creates its dependencies inside itself const catalog = new Map(); export function clearCatalog() { catalog.clear(); } export function lendBook(memberId, bookId, memberType) { const today = Math.floor(Date.now() / 86400000); const receiptId = `F-${String(Math.floor(Math.random() * 1000000)).padStart(6, '0')}`; if (catalog.has(bookId)) throw new Error(`book already on loan: ${bookId}`); const loanDays = memberType === 'student' ? 28 : 14; catalog.set(bookId, memberId); return { receiptId, memberId, bookId, borrowedDay: today, dueDay: today + loanDays }; } ``` The second version receives all three from outside. The business rule — the duration calculation, the double-loan check, the record shape — is identical. ```js // external.mjs — version whose dependencies are supplied from outside export function createLoanService({ clock, receiptGenerator, catalog }) { return { lendBook(memberId, bookId, memberType) { const today = clock(); const receiptId = receiptGenerator(); if (catalog.has(bookId)) throw new Error(`book already on loan: ${bookId}`); const loanDays = memberType === 'student' ? 28 : 14; catalog.set(bookId, memberId); return { receiptId, memberId, bookId, borrowedDay: today, dueDay: today + loanDays }; }, }; } ``` This second form is called **dependency injection**; the dependency is handed in from outside instead of being created inside. The name sounds like a framework feature, but as seen here it is only a signature decision: the code expects its dependency to be given to it instead of reaching out for it. #### The Cost of the Fixture Both versions' tests exercise the same rule; what differs is the fixture that has to be built to reach that rule. ```js // fixture.mjs — the fixture each version's test has to build import { clearCatalog, lendBook } from './internal.mjs'; import { createLoanService } from './external.mjs'; export function internalFixture() { const realNow = Date.now; const realRandom = Math.random; Date.now = () => 1000 * 86400000; Math.random = () => 0.123456; clearCatalog(); const restore = () => { Date.now = realNow; Math.random = realRandom; }; return { service: { lendBook }, restore }; } export function externalFixture() { const service = createLoanService({ clock: () => 1000, receiptGenerator: () => 'F-123456', catalog: new Map(), }); return { service, restore: () => {} }; } ``` The size of the fixtures can be measured. When both fixtures are run and the records they produce are compared, the difference turns out to be in the preparation, not the behavior. ```js // measure.mjs — the two fixtures' line count and the records they produce import { internalFixture, externalFixture } from './fixture.mjs'; const lines = (fn) => fn.toString().split('\n').length - 2; for (const [label, fixtureFn] of [['internal', internalFixture], ['external', externalFixture]]) { const { service, restore } = fixtureFn(); const record = service.lendBook('U-17', 'K-903', 'student'); restore(); console.log(`${label.padEnd(10)} fixture ${String(lines(fixtureFn)).padStart(2)} lines ${JSON.stringify(record)}`); } ``` ``` internal fixture 10 lines {"receiptId":"F-123456","memberId":"U-17","bookId":"K-903","borrowedDay":1000,"dueDay":1028} external fixture 6 lines {"receiptId":"F-123456","memberId":"U-17","bookId":"K-903","borrowedDay":1000,"dueDay":1028} ``` The two records are identical. The fixture cost is ten lines against six. But the real difference is not the line count — it is what those lines **do**. The externally-supplied version's fixture only builds an object. The internally-creating version's fixture replaces two global functions, clears the tested module's internal state, and takes on the job of undoing what it changed. The first lesson said that "a growing arrange section is a separate signal." This is the signal being measured here: as the fixture grows, the rule the test exercises does not shrink, but the risk entering the test grows. #### The Cost of an Undone Change The risk carried by a fixture that changes a global function is concrete. In the test file below, the first test freezes the clock and forgets to restore it. ```js // leak.test.mjs — an undone global change affects the next test import { test } from 'node:test'; import assert from 'node:assert/strict'; import { clearCatalog, lendBook } from './internal.mjs'; test('a student loan lasts twenty-eight days', () => { Date.now = () => 1000 * 86400000; clearCatalog(); const record = lendBook('U-17', 'K-903', 'student'); assert.equal(record.dueDay, 1028); }); test('the day number comes from the real clock', () => { const today = Math.floor(Date.now() / 86400000); assert.ok(today > 20000, `unexpected day number: ${today}`); }); ``` ```sh node --test --test-reporter=tap leak.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student loan lasts twenty-eight days not ok 2 - the day number comes from the real clock # tests 2 # pass 1 # fail 1 ``` The second test is correct on its own and passes when run alone. In the run where it fails, the day number it sees is one thousand, because the clock is still frozen where the first test left it. This is the same order dependence as in the previous lesson, but its source differs: the shared state is not sitting inside the library itself, it is accumulating in **the runtime's global objects**. Undoing the global change is one fix, but it is a fix that can be forgotten, and when it is forgotten, the test that breaks is not the test that made the mistake. In the externally-supplied version, this risk class never arises. ```js // clean.test.mjs — an externally-supplied dependency does not corrupt global state import { test } from 'node:test'; import assert from 'node:assert/strict'; import { createLoanService } from './external.mjs'; const setup = () => createLoanService({ clock: () => 1000, receiptGenerator: () => 'F-123456', catalog: new Map(), }); test('a student loan lasts twenty-eight days', () => { const service = setup(); const record = service.lendBook('U-17', 'K-903', 'student'); assert.equal(record.dueDay, 1028); }); test('the day number comes from the real clock', () => { const today = Math.floor(Date.now() / 86400000); assert.ok(today > 20000, `unexpected day number: ${today}`); }); test('the receipt number comes from an external source', () => { const service = setup(); const record = service.lendBook('U-17', 'K-903', 'member'); assert.equal(record.receiptId, 'F-123456'); }); ``` ```sh node --test --test-reporter=tap clean.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student loan lasts twenty-eight days ok 2 - the day number comes from the real clock ok 3 - the receipt number comes from an external source # tests 3 # pass 3 # fail 0 ``` The third test checks something neither of the first two versions could ever have checked: where the receipt number comes from. In the internally-creating version the receipt number comes from the randomness source, and verifying it would again require a global change. In the externally-supplied version the generator is a parameter, and the value the test gives it lands in the record exactly as given. #### Where Non-Deterministic Sources Belong Time and randomness are two typical examples of sources a test cannot control; the same class also includes process IDs, environment variables, network addresses, and the file system. What they share is that **they do not give the same output for the same input**. The way to make these testable is not to block the code from reaching these sources, but to gather that access at a single point that can then be supplied from outside. In the version above, `clock` and `receiptGenerator` are those points. Configuration running in production supplies the real sources; configuration running in tests supplies fixed values. This separation has a limit, and it should be said up front: if a piece of code genuinely has to read the time, that read has to be tested somewhere too. Supplying it from outside does not remove that test — it moves it out of the unit test, into an integration test. The gain is that the business rule becomes testable in a deterministic context. #### Summary - Testability is a property of the design, not the test; the same business rule can run correctly in two forms, but only one of them leaves a testable lever. - Supplying a dependency from outside is not a framework feature but a signature decision: the code expects its dependency to be given to it instead of reaching out for it. - In the measurement, both fixtures produced the same record; the internally-creating version's fixture was ten lines and the externally-supplied version's was six, and the difference was not just length but that one of them changed global state. - An undone global change fails the next test; the test that breaks is not the test that made the mistake. - Time, randomness, and similar non-deterministic sources are gathered at a single point that can be supplied from outside; this does not remove the testing of those sources, it moves it to an integration test. #### Next Step In this lesson, the dependencies supplied from outside were small functions that returned a fixed value: `clock` always gave a thousand, `receiptGenerator` always gave the same number. This is only one member of a family with four types. A dependency can also be supplied only to fill a parameter's place, to record calls when it is called, or to be a working but simplified copy of the real behavior. The next lesson writes these types out one by one and separates which question each one answers. ### Test Doubles Source: https://academia.sh/en/courses/unit-testing/mock-dependencies Hand-writing the dummy, stub, fake, spy, and mock types, separating the verification question each type answers, and measuring which type speaks up at the same change. In the previous lesson, the dependencies supplied from outside were small functions that returned a fixed value. That is a single member of a family. The common name for objects put in place of a real dependency in the tested code is **test double**, and the difference between them is not one of structure but of **purpose**: one only fills a parameter's place, one gives a specific answer, one is a working copy of the real behavior, one records calls, and one carries the expectation inside itself. These distinctions are not academic. Each type answers a different verification question, and when the wrong type is chosen, a test either verifies nothing or ends up bound to something it did not mean to verify. This lesson hand-writes all five types and, at the end, tests all of them against the same change. #### The Service Under Test The loan service takes four dependencies: the catalog holding records, the notification channel that informs the member, the transaction log, and the clock that gives the day number. ```js // service.mjs — loan service: catalog, notification, log, and clock dependencies export const RULES = { student: { loanDays: 28, maxLoans: 10 }, member: { loanDays: 14, maxLoans: 5 }, }; export function createLoanService({ catalog, notification, log, clock }) { return { lendBook(member, bookId) { const rule = RULES[member.type]; if (catalog.memberLoanCount(member.id) >= rule.maxLoans) { throw new Error(`loan limit exceeded: ${rule.maxLoans}`); } const today = clock(); const record = { memberId: member.id, bookId, borrowedDay: today, dueDay: today + rule.loanDays }; catalog.add(record); notification.send(member.id, `due date: ${record.dueDay}`); return record; }, checkIn(bookId) { const record = catalog.find(bookId); if (record === undefined) throw new Error(`no loan record: ${bookId}`); catalog.remove(bookId); log.write(`return ${bookId}`); return record; }, }; } ``` #### Writing the Five Types All five are a few lines each; none of them requires a library. ```js // doubles.mjs — four test-double types and an expectation-carrying mock export function dummy(name) { return new Proxy({}, { get: (_target, prop) => () => { throw new Error(`${name}.${String(prop)} should not have been called`); }, }); } export function stubCatalog(loanCount) { return { memberLoanCount: () => loanCount, add() {}, find: () => undefined, remove() {}, }; } export function fakeCatalog() { const records = new Map(); return { memberLoanCount: (memberId) => [...records.values()].filter((r) => r.memberId === memberId).length, add: (record) => records.set(record.bookId, record), find: (bookId) => records.get(bookId), remove: (bookId) => records.delete(bookId), }; } export function spy(...methods) { const calls = []; const obj = { calls }; for (const name of methods) obj[name] = (...args) => calls.push([name, ...args]); return obj; } export function mock(expected, ...methods) { const spyObj = spy(...methods); return { ...spyObj, verify() { const actual = JSON.stringify(spyObj.calls); const exp = JSON.stringify(expected); if (actual === exp) return; throw new Error(`expectation not met: expected ${exp}, actual ${actual}`); }, }; } ``` **Dummy** only fills a parameter's place; it is expected not to be called. The version here enforces that expectation: it throws an error whenever any of its methods is called. **Stub** fixes a value that the tested code reads. Its question is: which path does the code take with this input? The stub itself verifies nothing. **Fake** is a working but simplified copy of the real dependency. The catalog here uses a real map; adding, finding, and removing are meaningful. Multi-step scenarios can only be built with this type. **Spy** records calls and leaves them for the test to inspect afterward. Its question is: was this call actually made, and with which arguments? **Mock** carries the expectation inside itself. Its difference from a spy is that verification is the object's responsibility, not the test's. #### The Question Each Type Answers When the five types are used in five separate tests, their differences become visible. ```js // doubles.test.mjs — each test-double type answers a separate question import { test } from 'node:test'; import assert from 'node:assert/strict'; import { createLoanService } from './service.mjs'; import { dummy, stubCatalog, fakeCatalog, spy, mock } from './doubles.mjs'; const STUDENT = { id: 'U-17', type: 'student' }; const MEMBER = { id: 'U-42', type: 'member' }; test('dummy: lending never touches the log', () => { const service = createLoanService({ catalog: fakeCatalog(), notification: spy('send'), log: dummy('log'), clock: () => 1000, }); const record = service.lendBook(STUDENT, 'K-903'); assert.equal(record.dueDay, 1028); }); test('stub: a loan is rejected for a member at the limit', () => { const service = createLoanService({ catalog: stubCatalog(5), notification: dummy('notification'), log: dummy('log'), clock: () => 1000, }); assert.throws(() => service.lendBook(MEMBER, 'K-903'), /loan limit exceeded: 5/); }); test('fake: a returned book can be lent out again', () => { const service = createLoanService({ catalog: fakeCatalog(), notification: spy('send'), log: spy('write'), clock: () => 1000, }); service.lendBook(STUDENT, 'K-903'); service.checkIn('K-903'); const second = service.lendBook(MEMBER, 'K-903'); assert.equal(second.dueDay, 1014); }); test('spy: a single notification goes to the member', () => { const spyObj = spy('send'); const service = createLoanService({ catalog: fakeCatalog(), notification: spyObj, log: dummy('log'), clock: () => 1000, }); service.lendBook(STUDENT, 'K-903'); assert.deepEqual(spyObj.calls, [['send', 'U-17', 'due date: 1028']]); }); test('mock: no notification goes out on a rejected loan', () => { const mockObj = mock([], 'send'); const service = createLoanService({ catalog: stubCatalog(5), notification: mockObj, log: dummy('log'), clock: () => 1000, }); assert.throws(() => service.lendBook(MEMBER, 'K-903')); mockObj.verify(); }); ``` ```sh node --test --test-reporter=tap doubles.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - dummy: lending never touches the log ok 2 - stub: a loan is rejected for a member at the limit ok 3 - fake: a returned book can be lent out again ok 4 - spy: a single notification goes to the member ok 5 - mock: no notification goes out on a rejected loan # tests 5 # pass 5 # fail 0 ``` The five tests' assert sections differ from one another, and the difference comes from the type chosen. The first test's assertion concerns the returned record; the dummy only protects an assumption. The second test also looks at the return value; the stub supplies the input that makes that return possible. The third test builds a three-step scenario, and that can only happen with a stateful fake. The fourth and fifth tests look not at the return value but at **the call that was made**. The distinction between the last two tests is subtle. The test using a spy does its verification in its own body, and the assertion reads explicitly. The test using a mock sets up the expectation in the fixture and calls only `verify()` in its body. The second shortens the body when there are multiple call expectations; in exchange, where the expectation is set up moves to the start of the test. #### Who Speaks Up at the Same Change The types' real difference is which one makes noise when the code changes. The script below adds two new side effects to the service — every loan attempt is written to the log, and the member is now notified on a rejected request too — and applies the same three types to this new version. ```js // probe.mjs — which test double speaks up when the service gains a side effect import assert from 'node:assert/strict'; import { RULES } from './service.mjs'; import { dummy, stubCatalog, mock } from './doubles.mjs'; function createChangedService({ catalog, notification, log, clock }) { return { lendBook(member, bookId) { const rule = RULES[member.type]; log.write(`loan attempt ${bookId}`); if (catalog.memberLoanCount(member.id) >= rule.maxLoans) { notification.send(member.id, 'loan rejected'); throw new Error(`loan limit exceeded: ${rule.maxLoans}`); } const today = clock(); const record = { memberId: member.id, bookId, borrowedDay: today, dueDay: today + rule.loanDays }; catalog.add(record); notification.send(member.id, `due date: ${record.dueDay}`); return record; }, }; } const MEMBER = { id: 'U-42', type: 'member' }; const silentLog = { write() {} }; const silentNotification = { send() {} }; function attempt(label, body) { try { body(); console.log(`${label.padEnd(12)} silent`); } catch (error) { console.log(`${label.padEnd(12)} ${error.message.split('\n')[0]}`); } } attempt('stub', () => { const service = createChangedService({ catalog: stubCatalog(5), notification: silentNotification, log: silentLog, clock: () => 1000, }); assert.throws(() => service.lendBook(MEMBER, 'K-903'), /loan limit exceeded: 5/); }); attempt('dummy', () => { const service = createChangedService({ catalog: stubCatalog(0), notification: silentNotification, log: dummy('log'), clock: () => 1000, }); service.lendBook(MEMBER, 'K-903'); }); attempt('mock', () => { const mockObj = mock([], 'send'); const service = createChangedService({ catalog: stubCatalog(5), notification: mockObj, log: silentLog, clock: () => 1000, }); assert.throws(() => service.lendBook(MEMBER, 'K-903')); mockObj.verify(); }); ``` ``` stub silent dummy log.write should not have been called mock expectation not met: expected [], actual [["send","U-42","loan rejected"]] ``` The three lines tell, on their own, what the type choice means. The stub stayed **silent**, because the return value and the thrown error had not changed. This is not a defect: a stub only supplies input, it does not verify. A test that looks at the return value does not see the new side effects, and should not. The dummy **spoke up**, because a dependency that should not be called was called. This is this type's only function, and in exchange it turns the assumption the test rests on into documentation. The mock also **spoke up**, because its expectation was "no notification should go out" and a notification went out. A spy would have given the same signal; the difference is where the verification sits. The selection rule that follows from this has three parts. If what is to be verified is the **return value or the thrown error**, a stub is enough. If **a side effect happening** is to be verified, a spy or a mock is needed. If **a multi-step scenario** is to be built, a fake is needed; multi-step scenarios built with a stub quickly become inconsistent. Call verification carries a cost: the test becomes bound to **how** the tested code works. If a refactor moves the notification send to another layer, tests written with a spy or a mock fail, even though the externally observed behavior is the same. The eighth lesson measures this cost. #### The Capability the Runner Provides The built-in test runner provides a capability so that call-recording functions do not have to be hand-written. It does the same job as a hand-written spy; the record format comes ready-made. ```js // builtin.test.mjs — the mock-function capability the runner provides import { test, mock } from 'node:test'; import assert from 'node:assert/strict'; import { createLoanService } from './service.mjs'; import { dummy, fakeCatalog } from './doubles.mjs'; test('the built-in mock function keeps a call record', () => { const send = mock.fn(); const service = createLoanService({ catalog: fakeCatalog(), notification: { send }, log: dummy('log'), clock: () => 1000, }); service.lendBook({ id: 'U-17', type: 'student' }, 'K-903'); assert.equal(send.mock.callCount(), 1); assert.deepEqual(send.mock.calls[0].arguments, ['U-17', 'due date: 1028']); }); ``` ```sh node --test --test-reporter=tap builtin.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - the built-in mock function keeps a call record # tests 1 # pass 1 # fail 0 ``` The ready-made capability reduces the writing burden, but it does not make the type choice. Because `mock.fn()` keeps a call record, it can be used as a spy or as a mock; which one it is depends on how the test verifies it. The decision that chooses the type is still the writer's. #### Summary - Test doubles differ in purpose, not structure; the five types answer five separate verification questions. - A dummy only fills a place, a stub fixes input, a fake is a working copy, a spy records calls, and a mock carries the expectation itself. - When two side effects were added to the service, the stub stayed silent, the dummy reported the unexpected call, and the mock reported the unmet expectation. - If the return value is to be tested, a stub is enough; if a side effect is to be tested, a spy or a mock is needed; if a multi-step scenario is to be built, a fake is needed. - Call verification binds the test to how the code works; the ready-made capability the runner provides reduces the writing burden but does not make the type choice. #### Next Step In this lesson, what the tests verified was separated type by type, but the question of **how much** of the codebase they verified was never asked. The common answer to this question is a percentage: what fraction of the code ran during testing? The next lesson shows that this percentage is calculated in three separate ways and counts all three by hand. A bug that a codebase reaching one hundred percent statement coverage missed is shown, run in the same lesson. ### Coverage Measurement Source: https://academia.sh/en/courses/unit-testing/coverage-measurement Hand-counting statement, branch, and path coverage on the same function, showing with a run the bug a codebase with one hundred percent statement coverage missed, and the limit of using a coverage percentage as a target. The previous lessons separated **what** tests verify: which assertion catches which regression, which test double answers which question. The question that was never asked is this: how much of the codebase did the tests written so far touch? The common answer to this question is a percentage, and that percentage is calculated in at least three different ways. The three measurements give different numbers on the same code; the difference between them determines what the measurement says. This lesson counts all three by hand on the same function. #### The Function to Measure The late-fee calculation contains two conditions: the free-day allowance being exceeded, and the book being returned to a different location than the one it was borrowed from. ```js // fee.mjs — late fee and cross-location return fee export const LATE_FEE = { freeDays: 3, dailyFee: 2, cap: 40, crossLocationFee: 5 }; export function lateFee(record, checkin) { const delay = checkin.day - record.dueDay; let fee = 0; if (delay > LATE_FEE.freeDays) { fee = (delay - LATE_FEE.freeDays) * LATE_FEE.dailyFee; } if (checkin.location !== record.location) { fee += LATE_FEE.crossLocationFee; } return Math.min(fee, LATE_FEE.cap); } ``` Two tests have been written. Both pass. ```js // coverage.test.mjs — two tests, one hundred percent statement coverage import { test } from 'node:test'; import assert from 'node:assert/strict'; import { lateFee } from './fee.mjs'; const RECORD = { dueDay: 1014, location: 'central' }; test('a delayed return is charged per day', () => { assert.equal(lateFee(RECORD, { day: 1024, location: 'central' }), 14); }); test('a delayed cross-location return gets an extra fee', () => { assert.equal(lateFee(RECORD, { day: 1024, location: 'hillside' }), 19); }); ``` ```sh node --test --test-reporter=tap coverage.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a delayed return is charged per day ok 2 - a delayed cross-location return gets an extra fee # tests 2 # pass 2 # fail 0 ``` #### Counting the Three Measurements by Hand Coverage measurement is defined once the unit to be counted is chosen. **Statement coverage** is the ratio of executed lines to the total number of lines. **Branch coverage** counts whether both the true and the false outcome of each condition were seen. **Path coverage** counts the **combinations** of condition outcomes: a function with two conditions has four paths, one with three has eight. A tracing copy of the same function gives these three numbers directly. ```js // measure.mjs — the version of the same function that traces its statement, branch, and path coverage import { LATE_FEE } from './fee.mjs'; const statements = new Set(); const branches = new Set(); const paths = new Set(); function tracedFee(record, checkin) { statements.add('S1'); const delay = checkin.day - record.dueDay; statements.add('S2'); let fee = 0; statements.add('S3'); const d1 = delay > LATE_FEE.freeDays; branches.add(d1 ? 'D1-true' : 'D1-false'); if (d1) { statements.add('S4'); fee = (delay - LATE_FEE.freeDays) * LATE_FEE.dailyFee; } statements.add('S5'); const d2 = checkin.location !== record.location; branches.add(d2 ? 'D2-true' : 'D2-false'); if (d2) { statements.add('S6'); fee += LATE_FEE.crossLocationFee; } statements.add('S7'); paths.add(`${d1 ? 'T' : 'F'}${d2 ? 'T' : 'F'}`); return Math.min(fee, LATE_FEE.cap); } const RECORD = { dueDay: 1014, location: 'central' }; tracedFee(RECORD, { day: 1024, location: 'central' }); tracedFee(RECORD, { day: 1024, location: 'hillside' }); const ALL_STATEMENTS = ['S1', 'S2', 'S3', 'S4', 'S5', 'S6', 'S7']; const ALL_BRANCHES = ['D1-true', 'D1-false', 'D2-true', 'D2-false']; const ALL_PATHS = ['TT', 'TF', 'FT', 'FF']; const report = (label, all, seen) => { const missing = all.filter((x) => !seen.has(x)); const percent = Math.round(((all.length - missing.length) / all.length) * 100); console.log(`${label} ${all.length - missing.length}/${all.length} (${percent}%) missing: ${missing.join(', ') || 'none'}`); }; report('statement coverage', ALL_STATEMENTS, statements); report('branch coverage ', ALL_BRANCHES, branches); report('path coverage ', ALL_PATHS, paths); ``` ``` statement coverage 7/7 (100%) missing: none branch coverage 3/4 (75%) missing: D1-false path coverage 2/4 (50%) missing: FT, FF ``` Same two tests, same function, three different numbers. The statement measurement looks complete: seven statements out of seven ran. The branch measurement reports one missing: the false outcome of the first condition — that is, a return with no delay — was never tried. The path measurement reports two missing: neither location case for a non-delayed return ever occurred. The order of the three numbers is not a coincidence. Statement coverage is always greater than or equal to branch coverage, and branch coverage is always greater than or equal to path coverage. For a statement to run, entering one branch is enough, but seeing both outcomes of a branch needs separate inputs, and the combination of paths needs still more. #### What One Hundred Percent Statement Coverage Misses The two uncovered paths are not an empty possibility — each is a runnable input. ```js // missed.mjs — what the function returns on the uncovered paths import { lateFee } from './fee.mjs'; const RECORD = { dueDay: 1014, location: 'central' }; const CASES = [ ['FT not delayed, cross-location', { day: 1016, location: 'hillside' }, 0], ['FF not delayed, same location', { day: 1016, location: 'central' }, 0], ]; for (const [label, checkin, expected] of CASES) { const actual = lateFee(RECORD, checkin); console.log(`${label}: fee ${actual}, expected ${expected} -> ${actual === expected ? 'matches' : 'WRONG'}`); } ``` ``` FT not delayed, cross-location: fee 5, expected 0 -> WRONG FF not delayed, same location: fee 0, expected 0 -> matches ``` The written rule defines the cross-location return fee as an addition to the late fee: a book returned on time is free, whatever location it is left at. The implementation, however, applies the location check independently of the delay, and takes five units from a member who returns on time. This bug was sitting underneath a codebase with one hundred percent statement coverage. The measurement was not wrong; **the question it asked** was narrow. Statement measurement asks "did this line run," not "under what conditions did this line run." When the missing paths are turned into tests, the bug shows up in the report. ```js // coverage.test.mjs — version 2: all four paths are tested import { test } from 'node:test'; import assert from 'node:assert/strict'; import { lateFee } from './fee.mjs'; const RECORD = { dueDay: 1014, location: 'central' }; test('a delayed return is charged per day', () => { assert.equal(lateFee(RECORD, { day: 1024, location: 'central' }), 14); }); test('a delayed cross-location return gets an extra fee', () => { assert.equal(lateFee(RECORD, { day: 1024, location: 'hillside' }), 19); }); test('a return within the free-day allowance is free', () => { assert.equal(lateFee(RECORD, { day: 1016, location: 'central' }), 0); }); test('a cross-location return within the free-day allowance is free too', () => { assert.equal(lateFee(RECORD, { day: 1016, location: 'hillside' }), 0); }); ``` ```sh node --test --test-reporter=tap coverage.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a delayed return is charged per day ok 2 - a delayed cross-location return gets an extra fee ok 3 - a return within the free-day allowance is free not ok 4 - a cross-location return within the free-day allowance is free too # tests 4 # pass 3 # fail 1 ``` The fix separates a return within the free-day allowance with an early return. ```js // fee.mjs — version 2: the cross-location fee is only charged on a delayed return export const LATE_FEE = { freeDays: 3, dailyFee: 2, cap: 40, crossLocationFee: 5 }; export function lateFee(record, checkin) { const delay = checkin.day - record.dueDay; if (delay <= LATE_FEE.freeDays) return 0; let fee = (delay - LATE_FEE.freeDays) * LATE_FEE.dailyFee; if (checkin.location !== record.location) { fee += LATE_FEE.crossLocationFee; } return Math.min(fee, LATE_FEE.cap); } ``` ```sh node --test --test-reporter=tap coverage.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a delayed return is charged per day ok 2 - a delayed cross-location return gets an extra fee ok 3 - a return within the free-day allowance is free ok 4 - a cross-location return within the free-day allowance is free too # tests 4 # pass 4 # fail 0 ``` The early return has a second effect: for a return within the free-day allowance, the location condition is no longer evaluated at all. Four paths become three, because two paths merge into a single exit. Reducing the number of paths is another way to raise coverage besides writing tests, and it is usually cheaper. #### Reading the Percentage The correct reading of a coverage percentage runs one way only. A low coverage number gives **certain** information: a part of the code never ran. A high coverage number is proof of nothing; it says the code ran, not that it behaved correctly. The tautological test measured in the second lesson produces one hundred percent coverage and catches not a single regression. Two practical consequences follow from this. When coverage is used as a **target**, the metric breaks down: the cheapest way to raise the number is to write assertion-free tests, and that raises the percentage without producing any signal. When coverage is used as a **list**, it is directly useful: the uncovered branches and paths are the names of questions not yet asked. The bug above came straight out of that list. The built-in test runner has an option that produces a coverage report. That option's name, the report's format, and whether it is experimental depend on the runtime version; the table it produces contains absolute file paths and details of the environment it ran in. This is why coverage was counted by hand here: the **definition** of the measurement is independent of the tool, its output is not. #### Summary - Coverage measurement is defined once the unit to be counted is chosen; statement, branch, and path coverage give different numbers on the same code. - In the measurement, two tests brought statement coverage to one hundred percent, branch coverage to seventy-five percent, and path coverage to fifty percent. - Statement coverage is always greater than or equal to branch coverage, and branch coverage is always greater than or equal to path coverage. - One of the uncovered paths harbored a real bug: the cross-location fee was being applied to a book returned on time. - Low coverage gives certain information, high coverage is proof of nothing; coverage is read not as a target but as a list of unasked questions. #### Next Step In this lesson, tests were written without looking inside the code: input was given, the return value was checked. The spies and mocks in the sixth lesson, by contrast, looked inside the code — they verified which call was made. Both approaches can verify the same behavior, but they do not behave the same way in the face of a refactor. The next lesson actually performs a refactor and shows, with a run, which of the two test forms breaks. ### Flaky Tests Source: https://academia.sh/en/courses/unit-testing/flaky-tests Implementation-bound tests breaking under a refactor with no behavior change, behavior-bound tests staying green under the same refactor, and flakiness measured with the flake rate. In the previous lesson, tests were written without looking inside the code: input was given, the return value was checked. In the sixth lesson, by contrast, spies and mocks looked inside the code — they verified which call was made. Both approaches can verify the same behavior, but they do not behave the same way in the face of the same change. A test being **flaky** means it fails even though the behavior it tests has not changed. Flakiness has two sources: the test binding to something non-deterministic, and the test binding to the implementation. The fourth and fifth lessons showed the first. This lesson measures the second with a refactor. #### The Unit Under Test The reservation queue holds members waiting for a book that is out on loan. Its contract has four clauses: a member is added to the queue and learns their waiting position, the book is assigned to the first member in the queue, the same member cannot join the queue twice, and a notification goes to the member on every state change. Every operation is also written to a log. ```js // reservation.mjs — version 1: array-based reservation queue export function createReservationQueue({ notification, log }) { const queue = []; return { queue, add(memberId) { if (queue.includes(memberId)) throw new Error(`member already in queue: ${memberId}`); queue.push(memberId); log.write(`add ${memberId} position ${queue.length}`); notification.send(memberId, `queue: ${queue.length}`); return queue.length; }, next() { const memberId = queue.shift(); if (memberId === undefined) return undefined; log.write(`next ${memberId}`); notification.send(memberId, 'book assigned'); return memberId; }, length: () => queue.length, }; } ``` ```js // spy.mjs — small call-recording test double export function spy(...methods) { const calls = []; const obj = { calls }; for (const name of methods) obj[name] = (...args) => calls.push([name, ...args]); return obj; } ``` #### Two Test Sets The first set tests only what can be observed from outside: return values, the thrown error, and the notification sent to the member. The notification sent to the member is a side effect and is verified with a spy — but because it is part of the contract, it counts as behavior. ```js // behavior.test.mjs — tests only the externally observed behavior import { test } from 'node:test'; import assert from 'node:assert/strict'; import { createReservationQueue } from './reservation.mjs'; import { spy } from './spy.mjs'; const setup = () => { const notification = spy('send'); return { queue: createReservationQueue({ notification, log: spy('write') }), notification }; }; test('the order of adding determines the waiting position', () => { const { queue } = setup(); assert.equal(queue.add('U-17'), 1); assert.equal(queue.add('U-42'), 2); }); test('the book is assigned to the first member in the queue', () => { const { queue } = setup(); queue.add('U-17'); queue.add('U-42'); assert.equal(queue.next(), 'U-17'); assert.equal(queue.next(), 'U-42'); assert.equal(queue.next(), undefined); }); test('the same member cannot join the queue twice', () => { const { queue } = setup(); queue.add('U-17'); assert.throws(() => queue.add('U-17'), /member already in queue: U-17/); assert.equal(queue.length(), 1); }); test('a message telling the member their position is sent', () => { const { queue, notification } = setup(); queue.add('U-17'); queue.next(); assert.deepEqual(notification.calls, [ ['send', 'U-17', 'queue: 1'], ['send', 'U-17', 'book assigned'], ]); }); ``` The second set tests the same unit, but binds to the internal structure and the log text. Both are outside the contract: whether the queue is kept as an array or a map, and which sentence is written to the log, are of no concern to the member. ```js // implementation.test.mjs — tests bound to internal structure and log text import { test } from 'node:test'; import assert from 'node:assert/strict'; import { createReservationQueue } from './reservation.mjs'; import { spy } from './spy.mjs'; const setup = () => { const log = spy('write'); return { queue: createReservationQueue({ notification: spy('send'), log }), log }; }; test("the queue's internal array keeps the order of adding", () => { const { queue } = setup(); queue.add('U-17'); queue.add('U-42'); assert.deepEqual(queue.queue, ['U-17', 'U-42']); }); test('the lines written to the log have a specific format', () => { const { queue, log } = setup(); queue.add('U-17'); queue.next(); assert.deepEqual(log.calls, [ ['write', 'add U-17 position 1'], ['write', 'next U-17'], ]); }); ``` ```sh node --test --test-reporter=tap behavior.test.mjs implementation.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - the order of adding determines the waiting position ok 2 - the book is assigned to the first member in the queue ok 3 - the same member cannot join the queue twice ok 4 - a message telling the member their position is sent ok 5 - the queue's internal array keeps the order of adding ok 6 - the lines written to the log have a specific format # tests 6 # pass 6 # fail 0 ``` All six tests are green. At this point the difference between the two sets is invisible; both appear to verify the same code. #### The Refactor The queue is moved to a map, the internal array is no longer exposed, and the log lines are fit into a single pattern. All four clauses of the contract are preserved exactly: the return values, the thrown error, and the messages sent to the member do not change. ```js // reservation.mjs — version 2: queue moved to a map, log format collapsed to one pattern export function createReservationQueue({ notification, log }) { const entries = new Map(); let sequence = 0; const queued = () => [...entries.entries()] .sort((a, b) => a[1] - b[1]) .map(([memberId]) => memberId); return { add(memberId) { if (entries.has(memberId)) throw new Error(`member already in queue: ${memberId}`); sequence += 1; entries.set(memberId, sequence); const position = entries.size; log.write(`reservation member=${memberId} status=added position=${position}`); notification.send(memberId, `queue: ${position}`); return position; }, next() { const [first] = queued(); if (first === undefined) return undefined; entries.delete(first); log.write(`reservation member=${first} status=assigned`); notification.send(first, 'book assigned'); return first; }, length: () => entries.size, }; } ``` Same two files, same command, a refactored library. ```sh node --test --test-reporter=tap behavior.test.mjs implementation.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - the order of adding determines the waiting position ok 2 - the book is assigned to the first member in the queue ok 3 - the same member cannot join the queue twice ok 4 - a message telling the member their position is sent not ok 5 - the queue's internal array keeps the order of adding not ok 6 - the lines written to the log have a specific format # tests 6 # pass 4 # fail 2 ``` The measurement is clean: four behavior tests stayed green, two implementation tests failed. The library's contract did not change — no member sees a different result. The two failing tests did not report a regression; they reported that **their own assumptions** had broken. This is the cost of flakiness. Refactoring means improving the internal structure while preserving behavior; a test suite that fails every time this is done makes refactoring expensive. Once it is expensive enough, it stops happening, and the code becomes unimprovable because of the very tests meant to protect it. The boundary question that follows is this: is an assertion bound to something that is part of the contract? The fourth behavior test also uses a spy, and it too verifies a side effect — but the notification sent to the member is in the contract. What draws the line is not the type of tool but whether the thing being verified is something someone else observes. #### The Measure of Flakiness The other source of flakiness is non-deterministic input, and there the measure is the **flake rate**: the percentage of drops across a large number of runs of the same test on the same code. ```js // flake-rate.mjs — a flaky test's flake rate measured with a fixed seed function generator(seed) { let state = (seed * 2654435761) % 2147483647; return () => { state = (state * 48271) % 2147483647; return state / 2147483647; }; } // A batch job's duration varies by machine and load: between forty and sixty-nine. const durationFor = (random) => 40 + Math.floor(random() * 30); function budgetBoundTest(random) { const duration = durationFor(random); if (duration > 68) throw new Error(`duration budget exceeded: ${duration}`); } function behaviorBoundTest(random) { const duration = durationFor(random); if (Number.isInteger(duration) === false || duration < 0) throw new Error(`invalid duration: ${duration}`); } const RUNS = 1000; for (const [label, testFn] of [['duration-budget-bound', budgetBoundTest], ['behavior-bound', behaviorBoundTest]]) { const random = generator(20); let failed = 0; for (let i = 0; i < RUNS; i += 1) { try { testFn(random); } catch { failed += 1; } } const rate = ((failed / RUNS) * 100).toFixed(1); console.log(`${label.padEnd(21)} ${RUNS} runs, ${failed} drops, flake rate ${rate}%`); } ``` ``` duration-budget-bound 1000 runs, 27 drops, flake rate 2.7% behavior-bound 1000 runs, 0 drops, flake rate 0.0% ``` A rate of two point seven percent is dangerous precisely because it is hard to notice. A test that is green in forty-nine runs out of fifty gets called "flaky now and then," and the run where it fails gets re-run. At that point the meaning of the suite breaks down: a green run no longer says "the code is correct" — it starts saying "we got lucky this time." Once the flake rate is measured, the work to do is clear. The flaky test is put into **quarantine** — removed from the main run so the remaining tests' signal stays clean — and its source is fixed. In the example above, the fix is to stop loading a duration budget onto a unit test; duration is a performance metric and is measured separately. #### Summary - A flaky test is a test that fails even though the behavior it tests has not changed; its two sources are non-deterministic input and binding to the implementation. - When the queue array was moved to a map and the log format changed, four behavior tests stayed green and two implementation tests failed; the library's contract had not changed at all. - Implementation-bound tests make refactoring expensive; once expensive enough, the code becomes unimprovable because of the very tests meant to protect it. - The question that draws the line is scope, not tool: is the thing being verified part of the contract, or an internal detail? - The measure of non-deterministic flakiness is the flake rate; in the measurement, the test bound to the duration budget failed in twenty-seven of a thousand runs, and the test bound to behavior never failed. #### Next Step The Unit Testing Practice topic closes here. Up to this point, in every test the code came first: a bug report, a rule, or a refactor found an implementation already there, ready to be tested. The next topic reverses this order. When the test is written first, the test determines what code gets written; a cycle that starts with a red run ends, after reaching green, with a refactor. The next lesson runs these three steps one at a time. ## Test-Driven Development The steps of the red-green-refactor cycle, step size's effect on feedback time, defining a contract with examples, automating the acceptance criterion, and bringing legacy code under test by opening a seam. ### Red-Green-Refactor Source: https://academia.sh/en/courses/unit-testing/red-green-refactor Running each step of the three-step cycle in which the test is written before the code, generalizing a green passed with a fixed value through triangulation, and the guarantee condition of the refactor step. In the Unit Testing in Practice topic, the code came first in every test: a bug report, a rule, or a refactor found an implementation already there to test. The test verified that implementation, measured it, or exposed its fragility. This topic reverses that order. When the test is written first, it stops being a verification tool and becomes a **design tool**: the test decides which function will exist, under what signature, what input it takes, and what it returns. This reversal is called **test-driven development**, and it works through a three-step cycle. The new rule to build is loan renewal. A member can extend the loan on a book they are holding; a renewed loan's due day moves forward from the day of renewal by the loan period for that member's type. #### Red: Testing Code That Does Not Exist Yet The cycle begins with a failing test. For the test to run at all, the function under test must **exist**; so the first thing written is an empty implementation, just enough to compile. ```js // renewal.mjs — version 0: function exists, no behavior export function renew(record, today) { return record; } ``` The test itself describes behavior that has not been written yet. ```js // renewal.test.mjs — red step: single rule, no implementation yet import { test } from 'node:test'; import assert from 'node:assert/strict'; import { renew } from './renewal.mjs'; test('a student loan renewal moves the due day twenty-eight days past today', () => { const record = { memberNo: 'U-17', memberType: 'student', dueDay: 1028 }; const updated = renew(record, 1020); assert.equal(updated.dueDay, 1048); }); ``` ```sh node --test --test-reporter=tap renewal.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` not ok 1 - a student loan renewal moves the due day twenty-eight days past today # tests 1 # pass 0 # fail 1 ``` Three decisions were made in this step at once, and none of them while writing the implementation. The function's name is `renew`, its inputs are a loan record and a day number, its output is an updated record. This is the design side of test-driven development: the interface is derived from the code that **uses** it. The guarantee of the red run is the thing named in the first lesson: it was observed that the test can fail. A test that is green the moment it is written may be testing the wrong thing, or testing nothing at all; that possibility is ruled out only by watching a run fail. #### Green: The Shortest Path The second step has a single criterion: pass the test. Correct design, a general solution, or clean code are not this step's goal; those belong to the third step. ```js // renewal.mjs — version 1: shortest way to pass the test export function renew(record, today) { return { ...record, dueDay: 1048 }; } ``` ```sh node --test --test-reporter=tap renewal.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student loan renewal moves the due day twenty-eight days past today # tests 1 # pass 1 # fail 0 ``` This version, which returns a fixed value, is obviously wrong, and deliberately so. What it accomplishes is proving that the test really does check that value: the reason the run turns from red to green is the line that was written, and nothing else. This step is also a diagnostic tool. If the test still does not turn green even after the fixed value is written, the problem is not in the implementation — it is in the test itself. #### Triangulation What exposes the fixed value's wrongness is not code — it is a **second example**. Writing the same rule with a different input forces the implementation to generalize. This technique is called **triangulation**. ```js // renewal.test.mjs — version 2: second member type added import { test } from 'node:test'; import assert from 'node:assert/strict'; import { renew } from './renewal.mjs'; test('a student loan renewal moves the due day twenty-eight days past today', () => { const record = { memberNo: 'U-17', memberType: 'student', dueDay: 1028 }; const updated = renew(record, 1020); assert.equal(updated.dueDay, 1048); }); test('a standard member loan renewal moves the due day fourteen days past today', () => { const record = { memberNo: 'U-42', memberType: 'member', dueDay: 1014 }; const updated = renew(record, 1020); assert.equal(updated.dueDay, 1034); }); ``` ```sh node --test --test-reporter=tap renewal.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student loan renewal moves the due day twenty-eight days past today not ok 2 - a standard member loan renewal moves the due day fourteen days past today # tests 2 # pass 1 # fail 1 ``` The cycle is back to red, and this time what red points to is clear: the fixed value is not enough. Generalizing is no longer a guess — it is a necessity imposed by the failing test. ```js // renewal.mjs — version 2: rule table instead of a fixed value export const RULES = { student: { loanDays: 28 }, member: { loanDays: 14 } }; export function renew(record, today) { return { ...record, dueDay: today + RULES[record.memberType].loanDays }; } ``` ```sh node --test --test-reporter=tap renewal.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student loan renewal moves the due day twenty-eight days past today ok 2 - a standard member loan renewal moves the due day fourteen days past today # tests 2 # pass 2 # fail 0 ``` #### Refactor: Working on Top of Green The third step has a precise definition: **refactoring** is improving internal structure without changing observable behavior. No new rule is added, no new case is handled, no bug is fixed. Only structure changes. ```js // renewal.mjs — version 3: duration lookup moved into a named function export const RULES = { student: { loanDays: 28 }, member: { loanDays: 14 } }; export function loanPeriod(memberType) { return RULES[memberType].loanDays; } export function renew(record, today) { return { ...record, dueDay: today + loanPeriod(record.memberType) }; } ``` ```sh node --test --test-reporter=tap renewal.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student loan renewal moves the due day twenty-eight days past today ok 2 - a standard member loan renewal moves the due day fourteen days past today # tests 2 # pass 2 # fail 0 ``` What makes this step possible is the two steps before it. Without a green test suite, refactoring is a rewrite, and its correctness can only be checked by eye. Every structural change made while the tests are green is verified by the run itself. The measurement from the Flaky Tests lesson finds its counterpart here. If the tests are bound to the implementation, this step turns red and refactoring is punished. If the tests are bound to behavior, the third step is free. Because test-driven development derives its tests from the code that uses them, it makes writing behavior-bound tests easier — but it does not guarantee it; the line between a contract and an internal detail is still drawn by the person writing the test. #### The Questions the Three Steps Separate The cycle's value is that its steps separate three questions that would otherwise have to be considered all at once. In the red step there is a single question: **what** do I want? What is the rule, how will the interface be used, what is the expected result? Implementation is not considered at all. In the green step there is a single question: **how** do I make this test pass? The shortest path is enough; design concerns are a distraction in this step. In the refactor step there is a single question: **how does the code at hand get better**? Whether the answer to that question is correct is reported by the tests. Considering all three questions at once is a constant occurrence in an ordinary programming session, and its cost is a cost of attention. The cycle lowers that cost by splitting it into three. #### Summary - In test-driven development, the test is not a verification tool but a design tool: the function's name, inputs, and output are derived from the test that uses it. - The guarantee of the red step is seeing that the test can fail; a test that is green the moment it is written may be testing the wrong thing. - The green step has a single criterion: pass the test; a version that returns a fixed value proves that the line just written is the reason for the pass. - Triangulation forces the implementation to generalize with a second example; generalizing is not a guess but something the failing test demands. - Refactoring improves internal structure without changing behavior, and it is only safe on top of a green test suite. #### Next Step In this lesson the cycle turned twice, and each round handled a single rule. Step size is a choice: the same work could have been carried out in a single round covering three rules at once. That choice has a measurable cost — where a failing test points, how many lines changed between two runs, and the distance back to the last green point. The next lesson builds the same feature with two different step sizes and compares the two runs. ### Progressing in Small Steps Source: https://academia.sh/en/courses/unit-testing/progressing-in-small-steps Measuring the paths that reach the same rule set with two different step sizes, the relationship between the number of feedback points and the search space for a bug, and the cost of returning to the last green point. In the previous lesson the cycle turned twice, and each round handled a single rule. That was not a requirement — it was a choice: the same work could have been carried out in a single round covering all four rules at once. Step size is test-driven development's most debated and least measured variable. This lesson ties step size to something measurable. The same target is reached by two different paths, and two quantities are recorded at every stop: how many tests are failing, and how many lines have changed since the last stop. The second quantity matters, because when a test fails, that is exactly the space where the bug has to be searched for. #### The Target: Four Rules The renewal rules have grown to four. A renewal moves the due day forward from today by the loan period for the member's type, and it increments the renewal count. A reserved book cannot be renewed. A loan can be renewed at most twice. An overdue loan cannot be renewed; a renewal made on the due day itself is accepted. ```js // renewal.mjs — all four rules export const RULES = { student: { loanDays: 28 }, member: { loanDays: 14 } }; export const MAX_RENEWALS = 2; export const LIMIT_MESSAGE = `at most ${MAX_RENEWALS} renewals allowed`; export function renew(record, today) { if (record.reserved) throw new Error('a reserved book cannot be renewed'); if (record.renewalCount >= MAX_RENEWALS) throw new Error(LIMIT_MESSAGE); if (today > record.dueDay) throw new Error('an overdue loan cannot be renewed'); return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } ``` The four rules correspond to seven tests. The edge case — a renewal made on the due day itself — stands as a separate test, because choosing the wrong comparison operator shows up right there. ```js // renewal.test.mjs — seven tests for all four rules import { test } from 'node:test'; import assert from 'node:assert/strict'; import { renew } from './renewal.mjs'; const record = (extra = {}) => ({ memberNo: 'U-17', memberType: 'student', dueDay: 1028, renewalCount: 0, reserved: false, ...extra, }); test('a student renewal moves the due day twenty-eight days out from today', () => { assert.equal(renew(record(), 1020).dueDay, 1048); }); test('a standard member renewal moves the due day fourteen days out from today', () => { assert.equal(renew(record({ memberType: 'member' }), 1020).dueDay, 1034); }); test('the renewal count goes up by one', () => { assert.equal(renew(record(), 1020).renewalCount, 1); }); test('a reserved book cannot be renewed', () => { assert.throws(() => renew(record({ reserved: true }), 1020), /reserved/); }); test('at most two renewals are allowed', () => { assert.throws(() => renew(record({ renewalCount: 2 }), 1020), /at most 2/); }); test('an overdue loan cannot be renewed', () => { assert.throws(() => renew(record(), 1029), /overdue/); }); test('a renewal made on the due day is accepted', () => { assert.equal(renew(record(), 1028).dueDay, 1056); }); ``` ```sh node --test --test-reporter=tap renewal.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - a student renewal moves the due day twenty-eight days out from today ok 2 - a standard member renewal moves the due day fourteen days out from today ok 3 - the renewal count goes up by one ok 4 - a reserved book cannot be renewed ok 5 - at most two renewals are allowed ok 6 - an overdue loan cannot be renewed ok 7 - a renewal made on the due day is accepted # tests 7 # pass 7 # fail 0 ``` This is the destination. The question is how it is reached. #### Measuring the Two Paths The script below holds the path's intermediate stops as implementation versions. The big-step path has two stops: the version where the first rule is written and the finished version. The small-step path has five stops, and each stop adds one rule. The same seven checks run at every stop, and the number of body lines that changed between two consecutive stops is also counted. ```js // step-size.mjs — measuring the path to the same target with two step sizes import assert from 'node:assert/strict'; import { renew, RULES, MAX_RENEWALS, LIMIT_MESSAGE } from './renewal.mjs'; function S1(record, today) { return { ...record, dueDay: today + RULES[record.memberType].loanDays, }; } function S2(record, today) { return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } function S3(record, today) { if (record.reserved) throw new Error('a reserved book cannot be renewed'); return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } function S4(record, today) { if (record.reserved) throw new Error('a reserved book cannot be renewed'); if (record.renewalCount >= MAX_RENEWALS) throw new Error(LIMIT_MESSAGE); return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } function S5Hatali(record, today) { if (record.reserved) throw new Error('a reserved book cannot be renewed'); if (record.renewalCount >= MAX_RENEWALS) throw new Error(LIMIT_MESSAGE); if (today >= record.dueDay) throw new Error('an overdue loan cannot be renewed'); return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } const record = (extra = {}) => ({ memberNo: 'U-17', memberType: 'student', dueDay: 1028, renewalCount: 0, reserved: false, ...extra, }); const TESTS = { 'student period': (r) => assert.equal(r(record(), 1020).dueDay, 1048), 'member period': (r) => assert.equal(r(record({ memberType: 'member' }), 1020).dueDay, 1034), 'count increments': (r) => assert.equal(r(record(), 1020).renewalCount, 1), 'reserved rejection': (r) => assert.throws(() => r(record({ reserved: true }), 1020), /reserved/), 'renewal limit': (r) => assert.throws(() => r(record({ renewalCount: 2 }), 1020), /at most 2/), 'delay rejection': (r) => assert.throws(() => r(record(), 1029), /overdue/), 'due day acceptance': (r) => assert.equal(r(record(), 1028).dueDay, 1056), }; const TOTAL = Object.keys(TESTS).length; function failing(version) { return Object.entries(TESTS).filter(([, t]) => { try { t(version); return false; } catch { return true; } }).map(([name]) => name); } // The signature line is not counted; only body lines are compared. function changedLines(before, after) { const body = (f) => f.toString().split('\n').slice(1).map((s) => s.trim()).filter(Boolean); const remaining = body(before); let added = 0; for (const line of body(after)) { const i = remaining.indexOf(line); if (i === -1) added += 1; else remaining.splice(i, 1); } return added + remaining.length; } const PATHS = { 'big step': [S1, renew], 'small step': [S1, S2, S3, S4, renew] }; for (const [name, stops] of Object.entries(PATHS)) { console.log(name); let largest = 0; stops.forEach((version, i) => { const diff = i === 0 ? 0 : changedLines(stops[i - 1], version); largest = Math.max(largest, diff); console.log(` stop ${i + 1} failing ${failing(version).length}/${TOTAL} changed lines ${diff}`); }); console.log(` feedback points ${stops.length}, largest search space ${largest} lines`); } const wrongDiff = { 'big step': changedLines(S1, S5Hatali), 'small step': changedLines(S4, S5Hatali) }; console.log(`test failing when the wrong operator is written at the last stop: ${failing(S5Hatali).join(', ')}`); for (const [name, diff] of Object.entries(wrongDiff)) { console.log(` ${name}: search space to check ${diff} lines`); } ``` ``` big step stop 1 failing 4/7 changed lines 0 stop 2 failing 0/7 changed lines 4 feedback points 2, largest search space 4 lines small step stop 1 failing 4/7 changed lines 0 stop 2 failing 3/7 changed lines 1 stop 3 failing 2/7 changed lines 1 stop 4 failing 1/7 changed lines 1 stop 5 failing 0/7 changed lines 1 feedback points 5, largest search space 1 lines test failing when the wrong operator is written at the last stop: due day acceptance big step: search space to check 4 lines small step: search space to check 1 lines ``` #### Reading the Measurement Both paths arrive at the same place, and at the destination both tables pass all seven tests. The difference is in the path. On the big-step path, the failing-test count drops from four to zero in a single jump. Inside that jump are four lines of writing, and none of the four was verified on its own. If all four are correct there is no problem; if one is wrong, which line failed which test can only be found by picking through it by hand. On the small-step path, the failing-test count drops one at a time: four, three, two, one, zero. Every drop corresponds to a single line of writing. This is the measurement's real finding — the **search space** drops from four lines to one. The last three lines show this directly. When the comparison operator in the overdue check is chosen wrong — greater-than-or-equal instead of greater-than — only one test fails: the acceptance of a renewal made on the due day. The failing test is the same on both paths. What differs is the space that test points to: four lines on the big-step path, one line on the small-step path. This is where the concept of feedback time comes from. Feedback time is the distance between the moment a writing mistake is made and the moment it is noticed; its unit is not seconds but **the number of lines written in that interval**. As step size shrinks, that distance shortens. #### Returning to the Last Green Point The second gain of the small step is the cost of turning back. If a piece of writing turns unexpectedly complicated, the cheapest fix is often to throw it away and start over. The cost of that decision is the size of the work being thrown away. On the big-step path, the last green point is the first stop: turning back means throwing away all four lines. On the small-step path, the last green point is always one line back. The same holds for interpreting a failing test: on the small-step path, "the line I just wrote" is always a single statement. This has a cost, and it shows in the table: the small-step path has five feedback points, the big-step path has two. Every point means a run. If the run is cheap — fast and independent tests in the sense measured in the Fast and Independent Tests lesson — this cost is negligible. If the run is expensive, step size grows on its own and feedback time lengthens. That is the link between the two lessons: the speed of the test suite determines the development cycle's step size. #### Choosing a Step Size Step size is not a fixed rule; it is a variable adjusted to uncertainty. If what is being written is familiar, a big step is cheap and intermediate stops are wasted time. If what is being written is uncertain — a new domain, an unfamiliar interface, a complex edge case — the step shrinks. In practice, the criterion is this: if the number of failing tests at the end of a step comes out different from what was expected, the step is too big. In the small-step table above, every stop turns exactly one test green; that shows the step size fits the grain of the rule. #### Summary - Step size is measured by the number of lines written between two feedback points, and the space to search for a bug is exactly that number. - In the measurement, the big-step path dropped the failing-test count from four to zero in one jump; the small-step path dropped it one at a time. - The same writing mistake failed a single test on both paths, but the search space was four lines on the big-step path and one line on the small-step path. - On the small step, the last green point is always one line back; the cost of a turning-back decision is that distance. - The cost of the small step is more runs; if the test suite is slow, step size grows on its own and feedback time lengthens. #### Next Step The seven tests in this lesson described the rules with separate sentences, and each sentence became a code block. As rules multiply, this way of writing repeats itself: the same fixture, the same call, only the input and expected result change. The rule itself stays embedded in code and cannot be read while talking with the library clerk. The next lesson moves the rules into example tables, defines a format for those tables, and writes a parser that turns a table into a runnable test. ### Behavior-Driven Development Source: https://academia.sh/en/courses/unit-testing/behavior-driven-development Defining rules with example tables written in the domain's language, writing a format and a parser for that table, and sustaining the cycle with data-driven tests generated from the table. The seven tests in the previous lesson described the rules with separate sentences, and each sentence became a code block. As rules multiply, this way of writing repeats itself: the same fixture, the same call, only the input and expected result change. More importantly, the rule itself stays embedded in code. The person who sets the loan rules is the library clerk, but the place where the rules are written is a test file, and that file cannot be opened while talking with the clerk. **Behavior-driven development** aims to close that gap. The cycle is the same cycle — red, green, refactor — but the unit of conversation is not a test function, it is an **example** written in the domain's language. An example carries three parts: a given starting state, an action taken, and an expected result. This triad is the arrange, act, and assert sections from the Anatomy of a Unit Test lesson translated into the domain's language. #### The Format of the Example Table The format the examples are written in is not tied to a standard; the only thing it needs to carry is that both the clerk can read it and a program can parse it. The format below recognizes two kinds of line: a rule heading and the examples that belong to that rule. ```bash cat > examples.txt <<'EX' # renewal contract — every rule is defined by the examples that demonstrate it rule: A renewal moves the due day forward from today by the loan period for the member's type example: memberType=student today=1020 dueDay=1028 -> dueDay=1048 example: memberType=member today=1020 dueDay=1024 -> dueDay=1034 rule: A reserved book cannot be renewed example: today=1020 dueDay=1028 reserved=yes -> error=reserved rule: A loan can be renewed at most twice example: today=1020 dueDay=1028 renewalCount=1 -> renewalCount=2 example: today=1020 dueDay=1028 renewalCount=2 -> error=at most 2 rule: An overdue loan cannot be renewed, a renewal made on the due day is accepted example: today=1029 dueDay=1028 -> error=overdue example: today=1028 dueDay=1028 -> dueDay=1056 EX ``` The left side of the arrow is the given state, the right side is the expected result; the action taken is singular and is clear from the file's name. Unspecified fields take their default values, so each example shows only the fields relevant to its own rule. #### The Parser The format does not run on its own; it needs a parser that turns it into a data structure. A parser under fifty lines is enough for this job, and it matters that it throws on a line it does not recognize instead of staying silent — a silently skipped line means a rule that never runs. ```js // parser.mjs — turns the example file into a list of rules and examples const value = (text) => { if (text === 'yes') return true; if (text === 'no') return false; return /^-?\d+$/.test(text) ? Number(text) : text; }; const fields = (chunk) => Object.fromEntries( chunk.trim().split(/\s+/).filter(Boolean).map((pair) => { const [name, ...rest] = pair.split('='); return [name, value(rest.join('='))]; }), ); export function parse(text) { const rules = []; for (const raw of text.split('\n')) { const line = raw.trim(); if (line === '' || line.startsWith('#')) continue; if (line.startsWith('rule:')) { rules.push({ title: line.slice(5).trim(), examples: [] }); continue; } if (line.startsWith('example:')) { if (rules.length === 0) throw new Error(`example without a rule: ${line}`); const body = line.slice(8); const [input, output] = body.split('->'); if (output === undefined) throw new Error(`missing arrow: ${line}`); const expected = output.trim(); rules.at(-1).examples.push({ text: body.trim(), input: fields(input), error: expected.startsWith('error=') ? expected.slice(6) : undefined, result: expected.startsWith('error=') ? undefined : fields(expected), }); continue; } throw new Error(`unrecognized line: ${line}`); } return rules; } ``` The library under test is the version that implements the previous lesson's four rules. ```js // renewal.mjs — all four rules export const RULES = { student: { loanDays: 28 }, member: { loanDays: 14 } }; export const MAX_RENEWALS = 2; export const LIMIT_MESSAGE = `at most ${MAX_RENEWALS} renewals allowed`; export function renew(record, today) { if (record.reserved) throw new Error('a reserved book cannot be renewed'); if (record.renewalCount >= MAX_RENEWALS) throw new Error(LIMIT_MESSAGE); if (today > record.dueDay) throw new Error('an overdue loan cannot be renewed'); return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } ``` #### Tests Generated From the Table Tests are no longer written by hand; they are generated from the parsed table. This way of writing is called a **data-driven test**: the test body is singular, and the input set comes from the data. ```js // example.test.mjs — data-driven tests generated from the example table import { readFileSync } from 'node:fs'; import { describe, test } from 'node:test'; import assert from 'node:assert/strict'; import { parse } from './parser.mjs'; import { renew } from './renewal.mjs'; const DEFAULTS = { memberNo: 'U-17', memberType: 'student', dueDay: 1028, renewalCount: 0, reserved: false, nextInLine: 'U-99', }; const text = readFileSync(new URL('./examples.txt', import.meta.url), 'utf8'); for (const rule of parse(text)) { describe(rule.title, () => { for (const example of rule.examples) { test(example.text, () => { const { today, ...overrides } = example.input; const record = { ...DEFAULTS, ...overrides }; if (example.error !== undefined) { assert.throws(() => renew(record, today), new RegExp(example.error)); return; } const updated = renew(record, today); for (const [field, expected] of Object.entries(example.result)) { assert.equal(updated[field], expected, `${field} mismatch`); } }); } }); } ``` ```sh node --test --test-reporter=tap example.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - memberType=student today=1020 dueDay=1028 -> dueDay=1048 ok 2 - memberType=member today=1020 dueDay=1024 -> dueDay=1034 ok 1 - A renewal moves the due day forward from today by the loan period for the member's type ok 1 - today=1020 dueDay=1028 reserved=yes -> error=reserved ok 2 - A reserved book cannot be renewed ok 1 - today=1020 dueDay=1028 renewalCount=1 -> renewalCount=2 ok 2 - today=1020 dueDay=1028 renewalCount=2 -> error=at most 2 ok 3 - A loan can be renewed at most twice ok 1 - today=1029 dueDay=1028 -> error=overdue ok 2 - today=1028 dueDay=1028 -> dueDay=1056 ok 4 - An overdue loan cannot be renewed, a renewal made on the due day is accepted # tests 7 # pass 7 # fail 0 ``` The observation from the Test Naming lesson reaches its strongest form here: the list of passing tests is a rule document. The unindented lines give the rules, the indented lines give the examples that demonstrate that rule. This output and the example file carry the same text — one is the source's expression, the other is the run's. #### The Red Step Is Now a Line of Text Suppose a conversation with the clerk produces a new rule: a reserved book can be renewed if the member first in line for the reservation is the very member holding the book. This rule's red step no longer requires writing code. ```bash cat >> examples.txt <<'EX' rule: If the member first in line for the reservation is the one currently holding the loan, the renewal is accepted example: today=1020 dueDay=1028 reserved=yes nextInLine=U-17 -> dueDay=1048 EX ``` ```sh node --test --test-reporter=tap example.test.mjs | grep -E '^ *(not ok|# (tests|pass|fail))' ``` ``` not ok 1 - today=1020 dueDay=1028 reserved=yes nextInLine=U-17 -> dueDay=1048 not ok 5 - If the member first in line for the reservation is the one currently holding the loan, the renewal is accepted # tests 8 # pass 7 # fail 1 ``` Not a single line was added to the test file. What started the red step is two lines of text written in the domain's language. The green step, though, is still in code. ```js // renewal.mjs — version 2: rule added for the member first in line holding the loan export const RULES = { student: { loanDays: 28 }, member: { loanDays: 14 } }; export const MAX_RENEWALS = 2; export const LIMIT_MESSAGE = `at most ${MAX_RENEWALS} renewals allowed`; export function renew(record, today) { if (record.reserved && record.nextInLine !== record.memberNo) { throw new Error('a reserved book cannot be renewed'); } if (record.renewalCount >= MAX_RENEWALS) throw new Error(LIMIT_MESSAGE); if (today > record.dueDay) throw new Error('an overdue loan cannot be renewed'); return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } ``` ```sh node --test --test-reporter=tap example.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - memberType=student today=1020 dueDay=1028 -> dueDay=1048 ok 2 - memberType=member today=1020 dueDay=1024 -> dueDay=1034 ok 1 - A renewal moves the due day forward from today by the loan period for the member's type ok 1 - today=1020 dueDay=1028 reserved=yes -> error=reserved ok 2 - A reserved book cannot be renewed ok 1 - today=1020 dueDay=1028 renewalCount=1 -> renewalCount=2 ok 2 - today=1020 dueDay=1028 renewalCount=2 -> error=at most 2 ok 3 - A loan can be renewed at most twice ok 1 - today=1029 dueDay=1028 -> error=overdue ok 2 - today=1028 dueDay=1028 -> dueDay=1056 ok 4 - An overdue loan cannot be renewed, a renewal made on the due day is accepted ok 1 - today=1020 dueDay=1028 reserved=yes nextInLine=U-17 -> dueDay=1048 ok 5 - If the member first in line for the reservation is the one currently holding the loan, the renewal is accepted # tests 8 # pass 8 # fail 0 ``` The old reservation example is still green, because the next-in-line member defaults to someone else. The new rule did not invalidate the old one — it narrowed it, and what confirms that is that both still stand in the table. #### The Limits of Examples The example table has two gains, and both are limited. The first gain is a shared language: the rule is written so that someone who does not read code can verify it. Its limit is that the table only works this way as long as it is written in a spoken language. Once a class name replaces a domain word, or a method signature replaces a rule sentence, the table just turns into a more cumbersome test file. The second gain is that multiple examples for the same rule get cheaper. A new edge case is one line. Its limit is that not every behavior fits a table: for behavior that depends on run order, on time, or on a multi-step scenario, the table format falls short and a hand-written test is clearer. The two formats are used side by side, not as a replacement for each other. A third warning concerns the parser itself. The code that generates tests from the table is the single point every test's correctness depends on. A parser that silently skips a line, or resolves the expected result wrong, is as harmful as `assert.ok` in the Assertions lesson: everything looks green. This is why the parser throws on a line it does not recognize, and is tested itself. #### Summary - Behavior-driven development uses the same cycle, but the unit of conversation is an example written in the domain's language: a given state, an action taken, an expected result. - The example format is not tied to a standard; the only thing it needs to carry is that a person can read it and a program can parse it. - In a data-driven test the test body is singular and the input set comes from the data; the list of passing tests directly yields a rule document. - A new rule's red step is two lines added to the table; the test file is not touched. - The table format provides a shared language and cheap edge cases, but for multi-step behavior that depends on order or time, a hand-written test is clearer. #### Next Step In this lesson, examples described a single function's contract: the renewal rules. A user request is broader than that — the sentence "let a member extend their own book's loan" does not describe a function but an end-to-end flow, and it carries criteria that say it has been accepted. The next lesson turns those criteria into an automated check and builds a two-layer cycle whose outer loop runs on the acceptance criterion and whose inner loop runs on the unit test. ### Acceptance Test-Driven Development Source: https://academia.sh/en/courses/unit-testing/acceptance-test-driven-development Writing a user request's acceptance criteria in step form, automating them with a driver that binds the steps to the implementation, and a two-layer cycle whose outer loop runs on the acceptance criterion and whose inner loop runs on the unit test. In the previous lesson, examples described a single function's contract. A user request is broader than that. The sentence "let a member extend their own book's loan" does not describe a function but a flow: a record is found, a rule is applied, the result becomes permanent, and the member is notified. The criteria that say this flow is complete come along with the request itself. **Acceptance test-driven development** makes those criteria the starting point of development. The criterion is written first, turned into an automated check, and stays red; what says the work is done is that check turning green. Unit tests do not disappear — they get written where the acceptance criterion points. #### The Format of an Acceptance Criterion A criterion consists of three kinds of step. **Given** steps set up the starting state, a **when** step triggers a single action, **then** steps check the result. This split is the flow-level counterpart of the arrange, act, and assert sections from the Anatomy of a Unit Test lesson. ```bash cat > acceptance.txt <<'KABUL' # acceptance criteria — let a member extend their own loan story: Let a member extend the loan on the book they are holding criterion: A valid loan renewed moves the due day forward and sends the member a notification given: member no=U-17 type=student given: loan bookNo=K-903 memberNo=U-17 dueDay=1028 when: renewal memberNo=U-17 bookNo=K-903 today=1020 then: loan bookNo=K-903 dueDay=1048 renewalCount=1 then: notification memberNo=U-17 message="new due day: 1048" criterion: Renewing a reserved book is rejected and the due day does not change given: member no=U-42 type=member given: loan bookNo=K-101 memberNo=U-42 dueDay=1024 reserved=yes when: renewal memberNo=U-42 bookNo=K-101 today=1020 then: rejection reason="a reserved book cannot be renewed" then: loan bookNo=K-101 dueDay=1024 renewalCount=0 KABUL ``` There are two differences from the previous lesson's example lines. The steps are **ordered** and share state: a criterion tells a scenario, not a single call. Second, the words used in the steps are the application's concepts — member, loan, notification, rejection — not a specific function's signature. ```js // acceptance-parser.mjs — turns acceptance criteria into a list of steps const FIELD = /(\w+)=("[^"]*"|\S+)/g; const STEP_KEYS = ['given', 'when', 'then']; const convert = (raw) => { const value = raw.startsWith('"') ? raw.slice(1, -1) : raw; if (value === 'yes') return true; if (value === 'no') return false; return /^-?\d+$/.test(value) ? Number(value) : value; }; const fields = (text) => Object.fromEntries( [...text.matchAll(FIELD)].map(([, name, raw]) => [name, convert(raw)]), ); export function parse(text) { const criteria = []; for (const raw of text.split('\n')) { const line = raw.trim(); if (line === '' || line.startsWith('#') || line.startsWith('story:')) continue; if (line.startsWith('criterion:')) { criteria.push({ title: line.slice(10).trim(), steps: [] }); continue; } const [key, ...rest] = line.split(':'); if (STEP_KEYS.includes(key) === false) throw new Error(`unrecognized step: ${line}`); if (criteria.length === 0) throw new Error(`step without a criterion: ${line}`); const body = rest.join(':').trim(); criteria.at(-1).steps.push({ name: `${key} ${body.split(/\s+/)[0]}`, fields: fields(body), text: line, }); } return criteria; } ``` #### Implementation and the Driver The unit the criteria touch is not a single rule function but the service that runs the flow. The service loads the record, applies the rule, and saves the result. In this first version, sending the notification is **absent**; that is where the criterion stays red. ```js // renewal.mjs — the renewal rules export const RULES = { student: { loanDays: 28 }, member: { loanDays: 14 } }; export const MAX_RENEWALS = 2; export function renew(record, today) { if (record.reserved) throw new Error('a reserved book cannot be renewed'); if (record.renewalCount >= MAX_RENEWALS) throw new Error('at most 2 renewals allowed'); if (today > record.dueDay) throw new Error('an overdue loan cannot be renewed'); return { ...record, dueDay: today + RULES[record.memberType].loanDays, renewalCount: record.renewalCount + 1, }; } ``` ```js // service.mjs — version 1: loads the record, applies the rule, saves the result import { renew } from './renewal.mjs'; export function createService({ store, notification }) { return { renew(memberNo, bookNo, today) { const member = store.findMember(memberNo); const loan = store.findLoan(bookNo); if (member === undefined || loan === undefined || loan.memberNo !== memberNo) { return { status: 'rejected', reason: 'loan not found' }; } try { const updated = renew({ ...loan, memberType: member.type }, today); store.saveLoan(updated); return { status: 'accepted', loan: updated }; } catch (error) { return { status: 'rejected', reason: error.message }; } }, }; } ``` The link between the text and the implementation is the **driver**: a table that maps each step type to a piece of code. The driver builds a "world" with the test doubles introduced in the Test Doubles lesson, and runs the steps on that world. ```js // acceptance.test.mjs — turns acceptance criteria into a runnable test with the driver import { readFileSync } from 'node:fs'; import { describe, test } from 'node:test'; import assert from 'node:assert/strict'; import { parse } from './acceptance-parser.mjs'; import { createService } from './service.mjs'; function createWorld() { const members = new Map(); const loans = new Map(); const notifications = []; const store = { addMember: (member) => members.set(member.no, member), findMember: (no) => members.get(no), findLoan: (bookNo) => loans.get(bookNo), saveLoan: (loan) => loans.set(loan.bookNo, loan), }; const notification = { send: (memberNo, message) => notifications.push([memberNo, message]) }; return { store, notifications, service: createService({ store, notification }), result: undefined }; } const DEFAULT_LOAN = { renewalCount: 0, reserved: false }; const DRIVER = { 'given member': (w, a) => w.store.addMember({ no: a.no, type: a.type }), 'given loan': (w, a) => w.store.saveLoan({ ...DEFAULT_LOAN, ...a }), 'when renewal': (w, a) => { w.result = w.service.renew(a.memberNo, a.bookNo, a.today); }, 'then loan': (w, a) => { const { bookNo, ...expected } = a; const record = w.store.findLoan(bookNo); for (const [field, value] of Object.entries(expected)) { assert.equal(record[field], value, `${bookNo}.${field} mismatch`); } }, 'then notification': (w, a) => assert.deepEqual(w.notifications, [[a.memberNo, a.message]]), 'then rejection': (w, a) => { assert.equal(w.result.status, 'rejected'); assert.equal(w.result.reason, a.reason); }, }; const text = readFileSync(new URL('./acceptance.txt', import.meta.url), 'utf8'); for (const criterion of parse(text)) { describe(criterion.title, () => { const world = createWorld(); for (const step of criterion.steps) { test(step.text, () => { const driver = DRIVER[step.name]; if (driver === undefined) throw new Error(`step with no driver: ${step.name}`); driver(world, step.fields); }); } }); } ``` Every step runs as a separate test and shares the same world. The independence rule set up in the Fast and Independent Tests lesson is applied here at the criterion level: sharing stays **within a criterion**, and every criterion builds its own world. #### The Outer Loop: A Red Criterion ```sh node --test --test-reporter=tap acceptance.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - given: member no=U-17 type=student ok 2 - given: loan bookNo=K-903 memberNo=U-17 dueDay=1028 ok 3 - when: renewal memberNo=U-17 bookNo=K-903 today=1020 ok 4 - then: loan bookNo=K-903 dueDay=1048 renewalCount=1 not ok 5 - then: notification memberNo=U-17 message="new due day: 1048" not ok 1 - A valid loan renewed moves the due day forward and sends the member a notification ok 1 - given: member no=U-42 type=member ok 2 - given: loan bookNo=K-101 memberNo=U-42 dueDay=1024 reserved=yes ok 3 - when: renewal memberNo=U-42 bookNo=K-101 today=1020 ok 4 - then: rejection reason="a reserved book cannot be renewed" ok 5 - then: loan bookNo=K-101 dueDay=1024 renewalCount=0 ok 2 - Renewing a reserved book is rejected and the due day does not change # tests 10 # pass 9 # fail 1 ``` The second criterion is satisfied, the first is not. More valuable is that **which step** is not satisfied is visible: the due day advanced correctly, the record updated correctly, the only thing missing is the notification. The acceptance criterion thus became not a red step itself but the red step's **address**. #### The Inner Loop: A Unit Test An acceptance criterion is not translated directly into an implementation. A unit test is written where it points, and the cycle turns there. This two-layer structure is called the **outer loop** and the **inner loop**: the outer loop stays red for the length of a user request, the inner loop turns from red to green many times within that span. ```js // service.test.mjs — inner loop: the unit the acceptance criterion points to import { test } from 'node:test'; import assert from 'node:assert/strict'; import { createService } from './service.mjs'; function setup() { const loans = new Map([['K-903', { bookNo: 'K-903', memberNo: 'U-17', dueDay: 1028, renewalCount: 0, reserved: false }]]); const notifications = []; const store = { findMember: () => ({ no: 'U-17', type: 'student' }), findLoan: (bookNo) => loans.get(bookNo), saveLoan: (loan) => loans.set(loan.bookNo, loan), }; return { service: createService({ store, notification: { send: (m, i) => notifications.push([m, i]) } }), notifications }; } test('an accepted renewal notifies the member of the new due day', () => { const { service, notifications } = setup(); service.renew('U-17', 'K-903', 1020); assert.deepEqual(notifications, [['U-17', 'new due day: 1048']]); }); ``` ```sh node --test --test-reporter=tap service.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` not ok 1 - an accepted renewal notifies the member of the new due day # tests 1 # pass 0 # fail 1 ``` The green step is one line. ```js // service.mjs — version 2: an accepted renewal sends the member a notification import { renew } from './renewal.mjs'; export function createService({ store, notification }) { return { renew(memberNo, bookNo, today) { const member = store.findMember(memberNo); const loan = store.findLoan(bookNo); if (member === undefined || loan === undefined || loan.memberNo !== memberNo) { return { status: 'rejected', reason: 'loan not found' }; } try { const updated = renew({ ...loan, memberType: member.type }, today); store.saveLoan(updated); notification.send(memberNo, `new due day: ${updated.dueDay}`); return { status: 'accepted', loan: updated }; } catch (error) { return { status: 'rejected', reason: error.message }; } }, }; } ``` ```sh node --test --test-reporter=tap service.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' node --test --test-reporter=tap acceptance.test.mjs | grep -E '^(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - an accepted renewal notifies the member of the new due day # tests 1 # pass 1 # fail 0 ok 1 - A valid loan renewed moves the due day forward and sends the member a notification ok 2 - Renewing a reserved book is rejected and the due day does not change # tests 10 # pass 10 # fail 0 ``` The inner loop turned green, and the outer loop closed along with it. The second command filters to only the unindented lines; viewed at the criterion level, two lines are the entire information that says the work is done. #### The Division of Labor Between the Two Loops The difference between the outer and inner loop is speed and scope, and that difference is kept deliberately. The outer loop is **slow and wide**. A criterion touches the whole flow, so on its own it does not say where things broke when it fails — the step-by-step reporting in the run above partly covers that gap. It is few in number: a handful of criteria per user request. The inner loop is **fast and narrow**. A unit test checks a single rule and gives its address when it fails. It is many in number, and the signal strength measured in the Assertions lesson is what is sought there. Confusing the two loops leads to two typical mistakes. When acceptance criteria are multiplied like unit tests, the run slows down and the criteria list becomes unreadable; the property of being shareable with the user is lost. In the other direction, when no unit tests are written at all and everything is left to the acceptance criterion, a failing criterion starts a long search — the search-space measurement from the Progressing in Small Steps lesson holds here too. What says an acceptance criterion is done is also clear: when the criterion turns green, the work is considered **accepted**. This differs from a criterion of "the developer says it is done," because the definition of done is written together with the party requesting the work, and written in advance. #### Summary - An acceptance criterion is a check, written in advance, that says a user request has been satisfied; it consists of given, when, and then steps. - A criterion's steps are ordered and share state; the sharing stays within the criterion, and every criterion builds its own world. - The driver maps the step types in the text to implementation calls; a step with no driver is not silently skipped — it throws. - A red acceptance criterion is not the red step itself but its address; the unit test written in the inner loop goes to that address. - The outer loop is slow, wide, and few in number; the inner loop is fast, narrow, and many in number. Confusing the two produces either an unreadable criteria list or a long search space. #### Next Step Everything built in this topic started from a blank page: the test was written first, the code came after. That order cannot be established in an existing codebase. A function that has run for years, has no tests, and creates its dependencies internally must first have what it does put on record, then be made changeable. The next lesson does this work in three stages and closes the course. ### Bringing Legacy Code Under Test Source: https://academia.sh/en/courses/unit-testing/bringing-legacy-code-under-test Recording a seamless function's current behavior with characterization tests, making it changeable by opening a parameter seam, and the same tests staying green after the seam. Everything built in this topic started from a blank page: the test was written first, the code came after. That order cannot be established in an existing codebase. A function that has run for years is already there, its behavior is not written down, and most of the time nobody knows exactly what it does. The order for changing such a function is reversed: first **what it does** is put on record, then it is made **changeable**. The tool for the first step is characterization tests, for the second it is the seam. #### Seamless Code The function below produces a late report. It keeps records at module level, reads the day from the system clock, and writes the result directly to the output stream. ```js // legacy-report.mjs — the version that has run for years, with no tests const RECORDS = [ { bookNo: 'K-903', memberNo: 'U-17', dueDay: 1010, branch: 'central', returnBranch: 'central', returnDay: 1020 }, { bookNo: 'K-101', memberNo: 'U-42', dueDay: 1024, branch: 'central', returnBranch: 'hillside', returnDay: 1024 }, { bookNo: 'K-555', memberNo: 'U-58', dueDay: 1000, branch: 'hillside', returnBranch: 'hillside', returnDay: 1100 }, ]; export function writeLateReport() { const today = Math.floor(Date.now() / 86400000); let total = 0; console.log(`late report — day ${today}`); for (const record of RECORDS) { const late = record.returnDay - record.dueDay; let fee = 0; if (late > 3) fee = (late - 3) * 2; if (record.returnBranch !== record.branch) fee += 5; if (fee > 40) fee = 40; total += fee; console.log(`${record.bookNo} ${record.memberNo} late ${late} fee ${fee}`); } console.log(`total ${total}`); } ``` In a piece of code, a **seam** is a point where behavior can be altered without changing the source. This function has no such point. It has no input — the records are internal. It cannot take time from outside — it reads the clock itself. It does not return its result — it writes it. The measurement in the Testable Design lesson named exactly these three dependencies; here all three are combined in a single function. There is still one point of observation: the lines it writes. By mutating global objects, both the clock can be fixed and the writes can be collected. The Testable Design lesson said this path is expensive — here, temporarily, it is the only path. ```js // observation.mjs — recording the legacy function's output for today import { writeLateReport } from './legacy-report.mjs'; const realNow = Date.now; Date.now = () => 1020 * 86400000; try { writeLateReport(); } finally { Date.now = realNow; } ``` ``` late report — day 1020 K-903 U-17 late 10 fee 14 K-101 U-42 late 0 fee 5 K-555 U-58 late 100 fee 40 total 59 ``` There is something noticeable in the second line: the K-101 record has zero lateness and a fee of five. The Coverage Measurement lesson had determined this is a bug — the out-of-branch fee is being applied to a book returned on time. The characterization test's stance here is decisive. #### Characterization Tests A **characterization test** is a test that records not what the code should do, but what it **does right now**. It does not take its expected value from a specification; it takes it from a run. Its purpose is not to verify but to make a future change noticeable. This is a deliberate exception to the rule set in the Anatomy of a Unit Test lesson. There it was said that the expected value must come from a source independent of the code under test; here the expected value comes directly from the code under test. The difference is in purpose: these tests carry no claim of correctness — they establish a **baseline**. ```js // characterization.test.mjs — tests that record current behavior as it is import { test } from 'node:test'; import assert from 'node:assert/strict'; import { writeLateReport } from './legacy-report.mjs'; function capture(today) { const realNow = Date.now; const realLog = console.log; const lines = []; Date.now = () => today * 86400000; console.log = (line) => lines.push(line); try { writeLateReport(); } finally { Date.now = realNow; console.log = realLog; } return lines; } test('the report heading carries the day number for today', () => { assert.equal(capture(1020)[0], 'late report — day 1020'); }); test('the report lines are produced as they stand today', () => { assert.deepEqual(capture(1020).slice(1), [ 'K-903 U-17 late 10 fee 14', 'K-101 U-42 late 0 fee 5', 'K-555 U-58 late 100 fee 40', 'total 59', ]); }); test('the report content is not affected by the day number for today', () => { assert.deepEqual(capture(2000).slice(1), capture(1020).slice(1)); }); ``` ```sh node --test --test-reporter=tap characterization.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - the report heading carries the day number for today ok 2 - the report lines are produced as they stand today ok 3 - the report content is not affected by the day number for today # tests 3 # pass 3 # fail 0 ``` The second test also records the known bug: the line `K-101 U-42 late 0 fee 5` is inside the expected result. This is done on purpose. If the bug were fixed during the characterization stage, two changes would have been made at once — the structural change and the behavior change would blend together, and a failing test could not say which one it came from. The third test closes off an uncertainty: that the report's content does not depend on today is recorded by comparing results taken with two different day numbers. Questions like this are asked systematically while writing characterization tests, because an unknown dependency is what breaks the next step. #### Opening a Seam Once the baseline is established, the structure can change. The cheapest kind of seam is the **parameter seam**: a value read internally is moved to a parameter, and a thin wrapper that fills that parameter is left behind for old callers. ```js // legacy-report.mjs — version 2: parameter seam opened, behavior kept exactly const RECORDS = [ { bookNo: 'K-903', memberNo: 'U-17', dueDay: 1010, branch: 'central', returnBranch: 'central', returnDay: 1020 }, { bookNo: 'K-101', memberNo: 'U-42', dueDay: 1024, branch: 'central', returnBranch: 'hillside', returnDay: 1024 }, { bookNo: 'K-555', memberNo: 'U-58', dueDay: 1000, branch: 'hillside', returnBranch: 'hillside', returnDay: 1100 }, ]; export function reportLines(records, today) { const lines = [`late report — day ${today}`]; let total = 0; for (const record of records) { const late = record.returnDay - record.dueDay; let fee = 0; if (late > 3) fee = (late - 3) * 2; if (record.returnBranch !== record.branch) fee += 5; if (fee > 40) fee = 40; total += fee; lines.push(`${record.bookNo} ${record.memberNo} late ${late} fee ${fee}`); } lines.push(`total ${total}`); return lines; } export function writeLateReport() { const today = Math.floor(Date.now() / 86400000); for (const line of reportLines(RECORDS, today)) console.log(line); } ``` Three things were done at once: the records and the day were moved to parameters, writing the output was separated from the calculation, and the old external interface was preserved exactly. This is a refactor — behavior must not change. Its verification is rerunning the characterization tests **completely unchanged**. ```sh node --test --test-reporter=tap characterization.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - the report heading carries the day number for today ok 2 - the report lines are produced as they stand today ok 3 - the report content is not affected by the day number for today # tests 3 # pass 3 # fail 0 ``` The parameter seam is not the only option. An **object seam** supplies the dependency as an object field — the service setup in the Testable Design lesson is an example of this. A **module seam** makes the called module itself replaceable. What the three have in common is that they create a point where behavior can be altered without changing the source; what differs is cost and readability. #### Tests That Pass Through the Seam Tests written after the seam is opened do not touch global objects. The clock is a parameter, the records are a parameter, the result is a return value. ```js // seam.test.mjs — tests that pass through the seam: clock and records given from outside import { test } from 'node:test'; import assert from 'node:assert/strict'; import { reportLines } from './legacy-report.mjs'; const record = (extra) => ({ bookNo: 'K-903', memberNo: 'U-17', dueDay: 1010, branch: 'central', returnBranch: 'central', ...extra, }); test('lateness within the fee-free day allowance is not charged', () => { const lines = reportLines([record({ returnDay: 1013 })], 1020); assert.equal(lines[1], 'K-903 U-17 late 3 fee 0'); }); test('an on-time return to a different branch is charged five units', () => { const lines = reportLines([record({ returnDay: 1010, returnBranch: 'hillside' })], 1020); assert.equal(lines[1], 'K-903 U-17 late 0 fee 5'); }); test('an empty record list gives a total of zero', () => { assert.deepEqual(reportLines([], 1020), ['late report — day 1020', 'total 0']); }); ``` ```sh node --test --test-reporter=tap seam.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))' ``` ``` ok 1 - lateness within the fee-free day allowance is not charged ok 2 - an on-time return to a different branch is charged five units ok 3 - an empty record list gives a total of zero # tests 3 # pass 3 # fail 0 ``` The second test's name names the known bug **openly**. This is the final form of what carries over from the characterization stage: the behavior is no longer a hidden side effect but a named, tested rule. When the decision to fix it is made, the work to do is clear — the test's name and expectation change, then the implementation; and this is now just the ordinary red-green round from the Behavior-Driven Development lesson. This is why the order matters. In a seam-opening operation done without characterization tests, nothing says whether behavior was preserved. Tests are written first, the structure changes after, and the fix comes last. #### Summary - A seam is a point where a code's behavior can be altered without changing the source; in seamless code, the only point of observation may be global objects. - A characterization test records not what the code should do but what it does right now; it takes its expected value from a run, not a specification. - Known bugs are included in the baseline too; a fix and a structural change are not made at the same time. - A parameter seam moves an internally read value to a parameter and preserves the old interface with a thin wrapper; object seams and module seams are other, differently costly forms of the same purpose. - Tests that pass through the seam do not touch global objects; the order is always characterization, seam, fix. #### Course Wrap-Up This course took up a single question from two directions across two topics: what does a unit test prove, and how is that proof kept trustworthy? The Unit Testing in Practice topic turned the test into a measurement instrument. The test body was split into arrange, act, and assert sections; an assertion's signal strength was measured by the number of mutations it caught, a name's value by the information a failing run carries, independence by order permutation, fixture cost by line count, coverage by hand-counting three separate units, and flakiness by the number of tests that fail against a refactor. Five kinds of test double were written by hand, and the question each one answers was separated out. The Test-Driven Development topic reversed the order. The red-green-refactor cycle was run step by step, step size's effect on feedback time was measured by search space, rules were moved into example tables written in the domain's language, acceptance criteria were built as the outer loop, and finally, code with no tests was set on a baseline with characterization tests and opened with a seam. There is an assumption carried silently throughout the course, and it needs to be named. Every test here ran **in-process** and **without a real dependency**. The catalog was a map, the store was a fake, the notification was a spy, the clock was a function. These test doubles were assumed to match their real counterparts — but that was never tested anywhere. A fake catalog does not carry a real database's constraints; a spy notification does not produce a real notification channel's failures; the world the driver builds contains none of the boundaries that would be met in a real deployment. Testing that assumption is the subject of a separate test layer. The Integration, Contract and End-to-End Testing course takes up these boundaries by actually running against them: which dependency uses the real thing, how a test double and the real thing are assured to honor the same contract, and how tests that run across the whole application are kept stable.