Mocking External Services

In the previous lesson the dependency that ran for real sat on the same machine: its schema was on hand, its migration could be repeated. The library’s loan flow also depends on a bibliography service, and that service belongs to another team. It does not come up with the test’s command, it can give a different answer on every run, and on some runs it does not answer at all.

The approach used for this dependency is called record and replay: real responses are recorded once, and later runs are fed from the recording. The recording file is called a cassette. This lesson measures the defect classes the cassette catches and the ones it misses.

The External Service and the Transport

The bibliography service is represented here by a local process; in reality it sits on the other end of the network. It has two versions, and the difference between them is a single field name — that difference is measured at the end of the lesson.

// bibliography-service.mjs — the local process standing in for another team's bibliography service
import { createServer } from 'node:http';

const BOOKS = {
  'K-903': { author: 'Sabahattin Ali', year: 1943 },
  'K-904': { author: 'Halide Edip', year: 1922 },
};

export function createBibliographyService(port, version) {
  const server = createServer((request, response) => {
    const url = new URL(request.url, 'http://local');
    const no = url.searchParams.get('no');
    const book = BOOKS[no];
    if (url.pathname !== '/bibliography' || book === undefined) {
      response.writeHead(404, { 'content-type': 'application/json' });
      return response.end(`{"error":"not found: ${no}"}`);
    }
    // v2 renamed the title field to name
    const body = version === 'v2'
      ? { no, name: `book ${no}`, author: book.author, year: book.year }
      : { no, title: `book ${no}`, author: book.author, year: book.year };
    response.writeHead(200, { 'content-type': 'application/json' });
    response.end(JSON.stringify(body));
  });
  server.listen(port, '127.0.0.1');
  return server;
}

if (process.argv[2] !== undefined) createBibliographyService(Number(process.argv[2]), process.argv[3] ?? 'v1');

The transport runs in two modes. In record mode it goes to the real service and appends every interaction to the cassette; in replay mode it never goes out to the network, it looks the request up in the cassette. The key is built from the method and the full path — including the query fields.

// cassette.mjs — record and replay: record mode goes to the real service, replay mode reads from the file
import { existsSync, readFileSync, writeFileSync } from 'node:fs';

export function createCassetteClient({ mode, cassettePath, base }) {
  const cassette = existsSync(cassettePath) ? JSON.parse(readFileSync(cassettePath, 'utf8')) : [];
  let networkCalls = 0;
  return {
    async request(path) {
      const key = `GET ${path}`;
      if (mode === 'replay') {
        const entry = cassette.find((c) => c.key === key);
        if (entry === undefined) throw new Error(`no match in cassette: ${key}`);
        return { status: entry.status, body: entry.body };
      }
      const response = await fetch(`${base}${path}`);
      networkCalls += 1;
      const body = await response.json();
      cassette.push({ key, status: response.status, body });
      writeFileSync(cassettePath, `${JSON.stringify(cassette, null, 1)}\n`);
      return { status: response.status, body };
    },
    metrics: () => ({ networkCalls, interactions: cassette.length }),
  };
}

// ready.mjs — waits for the service to start listening
export async function waitUntilReady(base, maxAttempts = 100) {
  for (let i = 0; i < maxAttempts; i += 1) {
    try { await (await fetch(`${base}/bibliography?no=K-903`)).text(); return; }
    catch { await new Promise((c) => setTimeout(c, 20)); }
  }
  throw new Error('service did not become ready');
}

The bibliography client takes the query field’s name from a variable; the buggy version will change that name.

// client.mjs — the bibliography client; the FIELD_NAME variable picks the query field's name
export const QUERY_FIELD = process.env.FIELD_NAME ?? 'no';

export function createBibliographyClient(transport) {
  return {
    async getBibliography(bookId) {
      const { status, body } = await transport.request(`/bibliography?${QUERY_FIELD}=${bookId}`);
      if (status !== 200) throw new Error(`could not get bibliography: ${status}`);
      return body;
    },
  };
}

// bibliography.test.mjs — the same test in three modes: record, replay, and the live service
import test, { before } from 'node:test';
import assert from 'node:assert/strict';
import { createCassetteClient } from './cassette.mjs';
import { createBibliographyClient } from './client.mjs';
import { waitUntilReady } from './ready.mjs';

const mode = process.env.MODE ?? 'replay';
const cassettePath = process.env.CASSETTE ?? 'cassette.json';
const base = `http://127.0.0.1:${process.env.PORT ?? 8941}`;
const createClient = () => createBibliographyClient(createCassetteClient({ mode, cassettePath, base }));

before(async () => { if (mode === 'record') await waitUntilReady(base); });

test('the K-903 bibliography carries a title and an author', async () => {
  const bibliography = await createClient().getBibliography('K-903');
  assert.equal(bibliography.title, 'book K-903');
  assert.equal(bibliography.author, 'Sabahattin Ali');
});

test('the K-904 bibliography carries a year', async () => {
  const bibliography = await createClient().getBibliography('K-904');
  assert.equal(bibliography.year, 1922);
});

Record and Replay

Recording happens once and requires the real service.

node bibliography-service.mjs 8941 v1 & SERVICE=$!
MODE=record node --test --test-reporter=tap bibliography.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))'
kill $SERVICE
echo "cassette interactions: $(node -e "console.log(JSON.parse(require('fs').readFileSync('cassette.json')).length)")"

ok 1 - the K-903 bibliography carries a title and an author
ok 2 - the K-904 bibliography carries a year
# tests 2
# pass 2
# fail 0
cassette interactions: 2

Later runs never call the service at all.

node --test --test-reporter=tap bibliography.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))'
echo "bibliography service alive during the run: $(pgrep -f 'bibliography-service.mjs' | wc -l | tr -d ' ')"

ok 1 - the K-903 bibliography carries a title and an author
ok 2 - the K-904 bibliography carries a year
# tests 2
# pass 2
# fail 0
bibliography service alive during the run: 0

The cost paid in the second and third lessons is not paid here: no process comes up, there is no readiness probe, there is no teardown step. In exchange, a dependency on the cassette was born, and the cassette’s validity is an assumption.

The Caught Class: Request Drift

Let the bibliography client’s query field be changed from no to book. The real service would return 404 for this; the cassette finds no match.

FIELD_NAME=book node --test --test-reporter=tap bibliography.test.mjs \
  | grep -E "^ *(not ok|error:|# fail)" | sed 's/^ *//;s/^/buggy    /'
FIELD_NAME=no node --test --test-reporter=tap bibliography.test.mjs \
  | grep -E '^ *(ok|# (pass|fail))' | sed 's/^ *//;s/^/fixed    /'

buggy    not ok 1 - the K-903 bibliography carries a title and an author
buggy    error: 'no match in cassette: GET /bibliography?book=K-903'
buggy    not ok 2 - the K-904 bibliography carries a year
buggy    error: 'no match in cassette: GET /bibliography?book=K-904'
buggy    # fail 2
fixed    ok 1 - the K-903 bibliography carries a title and an author
fixed    ok 2 - the K-904 bibliography carries a year
fixed    # pass 2
fixed    # fail 0

The name of the caught defect class is request drift. The cassette fixes not only the responses but also the requests that fetch them; because the path and the query fields sit inside the key, every change on the request side goes unmatched. This is the cassette’s strongest side, and at the same time its noisiest: when the code’s call to the external service genuinely changes too, the same error comes out and the cassette has to be re-recorded — and re-recording requires the real service again.

The Missed Class: Frozen Response

The cassette freezes the service’s behavior at the moment of recording. When the team that owns the service changes the response body, the cassette knows nothing about it.

node --test --test-reporter=tap bibliography.test.mjs | grep -E '^ *(ok|not ok|# (pass|fail))' | sed 's/^/replay   /'
node bibliography-service.mjs 8942 v2 & SERVICE=$!
MODE=record CASSETTE=cassette-v2.json PORT=8942 node --test --test-reporter=tap bibliography.test.mjs \
  | grep -E '^ *(ok|not ok|# (pass|fail))' | sed 's/^/live v2  /'
kill $SERVICE
rm -f cassette-v2.json

replay   ok 1 - the K-903 bibliography carries a title and an author
replay   ok 2 - the K-904 bibliography carries a year
replay   # pass 2
replay   # fail 0
live v2  not ok 1 - the K-903 bibliography carries a title and an author
live v2  ok 2 - the K-904 bibliography carries a year
live v2  # pass 1
live v2  # fail 1

Same test, same code, two different responses. The service’s second version renamed the title field to name; the live run sees this, the run fed from the cassette does not. The name of the missed defect class is frozen response: the cassette carries yesterday’s behavior, and that behavior is no longer correct.

This class’s weight comes from being silent. Schema mismatch announced itself with a constraint violation, the migration defect with a failed assertion, request drift with a missing match. In frozen response, nothing fails — the suite is green, and because it is green, no one looks.

Cost

// cost.mjs — the cost of record and replay modes: process, network calls, file, and relative time
import { spawn } from 'node:child_process';
import { statSync, rmSync } from 'node:fs';
import { createCassetteClient } from './cassette.mjs';
import { createBibliographyClient } from './client.mjs';
import { waitUntilReady } from './ready.mjs';

const N = 100;
const base = 'http://127.0.0.1:8943';
const child = spawn(process.execPath, ['bibliography-service.mjs', '8943', 'v1'], { stdio: 'ignore' });
await waitUntilReady(base);

async function measure(mode, cassettePath) {
  const transport = createCassetteClient({ mode, cassettePath, base });
  const client = createBibliographyClient(transport);
  await client.getBibliography('K-903');
  const start = performance.now();
  for (let i = 0; i < N; i += 1) await client.getBibliography('K-903');
  return { time: performance.now() - start, ...transport.metrics() };
}

const live = await measure('record', 'cassette-cost.json');
const replay = await measure('replay', 'cassette.json');
child.kill('SIGTERM');
const size = statSync('cassette.json').size;
rmSync('cassette-cost.json', { force: true });

console.log(`record mode : 1 process, ${live.networkCalls} network calls (${N} requests + 1 warm-up)`);
console.log(`replay mode : 0 processes, ${replay.networkCalls} network calls, 1 file read`);
console.log(`cassette    : ${replay.interactions} interactions, ${size} bytes`);
console.log(`replay took less time               : ${replay.time < live.time}`);
console.log(`time difference at least twentyfold : ${live.time / replay.time >= 20}`);

record mode : 1 process, 101 network calls (100 requests + 1 warm-up)
replay mode : 0 processes, 0 network calls, 1 file read
cassette    : 2 interactions, 340 bytes
replay took less time               : true
time difference at least twentyfold : true

Replay mode’s run cost is the lowest of the three lessons: zero processes, zero network calls, a single file read. The cost has left the run and moved somewhere else — to the cassette’s freshness, at three hundred forty bytes. This cost is not paid on every run, it is paid when the service changes, and it stays invisible when it is not paid. It also grows as the cassette grows: two interactions can be checked by eye, two hundred interactions cannot.

Summary

• Record and replay records real responses once and feeds later runs from the cassette; in replay mode, processes and network calls are zero.
• The cassette fixes not only responses but requests too; when the query field’s name changed, both tests found no match in the cassette.
• The response the recording freezes can go stale: when the service’s second version renamed a field, the live run turned red and the run fed from the cassette stayed green.
• Frozen response is a silent defect class; because no test fails, no one looks.
• The cost moves from the run to the cassette’s freshness, and it becomes impossible to check by hand as the cassette grows.

Next Step

The five lessons in this topic asked the same question at different boundaries: which dependency runs for real, and what is the cost. The cassette reached the edge of the question. The recording freezes the external service’s yesterday’s behavior; when the team that owns the service changes its contract, the recording still stays green and the tests start lying. Refreshing the cassette is not a solution, because there is no signal that says when refreshing is needed.

What is missing is testing the contract itself that the two sides agreed on: do the fields the consumer expects match the fields the provider gives, and who notices when a field name changes, and when. The next topic opens with this question and takes up verifying the request and the response, carrying the expectation over to the provider, and checking for a breaking change.