Test Environment Management

In the previous lesson the catalog boundary ran for real, because the catalog’s real version could live in the same process. The notification channel’s real version cannot: it is a separate process, it listens on a port, and it reads its own data. This requires putting a test environment around the test — a fixture that is set up before the test, torn down after it, and whose cost is paid again on every run.

This lesson builds that fixture step by step and counts the steps. The environment it sets up catches the previous lesson’s missed defect class — recipient identity mismatch. The class it misses is just as visible, and is shown.

The Dependent Service

The notification channel is a small service that looks the recipient up in the member table. It returns 404 for a recipient it does not recognize and queues one it does with 202. The health endpoint exists so the environment can be asked whether it is ready.

// notification-service.mjs — the dependent service: reads the member table, returns 404 for an unknown recipient
import { createServer } from 'node:http';
import { DatabaseSync } from 'node:sqlite';

export function createNotificationService(port, catalogPath) {
  const db = new DatabaseSync(catalogPath);
  const queue = [];
  const server = createServer((request, response) => {
    if (request.url === '/health') {
      response.writeHead(200, { 'content-type': 'application/json' });
      return response.end('{"status":"ready"}');
    }
    if (request.method === 'GET' && request.url === '/notification') {
      response.writeHead(200, { 'content-type': 'application/json' });
      return response.end(JSON.stringify({ count: queue.length }));
    }
    let body = '';
    request.on('data', (p) => { body += p; });
    request.on('end', () => {
      const { recipient, text } = JSON.parse(body);
      const member = db.prepare('SELECT member_no FROM member WHERE member_no = ?').get(recipient);
      if (member === undefined) {
        response.writeHead(404, { 'content-type': 'application/json' });
        return response.end(`{"error":"unknown recipient: ${recipient}"}`);
      }
      queue.push({ recipient, text });
      response.writeHead(202, { 'content-type': 'application/json' });
      response.end('{"status":"queued"}');
    });
  });
  server.listen(port, '127.0.0.1');
  return server;
}

if (process.argv[2] !== undefined) createNotificationService(Number(process.argv[2]), process.argv[3]);

The Environment’s Steps

The environment consists of four steps and their order is binding: a temporary directory is opened, the schema and seed data are written, the process is started, and the readiness probe runs. Teardown is two steps.

// environment.mjs — test environment: temporary directory, schema, dependent process, and readiness probe
import { spawn } from 'node:child_process';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { DatabaseSync } from 'node:sqlite';

export const STEPS = ['temporary directory', 'schema and seed data', 'process startup', 'readiness probe'];
export const SCHEMA = `
CREATE TABLE member (member_no TEXT PRIMARY KEY, branch TEXT NOT NULL);
CREATE TABLE loan (
  book_no TEXT PRIMARY KEY, member_no TEXT NOT NULL, branch TEXT NOT NULL,
  borrow_day INTEGER NOT NULL, due_day INTEGER NOT NULL);
INSERT INTO member VALUES ('U-17', 'central');`;

async function waitUntilReady(base, maxAttempts = 100) {
  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
    try {
      const r = await fetch(`${base}/health`);
      if (r.ok) { await r.text(); return attempt; }
    } catch { /* process is not listening yet */ }
    await new Promise((c) => setTimeout(c, 20));
  }
  throw new Error('dependent service did not become ready');
}

export async function createEnvironment(port) {
  const root = mkdtempSync(join(tmpdir(), 'env-'));
  const catalogPath = join(root, 'catalog.db');
  const db = new DatabaseSync(catalogPath);
  db.exec(SCHEMA);
  db.close();
  const base = `http://127.0.0.1:${port}`;
  const start = () => spawn(process.execPath, ['notification-service.mjs', String(port), catalogPath],
    { stdio: 'ignore' });
  let child = start();
  await waitUntilReady(base);
  return {
    base, catalogPath, processCount: 1,
    restart: async () => {
      child.kill('SIGTERM');
      await new Promise((c) => setTimeout(c, 200));
      child = start();
      await waitUntilReady(base);
    },
    close: () => { child.kill('SIGTERM'); rmSync(root, { recursive: true, force: true }); },
  };
}

The fourth step can look unnecessary: if the process has started, it is ready. It is not. Starting a process does not mean the process has started listening on its port; between the two lies opening the interpreter, loading modules, and opening the database. What happens when the probe is skipped can be measured.

// not-ready.mjs — when the readiness probe is skipped, process startup races with the first request
import { spawn } from 'node:child_process';
import { mkdtempSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { DatabaseSync } from 'node:sqlite';
import { SCHEMA } from './environment.mjs';

const root = mkdtempSync(join(tmpdir(), 'not-ready-'));
const path = join(root, 'catalog.db');
const db = new DatabaseSync(path);
db.exec(SCHEMA);
db.close();

const child = spawn(process.execPath, ['notification-service.mjs', '8934', path], { stdio: 'ignore' });
try {
  await fetch('http://127.0.0.1:8934/health');
  console.log('no-probe first request: response received');
} catch (error) {
  console.log(`no-probe first request: ${error.cause?.code ?? error.code}`);
}
child.kill('SIGTERM');
rmSync(root, { recursive: true, force: true });

no-probe first request: ECONNREFUSED

This is the environment management’s own defect class and its name is readiness race. Putting a fixed wait in place of a probe does not solve the problem, it only hides it: when the wait is too short the test fails, when it is kept long every run pays that time. A probe has the environment itself answer for the duration.

Catching the Missed Class

The notification is no longer a spy object, it is a real client.

// client.mjs — the real implementation of the notification boundary: goes to the configured address
export function createNotificationClient(base) {
  return {
    send: async (recipient, text) => {
      const response = await fetch(`${base}/notification`, {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ recipient, text }),
      });
      if (response.status !== 202) {
        const body = await response.json();
        throw new Error(`notification rejected: ${response.status} ${body.error}`);
      }
    },
  };
}

Two versions of the loan service are tested: the previous lesson’s buggy version, which writes the member’s type as the recipient, and the fixed version, which writes the member’s id. Because the notification call now goes outside the process, lendBook is asynchronous now; this is a consequence of the boundary itself.

// services.mjs — two versions of the loan service: writes the member id or the member type as the recipient
import { DatabaseSync } from 'node:sqlite';

export const RULES = { student: { loanDays: 28 }, member: { loanDays: 14 } };

export function fileCatalog(path) {
  const db = new DatabaseSync(path);
  return {
    add: (r) => db.prepare('INSERT INTO loan VALUES (?, ?, ?, ?, ?)')
      .run(r.bookId, r.memberId, r.branch, r.borrowedDay, r.dueDay),
    find: (bookId) => db.prepare('SELECT member_no FROM loan WHERE book_no = ?').get(bookId),
  };
}

function createService({ catalog, notification, clock }, selectRecipient) {
  return {
    async lendBook(member, bookId) {
      const today = clock();
      const record = {
        memberId: member.id, bookId, branch: member.branch,
        borrowedDay: today, dueDay: today + RULES[member.type].loanDays,
      };
      catalog.add(record);
      await notification.send(selectRecipient(member), `due date: ${record.dueDay}`);
      return record;
    },
  };
}

export const buggyService = (deps) => createService(deps, (member) => member.type);
export const fixedService = (deps) => createService(deps, (member) => member.id);

The test hooks the environment to setup and teardown callbacks. The environment is set up once, a single test uses it, and teardown runs in every case.

// environment.test.mjs — the notification boundary runs for real; the SERVICE variable picks which version is tested
import test, { before, after } from 'node:test';
import assert from 'node:assert/strict';
import { createEnvironment } from './environment.mjs';
import { createNotificationClient } from './client.mjs';
import { fileCatalog, buggyService, fixedService } from './services.mjs';

const MEMBER = { id: 'U-17', type: 'member', branch: 'central' };
let environment;

before(async () => { environment = await createEnvironment(8931); });
after(() => environment.close());

test('a notification is accepted when a loan is made', async () => {
  const build = process.env.SERVICE === 'fixed' ? fixedService : buggyService;
  const service = build({
    catalog: fileCatalog(environment.catalogPath),
    notification: createNotificationClient(environment.base),
    clock: () => 1000,
  });
  await service.lendBook(MEMBER, 'K-903');
  const queue = await (await fetch(`${environment.base}/notification`)).json();
  assert.equal(queue.count, 1);
});

SERVICE=buggy node --test --test-reporter=tap environment.test.mjs | grep -E '^ *(ok|not ok|error:|# (tests|pass|fail))'

not ok 1 - a notification is accepted when a loan is made
  error: 'notification rejected: 404 unknown recipient: member'
# tests 1
# pass 0
# fail 1

In the previous lesson this defect stayed green: the spy object accepted whatever recipient it was given. The real service does not accept it, and it writes the reason for the rejection. The fixed version passes in the same environment.

SERVICE=fixed node --test --test-reporter=tap environment.test.mjs | grep -E '^ *(ok|not ok|# (tests|pass|fail))'

ok 1 - a notification is accepted when a loan is made
# tests 1
# pass 1
# fail 0

What This Environment Does Not See

The environment brings the process up once and takes it down after the test. Every run in between passes within a single process lifetime. This leaves one defect class untouched: accepted notifications sit in memory only.

// persistence.mjs — the state while the test is green versus the state after the process restarts
import { createEnvironment } from './environment.mjs';

const environment = await createEnvironment(8932);
const send = (recipient) => fetch(`${environment.base}/notification`, {
  method: 'POST', headers: { 'content-type': 'application/json' },
  body: JSON.stringify({ recipient, text: 'due date: 1014' }),
});
const queueCount = async () => (await (await fetch(`${environment.base}/notification`)).json()).count;

await send('U-17');
console.log(`queue while the test is green    : ${await queueCount()}`);
await environment.restart();
console.log(`queue after the process restarts : ${await queueCount()}`);
console.log(`catalog file survived            : ${environment.catalogPath.endsWith('catalog.db')}`);
environment.close();

queue while the test is green    : 1
queue after the process restarts : 0
catalog file survived            : true

The test is green, the queue is volatile. The name of the missed defect class is process-lifetime state: the dependency’s data does not survive a restart, and a single-lifetime environment never asks this in any run. Catching this class requires adding a fifth step to the environment — restarting the dependency mid-run — and that step’s cost is paid on every run too.

Cost

The environment’s cost is written as three run-independent quantities and one time comparison.

// cost.mjs — the real boundary's cost: process, setup step, and time relative to a spy object
import { createEnvironment, STEPS } from './environment.mjs';
import { createNotificationClient } from './client.mjs';

const N = 100;
const environment = await createEnvironment(8933);
const real = createNotificationClient(environment.base);
const spy = { send: async () => {} };

async function measure(notification) {
  const start = performance.now();
  for (let i = 0; i < N; i += 1) await notification.send('U-17', 'due date: 1014');
  return performance.now() - start;
}

await measure(real);
const spyTime = await measure(spy);
const realTime = await measure(real);
environment.close();

console.log(`process brought up : ${environment.processCount}`);
console.log(`setup steps        : ${STEPS.length} (${STEPS.join(', ')})`);
console.log(`teardown steps     : 2 (process termination, temporary directory removal)`);
console.log(`file per run       : 1 (catalog file)`);
console.log(`${N} notifications, real boundary took longer  : ${realTime > spyTime}`);
console.log(`time difference at least twentyfold           : ${realTime / spyTime >= 20}`);

process brought up : 1
setup steps        : 4 (temporary directory, schema and seed data, process startup, readiness probe)
teardown steps     : 2 (process termination, temporary directory removal)
file per run       : 1 (catalog file)
100 notifications, real boundary took longer  : true
time difference at least twentyfold           : true

One process, four setup steps, two teardown steps, and one file per run. Each of these six steps needs maintenance: the second step is rewritten when the schema changes, the third when the service’s startup interface changes, the fourth when the health endpoint changes.

A tool class exists that runs the same steps inside an isolated container. A container does not reduce the number of steps — a temporary container replaces the temporary directory, container startup replaces process startup — but it changes two things: the dependency’s version and configuration stay the same from run to run, and restarting the dependency becomes cheap. The missed defect class above becomes exactly the kind that can be caught there. In exchange, every run pays the cost of pulling an image or reading from a cache.

Summary

• A test environment is a fixture set up and torn down around the test; here it consists of four setup steps and two teardown steps.
• The readiness probe is a step: skipping it makes the first request come back with a connection refusal, and a fixed wait put in its place produces either flakiness or a fixed delay.
• The real notification boundary caught the previous lesson’s missed defect class: an unknown recipient was rejected with 404 and the test turned red.
• A single-lifetime environment cannot see process-lifetime state; accepted notifications vanished after a restart, and the test was still green.
• The cost is one process, six steps, and one file per run; a container-based tool class does not remove these steps, it trades them for version stability and cheap restarts.

Next Step

The seed data written together with this environment’s schema was a single row: one member, one branch. The tests passed because the questions asked could be answered with a single row. The catalog in production is not like this — the number of loans per member shows a wide distribution, records are linked to one another, and constraints are violated only in particular combinations. The next lesson takes up the test data itself: which property production-like data has to preserve, which property breaks when personal fields are masked, and what defect class a single-row seed cannot see.