Contract Tests

The machine-readable definition checks what the server produces, but does not know what the consumer uses. The definition says the loan record has five fields; does the shelf terminal read all of them, or only two? Without knowing this, every change gets planned for the worst case, and no field can be removed with confidence.

Contract tests close this gap. Every consumer writes down what it expects from the provider; the provider runs these expectations as part of its own test suite. Because expectations are derived from the consumer’s actual usage, they carry different information than the definition: the definition says what is given, the expectation says what is used.

Consumer Expectation

The expectation file contains the consumer’s name, the requests it makes, and the fields it reads from those requests. The loan service has two consumers: the barcode terminal on the shelves and members’ phone app.

{
  "consumer": "shelf-terminal",
  "interactions": [
    {
      "name": "the record's id and status are read",
      "request": { "method": "GET", "path": "/loans/O-1" },
      "expected": { "code": 200, "fields": { "/id": "string", "/status": "string" } }
    }
  ]
}

{
  "consumer": "mobile-app",
  "interactions": [
    {
      "name": "the loan record is shown on screen",
      "request": { "method": "GET", "path": "/loans/O-1" },
      "expected": { "code": 200, "fields": { "/member": "string", "/items/0/isbn": "string", "/returnDate": "string" } }
    },
    {
      "name": "an invalid member id is rejected with a field error",
      "request": { "method": "POST", "path": "/loans", "body": { "member": "1001", "items": [{ "isbn": "978-0262033848" }] } },
      "expected": { "code": 422, "fields": { "/type": "string", "/errors/0/path": "string", "/errors/0/code": "string" } }
    }
  ]
}

What the expectation does not contain matters as much as what it does. The shelf terminal never wrote down the member and items fields, because it does not read them. The mobile app did not write down the id field. The expectation file does not say “the response contains these fields,” it says “this consumer depends on these fields.” The difference determines who a field the provider removes will break.

The mobile app’s second expectation covers the error path too. The error contract is also part of the contract: the presence of the path and code fields in the 422 response’s errors array is the condition for that app being able to show errors next to their input boxes.

Running Expectations Against the Provider

The provider can run in two versions: v1, and v2, where the breaking change was made. The breaking change is the returnDate field being renamed dueDate and the member field turning into an object.

// provider.mjs — the loan service provider
// Usage: node provider.mjs <port> [v2]
//   v2: returnDate field becomes dueDate, member returns as an object (breaking change)
import { createServer } from "node:http";

const PORT = Number(process.argv[2] ?? 8438);
const V2 = process.argv[3] === "v2";

const RECORD = { id: "O-1", member: "U-1001", items: [{ isbn: "978-0262033848" }], returnDate: "2026-03-20", status: "open" };

const present = (k) => V2
  ? { id: k.id, member: { id: k.member }, items: k.items, dueDate: k.returnDate, status: k.status }
  : { ...k };

const readBody = (req) =>
  new Promise((resolve) => { let v = ""; req.on("data", (p) => (v += p)); req.on("end", () => resolve(v)); });

createServer(async (req, res) => {
  res.sendDate = false;
  const path = req.url.split("?")[0];
  const json = (code, g, type = "application/json") => {
    res.writeHead(code, { "content-type": `${type}; charset=utf-8` });
    res.end(JSON.stringify(g));
  };

  if (req.method === "GET" && path === "/loans/O-1") return json(200, present(RECORD));

  if (req.method === "POST" && path === "/loans") {
    const g = JSON.parse((await readBody(req)) || "{}");
    if (!/^U-\d{4}$/.test(g.member ?? "")) {
      return json(422, { type: "https://example.library/problems/validation", title: "Request body failed validation",
        status: 422, detail: "Member id format is invalid.", instance: "oc-0001",
        errors: [{ path: "/member", code: "format" }] }, "application/problem+json");
    }
    return json(201, present({ ...RECORD, id: "O-2", member: g.member, items: g.items ?? [] }));
  }

  json(404, { type: "https://example.library/problems/resource-not-found", title: "Resource not found",
    status: 404, detail: `${path} does not exist.`, instance: "oc-0002" }, "application/problem+json");
}).listen(PORT, "127.0.0.1", () => console.log(`provider 127.0.0.1:${PORT} ${V2 ? "v2" : "v1"}`));

The test file reads the expectations from the directory, starts the provider as a separate process, and actually sends every interaction.

// contract.test.mjs — runs consumer expectations against the provider
// Usage: node --test contract.test.mjs           (provider v1)
//        PROVIDER_VERSION=v2 node --test contract.test.mjs
import { test, before, after } from "node:test";
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { readdirSync, readFileSync } from "node:fs";

const PORT = 8438;
const BASE = `http://127.0.0.1:${PORT}`;
let child;

before(async () => {
  child = spawn("node", ["provider.mjs", String(PORT), process.env.PROVIDER_VERSION ?? "v1"], { stdio: "ignore" });
  for (let i = 0; i < 40; i++) {                       // wait until the provider comes up
    try { await fetch(`${BASE}/loans/O-1`); return; } catch { await new Promise((c) => setTimeout(c, 50)); }
  }
  throw new Error("provider failed to start");
});
after(() => child.kill());

const read = (body, path) => path.split("/").slice(1).reduce((d, p) => (d == null ? undefined : d[p]), body);
const typeOf = (d) => (Array.isArray(d) ? "array" : d === null ? "null" : typeof d);

for (const file of readdirSync("expectations").sort()) {
  const { consumer, interactions } = JSON.parse(readFileSync(`expectations/${file}`, "utf8"));
  for (const e of interactions) {
    test(`${consumer}: ${e.name}`, async () => {
      const response = await fetch(BASE + e.request.path, {
        method: e.request.method,
        headers: e.request.body ? { "content-type": "application/json" } : {},
        body: e.request.body ? JSON.stringify(e.request.body) : undefined,
      });
      assert.equal(response.status, e.expected.code, "status code");
      const body = await response.json();
      for (const [path, expectedType] of Object.entries(e.expected.fields)) {
        const value = read(body, path);
        assert.notEqual(value, undefined, `${path} field is not in the response`);
        assert.equal(typeOf(value), expectedType, `${path} field's type`);
      }
    });
  }
}

node --test contract.test.mjs 2>&1 | head -8

✔ mobile-app: the loan record is shown on screen (61.390833ms)
✔ mobile-app: an invalid member id is rejected with a field error (3.7515ms)
✔ shelf-terminal: the record's id and status are read (2.022584ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0

Timings vary by machine. Now the provider makes the breaking change:

PROVIDER_VERSION=v2 node --test contract.test.mjs 2>&1 | head -20

✖ mobile-app: the loan record is shown on screen (61.953125ms)
✔ mobile-app: an invalid member id is rejected with a field error (4.043333ms)
✔ shelf-terminal: the record's id and status are read (0.81875ms)
ℹ tests 3
ℹ suites 0
ℹ pass 2
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 123.58675

✖ failing tests:

test at contract.test.mjs:28:5
✖ mobile-app: the loan record is shown on screen (61.953125ms)
  AssertionError [ERR_ASSERTION]: /member field's type

  'object' !== 'string'

One of the three tests failed. The failing test names exactly which consumer’s which interaction broke in which field. The shelf terminal’s test passed, because it does not read the changed fields.

This is information validation against the definition cannot give. The definition check says “the response departed from the definition” and stops; the contract test says “the mobile app’s loan screen broke, the shelf terminal was unaffected.” The first reports a rule violation, the second reports a consequence.

Calculating the Impact Before the Change

For a test to fail, the change has to have already been made. Expectation files can also be read before the change: which field is read by which consumer is already written down.

// impact.mjs — from the expectation files, derives which change will break which consumer
import { readdirSync, readFileSync } from "node:fs";

// Consumer -> the field paths it depends on
const dependencies = new Map();
for (const file of readdirSync("expectations").sort()) {
  const { consumer, interactions } = JSON.parse(readFileSync(`expectations/${file}`, "utf8"));
  const paths = interactions.flatMap((e) => Object.keys(e.expected.fields).map((y) => `${e.request.path} ${y}`));
  dependencies.set(consumer, new Set(paths));
}

// All fields the definition declares in the GET /loans/O-1 response
const DEFINED = ["/id", "/member", "/items/0/isbn", "/returnDate", "/status"].map((y) => `/loans/O-1 ${y}`);

console.log("field                          dependent consumers");
for (const path of DEFINED) {
  const linked = [...dependencies].filter(([, k]) => k.has(path)).map(([t]) => t);
  console.log(`${path.replace("/loans/O-1 ", "").padEnd(30)} ${linked.length ? linked.join(", ") : "— (nobody reads this)"}`);
}

// Proposed change: returnDate is dropped, member becomes an object.
const CHANGE = ["/loans/O-1 /returnDate", "/loans/O-1 /member"];
const affected = [...dependencies].filter(([, k]) => CHANGE.some((d) => k.has(d))).map(([t]) => t);

console.log(`\nproposed change: ${CHANGE.map((d) => d.split(" ")[1]).join(", ")}`);
console.log(`affected consumers: ${affected.length}/${dependencies.size}  (${affected.join(", ")})`);
console.log(`fields never read: ${DEFINED.filter((y) => ![...dependencies.values()].some((k) => k.has(y))).length}`);

field                          dependent consumers
/id                            shelf-terminal
/member                        mobile-app
/items/0/isbn                  mobile-app
/returnDate                    mobile-app
/status                        shelf-terminal

proposed change: /returnDate, /member
affected consumers: 1/2  (mobile-app)
fields never read: 0

This table does the same job at the field level that the telemetry in the Deprecation Policy lesson did. Telemetry counted which consumer called which version; expectation files say which consumer reads which field. Together, the two turn a change’s cost from a guess into a calculation.

The last line is also a warning. “Fields never read: 0” looks reassuring, but it is only true for consumers who have written an expectation file. The conclusion that no one reads a field only holds once every consumer has written its expectations. This is contract testing’s most fragile spot: a missing expectation looks like a nonexistent dependency.

This is why contract testing is an arrangement both sides participate in. The consumer writes its expectation and hands it to the provider’s repository; the provider runs these files in its own test suite. A consumer that does not write an expectation is invisible in the provider’s decisions, and when it breaks, all it can do is report the fact. Setting up this arrangement for the library’s own shelf terminal is easy; collecting expectations for outside district libraries’ applications is the hardest part of the arrangement.

Summary

• The machine-readable definition says what is given, the consumer expectation says what is used; only the second tells you whom removing a field will break.
• Fields absent from an expectation file mean the consumer does not depend on them; the error response’s structure is part of the expectation too.
• When expectations are run against the provider, a breaking change names exactly which consumer’s which interaction broke in which field.
• The definition check reports a rule violation, the contract test reports a consequence; the two answer different questions.
• Expectation files can also be read before the change, and the number of consumers a proposed change affects can be calculated.
• The conclusion “a field nobody reads” only holds for consumers that have written their expectations; a missing expectation looks like a nonexistent dependency.

Next Step

Expectation files have been used one-directionally so far: the consumer wrote, the provider tested. The same files also work in the reverse direction. When a new consumer starts being developed, the provider may not have written that endpoint yet; but the consumer waiting is unnecessary, because the contract is already known. The next lesson writes a server that produces sample responses from the schema, develops a client against it, and shows that the same client code works unchanged once it moves to the real server.