---
title: 'Contract Tests'
source: 'https://academia.sh/en/courses/api-design/contract-testing'
course: 'Web API Design'
language: en
updated: '2026-08-19T05:19:27+00:00'
license: 'CC BY-SA 4.0'
---

# Contract Tests

Writing consumer expectations to a file, running them against the provider, showing which consumer's test fails on a breaking change, and calculating the impact scope before the change is made.

The machine-readable definition checks what the server produces, but does not know what
the consumer **uses**. The definition says the loan record has five fields; does the shelf
terminal read all of them, or only two? Without knowing this, every change gets planned
for the worst case, and no field can be removed with confidence.

Contract tests close this gap. Every consumer writes down what it expects from the
provider; the provider runs these expectations as part of its own test suite. Because
expectations are derived from the consumer's **actual usage**, they carry different
information than the definition: the definition says what is given, the expectation says
what is used.

## Consumer Expectation

The expectation file contains the consumer's name, the requests it makes, and the fields
it reads from those requests. The loan service has two consumers: the barcode terminal on
the shelves and members' phone app.

```json
{
  "consumer": "shelf-terminal",
  "interactions": [
    {
      "name": "the record's id and status are read",
      "request": { "method": "GET", "path": "/loans/O-1" },
      "expected": { "code": 200, "fields": { "/id": "string", "/status": "string" } }
    }
  ]
}
```

```json
{
  "consumer": "mobile-app",
  "interactions": [
    {
      "name": "the loan record is shown on screen",
      "request": { "method": "GET", "path": "/loans/O-1" },
      "expected": { "code": 200, "fields": { "/member": "string", "/items/0/isbn": "string", "/returnDate": "string" } }
    },
    {
      "name": "an invalid member id is rejected with a field error",
      "request": { "method": "POST", "path": "/loans", "body": { "member": "1001", "items": [{ "isbn": "978-0262033848" }] } },
      "expected": { "code": 422, "fields": { "/type": "string", "/errors/0/path": "string", "/errors/0/code": "string" } }
    }
  ]
}
```

What the expectation **does not** contain matters as much as what it does. The shelf
terminal never wrote down the `member` and `items` fields, because it does not read them.
The mobile app did not write down the `id` field. The expectation file does not say "the
response contains these fields," it says "this consumer depends on these fields." The
difference determines who a field the provider removes will break.

The mobile app's second expectation covers the error path too. The error contract is also
part of the contract: the presence of the path and code fields in the 422 response's
`errors` array is the condition for that app being able to show errors next to their input
boxes.

## Running Expectations Against the Provider

The provider can run in two versions: v1, and v2, where the breaking change was made. The
breaking change is the `returnDate` field being renamed `dueDate` and the `member` field
turning into an object.

```js
// provider.mjs — the loan service provider
// Usage: node provider.mjs <port> [v2]
//   v2: returnDate field becomes dueDate, member returns as an object (breaking change)
import { createServer } from "node:http";

const PORT = Number(process.argv[2] ?? 8438);
const V2 = process.argv[3] === "v2";

const RECORD = { id: "O-1", member: "U-1001", items: [{ isbn: "978-0262033848" }], returnDate: "2026-03-20", status: "open" };

const present = (k) => V2
  ? { id: k.id, member: { id: k.member }, items: k.items, dueDate: k.returnDate, status: k.status }
  : { ...k };

const readBody = (req) =>
  new Promise((resolve) => { let v = ""; req.on("data", (p) => (v += p)); req.on("end", () => resolve(v)); });

createServer(async (req, res) => {
  res.sendDate = false;
  const path = req.url.split("?")[0];
  const json = (code, g, type = "application/json") => {
    res.writeHead(code, { "content-type": `${type}; charset=utf-8` });
    res.end(JSON.stringify(g));
  };

  if (req.method === "GET" && path === "/loans/O-1") return json(200, present(RECORD));

  if (req.method === "POST" && path === "/loans") {
    const g = JSON.parse((await readBody(req)) || "{}");
    if (!/^U-\d{4}$/.test(g.member ?? "")) {
      return json(422, { type: "https://example.library/problems/validation", title: "Request body failed validation",
        status: 422, detail: "Member id format is invalid.", instance: "oc-0001",
        errors: [{ path: "/member", code: "format" }] }, "application/problem+json");
    }
    return json(201, present({ ...RECORD, id: "O-2", member: g.member, items: g.items ?? [] }));
  }

  json(404, { type: "https://example.library/problems/resource-not-found", title: "Resource not found",
    status: 404, detail: `${path} does not exist.`, instance: "oc-0002" }, "application/problem+json");
}).listen(PORT, "127.0.0.1", () => console.log(`provider 127.0.0.1:${PORT} ${V2 ? "v2" : "v1"}`));
```

The test file reads the expectations from the directory, starts the provider as a separate
process, and actually sends every interaction.

```js
// contract.test.mjs — runs consumer expectations against the provider
// Usage: node --test contract.test.mjs           (provider v1)
//        PROVIDER_VERSION=v2 node --test contract.test.mjs
import { test, before, after } from "node:test";
import assert from "node:assert/strict";
import { spawn } from "node:child_process";
import { readdirSync, readFileSync } from "node:fs";

const PORT = 8438;
const BASE = `http://127.0.0.1:${PORT}`;
let child;

before(async () => {
  child = spawn("node", ["provider.mjs", String(PORT), process.env.PROVIDER_VERSION ?? "v1"], { stdio: "ignore" });
  for (let i = 0; i < 40; i++) {                       // wait until the provider comes up
    try { await fetch(`${BASE}/loans/O-1`); return; } catch { await new Promise((c) => setTimeout(c, 50)); }
  }
  throw new Error("provider failed to start");
});
after(() => child.kill());

const read = (body, path) => path.split("/").slice(1).reduce((d, p) => (d == null ? undefined : d[p]), body);
const typeOf = (d) => (Array.isArray(d) ? "array" : d === null ? "null" : typeof d);

for (const file of readdirSync("expectations").sort()) {
  const { consumer, interactions } = JSON.parse(readFileSync(`expectations/${file}`, "utf8"));
  for (const e of interactions) {
    test(`${consumer}: ${e.name}`, async () => {
      const response = await fetch(BASE + e.request.path, {
        method: e.request.method,
        headers: e.request.body ? { "content-type": "application/json" } : {},
        body: e.request.body ? JSON.stringify(e.request.body) : undefined,
      });
      assert.equal(response.status, e.expected.code, "status code");
      const body = await response.json();
      for (const [path, expectedType] of Object.entries(e.expected.fields)) {
        const value = read(body, path);
        assert.notEqual(value, undefined, `${path} field is not in the response`);
        assert.equal(typeOf(value), expectedType, `${path} field's type`);
      }
    });
  }
}
```

```bash
node --test contract.test.mjs 2>&1 | head -8
```

```
✔ mobile-app: the loan record is shown on screen (61.390833ms)
✔ mobile-app: an invalid member id is rejected with a field error (3.7515ms)
✔ shelf-terminal: the record's id and status are read (2.022584ms)
ℹ tests 3
ℹ suites 0
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
```

Timings vary by machine. Now the provider makes the breaking change:

```bash
PROVIDER_VERSION=v2 node --test contract.test.mjs 2>&1 | head -20
```

```
✖ mobile-app: the loan record is shown on screen (61.953125ms)
✔ mobile-app: an invalid member id is rejected with a field error (4.043333ms)
✔ shelf-terminal: the record's id and status are read (0.81875ms)
ℹ tests 3
ℹ suites 0
ℹ pass 2
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 123.58675

✖ failing tests:

test at contract.test.mjs:28:5
✖ mobile-app: the loan record is shown on screen (61.953125ms)
  AssertionError [ERR_ASSERTION]: /member field's type

  'object' !== 'string'
```

One of the three tests failed. The failing test names exactly which **consumer's** which
**interaction** broke in which **field**. The shelf terminal's test passed, because it
does not read the changed fields.

This is information validation against the definition cannot give. The definition check
says "the response departed from the definition" and stops; the contract test says "the
mobile app's loan screen broke, the shelf terminal was unaffected." The first reports a
rule violation, the second reports a consequence.

## Calculating the Impact Before the Change

For a test to fail, the change has to have already been made. Expectation files can also
be read **before** the change: which field is read by which consumer is already written
down.

```js
// impact.mjs — from the expectation files, derives which change will break which consumer
import { readdirSync, readFileSync } from "node:fs";

// Consumer -> the field paths it depends on
const dependencies = new Map();
for (const file of readdirSync("expectations").sort()) {
  const { consumer, interactions } = JSON.parse(readFileSync(`expectations/${file}`, "utf8"));
  const paths = interactions.flatMap((e) => Object.keys(e.expected.fields).map((y) => `${e.request.path} ${y}`));
  dependencies.set(consumer, new Set(paths));
}

// All fields the definition declares in the GET /loans/O-1 response
const DEFINED = ["/id", "/member", "/items/0/isbn", "/returnDate", "/status"].map((y) => `/loans/O-1 ${y}`);

console.log("field                          dependent consumers");
for (const path of DEFINED) {
  const linked = [...dependencies].filter(([, k]) => k.has(path)).map(([t]) => t);
  console.log(`${path.replace("/loans/O-1 ", "").padEnd(30)} ${linked.length ? linked.join(", ") : "— (nobody reads this)"}`);
}

// Proposed change: returnDate is dropped, member becomes an object.
const CHANGE = ["/loans/O-1 /returnDate", "/loans/O-1 /member"];
const affected = [...dependencies].filter(([, k]) => CHANGE.some((d) => k.has(d))).map(([t]) => t);

console.log(`\nproposed change: ${CHANGE.map((d) => d.split(" ")[1]).join(", ")}`);
console.log(`affected consumers: ${affected.length}/${dependencies.size}  (${affected.join(", ")})`);
console.log(`fields never read: ${DEFINED.filter((y) => ![...dependencies.values()].some((k) => k.has(y))).length}`);
```

```
field                          dependent consumers
/id                            shelf-terminal
/member                        mobile-app
/items/0/isbn                  mobile-app
/returnDate                    mobile-app
/status                        shelf-terminal

proposed change: /returnDate, /member
affected consumers: 1/2  (mobile-app)
fields never read: 0
```

This table does the same job at the field level that the telemetry in the Deprecation
Policy lesson did. Telemetry counted which consumer called which **version**; expectation
files say which consumer reads which **field**. Together, the two turn a change's cost
from a guess into a calculation.

The last line is also a warning. "Fields never read: 0" looks reassuring, but it is only
true for consumers who have **written an expectation file**. The conclusion that no one
reads a field only holds once every consumer has written its expectations. This is
contract testing's most fragile spot: a missing expectation looks like a nonexistent
dependency.

This is why contract testing is an arrangement both sides participate in. The consumer
writes its expectation and hands it to the provider's repository; the provider runs these
files in its own test suite. A consumer that does not write an expectation is invisible in
the provider's decisions, and when it breaks, all it can do is report the fact. Setting up
this arrangement for the library's own shelf terminal is easy; collecting expectations for
outside district libraries' applications is the hardest part of the arrangement.

## Summary

- The machine-readable definition says what is given, the consumer expectation says what
  is used; only the second tells you whom removing a field will break.
- Fields absent from an expectation file mean the consumer does not depend on them; the
  error response's structure is part of the expectation too.
- When expectations are run against the provider, a breaking change names exactly which
  consumer's which interaction broke in which field.
- The definition check reports a rule violation, the contract test reports a consequence;
  the two answer different questions.
- Expectation files can also be read before the change, and the number of consumers a
  proposed change affects can be calculated.
- The conclusion "a field nobody reads" only holds for consumers that have written their
  expectations; a missing expectation looks like a nonexistent dependency.

## Next Step

Expectation files have been used one-directionally so far: the consumer wrote, the
provider tested. The same files also work in the reverse direction. When a new consumer
starts being developed, the provider may not have written that endpoint yet; but the
consumer waiting is unnecessary, because the contract is already known. The next lesson
writes a server that produces sample responses from the schema, develops a client against
it, and shows that the same client code works unchanged once it moves to the real server.