--- title: 'Schema Compatibility Checking' source: 'https://academia.sh/en/courses/integration-testing/schema-compatibility-checking' course: 'Integration, Contract and End-to-End Testing' language: en updated: '2026-08-23T14:25:14+00:00' license: 'CC BY-SA 4.0' --- # Schema Compatibility Checking A check that compares two schema versions and stops a breaking change before publication: change types falling into separate compatibility classes for the request and response directions, the expand–contract sequence turning the gate green, and a value-set widening the check counts as compatible breaking the consumer. The previous two checks both worked by running: a process comes up, a request goes out, the response gets examined. The provider's real question, though, is asked before the change is published, and it requires no run. Adding a field, removing a field, narrowing a type, making an optional field required: which of these four change types breaks the old consumer? The **definition** of a breaking change was established in the Web API Design course, and the **measurement of its cost** in the Application Layer and Service Interaction course; neither is repeated here. This lesson's subject is turning that knowledge into an automatic check: two schema versions are compared, every change is placed into a type and a compatibility class, and if there is a change in the breaking class, the suite turns red. ## Schema Versions and Proposals The published schema describes the book response and the checkout request together. There are three proposals: `A` carries all four change types at once, `B` meets the same need with the first step of the expand–contract sequence, and `C` only widens a value set. ```js // schema.mjs — the published schema and three proposals; each field carries a type, requiredness, and value set const A = (type, required, set) => (set ? { type, required, set } : { type, required }); export const PUBLISHED = { response: { isbn: A("string", true), title: A("string", true), shelfCount: A("number", true), status: A("string", true, ["shelved", "loaned", "lost"]), loanDays: A("number", true), tags: A("array", false), }, request: { memberNo: A("string", true), branch: A("string", false), count: A("number", false) }, }; // A: four change types at once. B: the expand step. C: only the set widens. export const PROPOSAL = { A: { response: { ...PUBLISHED.response, dueDay: A("string", false) }, request: { ...PUBLISHED.request, branch: A("string", true), count: A("integer", false) }, }, B: { response: { ...PUBLISHED.response, dueDay: A("string", false) }, request: { ...PUBLISHED.request } }, C: { response: { ...PUBLISHED.response, status: A("string", true, ["shelved", "loaned", "lost", "newly-shelved"]) }, request: { ...PUBLISHED.request }, }, }; delete PROPOSAL.A.response.tags; // field removal ``` The check itself has two parts: a comparison that sorts the diff into types, and a table that binds types to classes. The table having two columns is this lesson's central distinction — the same change type falls into a different class in the request direction than in the response direction. ```js // compatibility.mjs — sorts the changes between two schema versions into types and classes const NARROW = { number: "integer" }; export const CLASS = { "optional field addition": { response: "compatible", request: "compatible" }, "required field addition": { response: "compatible", request: "breaking" }, "field removal": { response: "breaking", request: "compatible" }, "type narrowing": { response: "compatible", request: "breaking" }, "type widening": { response: "breaking", request: "compatible" }, "making required": { response: "compatible", request: "breaking" }, "making optional": { response: "breaking", request: "compatible" }, "adding a value to the set": { response: "compatible", request: "breaking" }, "removing a value from the set": { response: "breaking", request: "compatible" }, }; export function diff(old, next) { const d = []; for (const [name, rule] of Object.entries(old)) { const n = next[name]; if (n === undefined) { d.push({ name, type: "field removal" }); continue; } if (n.type !== rule.type) d.push({ name, type: NARROW[rule.type] === n.type ? "type narrowing" : "type widening" }); if (n.required && !rule.required) d.push({ name, type: "making required" }); if (rule.required && !n.required) d.push({ name, type: "making optional" }); if ((n.set ?? []).some((v) => !(rule.set ?? []).includes(v))) d.push({ name, type: "adding a value to the set" }); if ((rule.set ?? []).some((v) => !(n.set ?? []).includes(v))) d.push({ name, type: "removing a value from the set" }); } for (const [name, n] of Object.entries(next)) { if (name in old) continue; d.push({ name, type: n.required ? "required field addition" : "optional field addition" }); } return d; } // Evaluates a proposal in both directions; each line of the returned list is one classified change. export const evaluate = (published, proposal) => ["response", "request"].flatMap((direction) => diff(published[direction], proposal[direction]).map((x) => ({ direction, ...x, class: CLASS[x.type][direction] }))); ``` ## The Distribution of Change Types ```js // distribution.mjs — how change types distribute over compatibility classes, and a count of the three proposals import { PUBLISHED, PROPOSAL } from "./schema.mjs"; import { CLASS, evaluate } from "./compatibility.mjs"; console.log(`${"change type".padEnd(32)}${"response".padStart(12)}${"request".padStart(12)}`); for (const [type, s] of Object.entries(CLASS)) { console.log(`${type.padEnd(32)}${s.response.padStart(12)}${s.request.padStart(12)}`); } for (const [name, o] of Object.entries(PROPOSAL)) { const d = evaluate(PUBLISHED, o); console.log(`\nproposal ${name}: ${d.length} changes, ${d.filter((x) => x.class === "breaking").length} breaking`); for (const x of d) console.log(` ${`${x.direction}.${x.name}`.padEnd(20)}${x.type.padEnd(30)}-> ${x.class}`); } ``` ``` change type response request optional field addition compatible compatible required field addition compatible breaking field removal breaking compatible type narrowing compatible breaking type widening breaking compatible making required compatible breaking making optional breaking compatible adding a value to the set compatible breaking removing a value from the set breaking compatible proposal A: 4 changes, 3 breaking response.tags field removal -> breaking response.dueDay optional field addition -> compatible request.branch making required -> breaking request.count type narrowing -> breaking proposal B: 1 changes, 0 breaking response.dueDay optional field addition -> compatible proposal C: 1 changes, 0 breaking response.status adding a value to the set -> compatible ``` The table's two columns mirror each other. Removing a field in the response is breaking, because the consumer is reading it; removing a field in the request is not breaking, because the old client keeps sending it and the field is ignored. In the same way, making a field required does no harm in the response — the field was always going to arrive anyway — but in the request it causes every request the old client sends to be rejected. Narrowing a type is compatible in the response, because an integer is still a number; it is breaking in the request, because the old client might send a decimal. This symmetry shows that the eight-row table is not a list to be memorized. There is a single question, and direction decides it: **is the old code on the other side still valid after the change?** In the response, the other side reads; in the request, it writes. ## The Acceptance Gate The check works as the **acceptance gate** defined in the Quality and Testing Fundamentals course: if there is a change in the breaking class, the release stops. ```js // compatibility.test.mjs — pre-publish gate: if there is a breaking change, the suite turns red // Usage: PROPOSAL=A|B|C node --test compatibility.test.mjs import { test } from "node:test"; import assert from "node:assert/strict"; import { PUBLISHED, PROPOSAL as PROPOSALS } from "./schema.mjs"; import { evaluate } from "./compatibility.mjs"; const NAME = process.env.PROPOSAL ?? "B"; test(`proposal ${NAME}: no breaking change against the published schema`, () => { const breaking = evaluate(PUBLISHED, PROPOSALS[NAME]).filter((x) => x.class === "breaking"); assert.equal(breaking.length, 0, `breaking change: ${breaking.map((x) => `${x.direction}.${x.name} ${x.type}`).join(", ")}`); }); ``` ```bash for o in A B C; do PROPOSAL=$o node --test --test-reporter=tap compatibility.test.mjs | grep -E '^(ok|not ok)|breaking change:' | sed 's/^ *//' done ``` ``` not ok 1 - proposal A: no breaking change against the published schema breaking change: response.tags field removal, request.branch making required, request.count type narrowing ok 1 - proposal B: no breaking change against the published schema ok 1 - proposal C: no breaking change against the published schema ``` **The defect class caught is the contract breaking before publication.** Proposal A turned red and named the three breaking changes along with their direction. The fix is to turn the same need into the first step of the expand–contract sequence: the new field is added alongside the old ones, and removal and making-required are left to the third step. Proposal B is exactly this, and the gate turns green. What sets this check apart from the previous two lessons is that it sees the break **without running a single consumer**. The contract test had to run the consumer's expectation and the provider's version together to show a break; the check here compares two texts. In exchange, what it knows is less: it cannot say which consumer breaks, only that one might. ## The Break the Check Counts as Compatible Proposal C passed through the gate. A new value was added to the defined value set for book status: books that have newly entered the collection and are on the shelf are now reported as `newly-shelved`. All of the old responses are still valid, no field was lost, and no type changed. ```js // missed.mjs — a change the check counts as compatible breaks the consumer import { PUBLISHED, PROPOSAL } from "./schema.mjs"; import { evaluate } from "./compatibility.mjs"; // The loan service's decision branch: an unknown status value counts as "not on shelf". const loanDecision = (book) => (book.status === "shelved" && book.shelfCount > 0 ? { granted: true, message: `${book.isbn} loaned` } : { granted: false, message: `not on shelf (status ${book.status}, shelfCount ${book.shelfCount})` }); const d = evaluate(PUBLISHED, PROPOSAL.C); console.log(`check: proposal C -> ${d.length} changes, ${d.filter((x) => x.class === "breaking").length} breaking, gate green`); const ON_SHELF = { isbn: "978-0262033848", status: "shelved", shelfCount: 2 }; const NEWLY_SHELVED = { ...ON_SHELF, status: "newly-shelved" }; // freshly arrived shelf: book is shelved and loanable for (const [label, book] of [["old value", ON_SHELF], ["new value", NEWLY_SHELVED]]) { const decision = loanDecision(book); console.log(`${label.padEnd(11)} -> granted=${String(decision.granted).padEnd(5)} ${decision.message}`); } console.log(`schema status values ${PROPOSAL.C.response.status.set.length}, values the consumer treats as loanable 1`); console.log(`a unit change does not change the schema -> ${evaluate(PUBLISHED, PUBLISHED).length} changes`); ``` ``` check: proposal C -> 1 changes, 0 breaking, gate green old value -> granted=true 978-0262033848 loaned new value -> granted=false not on shelf (status newly-shelved, shelfCount 2) schema status values 4, values the consumer treats as loanable 1 a unit change does not change the schema -> 0 changes ``` **The missed defect class is a change that alters meaning without breaking shape.** A book sitting on the shelf became impossible to loan; from the schema's point of view, no rule was violated. Adding a value to the set counts as compatible in the response direction, because it is, as far as schema validation goes — the old values keep their validity. What breaks is not the schema but the consumer's code, which **assumes it has exhausted** that set. This assumption is written in no schema. The last line shows the boundary even more plainly. The loan duration starting to be counted in weeks instead of days changes neither the field name nor its type; the two schemas stay byte-for-byte identical and the check reports zero changes. No check that only looks at the schema can see a change in unit, scale, or meaning. ## The Cost of the Check ```js // cost.mjs — the cost of a check that only looks at the schema: zero processes, zero requests import { createServer } from "node:http"; import { PUBLISHED, PROPOSAL } from "./schema.mjs"; import { evaluate } from "./compatibility.mjs"; const REPEAT = 200, proposalCount = Object.keys(PROPOSAL).length; const t0 = performance.now(); for (let i = 0; i < REPEAT; i += 1) for (const o of Object.values(PROPOSAL)) evaluate(PUBLISHED, o); const t1 = performance.now(); // Comparison point: a single local request's round trip on the same machine. const server = createServer((_, res) => res.end("{}")); await new Promise((c) => server.listen(0, "127.0.0.1", c)); const t2 = performance.now(); await (await fetch(`http://127.0.0.1:${server.address().port}/`)).text(); const t3 = performance.now(); server.close(); const fields = Object.keys(PUBLISHED.response).length + Object.keys(PUBLISHED.request).length; console.log(`schema ${fields} fields (response ${Object.keys(PUBLISHED.response).length}, request ${Object.keys(PUBLISHED.request).length}), ${proposalCount} proposals`); console.log(`check ${REPEAT * proposalCount} evaluations, 0 processes, 0 requests, 0 test data`); console.log(`one evaluation shorter than a single local request : ${(t1 - t0) / (REPEAT * proposalCount) < t3 - t2}`); console.log(`ratio at least fifty times : ${(t3 - t2) / ((t1 - t0) / (REPEAT * proposalCount)) >= 50}`); ``` ``` schema 9 fields (response 6, request 3), 3 proposals check 600 evaluations, 0 processes, 0 requests, 0 test data one evaluation shorter than a single local request : true ratio at least fifty times : true ``` Absolute times are machine-dependent; the direction of the ratio does not change. This check brings up no process, sends no request, and prepares no test data: its input is two texts, and its output is a classified list. Hundreds of evaluations fit inside the time of a single local request; that is why the check can run on every change proposal. The cost independent of the run sits in two places. First, the **published** version of the schema has to be stored somewhere: the comparison needs two texts, and the compatibility question cannot be asked with only one. Second is the upkeep of the class table — nine rows are hand-written, and when a new constraint is added to the schema language, a row has to be added to the table too. A missing row stays silent, and a check that stays silent is indistinguishable from a check that does not exist. ## Summary - A schema compatibility check compares two schema versions, sorts the diff into change types, and places each type into a compatibility class. - The same type falls into a different class in the request direction than in the response direction; the single question is whether the old code on the other side is still valid after the change. - The proposal carrying four change types at once turned red with three breaking changes; once the same need was turned into the first step of the expand–contract sequence, the gate turned green. - The missed class is a change that alters meaning without breaking shape: a value added to a value set was counted as compatible, yet a book sitting on the shelf became impossible to loan. With a unit change, the two schemas stay identical and the check reports zero changes. - Cost: zero processes, zero requests, zero test data; in return, the published schema has to be stored and the nine-row class table maintained by hand. - The check cannot say which consumer breaks; it can only say that one might. ## Next Step All three checks assumed the provider was ready: the schema check sent a request to the live service, the contract test brought up the provider's version, and the compatibility check read the published schema. Yet the moment a contract is most useful is the moment the provider is **not yet written**. The consumer team does not have to wait, because the contract is already settled. The next lesson builds a mock server that generates responses from the contract, develops the loan service against it, and measures a single question: at how many points does the mock server deviate from the real thing, and how many of those deviations can the checks built so far close.