Backward Compatibility

The previous lesson published two versions side by side but did not ask why the version number went up to two. The member field turning into an object was counted as breaking; adding a new option to the branch field was not discussed. When these questions are left to intuition, every team gives a different answer; the version number rests on opinion and gets skipped over with the sentence “this is a small change.”

This lesson defines breakingness, writes a differ, and derives the version number from the diff. The definition’s starting point is a simple observation: the same change can be compatible in the request direction while being breaking in the response direction.

Breakingness Determined by Direction

A contract has two directions. The client sends a body, the server produces a body. Because the client sends, the server defines an accepted set; because the server produces, it also defines a produced set.

If what the server accepts narrows, old clients are left out. If what is accepted widens, old clients are unaffected; what they send is still inside the set. So in the request direction, narrowing is breaking and widening is compatible.

If what the server produces widens, old clients encounter something they do not recognize. If what it produces narrows, clients are unaffected; the fields they read are still in place. So in the response direction, widening is breaking and narrowing is compatible.

The two directions’ rules are exactly each other’s opposite. This inverse relationship is why a single change, like removing the /isbn field, is compatible in the request and breaking in the response: an old client keeps sending the field removed from the request, and the server ignores it; but for the field removed from the response, the old client keeps looking for it and does not find it.

The two schemas below give the loan resource’s state across a version range. Each field states three things: its type, whether it is required, and, if any, the values it can take.

// schemas.mjs — two schema versions of the loan resource
// Field: { type, required, values }.  No values means the value set is unbounded.
export const V1 = {
  version: "1.4.2",
  request: {
    "/member":   { type: "string", required: true },
    "/isbn":     { type: "string", required: true },
    "/branch":   { type: "string", required: false, values: ["central", "shore"] },
    "/dayCount": { type: "number", required: false },
  },
  response: {
    "/id":         { type: "string", required: true },
    "/member":     { type: "string", required: true },
    "/isbn":       { type: "string", required: true },
    "/returnDate": { type: "string", required: true },
    "/status":     { type: "string", required: true, values: ["open", "closed"] },
  },
};

export const V2 = {
  request: {
    "/member":   { type: "string",  required: true },
    "/items":    { type: "array",   required: true },                                    // new required field
    "/branch":   { type: "string",  required: true, values: ["central", "shore", "hill"] },
    "/dayCount": { type: "integer", required: false },                                   // type narrowed
    "/note":     { type: "string",  required: false },                                   // new optional field
  },
  response: {
    "/id":         { type: "string", required: true },
    "/member":     { type: "object", required: true },                                    // type changed
    "/items":      { type: "array",  required: true },                                    // new field
    "/returnDate": { type: "string", required: false },                                   // now optional
    "/status":     { type: "string", required: true, values: ["open", "closed", "overdue"] },
  },
};

The Differ

The differ walks the two schemas, finds every difference, and applies the direction rule. Type changes require a small subtype relationship: every integer is a number, but the reverse is not true. Types with no such relationship between them are breaking in both directions.

// differ.mjs — compares two schema versions, detects breaking changes, derives the version
// Usage: node differ.mjs [tolerant]   ("tolerant": clients tolerate an unknown value)

import { V1, V2 } from "./schemas.mjs";

const TOLERANT = process.argv[2] === "tolerant";

// Type lattice: integer is a subtype of number. A subtype accepts a portion of what the supertype accepts.
const SUBTYPES = { integer: ["number"], date: ["string"] };
const isSubtype = (a, b) => a === b || (SUBTYPES[a] ?? []).includes(b);

// Two directions, two rules. Request: breaking if what the server accepts narrows.
// Response: breaking if what the server produces widens.
function compare(oldSchema, newSchema, direction) {
  const breaking = direction === "request"
    ? { narrowing: true, widening: false, requiredAdded: true, requiredRemoved: false, removal: false }
    : { narrowing: false, widening: true, requiredAdded: false, requiredRemoved: true, removal: true };
  const findings = [];
  const add = (kind, name, description) => findings.push({ kind, direction, name, description });

  for (const [name, o] of Object.entries(oldSchema)) {
    const n = newSchema[name];
    if (!n) { add(breaking.removal ? "breaking" : "compatible", name, "field removed"); continue; }
    if (!o.required && n.required) add(breaking.requiredAdded ? "breaking" : "compatible", name, "was optional, became required");
    if (o.required && !n.required) add(breaking.requiredRemoved ? "breaking" : "compatible", name, "was required, became optional");
    if (o.type !== n.type) {
      const narrowing = isSubtype(n.type, o.type), widening = isSubtype(o.type, n.type);
      const kind = narrowing ? (breaking.narrowing ? "breaking" : "compatible")
        : widening ? (breaking.widening ? "breaking" : "compatible")
        : "breaking";                                       // unrelated types are breaking in both directions
      add(kind, name, `type ${o.type} → ${n.type}` + (narrowing ? " (narrowing)" : widening ? " (widening)" : " (unrelated)"));
    }
    const oldValues = o.values, newValues = n.values;
    if (oldValues && newValues) {
      const removed = oldValues.filter((d) => !newValues.includes(d)), added = newValues.filter((d) => !oldValues.includes(d));
      if (removed.length) add(breaking.narrowing ? "breaking" : "compatible", name, `value set narrowed: ${removed.join(",")} removed`);
      if (added.length) {
        const isBreaking = breaking.widening && !(direction === "response" && TOLERANT);
        add(isBreaking ? "breaking" : "compatible", name, `value set widened: ${added.join(",")} added`);
      }
    }
  }
  for (const [name, n] of Object.entries(newSchema)) {
    if (oldSchema[name]) continue;
    const isBreaking = n.required && breaking.requiredAdded;
    add(isBreaking ? "breaking" : "addition", name, n.required ? "required field added" : "optional field added");
  }
  return findings;
}

const findings = [...compare(V1.request, V2.request, "request"), ...compare(V1.response, V2.response, "response")];
const ORDER = { breaking: 0, addition: 1, compatible: 2 };
findings.sort((a, b) => ORDER[a.kind] - ORDER[b.kind] || a.direction.localeCompare(b.direction));

console.log(`tolerance declaration: ${TOLERANT ? "present" : "absent"}\n`);
console.log("kind       direction  name          change");
for (const f of findings) console.log(`${f.kind.padEnd(10)} ${f.direction.padEnd(10)} ${f.name.padEnd(13)} ${f.description}`);

const counts = { breaking: 0, addition: 0, compatible: 0 };
for (const f of findings) counts[f.kind]++;
const [major, minor, patch] = V1.version.split(".").map(Number);
const next = counts.breaking ? `${major + 1}.0.0` : counts.addition ? `${major}.${minor + 1}.0` : `${major}.${minor}.${patch + 1}`;

console.log(`\nbreaking: ${counts.breaking}   addition: ${counts.addition}   compatible: ${counts.compatible}`);
console.log(`current version: ${V1.version}   derived version: ${next}`);
console.log(`if breaking changes were excluded: ${major}.${minor + 1}.0`);

tolerance declaration: absent

kind       direction  name          change
breaking   request    /branch       was optional, became required
breaking   request    /dayCount     type number → integer (narrowing)
breaking   request    /items        required field added
breaking   response   /member       type string → object (unrelated)
breaking   response   /isbn         field removed
breaking   response   /returnDate   was required, became optional
breaking   response   /status       value set widened: overdue added
addition   request    /note         optional field added
addition   response   /items        required field added
compatible request    /isbn         field removed
compatible request    /branch       value set widened: hill added

breaking: 7   addition: 2   compatible: 2
current version: 1.4.2   derived version: 2.0.0
if breaking changes were excluded: 1.5.0

The last two lines show the inverse relationship directly. Removing the /isbn field is compatible in the request, breaking in the response. Adding hill to the /branch field’s value set is compatible in the request; adding overdue to the /status field’s value set is breaking in the response. The same operation lands in the opposite class in the opposite direction.

The /items field in the addition rows is also a consequence of this rule: adding a required field to the response is compatible, because the old client was not reading that field anyway. Adding a required field to the request, however, is breaking, and the table shows exactly that.

The version number is no longer debated but derived. Seven breaking changes raise the major version. The last line shows a separate option: if the breaking changes were excluded from the release and only the additions were shipped, the version would be 1.5.0. These two lines turn the debate “what should we do about the version” into “what are we putting in the release”; the second is an answerable question.

Tolerance Declaration

Widening the value set in the response is the most contested item on the list. Does adding the overdue status break old clients? The answer depends on what the contract says. If the contract says “if an unknown status value arrives, the client must treat it as unrecognized,” the widening is not breaking; if it does not say that, it is breaking.

node differ.mjs tolerant | tail -4

breaking: 6   addition: 2   compatible: 3
current version: 1.4.2   derived version: 2.0.0
if breaking changes were excluded: 1.5.0

The breaking count drops from seven to six. The lesson here is that tolerance is a contract clause. It is not a decision made on the server side; it requires clients to actually be written that way, and that can only be expected if it was declared in the contract beforehand. A tolerance clause added to the contract afterward is itself a breaking change.

What the Diff Does Not Say

The differ tells you what could break, not what will break. To see the distinction, the same schemas are tested with a real old client.

// simulate.mjs — tests the classification by actually running an old client
// Old client sends a v1 body and reads a v1 response; the server is on the v2 schema.
import { V1, V2 } from "./schemas.mjs";

const OLD_REQUEST = { "/member": "U-1001", "/isbn": "978-0262033848", "/branch": "shore", "/dayCount": 14.5 };
const NEW_RESPONSE = { "/id": "O-1", "/member": { id: "U-1001" }, "/items": [{}], "/status": "overdue" };

const typeOf = (d) =>
  Array.isArray(d) ? "array" : typeof d === "object" ? "object"
    : typeof d === "number" ? (Number.isInteger(d) ? "integer" : "number") : "string";

// 1) The old client's body is run through the new schema.
const requestErrors = [];
for (const [name, field] of Object.entries(V2.request)) {
  const d = OLD_REQUEST[name];
  if (d === undefined) { if (field.required) requestErrors.push(`${name}: required field not sent`); continue; }
  if (field.type !== typeOf(d) && !(field.type === "number" && typeOf(d) === "integer")) requestErrors.push(`${name}: sent ${typeOf(d)}, expected ${field.type}`);
  if (field.values && !field.values.includes(d)) requestErrors.push(`${name}: ${d} is not accepted`);
}

// 2) The new response is read with the schema the old client expects.
const responseErrors = [];
for (const [name, field] of Object.entries(V1.response)) {
  const d = NEW_RESPONSE[name];
  if (d === undefined) { if (field.required) responseErrors.push(`${name}: expected field missing from response`); continue; }
  if (field.type !== typeOf(d)) responseErrors.push(`${name}: got ${typeOf(d)}, expected ${field.type}`);
  if (field.values && !field.values.includes(d)) responseErrors.push(`${name}: unrecognized value ${d}`);
}

console.log("— the old client's body on the new server —");
for (const h of requestErrors) console.log(`  ${h}`);
console.log(`  fields rejected: ${requestErrors.length}`);

console.log("\n— the new server's response on the old client —");
for (const h of responseErrors) console.log(`  ${h}`);
console.log(`  fields not readable: ${responseErrors.length}`);

— the old client's body on the new server —
  /items: required field not sent
  /dayCount: sent number, expected integer
  fields rejected: 2

— the new server's response on the old client —
  /member: got object, expected string
  /isbn: expected field missing from response
  /returnDate: expected field missing from response
  /status: unrecognized value overdue
  fields not readable: 4

The differ had found three breaking changes in the request direction; only two of them materialized on this client. The /branch field makes the difference: it went from optional to required, but this client was already sending it. The change is breaking by the contract — it raises the version number — but its cost for this client is zero.

Contract and cost are separate measures. The version number looks at the contract and rises even if not a single client is affected; the release plan looks at cost. The migration cost calculation in the Design Systems course established the same distinction: breakingness is the contract’s measure, the number of affected call sites is reality’s measure.

The four items in the response direction, however, materialized exactly. This shows another asymmetry between the directions: breakingness in the request direction depends on what the client sends, breakingness in the response direction depends on what it reads. Clients are generally not required to send every field in the schema, but the fields they read are fixed; this is why breaking changes in the response direction turn into real cost at a higher rate.

Summary

• The contract’s two directions are subject to opposite rules: it is breaking in the request direction if what the server accepts narrows, and breaking in the response direction if what it produces widens.
• The same change falls into opposite classes in the two directions; removing a field from the request is compatible, removing it from the response is breaking.
• Adding a required field to the response is compatible, adding a required field to the request is breaking; type narrowing is breaking in the request, type widening is breaking in the response.
• The version number is derived from the diff result; when breaking changes are excluded from the release, the derived number changes too, moving the debate from the version to the release scope.
• Tolerance for an unknown value is a contract clause and must be declared beforehand; adding it afterward is itself a breaking change.
• The differ tells you what could break; which client actually breaks is a separate measurement, and breakingness in the response direction turns into real cost at a higher rate.

Next Step

The differ detected seven breaking changes and raised the version to 2.0.0, but left one question unanswered: how long will the old version live? Raising the version number does not migrate old clients on its own; for them to migrate, they first have to learn about the situation, then find time for the transition. The next lesson turns this duration into a policy: it adds headers that announce deprecation in the response itself, models usage telemetry to calculate the share of consumers still on the old version at the end of the transition period, and shows how the sunset date is chosen based on that share.