Graceful Degradation

The bulkhead pattern shut a broken dependency inside its own pool: once the pool fills, the call fails fast, and other dependencies’ pools stay unaffected. What gets saved is the downstream side. Upstream, nothing has changed — the call site still gets an error object, the endpoint still throws it upward, the client still comes back empty. Even with four of the five fields in the panel response on hand, all five get thrown away because the fifth never arrived.

Graceful degradation closes this gap. Its effect was measured in the Resilience and Reliability course: which policy saves how many requests, the cost of a failure-free day, and what the pattern can do when a core dependency goes down were all counted there. What is measured here is something else: where this pattern sits in the code, how many call sites it covers, and which call sites it skips.

The Schema Object and Call Sites

The mechanism is five services of the split loan system. Each is a local node:http server, and refuses the connection when its flag is turned off — for the caller, this means an error object it has no control over.

// degradation/services.mjs — upstream services of the split loan system.
// Each is a local node:http process; when the RUNNING flag turns off, the connection is refused.
import http from "node:http";

export const PORT = {};
export const RUNNING = { membership: 1, loan: 1, billing: 1, notification: 1, catalog: 1 };
export const BODY = {
  membership: { name: "N.A.", status: "active" },
  loan: { open: 3, overdue: 1 },
  billing: { balance: 12.5 },
  notification: { unread: 4 },
  catalog: { suggestions: ["K-1180"] },
};
const SERVERS = [];

export async function start() {
  for (const name of Object.keys(BODY)) {
    const s = http.createServer((req, res) => {
      if (RUNNING[name] === 0) { res.socket.destroy(); return; }
      res.writeHead(200, { "content-type": "application/json" });
      res.end(JSON.stringify(BODY[name]));
    });
    await new Promise((r) => s.listen(0, "127.0.0.1", r));
    PORT[name] = s.address().port;
    SERVERS.push(s);
  }
}
export function stop() { for (const s of SERVERS) s.close(); }

// Plain call site: throws an error when the service is down. Every unwrapped call behaves this way.
export async function call(name) {
  const y = await fetch(`http://127.0.0.1:${PORT[name]}/`);
  if (y.ok === false) throw new Error(`${name}: HTTP ${y.status}`);
  return y.json();
}

RS10 — the panel’s field split. The member panel has five fields. member and loan are core: without these two the page shows nothing. balance, notification, and suggestions are optional, and each has a fallback value to stand in for it. RS11 — the receipt and summary endpoints were written before the panel and make their own calls; this is typical of the situation when the degradation pattern enters a system after the fact.

The split does not scatter through the code; it lives in a single schema object. Both placements read the same schema.

// degradation/panel.mjs — the panel response's schema and the pattern's two placements.
import { call } from "./services.mjs";

// RS10: the response's core/optional split lives in a schema object, not scattered through the code.
export const SCHEMA = {
  member: { service: "membership", core: 1 },
  loan: { service: "loan", core: 1 },
  balance: { service: "billing", core: 0, fallback: { balance: 0 } },
  notification: { service: "notification", core: 0, fallback: { unread: 0 } },
  suggestions: { service: "catalog", core: 0, fallback: { suggestions: [] } },
};

// Placement A — at the call site: every optional call carries its own try/catch and its own fallback.
export async function panelA() {
  const y = { member: await call("membership"), loan: await call("loan") };
  try { y.balance = await call("billing"); } catch { y.balance = { balance: 0 }; }
  try { y.notification = await call("notification"); } catch { y.notification = { unread: 0 }; }
  try { y.suggestions = await call("catalog"); } catch { y.suggestions = { suggestions: [] }; }
  return y;
}

// Placement B — shared wrapper: the fallback is read from the schema, the dropped field is marked in the response.
export async function optional(field, missing) {
  const t = SCHEMA[field];
  try { return await call(t.service); } catch { missing.push(field); return t.fallback; }
}
export async function panelB() {
  const missing = [];
  const y = { member: await call("membership"), loan: await call("loan") };
  y.balance = await optional("balance", missing);
  y.notification = await optional("notification", missing);
  y.suggestions = await optional("suggestions", missing);
  return { ...y, missing };
}

// The other two endpoints that touch the same services. Both were written before the panel.
export async function receipt() {
  const u = await call("membership");
  const b = await call("billing");
  return { member: u.name, balance: b.balance };
}
export async function summary() {
  const o = await call("loan");
  const b = await call("notification");
  return { open: o.open, unread: b.unread };
}

Source Scan and Run

The measurement has two parts. First the source text is scanned and every call site is classified: does it reach a core service, an optional service that is wrapped, or an optional service that is not wrapped. The third case is the uncovered call site. Then the same code runs under failure scenarios, and the scan is compared against the run.

// degradation/measurement.mjs — first the source is counted, then the same code runs under failure scenarios.
import { readFileSync } from "node:fs";
import { start, stop, RUNNING, BODY } from "./services.mjs";
import { SCHEMA, panelA, panelB, receipt, summary } from "./panel.mjs";

const OPTIONAL = new Set(Object.values(SCHEMA).filter((t) => !t.core).map((t) => t.service));
const sites = [], lines = {};            // every call site: which function, which service, wrapped or not
let current = null;
for (const s of readFileSync(new URL("./panel.mjs", import.meta.url), "utf8").split("\n")) {
  const head = s.match(/^export async function (\w+)/);
  if (head) current = head[1];
  if (current === null) continue;
  if (s.trim() !== "") lines[current] = (lines[current] ?? 0) + 1;
  for (const m of s.matchAll(/call\("(\w+)"\)/g))              // plain call site with the name written out
    sites.push({ fn: current, service: m[1], wrapped: s.includes("try {") });
  for (const m of s.matchAll(/optional\("(\w+)"/g))            // call site handed to the wrapper
    sites.push({ fn: current, service: SCHEMA[m[1]].service, wrapped: true });
  if (s.startsWith("}")) current = null;
}
const classOf = (y) => (OPTIONAL.has(y.service) ? (y.wrapped ? "wrapped" : "uncovered") : "core");

console.log("-- source scan --");
console.log(`${"function".padEnd(13)}${"lines".padStart(6)}${"call sites".padStart(11)}${"core".padStart(9)}${"wrapped".padStart(11)}${"uncovered".padStart(10)}`);
for (const i of [...new Set(sites.map((y) => y.fn))]) {
  const t = sites.filter((y) => y.fn === i);
  const n = (k) => String(t.filter((y) => classOf(y) === k).length);
  console.log(`${i.padEnd(13)}${String(lines[i]).padStart(6)}${String(t.length).padStart(11)}${n("core").padStart(9)}${n("wrapped").padStart(11)}${n("uncovered").padStart(10)}`);
}
console.log(`total ${sites.length} call sites; ${sites.filter((y) => classOf(y) === "wrapped").length} wrapped, ${sites.filter((y) => classOf(y) === "uncovered").length} uncovered`);
console.log(`placement A ${lines.panelA} lines, placement B ${lines.panelB}+${lines.optional}=${lines.panelB + lines.optional} lines\n`);

await start();
const ENDPOINT = { panelA, panelB, receipt, summary };
const SCENARIO = [["all healthy", []], ["billing down", ["billing"]],
  ["billing+notification+catalog", ["billing", "notification", "catalog"]], ["membership down", ["membership"]]];
console.log("-- run: fate of the call sites reaching the down service --");
console.log(`${"scenario".padEnd(31)}${"touched".padStart(10)}${"fell back".padStart(12)}${"threw".padStart(14)}${"endpoints answering".padStart(20)}`);
const body = {};
for (const [name, down] of SCENARIO) {
  for (const s of Object.keys(RUNNING)) RUNNING[s] = down.includes(s) ? 0 : 1;
  let answered = 0;
  for (const [ep, f] of Object.entries(ENDPOINT)) {
    try { body[`${name}|${ep}`] = JSON.stringify(await f()); answered += 1; } catch { /* endpoint fell */ }
  }
  const d = sites.filter((y) => down.includes(y.service));
  const fallenBack = d.filter((y) => classOf(y) === "wrapped").length;
  console.log(`${name.padEnd(31)}${String(d.length).padStart(10)}${String(fallenBack).padStart(12)}${String(d.length - fallenBack).padStart(14)}${`${answered}/4`.padStart(20)}`);
}

for (const s of Object.keys(RUNNING)) RUNNING[s] = 1;
BODY.billing.balance = 0;                            // a member whose balance is genuinely zero
const zeroA = JSON.stringify(await panelA()), zeroB = JSON.stringify(await panelB());
const downA = body["billing down|panelA"], downB = body["billing down|panelB"];
console.log("\n-- the fallback's silent falsehood: service down / balance genuinely 0 --");
console.log(`A down: ${downA}`);
console.log(`A zero: ${zeroA}`);
console.log(`same body: ${downA === zeroA}`);
console.log(`B down: ${downB.slice(downB.indexOf('"balance"'))}`);
console.log(`B zero: ${zeroB.slice(zeroB.indexOf('"balance"'))}`);
console.log(`same body: ${downB === zeroB}; the marker costs ${downB.length - downA.length} bytes, ${(((downB.length - downA.length) / downA.length) * 100).toFixed(1)}% of the response`);
stop();

-- source scan --
function      lines call sites     core    wrapped uncovered
panelA            7          5        2          3         0
panelB            8          5        2          3         0
receipt           5          2        1          0         1
summary           5          2        1          0         1
total 14 call sites; 6 wrapped, 2 uncovered
placement A 7 lines, placement B 8+4=12 lines

-- run: fate of the call sites reaching the down service --
scenario                          touched   fell back         threw endpoints answering
all healthy                             0           0             0                 4/4
billing down                            3           2             1                 3/4
billing+notification+catalog            8           6             2                 2/4
membership down                         3           0             3                 1/4

-- the fallback's silent falsehood: service down / balance genuinely 0 --
A down: {"member":{"name":"N.A.","status":"active"},"loan":{"open":3,"overdue":1},"balance":{"balance":0},"notification":{"unread":4},"suggestions":{"suggestions":["K-1180"]}}
A zero: {"member":{"name":"N.A.","status":"active"},"loan":{"open":3,"overdue":1},"balance":{"balance":0},"notification":{"unread":4},"suggestions":{"suggestions":["K-1180"]}}
same body: true
B down: "balance":{"balance":0},"notification":{"unread":4},"suggestions":{"suggestions":["K-1180"]},"missing":["balance"]}
B zero: "balance":{"balance":0},"notification":{"unread":4},"suggestions":{"suggestions":["K-1180"]},"missing":[]}
same body: false; the marker costs 22 bytes, 13.2% of the response

Two Placements and the Uncovered Call Site

Both placements do the same job and produce the same result at the panel endpoint: each wraps the same three optional calls, and each produces a response when billing goes down. The line cost differs — the call-site placement is 7 lines, the shared-wrapper placement is 8 plus 4, 12 lines total. At first glance, the call-site placement looks cheaper.

What makes the difference is not the line count but where the fallback is written. In the call-site placement, the expression { balance: 0 } is buried inside a catch block, with no link at all to the fallback field in the schema. The two can be edited separately and drift apart silently. In the shared wrapper, the fallback is read from a single place, the schema; the decision on whether a field is core lives there too.

The real measure is in the second table. When the billing service goes down, three call sites reach it: the two placements in the panel and the call inside the receipt endpoint. Two of them return a fallback, one still throws — the receipt endpoint cannot answer, three of the four endpoints stay up. When three services go down at once, the touched call sites rise to eight, six return a fallback, two throw, and the summary endpoint falls too.

The degradation pattern settled into the panel, not into the system. The schema object declares the balance field optional, but that declaration only binds the code path that reads it. The receipt endpoint calls the same service, gets the same error, and keeps throwing it upward. The pattern’s coverage is not where it was set up, it is the set of places where it is called. The source scan counts this set as six wrapped and two uncovered call sites; the two uncovered call sites turn into two fallen endpoints at run time.

The last row shows another side of the split. When the membership service goes down, all three call sites reaching it are classed as core, and none is wrapped — this is not a shortcoming, it is the design itself. Yet the summary endpoint stays up, because membership is not a dependency for it. Coreness is a property of the endpoint, not of the field: the same service is core at one endpoint and absent entirely at another.

The Fallback’s Silent Falsehood

The last section measures a flaw the pattern produces itself. The body the call-site placement produces while billing is down is compared against the body of a member whose balance is genuinely zero on a healthy day: byte for byte identical. The value balance: 0 means two different things, and the client has no way to tell them apart. The member’s screen says you have no balance due, while in reality there is a twelve-fifty balance, and no one sees an error.

This is the one point where the degradation pattern can be worse than throwing code. A thrown error is visible: the endpoint falls, it gets logged, it shows up in a counter. A fallback value is invisible, because it sits inside a successful response. The wrapped placement fixes this: the missing list carries which field was filled with a fallback right in the response itself, and the two bodies diverge. The cost is 22 bytes, 13.2 percent of this response.

These 22 bytes are not just one extra field, they are the pattern’s observable form. A degradation carrying no marker is invisible to every tool built in this course’s first two topics: no log is produced, no counter moves, no indicator shifts. Without a marker on the fallback value, degradation cannot be measured; and degradation that cannot be measured is, before long, mistaken for normal.

Summary

• Graceful degradation settles into the code as a schema object: which field is core, which is optional, and what its fallback is all live in one place. The pattern’s effect was measured in the Resilience and Reliability course; the measure here is the placement itself.
• Two placements were compared: try/catch at the call site is 7 lines, the shared wrapper is 12. What the line difference buys is the fallback being read from the schema instead of buried in a catch block.
• The source scan found 14 call sites: 6 wrapped, 2 uncovered, the rest core calls. The two uncovered call sites sit in the receipt and summary endpoints, both written before the panel.
• The run confirmed the scan: when billing went down, 2 of 3 call sites returned a fallback, 1 threw, and one of four endpoints fell. When three services went down at once, 6 of 8 call sites returned a fallback and two endpoints fell.
• An unmarked fallback silently produces false data: the body produced while the service is down is byte for byte identical to a member’s body whose balance is genuinely zero. The missing list separates the two and adds 22 bytes, 13.2 percent, to the response.
• Coreness is a property of the endpoint, not the field: the membership service is core for the panel and the receipt, and absent entirely for the summary endpoint.

Next Step

The five patterns so far all look in the same direction: what this system does when a downstream dependency breaks. The timeout cuts the waiting short, the retry asks again, the breaker stops asking, the bulkhead separates resources, degradation shrinks the response. All of them have the same subject: the caller.

The other side of the same system has never been addressed. These services do not only call, they also get called, and there is no limit on how many requests the calling side can send. One client’s badly written loop can send thousands of requests a minute to a single endpoint, and none of the patterns here can stop it — all of them regulate what happens to a request that has already arrived. The next lesson places the limit on the accepting side, and measures: which layer of the code the limiter sits in, whom the limit applies to, which paths stay outside it, and what happens when the wrong key is chosen.