Custom Resource Definitions

The previous lesson measured that a template produces new combinations from existing object types, but cannot produce a new type. The regional measurement network’s reading-collection arrangement does not sit neatly in any of the five objects either: a deployment, a quota, and a scheduled job together describe a single concept, and that concept’s name is written down nowhere.

A custom resource definition adds a new type to the list of object types the cluster recognizes. The delegated decision is plain: whether a declaration is valid gets decided not by a reviewing human but by a schema validator that runs at admission. What this lesson measures is that delegation’s payoff — as the schema grows, which class of error starts getting caught at admission, how many fields that costs to write in return, and how badly old declarations break under the new schema.

The measurement is again over the regional measurement network and is fictional. No real cluster is run; the defined type, the schema validator, and the pipeline’s execution are a model built with node.

• OO37. The fictional resource type is a reading pipeline, and it has eleven fields.
• OO38. Twelve declarations are hand-written; nine of them carry a flaw, and the flaws were not chosen at random.
• OO39. The run model processes 1200 readings a day and splits the result into three classes: correct, blew up, silently wrong.
• OO40. There are three schema versions: A checks only presence, B adds type and set membership to presence, C adds range and dependency rules on top of those.
• OO41. Writing burden is the required-field count multiplied by the declaration count.
• OO42. In the version-compatibility measurement, legacy declarations carry only schema A’s fields.

A Fictional Resource Type and Three Schemas

// resource.mjs — fictional resource type "ReadingPipeline": fields, three schema versions, twelve declarations.
export const FIELD = ["name", "source", "target", "batch-size", "window-min", "format",
  "concurrent-workers", "retry", "backoff-sec", "latency-limit-sec", "daily-cap"];

const S = (...a) => a;
export const DECLARATION = [
  //  name             source        target       batch  win  format       work retry back  latency  cap
  S("west-night",     "meter-queue", "verifier",   500,  15, "binary",      4,   3,    2,     960, 2000000),
  S("east-night",     "meter-queue", "verifier",   500,  15, "binary",      4,   3,    2,     960, 2000000),
  S("central-hourly", "meter-queue", "billing",    250,  60, "text",        2,   1,    5,    3700, 2000000),
  S("west-hourly",    "meter-queue",  null,        250,  60, "text",        2,   1,    5,    3700, 2000000),
  S("east-hourly",     null,        "billing",     250,  60, "text",        2,   1,    5,    3700, 2000000),
  S("north-night",    "meter-queue", "verifier", "fivehundred", 15, "binary", 4, 3,    2,     960, 2000000),
  S("south-night",    "meter-queue", "verifier",   500,  15, "raw",         4,   3,    2,     960, 2000000),
  S("west-standby",   "meter-queue", "verifier",   500,  15, "binary",    "-",   3,    2,     960, 2000000),
  S("east-standby",   "meter-queue", "verifier",   500,   0, "binary",      4,   3,    2,     960, 2000000),
  S("central-standby","meter-queue", "billing",    250,  60, "text",        2,   5, null,    3700, 2000000),
  S("west-fast",      "meter-queue", "work-order", 500,  15, "binary",      4,   3,    2,      60, 2000000),
  S("east-restricted","meter-queue", "work-order", 500,  15, "binary",      4,   3,    2,     960,    1000),
].map((d) => Object.fromEntries(FIELD.map((a, i) => [a, d[i]])));

const NUMBER = { "batch-size": "number", "window-min": "number", "concurrent-workers": "number" };
const NUMBER2 = { ...NUMBER, retry: "number", "backoff-sec": "number",
  "latency-limit-sec": "number", "daily-cap": "number" };
export const SCHEMA = {
  A: { required: FIELD.slice(0, 3), type: {}, set: {}, range: {}, rule: [] },
  B: { required: FIELD.slice(0, 7), type: NUMBER, set: { format: ["binary", "text", "compressed"] },
       range: {}, rule: [] },
  C: { required: FIELD.filter((a) => a !== "backoff-sec"), type: NUMBER2,
       set: { format: ["binary", "text", "compressed"] },
       range: { "window-min": [1, 1440], "batch-size": [1, 100000], "concurrent-workers": [1, 64],
                 retry: [0, 10] },
       rule: [
         { name: "backoff dependency", f: (b) => b.retry > 0 && b["backoff-sec"] == null },
         { name: "latency limit smaller than window", f: (b) => b["latency-limit-sec"] < b["window-min"] * 60 },
         { name: "daily cap smaller than daily volume",
           f: (b) => b["daily-cap"] < b["batch-size"] * (1440 / b["window-min"]) } ] },
};

A type definition is two parts: a field list and a schema. The field list determines what the type can say; the schema determines when what is said gets accepted. The three versions apply different degrees of strictness over the same eleven fields — A checks only whether a field exists, B additionally checks its type and set membership, C adds numeric range and cross-field dependency on top of those.

At Admission and at Runtime

// validate.mjs — schema validator (at admission) and the reading pipeline's fictional runtime model.
export function validate(b, s) {
  const h = [];
  for (const a of s.required) if (b[a] == null) h.push(`${a} missing`);
  for (const [a, t] of Object.entries(s.type))
    if (b[a] != null && t === "number" && typeof b[a] !== "number") h.push(`${a} is not a number`);
  for (const [a, k] of Object.entries(s.set))
    if (b[a] != null && !k.includes(b[a])) h.push(`${a} outside the set`);
  for (const [a, [lo, hi]] of Object.entries(s.range))
    if (typeof b[a] === "number" && (b[a] < lo || b[a] > hi)) h.push(`${a} outside the range`);
  for (const r of s.rule) if (r.f(b)) h.push(r.name);
  return h;
}

export function run(b) {                             // fictional reading pipeline: processes 1200 readings a day
  const TOTAL = 1200;
  if (!b.source || !b.target) throw new Error("source or target could not be resolved");
  if (!["binary", "text", "compressed"].includes(b.format)) throw new Error("unknown format");
  const windows = 1440 / b["window-min"];
  if (!Number.isFinite(windows)) throw new Error("window count could not be computed");
  if (b.retry > 0 && b["backoff-sec"] == null)
    throw new Error("backoff undefined, retry entered a loop");
  const batches = Math.ceil(TOTAL / b["batch-size"]);
  const processed = Number.isFinite(batches) ? Math.min(TOTAL, b["daily-cap"]) : 0;
  const latency = b["window-min"] * 60 + (batches * 2) / b["concurrent-workers"];
  const exceeded = !(latency <= b["latency-limit-sec"]);
  return { processed, dropped: TOTAL - processed, latency, exceeded,
    correct: processed === TOTAL && Number.isFinite(latency) && !exceeded };
}

// run13.mjs — across three schema versions, what is caught at admission, what blows up, what is silently wrong.
import { DECLARATION, SCHEMA } from "./resource.mjs";
import { validate, run } from "./validate.mjs";

const outcome = DECLARATION.map((b) => {              // the real run outcome, with no schema applied
  try { return run(b).correct ? "correct" : "silent"; } catch { return "blew up"; }
});
console.log("run of the twelve declarations with no schema: " +
  ["correct", "blew up", "silent"].map((t) => `${t} ${outcome.filter((x) => x === t).length}`).join("  "));
DECLARATION.forEach((b, i) => {
  if (outcome[i] === "correct") return;
  let n = "no error, wrong result";
  try { run(b); } catch (e) { n = e.message; }
  console.log(`   ${b.name.padEnd(17)} ${outcome[i].padEnd(9)} ${n}`);
});

const constraints = (s) => s.required.length + Object.keys(s.type).length + Object.keys(s.set).length +
  Object.keys(s.range).length + s.rule.length;
console.log("\nversion  required  constraints  burden  admission  blew up  silent  passing");
for (const [name, s] of Object.entries(SCHEMA)) {
  const rejected = DECLARATION.map((b) => validate(b, s).length > 0);
  const k = outcome.filter((_, i) => !rejected[i]);
  const count = (t) => String(k.filter((x) => x === t).length).padStart(2);
  console.log(`  ${name}         ${String(s.required.length).padStart(2)}           ${String(constraints(s)).padStart(2)}` +
    `          ${String(s.required.length * DECLARATION.length).padStart(3)}      ${String(rejected.filter(Boolean).length).padStart(2)}` +
    `       ${count("blew up")}      ${count("silent")}    ${count("correct")}`);
}

run of the twelve declarations with no schema: correct 3  blew up 5  silent 4
   west-hourly       blew up   source or target could not be resolved
   east-hourly       blew up   source or target could not be resolved
   north-night       silent    no error, wrong result
   south-night       blew up   unknown format
   west-standby      silent    no error, wrong result
   east-standby      blew up   window count could not be computed
   central-standby   blew up   backoff undefined, retry entered a loop
   west-fast         silent    no error, wrong result
   east-restricted   silent    no error, wrong result

version  required  constraints  burden  admission  blew up  silent  passing
  A          3            3           36       2        3       4     3
  B          7           11           84       5        2       2     3
  C         10           25          120       9        0       0     3

The first block gives the schema-less state: three of twelve declarations run correctly, five blow up at runtime, four produce a wrong result with no error at all. The four silent cases each have their own separate cause, and none of them shows up in any log: batch size is written as text, so zero readings get processed; worker count is text, so latency cannot be computed; the latency limit is smaller than the window, so the limit gets exceeded every round; the daily cap is smaller than the daily volume, so two hundred readings drop. All four are consistent with the sentence “I set up the pipeline, it is running.”

The table shows how this split shifts as the schema grows. Schema A rejects only two declarations at admission; the remaining seven flaws fall through to runtime. Schema B rises to five, but two silent flaws still get through. Schema C catches nine of the nine at admission, and nothing is left to blow up at runtime. Its cost sits in the same row: writing burden climbs from thirty-six to a hundred twenty fields, and the constraint count in the schema definition climbs from three to twenty-five. Every new error class caught at admission is a new field or a new rule written into all twelve declarations.

The direction of the relationship can also be read off: going from A to B catches three more errors and writing burden grows by forty-eight fields — sixteen fields per error. Going from B to C catches four more errors and the burden grows by thirty-six fields — nine fields per error. What gets cheaper is not the field count, it is the class of flaw caught: the three dependency rules C adds do not look at individual fields, they look at consistency across fields, and that is where two of the silent flaws get picked up.

Version Compatibility

// compat.mjs — declarations written under schema A moved to schema C: broken fields, and the cost of defaults.
import { FIELD, DECLARATION, SCHEMA } from "./resource.mjs";
import { validate, run } from "./validate.mjs";

const A_FIELD = SCHEMA.A.required, NEW_FIELD = FIELD.filter((a) => !A_FIELD.includes(a));
const legacy = DECLARATION.map((b) => Object.fromEntries(A_FIELD.map((a) => [a, b[a]])));
const DEFAULT = { "batch-size": 500, "window-min": 15, format: "binary", "concurrent-workers": 4,
  retry: 3, "backoff-sec": 2, "latency-limit-sec": 960, "daily-cap": 2000000 };
const filled = legacy.map((b) => ({ ...DEFAULT, ...b }));
const broken = (k) => k.reduce((t, b) => t + validate(b, SCHEMA.C).length, 0);

console.log(`legacy declarations ${legacy.length}, each with ${A_FIELD.length} fields;` +
  ` schema C wants ${SCHEMA.C.required.length} required fields (new fields ${NEW_FIELD.length})`);
console.log(`broken fields ${broken(legacy)} (${(broken(legacy) / legacy.length).toFixed(1)} per declaration);` +
  ` once defaults are defined ${broken(filled)}`);

let diverging = 0, declarations = 0;                 // how many fields the default diverges from the written value
for (let i = 0; i < legacy.length; i++) {
  const f = NEW_FIELD.filter((a) => DEFAULT[a] !== DECLARATION[i][a]);
  if (f.length) { diverging += f.length; declarations++; }
}
console.log(`the default diverges from the recorded value in ${declarations}/${legacy.length} declarations and ${diverging} fields`);

const outcome = (b) => { try { const r = run(b); return `read ${r.processed}, latency ${Math.round(r.latency)} sec`; }
  catch (e) { return `blew up (${e.message.split(",")[0]})`; } };
for (const i of [0, 2, 11]) console.log(`   ${legacy[i].name.padEnd(17)} written declaration: ${outcome(DECLARATION[i]).padEnd(30)}` +
  ` with default: ${outcome(filled[i])}`);

legacy declarations 12, each with 3 fields; schema C wants 10 required fields (new fields 8)
broken fields 86 (7.2 per declaration); once defaults are defined 2
the default diverges from the recorded value in 10/12 declarations and 34 fields
   west-night        written declaration: read 1200, latency 902 sec     with default: read 1200, latency 902 sec
   central-hourly    written declaration: read 1200, latency 3605 sec    with default: read 1200, latency 902 sec
   east-restricted   written declaration: read 1000, latency 902 sec     with default: read 1200, latency 902 sec

This is where extending the type’s cost shows up. Twelve declarations written under schema A carry three fields; schema C wants ten required fields. 7.2 fields break per declaration on average, eighty-six in total. This is the declaration owners’ work, not the type owner’s, and it gets paid once.

Defining a default drops the breakage from eighty-six to two — the two remaining breaks are fields that were already missing under schema A. But the default is not free: in ten of the twelve declarations, across thirty-four fields in total, the default value differs from what the declaration wrote. Sample rows show what kind of difference this is. The night pipeline behaves the same under the default, because the default was chosen from its own values in the first place. The hourly pipeline drops its window value from sixty to fifteen, and its latency falls from 3605 seconds to 902 — the pipeline keeps the same name, but it is now a different pipeline. The restricted pipeline loses its daily cap and processes twelve hundred readings instead of a thousand; the reason the cap existed has been erased from the declaration.

Both breakage and the default trace back to the same source: the old declaration says nothing about the new fields, and nobody knows what it does not say. Breakage makes this unknowability visible and loads its cost onto the declaration owner; the default closes the unknowability and absorbs its cost silently.

Delegation’s Balance Sheet

Measure	Schema A	Schema B	Schema C
required fields	3	7	10
constraints in schema definition	3	11	25
writing burden across twelve declarations (fields)	36	84	120
declarations caught at admission	2	5	9
blew up at runtime	3	2	0
silently wrong	4	2	0
passing and working correctly	3	3	3

The delegated decision is one sentence: the schema, not a human, decides a declaration’s validity. The items the system needs to know for this are countable — eleven fields’ names, three of their types or set memberships, four of their ranges, three dependency rules. Information’s cost is countable too: every item gets written into all twelve declarations, and the burden grows linearly as the declaration count grows.

What the system does not know also does something, and the table records it. Under schema A, four declarations get accepted, run, produce a result, and the result is wrong; the schema does not know the cap cannot be smaller than the volume, so it allows two hundred readings to drop. Once schema C learns that same rule, those four flaws bounce back at admission. Learning’s payoff is a hundred twenty fields of writing burden, and that burden falls equally on the three declarations that carried no flaw: a correctly written declaration has to write the same ten fields as an incorrectly written one.

Summary

• With no schema, 3 of 12 declarations ran correctly, 5 blew up at runtime, 4 produced a wrong result with no error.
• Schema A caught 2 of 9 flaws at admission, schema B 5, schema C 9; under schema C, the flaws falling through to runtime dropped to 0.
• The payoff’s cost is writing burden: required fields went from 3 to 10, the twelve declarations’ total burden from 36 to 120 fields, and the schema definition’s constraint count from 3 to 25.
• Going from A to B cost 16 fields per error, from B to C cost 9 fields per error; what made it cheaper was the 3 dependency rules that check cross-field consistency.
• Declarations written under schema A broke on 7.2 fields per declaration under schema C; a default cut the breakage from 86 to 2, but diverged from the written value in 10 of 12 declarations, across 34 fields.

Next Step

A schema says a declaration is valid; it does not say the declaration is correct. An accepted reading-pipeline declaration does nothing on its own in the cluster; something has to read it and bring the real state to the desired one, and today that something is an on-call human. The next lesson writes a controller that genuinely runs for this type: it reads the desired state, reads the real state, closes the gap, and does zero work on the second round. A manual operation’s closing time, in steps and minutes, is compared against the controller’s closing time, in rounds; the operating knowledge coded into the controller is counted item by item, and what the uncoded knowledge does in production is shown.