Variables, Locals, and Outputs

The previous lessons dealt with the record of a single environment. When the same definition has to run in three environments, two questions arise: which value should change by environment, and which should stay the same in every one. The structure that writes this has three parts — variables that take a value from outside, locals derived from variables, and outputs the definition hands to the outside. This lesson counts all three: how much of the input surface is genuinely environment-specific, where the odds of giving a wrong value go as the surface grows, and how many dependencies outputs create in other definitions.

The measurement runs on three environments of the regional measurement network and is fictional; the definition, the validation, and the consuming definitions are a model built with node.

DP19. The definition carries fourteen variables, three locals, four resources, and five outputs. DP20. A variable with no default is required; if an environment does not give a value, the default is used silently. DP21. Validation tests only the type and the value range or set membership; it does not test whether the value is correct for that environment. DP22. A variable counts as used if it is read directly or through a local; a variable never read is dead. DP23. Ten wrong inputs are chosen in advance; each is a typo or a unit mix-up that actually happens often. DP24. If an output changes, every definition that reads it gets replanned.

The Definition and Three Environments

// definition.mjs — a fictional definition applied to three environments: variables, locals, resources, outputs.
const D = (type, validate, default_) => ({ type, validate, default: default_ });
const oneOf = (...k) => (v) => k.includes(v);
const between = (a, b) => (v) => v >= a && v <= b;

export const VARIABLE = {                           // a variable with no default is required
  region: D("string", oneOf("west", "east", "central")),
  collectors: D("number", between(1, 64)),
  queue_partitions: D("number", between(1, 128)),
  disk_gb: D("number", between(20, 4000)),
  retention_days: D("number", between(1, 365), 30),
  threshold_ms: D("number", between(50, 5000), 900),
  cores: D("number", between(1, 64), 4),
  memory_gb: D("number", between(1, 256), 8),
  format: D("string", oneOf("binary", "text"), "binary"),
  monitoring: D("string", oneOf("on", "off"), "on"),
  backup_days: D("number", between(0, 35), 1),
  tag_owner: D("string", oneOf("metrics-team", "network-team"), "metrics-team"),
  certificate_days: D("number", between(1, 365), 90),
  legacy_flag: D("string", oneOf("on", "off"), "off"),
};

export const LOCAL = {                              // intermediate values derived from variables
  name_prefix: { input: ["region"], compute: (d) => `${d.region}-metrics` },
  total_cores: { input: ["collectors", "cores"], compute: (d) => d.collectors * d.cores },
  queue_name: { input: ["region"], compute: (d) => `${d.region}-meter-queue` },
};

export const RESOURCE = {                           // the variable and local each resource reads
  "machine.collector": ["region", "collectors", "cores", "memory_gb", "disk_gb", "monitoring",
    "tag_owner", "local.total_cores"],
  "queue.meter": ["queue_partitions", "retention_days", "format", "tag_owner", "local.queue_name"],
  "data_store.metrics": ["threshold_ms", "backup_days", "disk_gb", "local.name_prefix"],
  "object_store.archive": ["retention_days", "local.name_prefix"],
};

export const ENVIRONMENT = {                        // the values the three environments give
  west: { region: "west", collectors: 6, queue_partitions: 12, disk_gb: 200, retention_days: 30,
    threshold_ms: 900, cores: 4, memory_gb: 8 },
  east: { region: "east", collectors: 6, queue_partitions: 12, disk_gb: 200, retention_days: 30,
    threshold_ms: 900, cores: 4, memory_gb: 8 },
  central: { region: "central", collectors: 2, queue_partitions: 4, disk_gb: 60, retention_days: 7,
    threshold_ms: 900, cores: 4, memory_gb: 8 },
};

export const OUTPUT = {                             // values other definitions read
  queue_name: "local.queue_name", store_prefix: "local.name_prefix",
  total_cores: "local.total_cores", owner: "tag_owner", archive_days: "retention_days",
};
export const CONSUMER = {                           // which definition reads which output
  "pipeline-definition": ["queue_name", "store_prefix"],
  "alert-definition": ["queue_name", "archive_days"],
  "report-definition": ["store_prefix", "owner"],
};

export function resolve(given) {                    // effective value: given or default
  const d = {};
  for (const [name, t] of Object.entries(VARIABLE)) d[name] = name in given ? given[name] : t.default;
  for (const [name, l] of Object.entries(LOCAL)) d[`local.${name}`] = l.compute(d);
  return d;
}

A variable declaration carries three things: type, validation, and default. A local is a computation, and it names its input by name; resources read both variables and locals. An output, in turn, is just a routing table — it opens a value inside the definition to the outside by naming it.

The Input Surface’s Balance Sheet

// measure.mjs — input surface, values that vary vs. stay fixed by environment, dead variables, output dependencies.
import { VARIABLE, LOCAL, RESOURCE, ENVIRONMENT, OUTPUT, CONSUMER, resolve } from "./definition.mjs";

const ENVS = Object.keys(ENVIRONMENT), resolved = Object.fromEntries(ENVS.map((o) => [o, resolve(ENVIRONMENT[o])]));
const required = Object.entries(VARIABLE).filter(([, t]) => t.default === undefined).map(([a]) => a);
const used = new Set([...Object.values(RESOURCE).flat(),
  ...Object.values(LOCAL).flatMap((l) => l.input)]);
const dead = Object.keys(VARIABLE).filter((a) => !used.has(a));
const given = Object.keys(VARIABLE).filter((a) => ENVS.every((o) => a in ENVIRONMENT[o]));
const varying = given.filter((a) => new Set(ENVS.map((o) => ENVIRONMENT[o][a])).size > 1);

console.log(`variables ${Object.keys(VARIABLE).length}: ${required.length} required, ` +
  `${Object.keys(VARIABLE).length - required.length} defaulted, ${dead.length} dead (${dead.join(", ")})`);
console.log(`three environments write ${given.length * ENVS.length} values: ` +
  `${varying.length * ENVS.length} vary by environment (${varying.join(", ")}), ` +
  `${(given.length - varying.length) * ENVS.length} are the same across all three ` +
  `(${given.filter((a) => !varying.includes(a)).join(", ")})`);
console.log(`variables never given, left at their default: ` +
  `${Object.keys(VARIABLE).length - given.length}\n`);

const print = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 17 : 23)).join("").trimEnd());
print("resource", "input", "by environment", "always same");
for (const [k, inputs] of Object.entries(RESOURCE)) {
  const varies = inputs.filter((g) => new Set(ENVS.map((o) => resolved[o][g])).size > 1);
  print(k, inputs.length, varies.length, inputs.length - varies.length);
}

const inputs = [...new Set(Object.values(RESOURCE).flat())];
const varyingInputs = inputs.filter((g) => new Set(ENVS.map((o) => resolved[o][g])).size > 1);
console.log(`  ${inputs.length} unique inputs total, ${varyingInputs.length} of them vary by environment`);

const missing = { ...ENVIRONMENT.central };                  // if the central environment forgets a value
delete missing.retention_days;
const resolved2 = resolve(missing), shifted = Object.keys(resolved.central).filter((k) => resolved.central[k] !== resolved2[k]);
console.log(`\nif central does not give retention_days, the default ${VARIABLE.retention_days.default} takes hold ` +
  `(the environment's value is ${ENVIRONMENT.central.retention_days}): ${shifted.length} values shift, ` +
  `${Object.entries(RESOURCE).filter(([, g]) => g.some((x) => shifted.includes(x))).length} resources affected, ` +
  `validation says nothing at all`);

console.log(`\noutputs ${Object.keys(OUTPUT).length}, consuming definitions ${Object.keys(CONSUMER).length}`);
for (const c of Object.keys(OUTPUT)) {
  const t = Object.entries(CONSUMER).filter(([, l]) => l.includes(c)).map(([a]) => a);
  console.log(`  ${c.padEnd(17)}${t.length} consumers  ${t.join(", ") || "(dead output)"}`);
}
const linked = Object.values(CONSUMER).flat().length;
console.log(`total dependencies ${linked}; if an output's value changes, on average ` +
  `${(linked / Object.keys(OUTPUT).length).toFixed(1)} definitions get replanned`);

variables 14: 4 required, 10 defaulted, 2 dead (certificate_days, legacy_flag)
three environments write 24 values: 15 vary by environment (region, collectors, queue_partitions, disk_gb, retention_days), 9 are the same across all three (threshold_ms, cores, memory_gb)
variables never given, left at their default: 6

resource               input            by environment   always same
machine.collector      8                4                4
queue.meter            5                3                2
data_store.metrics     4                2                2
object_store.archive   2                2                0
  15 unique inputs total, 8 of them vary by environment

if central does not give retention_days, the default 30 takes hold (the environment's value is 7): 1 values shift, 2 resources affected, validation says nothing at all

outputs 5, consuming definitions 3
  queue_name       2 consumers  pipeline-definition, alert-definition
  store_prefix     2 consumers  pipeline-definition, report-definition
  total_cores      0 consumers  (dead output)
  owner            1 consumers  report-definition
  archive_days     1 consumers  alert-definition
total dependencies 6; if an output's value changes, on average 1.2 definitions get replanned

The first line gives parameterization’s quiet cost. Four of fourteen variables are required, ten have defaults; two are dead — no resource and no local reads them. A dead variable looks harmless, but it is part of the input surface: it sits in the documentation, gets read during review, gets argued over when a new environment is added about whether to fill it in, and changes nothing.

The second line is this lesson’s main measure. Three environments write twenty-four values in total. Fifteen of these genuinely differ by environment — region, collector count, queue partitions, disk size, retention. The remaining nine are the same across all three: threshold time, cores, and memory are written with the same value in every environment. These nine values do not have to be variables; they could stand inside the definition as a constant or a default. The difference between them shows up in one thing: each has been hand-written three times, and all three carry the chance of being written wrong. Six variables, meanwhile, are never given in any environment and stay at their default; these six are already constants, but they show up in the record as variables.

The third table gives the load’s distribution across resources. Four of the collector machine’s eight inputs vary by environment, four always stay the same; both of the archive object store’s two inputs vary by environment. How much a resource diverges across environments depends on how many of the inputs it reads actually change — not on the number of resources. In total, four resources read fifteen unique inputs, and eight of these vary by environment; so the entire difference between three environments is concentrated at eight points. This is parameterization’s real width, not fourteen variables.

The next line shows the default’s silent side. If the central environment forgets to give retention, the value does not stay empty: the definition’s default of thirty days quietly takes effect, even though that environment wants seven. One value shifts, two resources are set up wrong, and validation says nothing, because thirty days is flawless on type and range. Rather than turning a missing input into an error, the default turns it into a silent decision; giving a default to a variable that should vary by environment is, for this reason, the same as making that variable no longer required.

The last section counts outputs. Four of five outputs are consumed, one is a dead output: no definition reads the total core count. Three consuming definitions establish six dependencies in total; the queue name and the store prefix are each read by two definitions. What this means is: a change to the queue name forces not just its own definition but two other definitions to be replanned as well. An output becomes a promise the definition makes to the outside, and the cost of that promise is however many definitions read it.

Giving a Wrong Value

// input.mjs — as the input surface grows, giving a wrong value: ten fictional mistakes, how many validation catches.
import { VARIABLE, LOCAL, RESOURCE, ENVIRONMENT } from "./definition.mjs";

const WRONG = [                                    // chosen in advance, not random
  ["region typo", "region", "wes"], ["number instead of string", "collectors", "6"],
  ["extra zero", "collectors", 600], ["hours instead of days", "retention_days", 720],
  ["missing digit", "queue_partitions", 2], ["missing zero", "threshold_ms", 90],
  ["unit mix-up", "disk_gb", 20], ["extra digit", "cores", 40],
  ["out-of-set format", "format", "raw"], ["sign", "memory_gb", -8],
];

const validate = (name, v) => {                        // type and value validation
  const t = VARIABLE[name];
  if (typeof v !== (t.type === "number" ? "number" : "string")) return `${t.type} expected`;
  return t.validate(v) ? null : "value not accepted";
};
const affected = (name) => Object.entries(RESOURCE).filter(([, g]) => g.includes(name) ||
  g.some((x) => x.startsWith("local.") && LOCAL[x.slice(6)].input.includes(name))).map(([k]) => k);

console.log(`input surface: ${Object.keys(ENVIRONMENT).length} environments x ` +
  `${Object.keys(ENVIRONMENT.west).length} given variables = ` +
  `${Object.keys(ENVIRONMENT).length * Object.keys(ENVIRONMENT.west).length} hand-written values\n`);
console.log(`${"mistake".padEnd(29)}${"variable".padEnd(18)}${"value".padEnd(9)}${"validation".padEnd(23)}affected resource`);
let passed = 0;
for (const [name, variable, value] of WRONG) {
  const h = validate(variable, value);
  if (!h) passed++;
  console.log(`  ${name.padEnd(27)}${variable.padEnd(18)}${String(value).padEnd(9)}` +
    `${(h ?? "passed").padEnd(23)}${h ? "" : affected(variable).length}`);
}
console.log(`\n${WRONG.length - passed} of ${WRONG.length} mistakes were caught by validation, ` +
  `${passed} fell through to apply`);
const required = Object.entries(VARIABLE).filter(([, t]) => t.default === undefined).length;
console.log(`everything that is not caught is fine on type and range; the only way to tell it is ` +
  `wrong is from the environment's expectation`);
console.log(`if the input surface grows from ${Object.keys(ENVIRONMENT).length} environments to ${Object.keys(ENVIRONMENT).length * 2}, ` +
  `hand-written values go from ${Object.keys(ENVIRONMENT.west).length * Object.keys(ENVIRONMENT).length} to ` +
  `${Object.keys(ENVIRONMENT.west).length * Object.keys(ENVIRONMENT).length * 2}, and required values ` +
  `from ${required * Object.keys(ENVIRONMENT).length} to ${required * Object.keys(ENVIRONMENT).length * 2}`);

input surface: 3 environments x 8 given variables = 24 hand-written values

mistake                      variable          value    validation             affected resource
  region typo                region            wes      value not accepted     
  number instead of string   collectors        6        number expected        
  extra zero                 collectors        600      value not accepted     
  hours instead of days      retention_days    720      value not accepted     
  missing digit              queue_partitions  2        passed                 1
  missing zero               threshold_ms      90       passed                 1
  unit mix-up                disk_gb           20       passed                 2
  extra digit                cores             40       passed                 1
  out-of-set format          format            raw      value not accepted     
  sign                       memory_gb         -8       value not accepted     

6 of 10 mistakes were caught by validation, 4 fell through to apply
everything that is not caught is fine on type and range; the only way to tell it is wrong is from the environment's expectation
if the input surface grows from 3 environments to 6, hand-written values go from 24 to 48, and required values from 12 to 24

The input surface is twenty-four hand-written values for three environments. Six of ten fictional mistakes get caught by validation: a misspelled region name, a string given instead of a number, six hundred written instead of six, hours entered instead of days, an out-of-set format, negative memory. Four of them, though, pass — two written instead of twelve for queue partitions, ninety instead of nine hundred for the threshold, twenty instead of two hundred for disk, forty instead of four for cores. What the four have in common is that they are flawless on type and range. Validation tests a value’s form, it cannot test whether it is correct for that environment; because the standard of correctness lives not inside the definition but in the environment’s expectation.

The impact of the four that pass is also counted: three set up one resource wrong, one sets up two. How many resources an input touches depends on the resources that read it directly and the resources that the locals reading it pass through — since disk size feeds into two resources at once, a single wrong digit shifts both of them together.

The last line shows how the surface grows. If the environment count rises from three to six, hand-written values go from twenty-four to forty-eight, and required values from twelve to twenty-four. The opportunity for a wrong entry grows linearly with the environment count, and three of every eight of that growth come from values written the same across all three environments — that is, values sitting on the surface for no reason. The cheapest way to shrink the input surface is not writing a new validation, it is taking a value that does not vary by environment out of being a variable at all.

Summary

• The definition carried 14 variables: 4 required, 10 defaulted, 2 dead. 6 variables were never given in any environment and stayed at their default.
• Three environments wrote 24 values; 15 genuinely differed by environment, 9 were the same across all three. Those 9 values did not have to be variables, but they were hand-written three times.
• Four resources read 15 unique inputs; the entire difference between three environments was concentrated in 8 of these inputs. When the central environment forgot to give a single value, the default silently kicked in and set up 2 resources wrong.
• 6 of 10 wrong inputs were caught by validation, 4 passed; every one of the four was flawless on type and range and set up 3 resources with a wrong value in 5 places.
• 4 of 5 outputs were consumed, 1 was a dead output; 3 consuming definitions established 6 dependencies, and two outputs were each read by two definitions.
• If the environment count rises from 3 to 6, hand-written values go from 24 to 48, and required values from 12 to 24.

Next Step

Variables gave the definition values from outside, locals derived them, outputs opened the result to other definitions. But these three structures never said one thing: which resource has to be created before which. If the queue name depends on a machine, the endpoint on a data store, the network rule on a subnet, apply cannot try them in a random order. The next lesson takes on this ordering: the difference between the implicit dependency born from one resource reading another resource’s value and the hand-written explicit dependency, how the ordering gets resolved, and where apply stalls when the order is wrong.