Environments

The previous topic built feedback loops and counted how many steps an occurrence took to reach a role. Every one of those measurements carried a single silent assumption: the software running on a developer’s machine and the software running in production at midnight are the same software. If the assumption held, a defect first seen in production would have to be seen on the development machine too.

The difference is hiding somewhere, and it is not measured until it is named. This lesson takes up the first place the difference hides: the environment the software runs in. Here, an environment is not just where testing happens — it is a link in the chain leading to production, and the question this lesson asks is what each link can see.

One fictional setup runs through the whole topic: a municipality’s regional measurement network. The software collects readings from water meters, verifies the readings, converts them into invoices, and opens work orders for field crews. A nightly batch job processes the whole day’s readings. It is fiction; the numbers below come from this fiction’s model, not from a measured setup.

DC1. The team that writes the network and the team that operates it are separate. The network runs three environments: development, test, and production. The fourth link, the staging environment, is not set up; what this lesson measures is exactly what that link would add.

The Chain’s Four Links

An environment is a stop a change passes through on its way to production. The four stops are distinguished by purpose, not by name.

The development environment is the writer’s machine. A single process runs, every external dependency is faked, and the data is a few hundred hand-generated records. Its purpose is to see the shape of the change: does the code compile, does the function return the expected value.

The test environment runs the change independently of its writer. Automated tests run here, the data has a realistic shape but is small, and some of the dependencies are real. Its purpose is to verify, independently of the writer’s machine, whether the change is correct on its own.

The staging environment is the link closest to production. External dependencies are real, resource limits are production’s limits, versions are production’s versions. Its purpose is not verification — it is seeing production-specific defect classes before they reach production.

The production environment is where real meter readings are processed, real invoices are produced, and real work orders are opened for real field crews. The cost of a defect here is a wrong invoice.

Which of these four stops can see what depends on how closely that stop resembles production. The measure of that resemblance is called environment parity; the name for it becoming unmeasurable is configuration drift: the links drift apart from one another over time, and where they drift apart goes unrecorded.

What Parity Is Measured By

For parity to be counted, its dimensions must be named. The fictional network’s environment document declares seven dimensions: two package versions, one resource limit, one concurrency setting, one data volume, and whether two external dependencies are real or fake.

DC2. The seven dimensions are written in the environment manifest, and both teams can read this document. DC3. Three more dimensions exist — timezone, the file system’s case sensitivity, network latency — and none of these is written in any document; they can only be measured while the process is running.

// measurement-network/environments.mjs — the four environments' manifest and runtime probe (model, fictional data)

// Seven dimensions written in the environment document. Values belong to the regional measurement network fiction.
export const MANIFEST = {
  development: { resolverVersion: "3.2.1", rulePackageVersion: "9.1.0", memoryMB: 512,
    concurrentWorkers: 1, meterCount: 120, billingGateway: "fake", workOrderQueue: "fake" },
  test: { resolverVersion: "3.2.1", rulePackageVersion: "9.1.0", memoryMB: 1024,
    concurrentWorkers: 2, meterCount: 5000, billingGateway: "fake", workOrderQueue: "real" },
  staging: { resolverVersion: "3.4.0", rulePackageVersion: "9.1.0", memoryMB: 4096,
    concurrentWorkers: 8, meterCount: 180000, billingGateway: "real", workOrderQueue: "real" },
  production: { resolverVersion: "3.4.0", rulePackageVersion: "9.1.0", memoryMB: 4096,
    concurrentWorkers: 8, meterCount: 1240000, billingGateway: "real", workOrderQueue: "real" },
};

// Three dimensions absent from the manifest, measurable only from inside the running process.
export const PROBE = {
  development: { timezone: "+03:00", caseSensitiveFileSystem: false, networkLatencyMs: 0 },
  test: { timezone: "+03:00", caseSensitiveFileSystem: true, networkLatencyMs: 2 },
  staging: { timezone: "+00:00", caseSensitiveFileSystem: true, networkLatencyMs: 11 },
  production: { timezone: "+00:00", caseSensitiveFileSystem: true, networkLatencyMs: 34 },
};

// The defect class each dimension conceals when it differs from production.
export const CONCEALED = {
  resolverVersion: "resolution drift", rulePackageVersion: "amount error",
  memoryMB: "memory exhaustion", concurrentWorkers: "race condition",
  meterCount: "scale collapse", billingGateway: "contract drift",
  workOrderQueue: "contract drift", timezone: "day-boundary shift",
  caseSensitiveFileSystem: "path resolution error", networkLatencyMs: "timeout",
};

export const DECLARED = Object.keys(MANIFEST.production);
export const PROBED = Object.keys(PROBE.production);
export const DIMENSIONS = [...DECLARED, ...PROBED];
export const full = (env) => ({ ...MANIFEST[env], ...PROBE[env] });
export const differences = (env) =>
  DIMENSIONS.filter((d) => full(env)[d] !== full("production")[d]);

The measurement compares every environment against production and writes the difference dimension by dimension.

// measurement-network/parity.mjs — counts environment parity dimension by dimension
import { DIMENSIONS, PROBED, CONCEALED, full, differences } from "./environments.mjs";

const RING = ["development", "test", "staging"];
const star = (d) => (PROBED.includes(d) ? d + " *" : d);

console.log("parity against production (* = absent from manifest, visible only at runtime)");
console.log("dimension".padEnd(26) + RING.map((o) => o.padEnd(13)).join("") + "concealed defect class");
for (const d of DIMENSIONS) {
  const cell = RING.map((o) => (full(o)[d] === full("production")[d] ? "SAME" : "DIFF").padEnd(13));
  console.log(star(d).padEnd(26) + cell.join("") + CONCEALED[d]);
}

const count = RING.map((o) => differences(o));
const total = count.flat().length;
const atRuntime = count.flat().filter((d) => PROBED.includes(d)).length;
console.log("\ndifference count: " + RING.map((o, i) => `${o} ${count[i].length}/${DIMENSIONS.length}`).join(", "));
console.log(`total ${total} differences; declared in manifest ${total - atRuntime}, visible only at runtime ${atRuntime}`);
console.log("dimensions where staging diverges from production: " + differences("staging").join(", "));
console.log("dimensions with no environment at parity with production: " +
  DIMENSIONS.filter((d) => RING.every((o) => full(o)[d] !== full("production")[d])).join(", "));

parity against production (* = absent from manifest, visible only at runtime)
dimension                 development  test         staging      concealed defect class
resolverVersion           DIFF         DIFF         SAME         resolution drift
rulePackageVersion        SAME         SAME         SAME         amount error
memoryMB                  DIFF         DIFF         SAME         memory exhaustion
concurrentWorkers         DIFF         DIFF         SAME         race condition
meterCount                DIFF         DIFF         DIFF         scale collapse
billingGateway            DIFF         DIFF         SAME         contract drift
workOrderQueue            DIFF         SAME         SAME         contract drift
timezone *                DIFF         DIFF         SAME         day-boundary shift
caseSensitiveFileSystem * DIFF         SAME         SAME         path resolution error
networkLatencyMs *        DIFF         DIFF         DIFF         timeout

difference count: development 9/10, test 7/10, staging 2/10
total 18 differences; declared in manifest 12, visible only at runtime 6
dimensions where staging diverges from production: meterCount, networkLatencyMs
dimensions with no environment at parity with production: meterCount, networkLatencyMs

There are 18 differences between the three environments and production. 12 of these are declared in the manifest: they are visible to a reader, can be discussed, and can be closed. The remaining 6 are written in no document; they can only be seen while the process is running in that environment. This distinction is the lesson’s measurement axis: a declared difference is a matter for a decision, while a difference visible only at runtime is a surprise.

None of these 18 differences was put there by a decision. The environments were set up at different times, upgraded at different moments, and maintained separately: when production’s package version was upgraded, the development machine was left behind; when the test environment was connected to a real work order queue, its billing gateway was left fake. This is what configuration drift means — the difference accumulates wherever no one decided anything. And there is exactly one condition for drift to be measurable: the dimensions must be named and the manifest must be readable.

The gradient is also readable: development diverges from production on nine of the ten dimensions, test on seven, staging on two. The number of differences drops as the links approach production — that is why the chain is ordered. Two dimensions — data volume and network latency — are also at parity with production in no environment. No link goes without carrying the gap between a meter count of one and a half million and one of a hundred twenty.

The Defect Class the Difference Conceals

The number of differences alone carries no decision; what the difference conceals does. The model below binds a defect to each dimension and computes which link in the chain will first see that defect.

DC4. The model has one rule: a defect is visible if the environment carries the same value as production on the dimension it is bound to, and invisible otherwise. DC5. Each dimension carries exactly one defect class; ten dimensions means ten defects. DC6. The chain is walked in order, and a defect stops at the first link where it is seen.

// measurement-network/chain.mjs — counts which link in the chain first sees a defect
import { CONCEALED, full } from "./environments.mjs";

// Model rule: a defect is visible if the environment carries the same value as production on its bound dimension.
const DEFECTS = [
  ["empty read field fails to parse", "resolverVersion"],
  ["new tariff bracket rounds incorrectly", "rulePackageVersion"],
  ["batch job loads every reading into memory", "memoryMB"],
  ["two invoice lines for the same meter", "concurrentWorkers"],
  ["verification query is quadratic in meter count", "meterCount"],
  ["billing gateway rejection code goes unhandled", "billingGateway"],
  ["same work order enters the queue twice", "workOrderQueue"],
  ["nightly job cuts the day boundary by local time", "timezone"],
  ["tariff file is looked up in uppercase", "caseSensitiveFileSystem"],
  ["aggregator response exceeds the timeout", "networkLatencyMs"],
];

const FOUR = ["development", "test", "staging", "production"];
const THREE = ["development", "test", "production"];
const firstToSee = (chain, dim) =>
  chain.find((o) => full(o)[dim] === full("production")[dim]);

console.log("defect".padEnd(49) + "four links".padEnd(13) + "three links");
for (const [name, dim] of DEFECTS) {
  console.log(name.padEnd(49) + firstToSee(FOUR, dim).padEnd(13) + firstToSee(THREE, dim));
}

const tally = (chain) => {
  const d = Object.fromEntries(chain.map((o) => [o, 0]));
  for (const [, dim] of DEFECTS) d[firstToSee(chain, dim)] += 1;
  return chain.map((o) => `${o} ${d[o]}`).join(", ");
};
console.log("\nfour-link chain: " + tally(FOUR));
console.log("three-link chain: " + tally(THREE));

const escaping = (chain) => DEFECTS.filter(([, d]) => firstToSee(chain, d) === "production");
const fourEscaping = escaping(FOUR).map(([, d]) => CONCEALED[d]);
const threeEscaping = escaping(THREE).map(([, d]) => CONCEALED[d]);
const added = threeEscaping.filter((s) => !fourEscaping.includes(s));
console.log(`escaping to production even with staging in place, ${fourEscaping.length} classes: ` + fourEscaping.join(", "));
console.log(`escaping when staging is skipped, ${threeEscaping.length} classes; added ${added.length}:`);
for (let i = 0; i < added.length; i += 3) console.log("  " + added.slice(i, i + 3).join(", "));

console.log("\nchain length scanned (links added in order):");
let previous = 0;
for (let n = 0; n <= 3; n++) {
  const chain = [...FOUR.slice(0, n), "production"];
  const caught = DEFECTS.filter(([, d]) => firstToSee(chain, d) !== "production").length;
  console.log(`${n} links + production`.padEnd(22) + `caught before production ${caught}` +
    `  (last link's contribution ${caught - previous})`);
  previous = caught;
}

defect                                           four links   three links
empty read field fails to parse                  staging      production
new tariff bracket rounds incorrectly            development  development
batch job loads every reading into memory        staging      production
two invoice lines for the same meter             staging      production
verification query is quadratic in meter count   production   production
billing gateway rejection code goes unhandled    staging      production
same work order enters the queue twice           test         test
nightly job cuts the day boundary by local time  staging      production
tariff file is looked up in uppercase            test         test
aggregator response exceeds the timeout          production   production

four-link chain: development 1, test 2, staging 5, production 2
three-link chain: development 1, test 2, production 7
escaping to production even with staging in place, 2 classes: scale collapse, timeout
escaping when staging is skipped, 7 classes; added 5:
  resolution drift, memory exhaustion, race condition
  contract drift, day-boundary shift

chain length scanned (links added in order):
0 links + production  caught before production 0  (last link's contribution 0)
1 links + production  caught before production 1  (last link's contribution 1)
2 links + production  caught before production 3  (last link's contribution 2)
3 links + production  caught before production 8  (last link's contribution 5)

The table makes one rule visible: an environment can only see the defects on dimensions where it is the same as production. The development environment catches only one of ten defects, because it is the same as production on only a single dimension — the rule package version. Test catches two defects; both are bound to dimensions where test is the same as production: a real work order queue and a case-sensitive file system.

Reading it the other way says more: the list of defect classes an environment fails to catch is the list of dimensions where that environment diverges from production. Staging diverged on two dimensions; the two defect classes that escape to production even with staging in place are bound to exactly those two dimensions. This is not an inference from the model — it is a consequence of its definition. But because it turns the definition into a number, it takes the environment debate out of opinion: the question “do we need a staging environment” is replaced by “which dimensions are we buying parity on.”

The last part of the output scans the chain length: as links are added in order, the number of classes caught before production is 0, 1, 3, and 8. The contribution per link is not equal — the first link adds one class, the second two, the third five. The ordering also explains why: each link closes the dimensions the previous one could not see, and the link closest to production closes the most dimensions. Adding a new environment to the front of the chain is investing where the contribution is smallest; the number of dimensions a link closes is read from its manifest, not from its name.

The Cost of Skipping a Link

The fictional network was not running a staging environment. The model’s two runs count the cost of that decision.

In the four-link chain, two defect classes escape to production: scale collapse and timeout. In the three-link chain, that climbs to seven. The staging link catches five defect classes ahead of production: resolution drift, memory exhaustion, race condition, contract drift, and day-boundary shift. As long as the network does not set up staging, these five are classes that first appear while a real invoice is being produced.

What skipping gains can be counted too: the chain drops from four stops to three, which means one less wait per release and one less handoff. This gain is written to lead time; the cost is written to change failure rate and time to restore, because each of the five escaping classes produces a change to roll back and an outage to repair when it is seen in production. A sentence that looks at a single metric carries no decision here: the speed-up is taken from one place and set down in another.

The two escaping classes stay in place even if staging is set up, because no link carries a data volume of one and a quarter million meters or a network latency of thirty-four milliseconds. There is a way to close these two dimensions — a production-scale-shaped data set and injected artificial latency — and each has a cost: the environment’s setup cost and its run time. The measurement makes this trade-off concrete too: there are exactly two dimensions to close, both are named, and which defect class gets headed off once they are closed is written down.

This, finally, is what the six differences between the manifest and the runtime mean. The twelve differences written in the manifest can come up in a meeting; the six that are not written cannot. Timezone diverging between development and production shows up as the nightly batch job cutting the day boundary in the wrong place, and it occurs to no one that the cause is an environment difference until the defect is found. Widening the environment manifest’s scope — declaring all three of the runtime-measured dimensions too — makes all six of these differences discussable.

Summary

• An environment is a link in the chain leading to production; the four links are distinguished by purpose, and each link can only see the defects on dimensions where it is the same as production.
• Environment parity is counted dimension by dimension. The fictional model has ten dimensions; 18 differences were measured between the three environments and production: development 9, test 7, staging 2.
• 12 of these 18 differences are declared in the environment manifest, 6 are visible only while the process is running; an undeclared difference cannot be discussed and so cannot be a matter for a decision.
• Every difference conceals a defect class, and the list of classes an environment fails to catch equals the list of dimensions where that environment diverges from production.
• The staging link catches five defect classes ahead of production; skipping it carries those five to production. Data volume and network latency are at parity with production in no link, so two classes reach production regardless.

Next Step

This lesson’s chain assumed one thing: the object that travels between links is always the same object. But the environment manifests do not confirm this — each link has its own package versions, its own file paths, its own environment variables. If the change is recompiled from source at every stop, then it is not a single object circulating through the chain but four separate objects, and which one is the same as which has not been measured. The next lesson names and measures this object: is output produced twice from the same source actually the same, and if not, which sources introduce the difference, and how many separate binaries does recompiling per environment produce?