--- title: 'Environment Promotion' source: 'https://academia.sh/en/courses/ci-cd/environment-promotion' course: 'Continuous Integration and Delivery' language: en updated: '2026-09-13T05:08:18+00:00' license: 'CC BY-SA 4.0' objectives: - 'Measures the scheme where the same artifact promotes against the scheme where every environment rebuilds: artifact identity, drifted dependency versions, environment-specific defects' - "Counts the promotion gates' marginal value and, with the numbers, makes the case for removing a gate whose marginal value comes out near zero" - 'Scans what drift is a function of: how identity gets lost as the interval between the staging build and the production build grows' --- # Environment Promotion The scheme where the same build artifact promotes between environments is run against the scheme where every environment rebuilds, on the same forty-candidate sequence: environment-specific defects, what the promotion gate catches, whether the artifact's identity is preserved across environments, and how long an artifact waits at the promotion gate are counted. The change entered the shared trunk and passed the integration gate. This does not mean the change is in production. At least one build artifact and a few environments stand between: development, staging, production. This lesson's question is **how** that artifact advances between environments. There are two schemes. In the first, the artifact is produced once and the same object is promoted to every environment. In the second, every environment does its own build; the source is the same, the artifact does not have to be. Whether environments are equivalent to one another is a separate measure's subject and is not repeated here. This lesson measures the difference that remains even when equivalence is achieved: **the difference in the running object itself**. In this lesson the promotion gate is an approval decision — it either lets an artifact move to the next environment or it does not — and the gate's value depends on whether the object it approves is the same object it tested. - **RS7.** Forty release candidates are generated for the fictional regional measurement network's verification service; there are three environments, and the candidates spread across a forty-day horizon. Both the network and the figures are fictional. - **RS8.** Environments are taken as equivalent in configuration and data. Environment-equivalence measurement is not repeated; the only source of difference here is the artifact itself. - **RS9.** Dependency resolution picks the newest version at the moment of resolution. The version schedule for the eight dependencies is fictional; lockfiles, mirroring, and manual version pinning are outside the model. - **RS10.** The defect classes are three: a **source defect** (present in every artifact), a **dependency-version defect** (present only in an artifact that includes that version), and a **production-scale-dependent defect** (invisible at every promotion gate). - **RS11.** The promotion gate runs in a window that opens every four hours; production promotion also waits zero to eight hours for approval. A gate run takes eighteen minutes, a build twenty-two. - **RS12.** Artifact identity is the first twelve hexadecimal digits of the digest of the input record. Digest collision is outside the model. ## Measurement Setup What is measured is not a real pipeline but a model set up with `node`. The model produces a dependency version schedule, resolves the candidates against that schedule, and actually computes artifact identity with `node:crypto`. The random generator is written by hand, and its seed is visible. ```js // promotion.mjs — model of the scheme where the same artifact promotes vs. every environment rebuilding import { createHash } from 'node:crypto'; export const SEED = 20260902; export const CANDIDATES = 40, DEPENDENCIES = 8, HORIZON_HOURS = 960, INTERVAL_HOURS = 24; export const BUILD_MIN = 22, GATE_MIN = 18, WINDOW_HOURS = 4; export const SOURCE = 0.25, SCALE = 0.10, VERSION_DEFECT = 0.12, FLAKY = 0.05; export const DEV_CATCH = 0.45, STAGING_CATCH = 0.75; export function generator(seed) { // linear congruential generator, seed visible let x = seed >>> 0; return () => { x = (Math.imul(x, 1664525) + 1013904223) >>> 0; return x / 4294967296; }; } export const digest = (candidate, resolved) => // the build artifact's identity: a digest of its inputs createHash('sha256').update(JSON.stringify({ candidate, resolved: resolved.map((v) => v.name) })).digest('hex').slice(0, 12); // Every dependency has a version schedule; resolution picks the newest version at that moment. export function schedule(rnd) { return Array.from({ length: DEPENDENCIES }, (_, i) => { const versions = [{ hour: 0, name: `d${i}v1`, defect: false }]; for (let hour = 0; ;) { hour += 40 + Math.floor(rnd() * 200); if (hour >= HORIZON_HOURS) break; versions.push({ hour, name: `d${i}v${versions.length + 1}`, defect: rnd() < VERSION_DEFECT }); } return versions; }); } export const resolve = (sched, hour) => sched.map((v) => v.filter((x) => x.hour <= hour).pop()); export function candidates(rnd) { return Array.from({ length: CANDIDATES }, (_, i) => ({ no: i + 1, hour: Math.floor((i + 0.5) * (HORIZON_HOURS / CANDIDATES) + rnd() * 8), source: rnd() < SOURCE, scale: rnd() < SCALE, devRoll: rnd(), stagingRoll: rnd(), depRoll: rnd(), approval: Math.floor(rnd() * 9), })); } export const wait = (hour) => (WINDOW_HOURS - (hour % WINDOW_HOURS)) % WINDOW_HOURS; // rebuild=false: one build, the same artifact promotes. rebuild=true: every environment builds its own. export function run(rebuild, interval = INTERVAL_HOURS) { const rnd = generator(SEED), sched = schedule(rnd), cand = candidates(rnd), flakyRnd = generator(SEED + 3); const s = { digestSame: 0, sourceEscaped: 0, scaleEscaped: 0, devMarginal: 0, stagingMarginal: 0, uniqueEscaped: 0, testedEscaped: 0, caught: 0, wronglyHeld: 0, versionDrift: 0, min: 0, wait: 0, wait1: 0, peak: 0 }; for (const c of cand) { const staging = rebuild ? resolve(sched, c.hour + 6) : resolve(sched, c.hour); const prod = rebuild ? resolve(sched, c.hour + 6 + interval) : resolve(sched, c.hour); if (digest(c.no, staging) === digest(c.no, prod)) s.digestSame += 1; if (c.source) { // a source defect is present in every build const g = c.devRoll < DEV_CATCH, h = c.stagingRoll < STAGING_CATCH; if (g || h) s.caught += 1; else s.sourceEscaped += 1; if (g && !h) s.devMarginal += 1; // caught only at the dev gate if (h && !g) s.stagingMarginal += 1; // caught only at the staging gate } if (c.scale) s.scaleEscaped += 1; // a production-scale defect: invisible at every gate const tested = new Set(staging.map((v) => v.name)); s.versionDrift += prod.filter((v) => !tested.has(v.name)).length; for (const v of prod.filter((x) => x.defect)) { if (!tested.has(v.name)) s.uniqueEscaped += 1; // present in production, absent from the tested artifact else if (c.depRoll < STAGING_CATCH) s.caught += 1; else s.testedEscaped += 1; } for (let k = 0; k < 2; k += 1) if (flakyRnd() < FLAKY) s.wronglyHeld += 1; s.min += (rebuild ? 3 : 1) * BUILD_MIN + 2 * GATE_MIN; const ready = c.hour + (rebuild ? BUILD_MIN / 60 : 0), w1 = wait(ready); const w2 = wait(ready + w1 + GATE_MIN / 60 + c.approval + (rebuild ? BUILD_MIN / 60 : 0)); s.wait += w1 + w2; s.wait1 += w1; s.peak = Math.max(s.peak, w1 + w2); } return { ...s, avgWait: s.wait / CANDIDATES }; } ``` The model has a single switch: `rebuild`. When it is off, the artifact resolves once, the moment the candidate is ready, and that resolution holds for all three environments. When it is on, staging does its own build at hour six, production twenty-four hours later. The source is the same across all three builds; the only thing that changes is **when** it gets resolved. ## Two Schemes, One Candidate Sequence ```js // run1.mjs — the same candidate sequence is run through two promotion schemes import { run, digest, schedule, candidates, generator, resolve, SEED, CANDIDATES, INTERVAL_HOURS } from './promotion.mjs'; console.log(`seed ${SEED} | ${CANDIDATES} candidates, 8 dependencies, three environments, production build runs ${INTERVAL_HOURS} h after staging`); console.log('scheme digest version env-uniq tested source scale wrongly pipeline avg peak'); console.log(' match drift escape escape escape escape held min wait/h (h)'); for (const [label, rebuild] of [['same artifact', false], ['rebuild each env', true]]) { const s = run(rebuild); console.log(`${label.padEnd(16)} ${`${s.digestSame}/${CANDIDATES}`.padEnd(7)} ${String(s.versionDrift).padEnd(8)} ` + `${String(s.uniqueEscaped).padEnd(9)} ${String(s.testedEscaped).padEnd(8)} ${String(s.sourceEscaped).padEnd(7)} ` + `${String(s.scaleEscaped).padEnd(7)} ${String(s.wronglyHeld).padEnd(8)} ${String(s.min).padEnd(9)} ` + `${s.avgWait.toFixed(2).padEnd(8)} ${s.peak.toFixed(2)}`); } const t = run(false); console.log(`\nmarginal value of the promotion gates (the same source-defect set under both schemes):`); console.log(` caught only at the dev gate: ${t.devMarginal}`); console.log(` caught only at the staging gate: ${t.stagingMarginal}`); console.log(` the dev gate's bill: ${CANDIDATES * 18} min of gate time + ${t.wait1.toFixed(1)} h of window waiting`); const rnd = generator(SEED), sched = schedule(rnd), cand = candidates(rnd); const c = cand.find((x) => digest(x.no, resolve(sched, x.hour + 6)) !== digest(x.no, resolve(sched, x.hour + 6 + INTERVAL_HOURS))); const staging = resolve(sched, c.hour + 6), prod = resolve(sched, c.hour + 6 + INTERVAL_HOURS); console.log(`\ncandidate ${c.no} (rebuild scheme):`); console.log(` artifact tested in staging: ${digest(c.no, staging)}`); console.log(` artifact that reached production: ${digest(c.no, prod)}`); console.log(` drifted dependency: ${staging.map((v, i) => v.name === prod[i].name ? null : `${v.name} -> ${prod[i].name}`).filter(Boolean).join(', ')}`); ``` ``` seed 20260902 | 40 candidates, 8 dependencies, three environments, production build runs 24 h after staging scheme digest version env-uniq tested source scale wrongly pipeline avg peak match drift escape escape escape escape held min wait/h (h) same artifact 40/40 0 0 5 3 4 5 2320 3.93 6.70 rebuild each env 9/40 56 2 4 3 4 5 4080 4.19 6.97 marginal value of the promotion gates (the same source-defect set under both schemes): caught only at the dev gate: 1 caught only at the staging gate: 7 the dev gate's bill: 720 min of gate time + 59.0 h of window waiting candidate 2 (rebuild scheme): artifact tested in staging: 7f8036842703 artifact that reached production: 8c914f3974ca drifted dependency: d6v1 -> d6v2 ``` The first column is this lesson's core. In the same-artifact scheme, in all forty of the forty candidates, the object in production is the object tested in staging. In the rebuild scheme this number drops to nine: **in thirty-one candidates, production ran an artifact no gate had ever seen**. Drifted dependency versions number fifty-six — across the forty candidates, fifty-six times a dependency in the artifact that reached production differed from the one in the artifact tested in staging. The damage from this drift shows up in the third column: two environment-unique defects. The number looks small, but its nature is different. The four tested-and-escaped defects are defects the gate **looked at and could not see**; they fall as the gate's catch rate rises. The two environment-unique defects, by contrast, are defects the gate **never looked at** — that version never ran in any environment. Raising the gate's catch rate to one hundred percent does not zero out either of these two. The source-defect and scale-defect columns are the same under both schemes: three and four. This is the expected result, and it is the measurement's consistency check. The promotion scheme neither reduces nor multiplies the source's own defects; it only forges or breaks the link between the tested object and the running object. The block below shows a single candidate. Candidate two's staging-tested artifact digest is `7f8036842703`, the one that reached production is `8c914f3974ca`. The source is the same, the candidate is the same; the only difference is that the sixth dependency resolved to `d6v2` instead of `d6v1`. No one made a decision — twenty-four hours passed, nothing more. ## What Drift Is a Function Of ```js // run2.mjs — the interval between the staging build and the production build is scanned import { run, CANDIDATES } from './promotion.mjs'; console.log('interval digest version env-uniq tested pipeline avg'); console.log('(h) match drift escape escape min wait/h'); for (const a of [2, 6, 12, 24, 48, 96]) { const s = run(true, a); console.log(`${String(a).padEnd(9)} ${`${s.digestSame}/${CANDIDATES}`.padEnd(7)} ${String(s.versionDrift).padEnd(8)} ` + `${String(s.uniqueEscaped).padEnd(9)} ${String(s.testedEscaped).padEnd(8)} ${String(s.min).padEnd(9)} ${s.avgWait.toFixed(2)}`); } console.log('\nin the same-artifact scheme, every one of these rows reads: 40/40 digest match, 0 version drift, 0 env-unique escape'); ``` ``` interval digest version env-uniq tested pipeline avg (h) match drift escape escape min wait/h 2 36/40 4 0 5 4080 4.19 6 24/40 18 0 4 4080 4.19 12 17/40 31 1 4 4080 4.19 24 9/40 56 2 4 4080 4.19 48 0/40 104 4 4 4080 4.19 96 0/40 192 8 1 4080 4.19 in the same-artifact scheme, every one of these rows reads: 40/40 digest match, 0 version drift, 0 env-unique escape ``` The scan says what drift is a function of: **elapsed time**. Even at a two-hour interval, four of the forty candidates lose their identity. At forty-eight hours, no candidate keeps its identity. No one changes a version, no one touches the configuration; resolution happens twice, and the world moves on in between. The last row shows a quiet shift. At the ninety-six-hour interval, tested-and-escaped defects drop from five to one — not because the gate is working better, but because those defects have moved into the **untested** column instead. A drop in what a gate misses is not, by itself, good news; the set the gate is looking at may have shrunk instead. The pipeline-minutes and wait columns stay constant across the scan: 4,080 minutes, and four hours eleven minutes per candidate. Shortening the interval buys neither machine time nor wait time. In the rebuild scheme, the only way to reduce drift is to bring the two builds closer together, and once that closing reaches zero, what is left is a single build. So the scan shows there is no middle ground between the two schemes: however small the interval gets, the probability of drift never reaches zero — the only point where it does is the point where the second build never happens at all. ## The Gate's Three Numbers, and a Removable Gate **What it catches:** across the forty-candidate sequence, the two promotion gates together caught ten of thirteen source defects and most of the dependency defects in the tested artifact. **What only it catches:** the staging gate seven, the dev gate one. **What it misses:** three source defects, four scale defects, plus two environment-unique defects in the rebuild scheme. **What it wrongly holds:** five clean candidates. **The time it adds:** an average of three hours fifty-six minutes of wait per candidate, a peak of six hours forty-two minutes; pipeline time of 2,320 minutes, 4,080 in the rebuild scheme. The dev gate's arithmetic writes itself: across forty candidates, it caught only one defect on its own, and in exchange asked for 720 minutes of gate time and fifty-nine hours of window waiting. The staging gate delivers seven defects for the same span of time. With these numbers, removing the dev gate is a defensible decision: in exchange for one defect being caught later, across forty candidates, every candidate advances one window earlier. What decides this is not an opinion about the gate, but its marginal value. The same arithmetic gives a different result for the rebuild scheme. There, the staging gate's seven-defect marginal value holds **for the object it tested**; in thirty-one candidates, the object that reaches production is a different one. The gate works, and its numbers look good — but its evidence does not travel with the object. What needs removing is not the gate, but the second build that invalidates the gate's evidence. ## Summary - In the same-artifact scheme, in all forty of the forty candidates production ran the object tested in staging; in the rebuild scheme this number was nine, and drifted dependency versions numbered fifty-six. - Drift produced two environment-unique defects: a defect class the gate never looks at, and one that will not shrink just because the catch rate rises. - Source defects three, scale defects four: the same under both schemes. The promotion scheme does not change the source's own defects; it only forges or breaks the link between the tested object and the running object. - Drift is a function of time: at a two-hour interval, four of the forty candidates lost their identity; at forty-eight hours, all of them did. At ninety-six hours, the drop in escaped defects is not the gate improving — it is the set it looks at shrinking. - The dev promotion gate caught only one defect on its own across forty candidates, and asked for 720 minutes plus fifty-nine hours of window waiting; with these numbers, removing it is defensible. - The promotion gate's waiting bill runs an average of three hours fifty-six minutes per candidate, with a peak of six hours forty-two minutes; the rebuild scheme adds two extra builds per candidate on top of that. ## Next Step In this measurement, promotion to production was a single decision: the artifact either passed or it did not, and when it passed, it opened to everyone at once. But promotion does not have to happen in a single step. A new version can first open to a small percentage of requests, be watched for a while, and then have that percentage raised. This turns the gate from a single decision into a **sequence of decisions**, and it opens a new question: what metric drives the decision to raise the percentage, over what window is that metric collected, and what is its threshold. The next lesson scans these three settings, and also counts how often a clean version gets stopped while the metric is noisy.