Build Environment Isolation

The previous lesson counted that seven of the provenance document’s fourteen fields can never be tested under any condition. Four of those seven described a single subject: runner type, isolation level, network access, and environment-variable names. All four were written in the document, all four were attested, and their values depended only on that environment actually having been set up that way.

This lesson counts that environment item by item. The measure is the difference between the number of items the build environment accesses and the number it actually needs. In the Continuous Integration and Delivery course, pipeline security and secret management were measured on the deployment axis; that measurement is not repeated here. The object here is the pipeline’s own permissions. The count sits on the defensive side: which item is reachable from which step is computed from the inventory, not how the access would be used.

SC15. The fictional pipeline has five steps, and the access items are eighteen items in four types; the set is modeled with node. Which step actually needs which item is also fictional. SC16. On a shared runner, every step reaches every item; the unit of measure is the access pair — a step being able to reach an item is one pair.

What the Build Environment Reaches

// measurement-network/access-inventory.mjs — which items the build environment reaches, how many it needs
import { mkdirSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const ROOT = join(process.env.TMPDIR ?? "/tmp", "measurement-network-sc");
mkdirSync(ROOT, { recursive: true });

// Fictional regional measurement network: the subscriber portal's pipeline is five steps.
const STEPS = ["fetch-dependencies", "build", "test", "package", "publish"];

// Item inventory is the model: type, name, and the steps that ACTUALLY need the item.
const ITEMS = [
  ["secret", "artifact-repository-token", ["publish"]],
  ["secret", "source-repository-key", []],
  ["secret", "signing-private-key", ["publish"]],
  ["secret", "cloud-identity", []],
  ["secret", "database-password", ["test"]],
  ["secret", "notification-token", ["publish"]],
  ["network", "package-mirror", ["fetch-dependencies"]],
  ["network", "artifact-repository", ["publish"]],
  ["network", "signing-service", ["publish"]],
  ["network", "public-internet", []],
  ["network", "internal-service-network", ["test"]],
  ["path", "source-tree", ["build", "test", "package"]],
  ["path", "cache", ["fetch-dependencies", "build"]],
  ["path", "previous-job-artifacts", []],
  ["path", "runner-root", []],
  ["identity", "pipeline-account", ["publish"]],
  ["identity", "container-registry", ["package"]],
  ["identity", "control-plane", []],
];
writeFileSync(join(ROOT, "items.json"), JSON.stringify({ STEPS, ITEMS }));

// On a shared runner every step reaches every item: access pair = step x item.
const TYPES = ["secret", "network", "path", "identity"];
const S = [10, 8, 16, 15, 0];
const row = (s) => console.log(s.map((v, i) => String(v).padEnd(S[i])).join(""));
row(["type", "items", "never needed", "access pairs", "required pairs"]);
let totalPairs = 0, totalRequired = 0, neverNeeded = 0;
for (const t of TYPES) {
  const set = ITEMS.filter(([type]) => type === t);
  const pairs = set.length * STEPS.length;
  const required = set.reduce((n, [, , g]) => n + g.length, 0);
  const unused = set.filter(([, , g]) => g.length === 0).length;
  totalPairs += pairs; totalRequired += required; neverNeeded += unused;
  row([t, set.length, unused, pairs, required]);
}
console.log("");
console.log(`steps ${STEPS.length}, items ${ITEMS.length}; access pairs ${totalPairs}, ` +
  `required ${totalRequired}, unneeded ${totalPairs - totalRequired}`);
console.log(`items never needed by any step ${neverNeeded}: ` +
  ITEMS.filter(([, , g]) => g.length === 0).map(([, name]) => name).join(", "));

type      items   never needed    access pairs   required pairs
secret    6       2               30             4
network   5       1               25             4
path      4       2               20             5
identity  3       1               15             2

steps 5, items 18; access pairs 90, required 15, unneeded 75
items never needed by any step 6: source-repository-key, cloud-identity, public-internet, previous-job-artifacts, runner-root, control-plane

On a shared runner, five steps reach eighteen items, and that comes to 90 access pairs. Of these pairs, 15 are required; the remaining 75 serve no step at all, and none of them is the result of a decision. Nobody said “let the build step see the signing private key”; the steps saw it because they ran on the same runner.

The sharpest number is in the third column: six of eighteen items are never needed by any step. The source repository key and the cloud identity are left over from an earlier version of the pipeline, no step lists public internet access as a target, previous job artifacts and the runner root belong to the shared machine itself, and the control-plane identity is a broadly scoped token that has never been used. Six items sit across the entire pipeline without serving any part of it.

These six items become visible the moment the inventory is taken. This is the measure’s real work: scoping permissions, stated as a principle, carries no decision — only when counted as pairs does it become writable which item is useful in which step. If an item’s list of needing steps is empty, the item is dropped; if the list is not empty, it is already the target of scoping.

What Remains After Isolation

Isolation does two separate things: dropping an unneeded item entirely, and narrowing a remaining item’s access to the step that needs it. The second is not equally possible for every item.

SC17. The level scoping attaches to depends on how the item is provisioned, and it takes one of two values: step level or job level. This is a model. SC18. The durations are given as data, not measured: a new runner per job takes 38 seconds, secret scoping takes 6 seconds, a network allowlist takes 2 seconds.

// measurement-network/isolation.mjs — how many items remain after isolation, and why
import { readFileSync } from "node:fs";
import { join } from "node:path";

const ROOT = join(process.env.TMPDIR ?? "/tmp", "measurement-network-sc");
const { STEPS, ITEMS } = JSON.parse(readFileSync(join(ROOT, "items.json")));

// The level scoping ATTACHES to depends on how the item is provisioned; this is a model.
// "step": the item can be handed only to the step that needs it. "job": the mechanism attaches at job level.
const LEVEL = {
  "artifact-repository-token": "step", "signing-private-key": "step",
  "notification-token": "step", "container-registry": "step", "pipeline-account": "step",
  "database-password": "job",         // put on the whole job as an environment variable
  "package-mirror": "job", "artifact-repository": "job",  // allowlist is at runner level
  "signing-service": "job", "internal-service-network": "job",
  "source-tree": "job", "cache": "job",             // workspace mount is at job level
};

// Isolation does two things: an unneeded item is dropped entirely, a remaining item is scoped.
const remaining = ITEMS.filter(([, , g]) => g.length > 0);
const reached = ([, name, g]) => (LEVEL[name] === "step" ? g.length : STEPS.length);

const TYPES = ["secret", "network", "path", "identity"];
const S = [10, 20, 14, 14, 0];
const row = (s) => console.log(s.map((v, i) => String(v).padEnd(S[i])).join(""));
row(["type", "items before/after", "pairs before", "pairs after", "remaining unneeded"]);
let before = 0, after = 0, required = 0;
for (const t of TYPES) {
  const all = ITEMS.filter(([type]) => type === t);
  const k = remaining.filter(([type]) => type === t);
  const p1 = all.length * STEPS.length;
  const p2 = k.reduce((n, e) => n + reached(e), 0);
  const g = k.reduce((n, [, , gg]) => n + gg.length, 0);
  before += p1; after += p2; required += g;
  row([t, `${all.length}/${k.length}`, p1, p2, p2 - g]);
}
console.log("");
console.log(`access pairs ${before} -> ${after}; required ${required}, ` +
  `remaining unneeded ${after - required}`);
console.log("still bound at job level: " +
  remaining.filter(([, name]) => LEVEL[name] === "job").map(([, name]) => name).join(", "));

// If the network were closed entirely: every step that needs a network item would break.
const networkedSteps = [...new Set(remaining.filter(([t]) => t === "network").flatMap(([, , g]) => g))];
console.log(`if the network is closed entirely, broken steps ${networkedSteps.length}/${STEPS.length}: ` +
  networkedSteps.join(", "));

// Durations are given as DATA, not measured.
const [NEW_RUNNER, SECRET_SCOPING, ALLOWLIST, RUNS] = [38, 6, 2, 30];
const extra = NEW_RUNNER + SECRET_SCOPING + ALLOWLIST;
console.log(`isolation's extra time per run ${extra} s (${NEW_RUNNER}+${SECRET_SCOPING}+` +
  `${ALLOWLIST}), across ${RUNS} runs ${extra * RUNS} s; durations were given as data`);

type      items before/after  pairs before  pairs after   remaining unneeded
secret    6/4                 30            8             4
network   5/4                 25            20            16
path      4/2                 20            10            5
identity  3/2                 15            2             0

access pairs 90 -> 40; required 15, remaining unneeded 25
still bound at job level: database-password, package-mirror, artifact-repository, signing-service, internal-service-network, source-tree, cache
if the network is closed entirely, broken steps 3/5: fetch-dependencies, publish, test
isolation's extra time per run 46 s (38+6+2), across 30 runs 1380 s; durations were given as data

Access pairs drop from 90 to 40, and the required amount is 15. The gain is real: six of eighteen items are dropped, and the remaining twelve have their access scoped. But the remaining unneeded pair count is not zero, it is 25, and this row is the lesson’s real question: why do they remain?

The answer varies by type. In the identity row, remaining-unneeded is zero, because a token can be handed to a specific step: the pipeline account sits only in the publish step, the registry identity only in the package step. In the secret row, remaining-unneeded is four, and all four come from a single item — the database password is given to the test step as an environment variable, and an environment variable is placed on the whole job. One item’s provisioning method opens that item to all five steps.

The network row is the worst: four targets remain, all bound at job level, producing 16 unneeded pairs. The reason is that the network allowlist is enforced at the runner level; the list can say “this job may reach these four targets,” it cannot say “the build step may reach only the package mirror.” Workspace mounting works the same way: the source tree and the cache stay open to steps that do not need them too. The limit of scoping is not in the principle, it is in the item’s provisioning mechanism.

The third output line ties back to the previous lesson. In the provenance document, the network access field said “closed” and could not be tested. Here the number says: if the network is closed entirely, three of five steps break — fetch dependencies, test, and publish. That field is not merely unverifiable; as written, it cannot even be correct. The only form that could be correct is something like a list of “allowed to four targets,” and that list is not evidence either, it is again an attested statement.

The last line writes the price. Isolation adds 46 seconds per run, 1,380 seconds across thirty runs. That time buys the closing of 50 access pairs; nothing is paid for the remaining 25, because those close through mechanism, not through time.

A Change from Outside the Repository

The access pair does not care whose change is running. The same items are open to the same steps whether the change comes from inside the repository or from an outside contribution.

SC19. Eight of thirty runs are triggered by a change from outside the repository; in isolated configurations the publish step is skipped for an outside contribution. A step cannot run if one of the items it needs does not exist at all on that pipeline.

// measurement-network/external-contribution.mjs — how many secrets are reachable when a change from outside the repository runs
import { readFileSync } from "node:fs";
import { join } from "node:path";

const ROOT = join(process.env.TMPDIR ?? "/tmp", "measurement-network-sc");
const { STEPS, ITEMS } = JSON.parse(readFileSync(join(ROOT, "items.json")));
const LEVEL = {
  "artifact-repository-token": "step", "signing-private-key": "step",
  "notification-token": "step", "container-registry": "step", "pipeline-account": "step",
  "database-password": "job", "package-mirror": "job", "artifact-repository": "job",
  "signing-service": "job", "internal-service-network": "job", "source-tree": "job", "cache": "job",
};

// Fictional: eight of thirty runs are triggered by a change from outside the repository.
const RUNS = 30, EXTERNAL = 8;
const remaining = ITEMS.filter(([, , g]) => g.length > 0);
const SECRETS = ITEMS.filter(([t]) => t === "secret").map(([, name]) => name);

// Three configurations: internal change set, external contribution set, scoped or not, skipped step.
const CONFIGS = [
  ["shared runner", ITEMS, ITEMS, false, []],
  ["isolated pipeline", remaining, remaining, true, ["publish"]],
  ["secret-free pipeline for external contributions", remaining,
    remaining.filter(([t, name]) => t !== "secret" && name !== "internal-service-network"), true, ["publish"]],
];

// An item is reachable if the intersection of its access level and the running steps is non-empty.
const reachable = (set, scoped, running, secret) => {
  const item = set.find(([, name]) => name === secret);
  if (item === undefined) return false;
  return (scoped && LEVEL[secret] === "step" ? item[2] : STEPS).some((s) => running.includes(s));
};

const S = [49, 14, 14, 17, 0];
const row = (s) => console.log(s.map((v, i) => String(v).padEnd(S[i])).join(""));
row(["configuration", "int: secrets", "ext: secrets", "ext: not running", "secret-runs across eight"]);
const results = [];
for (const [name, internalSet, externalSet, scoped, skipped] of CONFIGS) {
  const externalRunning = STEPS.filter((s) => !skipped.includes(s));
  const internal = SECRETS.filter((s) => reachable(internalSet, scoped, STEPS, s)).length;
  const external = SECRETS.filter((s) => reachable(externalSet, scoped, externalRunning, s)).length;
  // A step cannot run if one of the items it needs does not exist at all on that pipeline.
  const broken = externalRunning.filter((s) =>
    ITEMS.some(([, itemName, g]) => g.includes(s) && !externalSet.some(([, n]) => n === itemName)));
  results.push([name, broken]);
  row([name, internal, external, [...skipped, ...broken].join(", ") || "-", external * EXTERNAL]);
}
console.log("");
console.log(`runs ${RUNS}, from outside the repository ${EXTERNAL}; secrets in the pipeline ${SECRETS.length}, ` +
  `remaining after isolation ${remaining.filter(([t]) => t === "secret").length}`);
const [lastName, lastBroken] = results.at(-1);
console.log(`${lastName}: step that cannot run ${lastBroken.join(", ")}; ${EXTERNAL} external contributions ` +
  `enter the review queue without seeing integration testing`);

configuration                                    int: secrets  ext: secrets  ext: not running secret-runs across eight
shared runner                                    6             6             -                48
isolated pipeline                                4             1             publish          8
secret-free pipeline for external contributions  4             0             publish, test    0

runs 30, from outside the repository 8; secrets in the pipeline 6, remaining after isolation 4
secret-free pipeline for external contributions: step that cannot run test; 8 external contributions enter the review queue without seeing integration testing

The first row states the number in its plainest form: on a shared runner, a change from outside the repository finds all six of the six secrets reachable, and across eight contributions that comes to 48 secret-run pairs. What the change does does not matter; the environment does not separate it from its own change, because no item exists to make that separation.

In the second row, reachable secrets drop to 1, and the drop’s source splits in two. Two secrets are already dropped, three are bound to the publish step, and that step does not run for an outside contribution. The one secret left is again the database password: because it is bound at job level, it comes along with the test step. Isolation’s gain in this configuration is dropping from 48 pairs to 8, and all 8 remaining come from a single item’s provisioning method.

The third row removes that item too, and its price shows immediately. On the secret-free pipeline, reachable secrets are zero, but the test step cannot run — the database password and the internal service network are not there. Eight external contributions enter the review queue without seeing integration testing. This is not the place where isolation fails; it is the place where its price is paid by a different item. Zero access and tested contributions are not obtained at the same time, and the choice between them is a decision that depends on how thoroughly the eight contributions get reviewed, not a question of principle.

Summary

• The five-step pipeline reaches eighteen items: only 15 of 90 access pairs are required. 6 of the eighteen items are never needed by any step and sit across the entire pipeline.
• Isolation drops the access pair count from 90 to 40; 25 pairs remain unneeded. Remaining unneeded is 0 for the identity type and 16 for the network type.
• The reason items remain is not in the principle but in the mechanism: the network allowlist is enforced at runner level, the workspace mount at job level, and a secret is placed on the whole job as an environment variable.
• If the network is closed entirely, 3 of five steps break; the provenance document’s “network access: closed” statement is not merely unverifiable, it cannot even be correct as written. Isolation adds 46 seconds per run, 1,380 seconds across thirty runs.
• A change from outside the repository reaches 6 secrets on a shared runner (48 pairs across eight contributions), 1 secret on an isolated pipeline (8 pairs), 0 secrets on a secret-free pipeline — but on the secret-free pipeline the test step does not run, and eight contributions enter review untested.

Next Step

The build environment’s permissions are counted: how many items are accessed, how many are needed, what remains after isolation and why it remains. All of this measurement was on the pipeline’s own items. But that same pipeline’s first step is fetching dependencies, and that step is the one that brings the inventory’s seventy-three records into the pipeline. Scoping permissions does not touch any of those records: once a component enters the pipeline, what determines the outcome is not how narrow the environment is, but what that component is.

The next lesson turns this question into a rule set. The inventory’s records are run through a policy, and the measure is again two numbers: how many components does the policy flag, and how many of the flags are wrong? The abandoned-package rule produces the most flags of any rule; how many of its flags point to a real problem, and how many merely show that a date field has aged?