Dependency Policy

The previous lesson counted the build environment’s permissions item by item and left the pipeline’s first step out of scope. The dependency-fetch step is the step that brings the inventory’s seventy-three records into the pipeline, and scoping permissions does not touch any of those records. However narrow the environment is, what comes in as a component is a separate question.

This lesson turns that question into a rule set. The measure is two numbers: how many components the policy flags, and how many of the flags are wrong. Dependency risk and version pinning were already measured in two earlier courses; that exercise is not repeated here. The object here is the rule set itself — which rule runs against which field, and what its flag is actually worth.

SC20. The inventory is reproduced from the same seed as the model in the first lesson: 73 records, 48 unique components. The policy runs on the component; records are grouped under a unique name. SC21. The policy joins two sources: five fields from the inventory and two fields from the external package registry — months since last release and maintainer count. Neither is in the inventory; the external registry’s seed 660418 is visible.

Running the Policy as a Rule Set

// measurement-network/policy.mjs — the dependency policy run as a rule set: how many components it flags
import { mkdirSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const ROOT = join(process.env.TMPDIR ?? "/tmp", "measurement-network-sc");
mkdirSync(ROOT, { recursive: true });
const rng = (seed) => () =>
  (seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648;
const r = rng(20260731);                                     // inventory seed is visible
const pick = (d) => d[Math.floor(r() * d.length)];

// The inventory is reproduced from the same seed as the first lesson's model: 73 records, 48 unique components.
const REPOS = [["subscriber-portal", 11], ["meter-intake", 9], ["shared-library", 6]];
const POOL = Array.from({ length: 70 }, (_, i) => `component-${String(i + 1).padStart(2, "0")}`);
const VERSIONS = ["1.2", "1.9", "2.0", "2.4", "3.0", "3.6", "4.1"];
const LICENSES = ["permissive", "permissive", "permissive", "weak-copyleft", "copyleft"];
const records = [];
for (const [repo, directCount] of REPOS) {
  const taken = new Set();
  const next = () => { let a; do { a = pick(POOL); } while (taken.has(a)); taken.add(a); return a; };
  const add = (name, introducer) => records.push({ name, version: pick(VERSIONS), repo,
    license: r() < 0.18 ? "" : pick(LICENSES),
    introducedBy: introducer === null ? repo : (r() < 0.12 ? "" : introducer) });
  for (let i = 0; i < directCount; i++) {
    const rootName = next();
    add(rootName, null);
    for (let k = 0, n = Math.floor(r() * 4); k < n; k++) {
      const child = next();
      add(child, rootName);
      if (r() < 0.35) add(next(), child);
    }
  }
}

// Component view: seventy-three records are grouped under a unique name.
const REPO_NAMES = new Set(REPOS.map(([name]) => name));
const COMPONENTS = new Map();
for (const rec of records) {
  const c = COMPONENTS.get(rec.name) ?? { name: rec.name, license: "", direct: false, introducerEmpty: false };
  if (c.license === "") c.license = rec.license;
  if (REPO_NAMES.has(rec.introducedBy)) c.direct = true;
  if (rec.introducedBy === "") c.introducerEmpty = true;
  COMPONENTS.set(rec.name, c);
}

// The external package registry is a separate source: months since last release and maintainer count are not in the inventory.
const d = rng(660418);                                       // external registry seed is visible
for (const c of COMPONENTS.values()) {
  c.inExternalRegistry = d() < 0.9;
  c.lastReleaseMonths = 1 + Math.floor(d() * 47);
  c.maintainers = 1 + Math.floor(d() * d() * 5);
  c.hasReplacement = d() < 0.55;                              // hidden: the rules do not see this
  c.maintenanceTransferred = d() < 0.07;                      // hidden: no field shows this
  c.hiddenLicense = c.license === "" ? (d() < 0.45 ? "copyleft" : "permissive") : c.license;
}
const C = [...COMPONENTS.values()];

// Seven rules. Each rule declares the field it needs; without the field, it cannot run.
const FIELDS = new Set(["name", "version", "repo", "license", "introducedBy", "lastReleaseMonths", "maintainers"]);
const RULES = [
  ["restricted license", ["license"], (c) => c.license === "copyleft"],
  ["license unknown", ["license"], (c) => c.license === ""],
  ["abandoned package", ["lastReleaseMonths", "maintainers"],
    (c) => c.inExternalRegistry && c.lastReleaseMonths >= 24 && c.maintainers <= 1],
  ["single maintainer", ["maintainers"], (c) => c.inExternalRegistry && c.maintainers === 1],
  ["no external registry match", ["lastReleaseMonths"], (c) => c.inExternalRegistry === false],
  ["introducer edge empty", ["introducedBy"], (c) => c.introducerEmpty],
  ["restricted license called at runtime", ["calledAtRuntime", "license"], null],
];

const S = [38, 31, 11, 0];
const row = (s) => console.log(s.map((v, i) => String(v).padEnd(S[i])).join(""));
row(["rule", "required field", "could run", "flagged components"]);
const flags = {};
for (const [name, needed, test] of RULES) {
  const runs = needed.every((f) => FIELDS.has(f));
  if (runs) flags[name] = C.filter(test).map((c) => c.name);
  row([name, needed.join("+"), runs ? "yes" : "NO",
    runs ? flags[name].length : "no field"]);
}
const union = new Set(Object.values(flags).flat());
writeFileSync(join(ROOT, "policy.json"), JSON.stringify({ C, flags }));
console.log("");
console.log(`records ${records.length}, unique components ${C.length}; with an external registry match ` +
  `${C.filter((c) => c.inExternalRegistry).length}; seeds 20260731 and 660418`);
console.log(`${Object.keys(flags).length} of seven rules run, one does not run because the field is ` +
  `missing from the inventory; flags ${Object.values(flags).flat().length}, ` +
  `flagged unique components ${union.size}/${C.length}`);

rule                                  required field                 could run  flagged components
restricted license                    license                        yes        5
license unknown                       license                        yes        6
abandoned package                     lastReleaseMonths+maintainers  yes        9
single maintainer                     maintainers                    yes        18
no external registry match            lastReleaseMonths              yes        2
introducer edge empty                 introducedBy                   yes        6
restricted license called at runtime  calledAtRuntime+license        NO         no field

records 73, unique components 48; with an external registry match 46; seeds 20260731 and 660418
6 of seven rules run, one does not run because the field is missing from the inventory; flags 46, flagged unique components 30/48

Six of seven rules run, one cannot. The one that cannot run is not missing because of a configuration gap: the “restricted license called at runtime” rule needs the calledAtRuntime field, and that field is in neither the inventory nor the external registry. The first lesson counted this same question as one of four unanswerable questions; here the same gap shows up as a rule that can never be written at all. A policy is only as writable as the field set it rests on.

The six rules that run produce 46 flags, and those flags fall on 30 of 48 components: two-thirds of the inventory is flagged. The distribution is not even — the single-maintainer rule alone produces 18 flags. The join is not clean either: two components have no external registry match at all, and for those two the abandoned-package and single-maintainer rules can never be evaluated.

How Many of the Flags Are Wrong

Thirty flagged components is not a number on its own. The second half of the measure is how many of these flags actually correspond to a genuine policy violation.

SC22. The genuinely deviant set is fictional and comes from four sources: a copyleft-licensed direct dependency, an abandoned package with no replacement version, a component with an unknown license that is actually restricted, and a package whose maintenance has been transferred. The rules do not see this set. The transfer happened to three components, and no source has a field showing the transfer; those three are chosen from the set no rule flags.

// measurement-network/false-flags.mjs — how many of the policy's flags are real, how many are wrong
import { readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const ROOT = join(process.env.TMPDIR ?? "/tmp", "measurement-network-sc");
const { C, flags } = JSON.parse(readFileSync(join(ROOT, "policy.json")));
const union = new Set(Object.values(flags).flat());

// Fictional ground truth. The rules do not see this set; hidden fields are read only here.
const ACTUAL = new Set();
const REASON = { license: 0, unmaintained: 0, hiddenLicense: 0, transfer: 0 };
for (const c of C) {
  // (a) Copyleft-licensed and a direct dependency: the license obligation genuinely arises.
  if (c.license === "copyleft" && c.direct) { ACTUAL.add(c.name); REASON.license++; }
  // (b) Abandoned and no replacement version exists: the lack of maintenance is a real risk.
  else if (c.inExternalRegistry && c.lastReleaseMonths >= 24 && c.maintainers <= 1 && !c.hasReplacement) {
    ACTUAL.add(c.name); REASON.unmaintained++;
  }
  // (c) A component with an unknown license that is actually restricted: the state behind the blank field.
  else if (c.license === "" && c.hiddenLicense === "copyleft") {
    ACTUAL.add(c.name); REASON.hiddenLicense++;
  }
}
// (d) Fictional: three components from the set no rule flags have had a maintenance transfer.
// No field, in the inventory or the external registry, shows the transfer: it is outside every rule's field of view.
const cleanLooking = C.filter((c) => !union.has(c.name)).map((c) => c.name);
for (const name of [0, 6, 12].map((i) => cleanLooking[i])) { ACTUAL.add(name); REASON.transfer++; }

writeFileSync(join(ROOT, "actual.json"), JSON.stringify([...ACTUAL]));

const S = [30, 9, 9, 15, 0];
const row = (s) => console.log(s.map((v, i) => String(v).padEnd(S[i])).join(""));
row(["rule", "flags", "actual", "false mark", "false rate"]);
for (const [name, list] of Object.entries(flags)) {
  const actual = list.filter((a) => ACTUAL.has(a)).length;
  row([name, list.length, actual, list.length - actual,
    `${Math.round((100 * (list.length - actual)) / list.length)}%`]);
}

const seen = [...ACTUAL].filter((a) => union.has(a));
const unseen = [...ACTUAL].filter((a) => !union.has(a));
console.log("");
console.log(`genuinely deviant ${ACTUAL.size}/${C.length} components: license ${REASON.license}, ` +
  `unmaintained ${REASON.unmaintained}, unknown license ${REASON.hiddenLicense}, transfer ${REASON.transfer}`);
console.log(`flags ${Object.values(flags).flat().length}, flagged unique components ` +
  `${union.size}: ${seen.length} real, ${union.size - seen.length} false marks`);
console.log(`genuinely deviant seen by no rule ${unseen.length}/${ACTUAL.size}; ` +
  `drawn from the ${cleanLooking.length} clean-looking components`);

rule                          flags    actual   false mark     false rate
restricted license            5        3        2              40%
license unknown               6        1        5              83%
abandoned package             9        6        3              33%
single maintainer             18       7        11             61%
no external registry match    2        0        2              100%
introducer edge empty         6        1        5              83%

genuinely deviant 13/48 components: license 3, unmaintained 6, unknown license 1, transfer 3
flags 46, flagged unique components 30: 10 real, 20 false marks
genuinely deviant seen by no rule 3/13; drawn from the 18 clean-looking components

Of the thirty flagged components, 10 are real, 20 are false marks. The policy flags two-thirds of the inventory, but the genuinely deviant amount is one-eighth of it.

The per-rule distribution gives a ranking. The abandoned-package rule is the least noisy: 6 of 9 flags are real, because it looks for two fields together — both release age and maintainer count. The wrong three are packages whose release is old but that have a replacement version; an aged date field alone does not mean lack of maintenance. The single-maintainer rule is the loudest, with 18 flags, and 11 are wrong: maintainer count is not a deviation, it is a distribution property.

The last line writes the measure’s missing half. Of 13 genuinely deviant components, 3 are seen by no rule at all, and all three are among the 18 components the rules consider “clean.” The reason is not a threshold setting: no source has a field showing a maintenance transfer, and so a rule to show those three cannot be written. A policy measurement with an unwritten false negative presents the thirty components it flags as if that were a guarantee.

When the Rule Becomes a Gate

A build that stops with a flag list is not the same thing. Stopped work can only be measured once a rule turns into a gate.

SC23. Thirty builds are fictional; each build touches one to four components, the seed 9143205 is visible. If a build stops only on false marks, the team writes those components to an exception list and the component never stops the build again. SC24. Durations are data: evaluation takes 9 seconds, intervention per stopped build takes 4 minutes.

// measurement-network/gate.mjs — when policy becomes a gate: how many builds stop, how many stop in the wrong place
import { readFileSync } from "node:fs";
import { join } from "node:path";

const ROOT = join(process.env.TMPDIR ?? "/tmp", "measurement-network-sc");
const { C, flags } = JSON.parse(readFileSync(join(ROOT, "policy.json")));
const ACTUAL = new Set(JSON.parse(readFileSync(join(ROOT, "actual.json"))));
const flaggedBy = new Map(C.map((c) => [c.name, Object.entries(flags)
  .filter(([, l]) => l.includes(c.name)).map(([k]) => k)]));
const direct = new Map(C.map((c) => [c.name, c.direct]));

const rng = (seed) => () =>
  (seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648;
// Fictional: thirty builds, each touching one to four components. Seed is visible.
const r = rng(9143205);
const BUILDS = Array.from({ length: 30 }, () =>
  Array.from({ length: 1 + Math.floor(r() * 4) }, () => C[Math.floor(r() * C.length)].name));

// Durations are given as DATA: evaluation 9 s, intervention per stopped build 4 min.
const [EVAL_S, INTERVENTION_MIN] = [9, 4];
const MODES = [
  ["warn only", () => false],
  ["stop on direct dependency", (a) => direct.get(a) && flaggedBy.get(a).length > 0],
  ["stop on every flag", (a) => flaggedBy.get(a).length > 0],
];

const S = [28, 15, 17, 12, 0];
const row = (s) => console.log(s.map((v, i) => String(v).padEnd(S[i])).join(""));
row(["gate mode", "builds stopped", "wrongly stopped", "exceptions", "pipeline + intervention"]);
for (const [name, blocks] of MODES) {
  const exception = new Set();
  let stopped = 0, wronglyStopped = 0;
  for (const components of BUILDS) {
    const cause = components.filter((a) => blocks(a) && !exception.has(a));
    if (cause.length === 0) continue;
    stopped++;
    if (cause.every((a) => !ACTUAL.has(a))) {           // all are false marks: written to the exception list
      wronglyStopped++;
      for (const a of cause) exception.add(a);
    }
  }
  row([name, stopped, wronglyStopped, exception.size,
    `${BUILDS.length * EVAL_S} s + ${stopped * INTERVENTION_MIN} min`]);
}

// If a rule with a high false rate is suppressed wholesale, how many real deviations remain seen.
const THRESHOLD = 0.6;
const remainingRules = Object.entries(flags).filter(([, l]) =>
  l.filter((a) => !ACTUAL.has(a)).length / l.length <= THRESHOLD);
const remainingFlagged = new Set(remainingRules.flatMap(([, l]) => l));
const allFlagged = new Set(Object.values(flags).flat());
console.log("");
console.log(`thirty builds touch ${BUILDS.flat().length} components; flagged components ` +
  `${[...flaggedBy.values()].filter((k) => k.length > 0).length}/${C.length}; seed 9143205`);
console.log(`if a rule with a false rate above ${THRESHOLD * 100}% is suppressed wholesale: rules ` +
  `${Object.keys(flags).length} -> ${remainingRules.length}, flagged components ` +
  `${allFlagged.size} -> ${remainingFlagged.size}`);
console.log(`genuinely deviant seen ${[...ACTUAL].filter((a) => allFlagged.has(a)).length} -> ` +
  `${[...ACTUAL].filter((a) => remainingFlagged.has(a)).length}/${ACTUAL.size}`);

gate mode                   builds stopped wrongly stopped  exceptions  pipeline + intervention
warn only                   0              0                0           270 s + 0 min
stop on direct dependency   16             5                5           270 s + 64 min
stop on every flag          24             9                14          270 s + 96 min

thirty builds touch 85 components; flagged components 30/48; seed 9143205
if a rule with a false rate above 60% is suppressed wholesale: rules 6 -> 2, flagged components 30 -> 14
genuinely deviant seen 10 -> 9/13

Three rows are three settings of the same rule set. Warn-only mode takes 270 seconds from the pipeline and stops no build; everything it finds is written to a list, and reading the list is nobody’s job. Stop-on-every-flag mode stops 24 of thirty builds, costs 96 minutes of developer time, and leaves a 14-component exception list. The mode limited to direct dependencies sits in between: 16 builds stopped, 5 wrongly stopped, 5 exceptions.

The exception column carries the real result. Fourteen components are written to the list because they wrongly stopped a build, and the list never shrinks again. A rule shuts itself off with its own false marks: if twenty of forty-six flags are wrong, a gate becomes less effective the more it runs.

The last two rows measure this in aggregate. If the four rules whose false rate exceeds 60% are suppressed wholesale, flagged components drop from 30 to 14, and the genuinely deviant amount seen drops from 10 to 9. The four noisy rules were showing nothing new besides a single real component; fifteen of the sixteen components that fall off the list were false marks. A rule’s value is not in its flag count, but in the number of genuine items that no rule but itself shows.

Summary

• 6 of seven rules run, one cannot: the calledAtRuntime field is in neither the inventory nor the external registry. A policy is only as writable as the field set it rests on.
• The six rules that run produce 46 flags and fall on 30 of 48 components; two components have no external registry match, and for those two, two rules can never be evaluated.
• Of the 30 flagged components, 10 are real, 20 are false marks. The abandoned-package rule is the least noisy because it looks for two fields together (33%); the single-maintainer rule is wrong in 11 of 18 flags.
• 3 of 13 genuinely deviant components are seen by no rule at all, because no source has a field showing the maintenance transfer.
• The stop-on-every-flag mode stops 24 of thirty builds, requires 96 minutes of intervention, and leaves a 14-component exception list. Suppressing the four noisy rules drops flags from 30 to 14, while the genuinely deviant amount seen drops only from 10 to 9.

Next Step

The policy has run, the flags are counted, and what each flag is worth is written down. What remains is thirty flagged components, ten genuine deviations, and a fourteen-component exception list. This course’s earlier lessons each left their own list too: check findings, dependency match items, the inventory’s missing fields, the provenance document’s untestable attestations. All of them are separate lists, and all of them land in the same developer queue.

Since the queue is finite, an order has to come from somewhere. The next lesson gathers these findings into a single set and ranks them three times: first by severity, then by exploitability, then by exposure. The measure is not the ranking itself, it is the difference between rankings — how many places do the top ten items move, and which item stays on top in all three?