Policy, Standard, and Procedure

Everything measured in the previous lesson’s last two tables rested on a document: the inventory is a document, the retention period is written in a document, the steps of the request flow are defined in a document. Up to this point, documents were treated as a single mass. From an audit standpoint, however, there is a sharp difference between them, and this lesson measures that difference.

The document hierarchy has three levels. A policy states intent and responsibility; it says who owns what, and gives no measurable threshold. A standard gives the threshold: a number, a duration, a configuration value. A procedure defines a task step by step and produces a record every time it runs. Evidence comes only from the last level. The cost of turning an architectural rule into a runnable check was measured in M20/K04; the question here is different — which document level counts as an item in front of an auditor, and which one is merely read and passed over.

• FW33. The document set is 26 items and is spread across eight subject domains. The documents, record counts, and ages are fictional.
• FW34. Every document has a level (policy, standard, procedure), whether or not it carries a measurable threshold, the number of records it produced in the period, and the number of days since its last update.
• FW35. A requirement is written as a chain linking the three levels together: policy → standard → procedure. An empty cell is a missing link.
• FW36. Evidence rule: a requirement is considered provable in an audit only if it has a record-producing procedure. A policy and a standard are not evidence on their own.
• FW37. The cost of producing a missing link varies by level: policy 3, standard 8, procedure 15, the procedure’s record flow 10 person-days (fictional).
• FW38. A procedure can appear in more than one chain; in that case the record item is shared too.

// document/hierarchy.mjs — separating policy, standard, and procedure levels along the axis of evidence production.
// MODEL: regional measurement network (fictional); documents, record counts, and ages are fictional.
export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|"));
export const gen = (B) => (...h) => h.map((v, j) =>
  (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join("");

// code | level | domain | measurable threshold | records produced in period | days since last update
export const DOCUMENT = Object.fromEntries(row(`
  P1|policy|access|no|0|420
  P2|policy|data|no|0|180
  P3|policy|change|no|0|900
  P4|policy|incident|no|0|150
  P5|policy|vendor|no|0|640
  P6|policy|awareness|no|0|300
  S1|standard|access|yes|0|120
  S2|standard|access|yes|0|540
  S3|standard|data|yes|0|90
  S4|standard|data|no|0|380
  S5|standard|change|yes|0|75
  S6|standard|incident|yes|0|200
  S7|standard|vendor|no|0|700
  S8|standard|continuity|yes|0|260
  S9|standard|awareness|yes|0|340
  Y1|procedure|access|yes|240|60
  Y2|procedure|access|yes|48|110
  Y3|procedure|data|yes|365|30
  Y4|procedure|data|yes|30|45
  Y5|procedure|change|yes|310|20
  Y6|procedure|incident|yes|18|95
  Y7|procedure|incident|no|0|410
  Y8|procedure|vendor|yes|3|280
  Y9|procedure|continuity|yes|4|150
  Y10|procedure|awareness|yes|4|85
  Y11|procedure|asset|yes|12|40`)
  .map(([code, level, domain, threshold, records, age]) =>
    [code, { code, level, domain, threshold: threshold === "yes", records: +records, age: +age }]));

// requirement | policy | standard | procedure  (an empty cell is a missing link)
export const CHAIN = row(`
  Z01 access approval|P1|S1|Y1
  Z02 privileged access|P1|S2|Y2
  Z03 authentication rules|P1|S2|
  Z04 data classification|P2|S3|Y3
  Z05 deletion request|P2|S4|Y4
  Z06 retention period|P2|S3|Y3
  Z07 change approval|P3|S5|Y5
  Z08 incident notification|P4|S6|Y6
  Z09 post-incident review|P4|S6|Y7
  Z10 vendor assessment|P5|S7|Y8
  Z11 vendor offboarding|P5||
  Z12 continuity exercise||S8|Y9
  Z13 awareness training|P6|S9|Y10
  Z14 asset inventory|||Y11
  Z15 record integrity|||`)
  .map(([name, p, s, y]) => ({ name, code: name.split(" ")[0], p, s, y }));

export const LEVEL = ["policy", "standard", "procedure"];
// Evidence rule: a requirement is provable only with a record-producing procedure.
export const provable = (z) => Boolean(z.y) && DOCUMENT[z.y].records > 0;
export const breakPoint = (z) => !z.p ? "no policy" : !z.s ? "no standard"
  : !z.y ? "no procedure" : DOCUMENT[z.y].records === 0 ? "produces no records" : "complete";

if (import.meta.url === `file://${process.argv[1]}`) {
  const all = Object.values(DOCUMENT);
  console.log(`model: ${all.length} documents, ${CHAIN.length} requirement chains, ` +
    `${new Set(all.map((b) => b.domain)).size} subject domains`);

  const y = gen([11, 6, 11, 11, 16, 14]);
  console.log("\n" + y("level", "docs", "threshold", "producing", "period records", "avg age days"));
  for (const d of LEVEL) {
    const k = all.filter((b) => b.level === d);
    console.log(y(d, k.length, k.filter((b) => b.threshold).length,
      k.filter((b) => b.records > 0).length, k.reduce((a, b) => a + b.records, 0),
      (k.reduce((a, b) => a + b.age, 0) / k.length).toFixed(0)));
  }
  console.log(`of the ${all.length} documents in total, ${all.filter((b) => b.records > 0).length} ` +
    `produce evidence in an audit; ${all.filter((b) => !b.threshold).length} do not carry a ` +
    `measurable threshold`);

  const z = gen([21, 7, 9, 37]);
  console.log("\n" + z("chain status", "count", "percent", "requirements"));
  const STATUS = ["complete", "produces no records", "no procedure", "no standard", "no policy"];
  for (const d of STATUS) {
    const k = CHAIN.filter((x) => breakPoint(x) === d);
    console.log(z(d, k.length, (100 * k.length / CHAIN.length).toFixed(0),
      k.map((x) => x.code).join(" ") || "-"));
  }
  console.log(`\nprovable requirement ${CHAIN.filter(provable).length}/${CHAIN.length}; ` +
    `with a policy ${CHAIN.filter((x) => x.p).length}, with a standard ` +
    `${CHAIN.filter((x) => x.s).length}, with a procedure ${CHAIN.filter((x) => x.y).length}`);
  const stale = CHAIN.filter((x) => x.p && x.y && DOCUMENT[x.p].age > DOCUMENT[x.y].age + 180);
  console.log(`chains where the policy is more than 180 days older than its procedure ${stale.length}: ` +
    `${stale.map((x) => `${x.code} (${DOCUMENT[x.p].age}/${DOCUMENT[x.y].age} days)`).join(", ")}`);
}

model: 26 documents, 15 requirement chains, 8 subject domains

level        docs  threshold  producing  period records  avg age days
policy          6          0          0               0           432
standard        9          7          0               0           301
procedure      11         10         10            1034           120
of the 26 documents in total, 10 produce evidence in an audit; 9 do not carry a measurable threshold

chain status           count  percent                         requirements
complete                   9       60  Z01 Z02 Z04 Z05 Z06 Z07 Z08 Z10 Z13
produces no records        1        7                                  Z09
no procedure               1        7                                  Z03
no standard                1        7                                  Z11
no policy                  3       20                          Z12 Z14 Z15

provable requirement 11/15; with a policy 12, with a standard 12, with a procedure 12
chains where the policy is more than 180 days older than its procedure 5: Z01 (420/60 days), Z02 (420/110 days), Z07 (900/20 days), Z10 (640/280 days), Z13 (300/85 days)

The first table separates the hierarchy level by level, and reading it requires taking in three columns at once. None of the six policies carry a measurable threshold, and none produce records; seven of the nine standards carry a threshold, but again none of them produce records; ten of the eleven procedures both carry a threshold and produce records, and put out 1,034 items in the period. Ten of the twenty-six documents produce evidence in an audit. The remaining sixteen can be placed in the audit file, read, and accepted — but they do not show that work was done, only that it should be done.

The age column explains why this is so. The average age of policies is 432 days, of standards 301, of procedures 120. The direction of this ranking is not a coincidence: a record-producing document meets reality on every run and changes when it does not hold up; a document that never produces a record faces no pressure of its own accord. The last row gives this its sharpest form — in five chains, the policy is more than 180 days older than its own procedure; in the change domain, the policy is 900 days old and the procedure 20. In these chains, two documents may be describing the same work differently, and the contradiction only becomes visible if a reader happens to notice it.

The second table runs the chains. Nine of the fifteen requirements are complete chains. Where the remaining six chains break is quite different from one another and produces four separate outcomes: Z09 has a procedure but it produces no records, Z03 has a standard but no procedure, Z11 has only a policy, and Z12, Z14, and Z15 have no policy.

The number just before the last line gives this lesson’s main distinction: the provable requirement count is 11, while the complete-chain count is 9. The two requirements in between — the continuity exercise and the asset inventory — have no policy, but they do have a record-producing procedure. Both are proven in an audit. In other words, evidence production does not depend on the upper levels of the hierarchy; a record-producing procedure shows that work was done even with no policy sitting above it. The reverse is not true: twelve chains have a policy, but none of the policies alone prove any requirement.

The Cost of the Missing Link

// document/completion.mjs — the cost of completing a broken link, and evidence production per domain.
import { DOCUMENT, CHAIN, LEVEL, gen, provable, breakPoint } from "./hierarchy.mjs";

// The cost of producing a missing link (person-days, fictional)
const DAYS = { policy: 3, standard: 8, procedure: 15, records: 10 };
const missingPieces = (z) => [...(!z.p ? ["policy"] : []), ...(!z.s ? ["standard"] : []),
  ...(!z.y ? ["procedure", "records"] : DOCUMENT[z.y].records === 0 ? ["records"] : [])];
const cost = (z) => missingPieces(z).reduce((a, e) => a + DAYS[e], 0);

const incompleteChain = CHAIN.filter((z) => missingPieces(z).length > 0);
const y = gen([26, 35, 6, 15, 13]);
console.log(`${incompleteChain.length} chains have a missing link; provable requirement ` +
  `${CHAIN.filter(provable).length}/${CHAIN.length}`);
console.log("\n" + y("requirement", "missing link", "days", "evidence gain", "days / gain"));
for (const z of [...incompleteChain].sort((a, b) => cost(a) - cost(b))) {
  const gain = provable(z) ? 0 : 1;
  console.log(y(z.name, missingPieces(z).join("+"), cost(z), gain,
    gain ? cost(z) : "no gain"));
}
const noEvidence = incompleteChain.filter((z) => !provable(z));
const appearanceOnly = incompleteChain.filter((z) => provable(z));
console.log(`\ntotal cost of the ${noEvidence.length} chains that gain evidence ` +
  `${noEvidence.reduce((a, z) => a + cost(z), 0)} person-days; cost of the ` +
  `${appearanceOnly.length} chains that gain no evidence at all ` +
  `${appearanceOnly.reduce((a, z) => a + cost(z), 0)} person-days`);
console.log(`cheapest to costliest: ` +
  `${[...incompleteChain].sort((a, b) => cost(a) - cost(b)).map((z) => z.code).join(" < ")}`);
console.log(`ranked by evidence gain: ` +
  `${[...incompleteChain].sort((a, b) => (provable(a) ? 1 : 0) - (provable(b) ? 1 : 0) ||
    cost(a) - cost(b)).map((z) => z.code).join(" < ")}`);

// Document count vs. evidence production per domain
const domains = [...new Set(Object.values(DOCUMENT).map((b) => b.domain))];
const docsIn = (a) => Object.values(DOCUMENT).filter((b) => b.domain === a);
const z2 = gen([12, 6, 8, 10, 11, 16]);
console.log("\n" + z2("domain", "docs", ...LEVEL, "period records"));
for (const a of domains) {
  const k = docsIn(a);
  console.log(z2(a, k.length, ...LEVEL.map((d) => k.filter((b) => b.level === d).length),
    k.reduce((x, b) => x + b.records, 0)));
}
const byDocCount = [...domains].sort((a, b) => docsIn(b).length - docsIn(a).length ||
  a.localeCompare(b));
const byRecordCount = [...domains].sort((a, b) => docsIn(b).reduce((x, i) => x + i.records, 0) -
  docsIn(a).reduce((x, i) => x + i.records, 0) || a.localeCompare(b));
console.log(`\ntop three by document count : ${byDocCount.slice(0, 3).join(", ")}`);
console.log(`top three by evidence production: ${byRecordCount.slice(0, 3).join(", ")}`);
console.log(`shared: ${byRecordCount.slice(0, 3).filter((a) => byDocCount.slice(0, 3).includes(a)).length}/3; ` +
  `the single-document asset domain produces ${docsIn("asset").reduce((x, b) => x + b.records, 0)} records, ` +
  `the four-document incident domain produces ${docsIn("incident").reduce((x, b) => x + b.records, 0)} records`);
console.log(`break points: ` +
  `${incompleteChain.map((z) => `${z.code} ${breakPoint(z)}`).join("; ")}`);

6 chains have a missing link; provable requirement 11/15

requirement                                      missing link  days  evidence gain  days / gain
Z12 continuity exercise                                policy     3              0      no gain
Z09 post-incident review                              records    10              1           10
Z14 asset inventory                           policy+standard    11              0      no gain
Z03 authentication rules                    procedure+records    25              1           25
Z11 vendor offboarding             standard+procedure+records    33              1           33
Z15 record integrity        policy+standard+procedure+records    36              1           36

total cost of the 4 chains that gain evidence 104 person-days; cost of the 2 chains that gain no evidence at all 14 person-days
cheapest to costliest: Z12 < Z09 < Z14 < Z03 < Z11 < Z15
ranked by evidence gain: Z09 < Z03 < Z11 < Z15 < Z12 < Z14

domain        docs  policy  standard  procedure  period records
access           5       1         2          2             288
data             5       1         2          2             395
change           3       1         1          1             310
incident         4       1         1          2              18
vendor           3       1         1          1               3
awareness        3       1         1          1               4
continuity       2       0         1          1               4
asset            1       0         0          1              12

top three by document count : access, data, incident
top three by evidence production: data, change, access
shared: 2/3; the single-document asset domain produces 12 records, the four-document incident domain produces 18 records
break points: Z03 no procedure; Z09 produces no records; Z11 no standard; Z12 no policy; Z14 no policy; Z15 no policy

When sorted from cheapest to costliest, the first table carries two traps right at the top. The continuity exercise becomes a “complete chain” at 3 person-days, the asset inventory at 11; neither produces any new evidence, because both are already in a provable state. The total of 14 person-days fixes the table’s appearance in the audit report and does not change the number of items shown in the audit. Setting up the post-incident review’s record flow, by contrast, costs 10 person-days and turns an unprovable requirement into a provable one.

This is exactly where the two rankings diverge. The cheapest-to-costliest order starts with Z12; the ranking by evidence gain starts with Z09, and Z12 and Z14 fall to the very end. Completing missing links by looking at the level — “the policies should be written first, so the hierarchy is complete” — means spending 14 person-days on appearance alone. The total cost of the four chains that gain evidence is 104 person-days, and 36 of that goes to a single chain, record integrity, which has no level at all.

The last table shows the same divergence at the domain level. The top three by document count are the access, data, and incident domains; the top three by evidence production are data, change, and access. The incident domain carries four documents and produces 18 records in the period; the asset domain has a single document and produces 12. How many documents a domain has says nothing about how much evidence that domain produces. Counting documents is a measure of effort; counting records is a measure of outcome, and there is no structural link between these two numbers.

Summary

• The document hierarchy has three levels: a policy states intent, a standard gives a measurable threshold, a procedure produces a record every time it runs. In an audit, evidence comes only from the last level.
• Ten of the twenty-six documents produce evidence; none of the six policies carry a threshold or produce records, and seven of the nine standards carry a threshold but none of them produce records.
• A record-producing document is fresher: the average age of policies is 432 days, of procedures 120. In five chains, the policy is more than 180 days older than its own procedure.
• The provable requirement count is 11, the complete-chain count 9: two requirements have no policy, yet are proven thanks to their record-producing procedure. The reverse is not true — no policy proves a requirement on its own.
• Completing missing links by level is misleading: 14 person-days make two chains look “complete” and produce no new evidence at all; the cost of the four chains that gain evidence is 104 person-days, and the cheapest of them is 10 days.
• The number of documents in a subject domain says nothing about that domain’s evidence production: the four-document incident domain produces 18 records, the single-document asset domain 12.

Next Step

Every number in this lesson rests on an assumption: that the cells said to be filled in the chain really are filled, that the procedure said to produce records really does produce them, that a 900-day-old policy is still valid. The source of this assumption is clear — the party filling in the table, writing the documents, and reporting that the records were produced is the same party. Everything measured across five lessons has been run on the institution’s own self-declaration.

The next lesson closes out the topic and removes this assumption: how many findings never surface at all if the same person both builds the control and audits it, how many items separating the builder from the auditor uncovers, and how many items of extra work that very separation brings. Independent assurance is not an adjective; it is an arrangement that can be counted.