--- title: 'Keeping Documentation Current' source: 'https://academia.sh/en/courses/architectural-documentation/keeping-documentation-current' course: 'Architectural Decisions and Documentation' language: en updated: '2026-08-23T07:01:04+00:00' license: 'CC BY-SA 4.0' --- # Keeping Documentation Current Actually measuring documentation drift: comparing the reality extracted from a module import graph against the document's claim, watching the drift grow across eight changes, and comparing three keeping-current regimes by the drift they catch, the maintenance cost they impose, and the false alarms they produce. The previous lesson made quality requirements measurable and produced a document; that document was in step with the code on the day it was written. This lesson measures what comes after: the code keeps changing, the document stays put, and the gap between them grows. This gap's name is **documentation drift**, and it is found not by guessing but by counting. The measurement method is borrowed. The Service Architectures course's Boundary-Drawing Criteria lesson extracted import edges from source files and counted boundary violations. The same extraction is used here to compare the graph read from code against the document's claim; the measured quantity is different. ## Code, Document, and Audit The block below defines three things: the regional library network system's module map, a fourteen-claim document, and the audit that tests the claims against the code. The module map is a **model**; each module is written to disk as a source file, and the graph is extracted from those files — the measurement happens on real files. Splitting the claims into three is the lesson's axis: a **rule** claim states architectural intent, and if the code does not comply, there is drift; a **snapshot** claim is a count of the code's current state on that day, and it goes stale when the code changes; an **unverifiable** claim cannot be checked against the code at all. ```js // environment.mjs — defines the codebase, the document, and the audit; when run directly, // reports the starting state. The next two blocks import this file. import { mkdirSync, rmSync, writeFileSync, readdirSync, readFileSync } from "node:fs"; // VM19: the six domain modules of the regional library network system. Each module's source // carries which modules it imports, which tables it writes to, and which table it owns. export const START = { branch: { imports: [], writes: ["branch"], owns: ["branch"] }, catalog: { imports: [], writes: ["book", "copy"], owns: ["book", "copy"] }, membership: { imports: [], writes: ["member"], owns: ["member"] }, loan: { imports: ["catalog", "membership"], writes: ["loan"], owns: ["loan"] }, notification: { imports: ["membership", "loan"], writes: ["notification"], owns: ["notification"] }, fee: { imports: ["loan"], writes: ["fee"], owns: ["fee"] }, }; export const write = (m) => { // the code/ directory is rebuilt from scratch on every write rmSync("code", { recursive: true, force: true }); mkdirSync("code", { recursive: true }); for (const [name, x] of Object.entries(m)) { writeFileSync(`code/${name}.mjs`, `// owns: ${x.owns.join(", ")}\n` + x.imports.map((i) => `import { ${i}Op } from "./${i}.mjs";\n`).join("") + `export const write = (t, v) => ({ t, v });\n` + `export function ${name}Op(v) {\n` + x.writes.map((t) => ` write("${t}", v);\n`).join("") + "}\n"); } }; // The import graph is extracted from source files. The method is borrowed from the Service Architectures course's // Boundary-Drawing Criteria lesson, where it measured boundary violations; here it measures documentation drift. export const graph = () => Object.fromEntries(readdirSync("code").map((d) => { const t = readFileSync(`code/${d}`, "utf8"); return [d.replace(".mjs", ""), { imports: [...t.matchAll(/from "\.\/(\w+)\.mjs"/g)].map((x) => x[1]), writes: [...t.matchAll(/write\("(\w+)"/g)].map((x) => x[1]), owns: t.match(/^\/\/ owns: (.+)$/m)[1].split(", "), }]; })); const importerOf = (g, a) => Object.entries(g).filter(([, m]) => m.imports.includes(a)).map(([x]) => x); const writerOf = (g, t) => Object.entries(g).filter(([, m]) => m.writes.includes(t)).map(([x]) => x); // The document carries three kinds of claim. RULE: architectural intent — a violation in the code is drift. SNAPSHOT: a count // of the code's current state — it goes stale when the code changes. UNVERIFIABLE: cannot be checked against the code at all. export const RULE = [ ["K1", "membership", "Membership does not import any domain module.", (g) => g.membership.imports.length === 0], ["K2", "catalog", "Only loan may import catalog.", (g) => importerOf(g, "catalog").every((a) => a === "loan")], ["K3", "notification", "No module may depend on notification.", (g) => importerOf(g, "notification").length === 0], ["K4", "fee", "Fee may only import the loan module.", (g) => g.fee.imports.every((a) => a === "loan")], ["K5", "branch", "Branch does not import any domain module.", (g) => g.branch.imports.length === 0], ["K6", "loan", "Only loan may write to the loan table.", (g) => writerOf(g, "loan").every((a) => a === "loan")], ]; export const SNAPSHOT = [ ["A1", "system", "The system consists of six domain modules.", (g) => Object.keys(g).length], ["A2", "loan", "Loan imports the catalog and membership modules.", (g) => g.loan.imports], ["A3", "system", "The graph has five import edges.", (g) => Object.values(g).reduce((s, m) => s + m.imports.length, 0)], ["A4", "notification", "Notification imports the membership and loan modules.", (g) => g.notification.imports], ["A5", "fee", "Fee writes only to the fee table.", (g) => g.fee.writes], ["A6", "catalog", "Catalog owns the book and copy tables.", (g) => g.catalog.owns], ]; export const UNVERIFIABLE = [ ["D1", "catalog", "The catalog system is sourced externally."], ["D2", "system", "The nightly maintenance window is 02:00-04:00."], ]; // VM20: eight changes applied over the next two quarters. The intent label says whether the change // complies with an architectural rule; the label is not asserted, it is compared against the audit result. export const CHANGE = [ ["reservation module is added", "compliant", "reservation", (m) => { m.reservation = { imports: ["membership"], writes: ["reservation"], owns: ["reservation"] }; }], ["loan imports notification", "non-compliant", "loan", (m) => m.loan.imports.push("notification")], ["fee imports membership", "non-compliant", "fee", (m) => m.fee.imports.push("membership")], ["notification drops its membership import", "compliant", "notification", (m) => { m.notification.imports = m.notification.imports.filter((a) => a !== "membership"); }], ["reservation writes to the loan table", "non-compliant", "reservation", (m) => m.reservation.writes.push("loan")], ["catalog takes over the tag table", "compliant", "catalog", (m) => { m.catalog.owns.push("tag"); m.catalog.writes.push("tag"); }], ["fee writes to the penalty table", "compliant", "fee", (m) => m.fee.writes.push("penalty")], ["branch imports membership", "non-compliant", "branch", (m) => m.branch.imports.push("membership")], ]; if (process.argv[1]?.endsWith("environment.mjs")) { write(structuredClone(START)); const g = graph(); const row = [ ...RULE.map(([ad, ozne, metin, sina]) => [ad, "rule", ozne, sina(g) ? "true" : "false", metin]), ...SNAPSHOT.map(([ad, ozne, metin, oku]) => [ad, "snapshot", ozne, String(oku(g)), metin]), ...UNVERIFIABLE.map(([ad, ozne, metin]) => [ad, "unverifiable", ozne, "-", metin]), ]; console.log(`${"claim".padEnd(6)}${"type".padEnd(13)}${"subject".padEnd(13)}${"t=0".padEnd(21)}text`); for (const [a, b, c, d, e] of row) { console.log(`${a.padEnd(6)}${b.padEnd(13)}${c.padEnd(13)}${d.padEnd(21)}${e}`); } console.log(`\n${Object.keys(g).length} modules, ` + `${Object.values(g).reduce((s, x) => s + x.imports.length, 0)} import edges; the document carries ${row.length} ` + `claims (${RULE.length} rule, ${SNAPSHOT.length} snapshot, ${UNVERIFIABLE.length} unverifiable)`); console.log(`t=0 drift: 0 false rule claims, 0 stale snapshot claims ` + `(the document was written the same day as the code)`); } ``` ``` claim type subject t=0 text K1 rule membership true Membership does not import any domain module. K2 rule catalog true Only loan may import catalog. K3 rule notification true No module may depend on notification. K4 rule fee true Fee may only import the loan module. K5 rule branch true Branch does not import any domain module. K6 rule loan true Only loan may write to the loan table. A1 snapshot system 6 The system consists of six domain modules. A2 snapshot loan catalog,membership Loan imports the catalog and membership modules. A3 snapshot system 5 The graph has five import edges. A4 snapshot notification membership,loan Notification imports the membership and loan modules. A5 snapshot fee fee Fee writes only to the fee table. A6 snapshot catalog book,copy Catalog owns the book and copy tables. D1 unverifiable catalog - The catalog system is sourced externally. D2 unverifiable system - The nightly maintenance window is 02:00-04:00. 6 modules, 5 import edges; the document carries 14 claims (6 rule, 6 snapshot, 2 unverifiable) t=0 drift: 0 false rule claims, 0 stale snapshot claims (the document was written the same day as the code) ``` ## How Drift Grows The document starts with zero drift. The next block applies the eight changes in order, retests all fourteen claims after each step, and never touches the document. ```js // drift.mjs — imports environment.mjs; measures the drift of an unupdated document, step by step import { START, CHANGE, RULE, SNAPSHOT, UNVERIFIABLE, write, graph } from "./environment.mjs"; const m = structuredClone(START); write(m); const initial = graph(); const expected = Object.fromEntries(SNAPSHOT.map(([ad, , , oku]) => [ad, JSON.stringify(oku(initial))])); const driftMeasure = (g) => ({ rule: RULE.filter(([, , , test]) => !test(g)).map(([ad]) => ad), snapshot: SNAPSHOT.filter(([ad, , , oku]) => JSON.stringify(oku(g)) !== expected[ad]).map(([ad]) => ad), }); console.log(`${"t".padStart(2)} ${"change".padEnd(42)}${"intent".padEnd(15)}` + `${"claim dropped".padEnd(16)}${"false rules".padStart(12)}${"stale snapshots".padStart(16)}` + `${"drift".padStart(7)}`); let previous = []; CHANGE.forEach(([ad, intent, , apply], t) => { apply(m); write(m); const s = driftMeasure(graph()); const all = [...s.rule, ...s.snapshot]; const fresh = all.filter((a) => !previous.includes(a)); previous = all; console.log(`${String(t + 1).padStart(2)} ${ad.padEnd(42)}${intent.padEnd(15)}` + `${(fresh.join(",") || "-").padEnd(16)}${String(s.rule.length).padStart(12)}` + `${String(s.snapshot.length).padStart(16)}${String(all.length).padStart(7)}`); }); const final = driftMeasure(graph()); const total = RULE.length + SNAPSHOT.length + UNVERIFIABLE.length; console.log(`\nafter eight changes, ${final.rule.length + final.snapshot.length} of ${total} claims are ` + `wrong, ${UNVERIFIABLE.length} are unverifiable, ` + `${total - final.rule.length - final.snapshot.length - UNVERIFIABLE.length} are correct ` + `(correct ones remaining: ${RULE.filter(([a]) => !final.rule.includes(a)).map(([a]) => a).join(", ")})`); console.log(`${CHANGE.filter(([, n]) => n === "compliant").length} compliant changes ` + `dropped no rule claim; ` + `${CHANGE.filter(([, n]) => n === "non-compliant").length} non-compliant changes ` + `dropped ${final.rule.length} rule claims`); ``` ``` t change intent claim dropped false rules stale snapshots drift 1 reservation module is added compliant A1,A3 0 2 2 2 loan imports notification non-compliant K3,A2 1 3 4 3 fee imports membership non-compliant K4 2 3 5 4 notification drops its membership import compliant A4 2 4 6 5 reservation writes to the loan table non-compliant K6 3 4 7 6 catalog takes over the tag table compliant A6 3 5 8 7 fee writes to the penalty table compliant A5 3 6 9 8 branch imports membership non-compliant K5 4 6 10 after eight changes, 10 of 14 claims are wrong, 2 are unverifiable, 2 are correct (correct ones remaining: K1, K2) 4 compliant changes dropped no rule claim; 4 non-compliant changes dropped 4 rule claims ``` Drift starts at the first change and grows in one direction: 2, 4, 5, 6, 7, 8, 9, 10. By the end, ten of the fourteen claims are wrong and two are unverifiable; the two that stay correct are rule claims. The document keeps talking, but most of it no longer matches the code. The two claim types behave differently. None of the four compliant changes dropped a rule claim; all of them only staled snapshot claims. All four non-compliant changes dropped a rule claim. **A wrong rule claim is a code defect; a stale snapshot claim is a document defect.** Both show up in the same table, but one is fixed in the code, the other in the document. ## Three Keeping-Current Regimes The same eight changes now run under three regimes. **Manual update**: whoever makes the change reviews only the claims of the module they touched. **Generated audit**: after every change, every claim is tested against the code. **Rule-only document**: the document never carries a snapshot claim. ```js // regime.mjs — imports environment.mjs; compares three keeping-current regimes on the same eight changes import { START, CHANGE, RULE, SNAPSHOT, UNVERIFIABLE, write, graph } from "./environment.mjs"; // manual: whoever makes the change reviews the claims of the module they touched. // audit: after every change, every claim is automatically tested against the code. // rule-only: the document carries only rule claims; no snapshot claim is ever written. const run = (regime) => { const m = structuredClone(START); write(m); const doc = regime === "rule-only" ? [] : SNAPSHOT.map(([ad, ozne, , oku]) => ({ ad, ozne, oku, value: JSON.stringify(oku(graph())) })); const caught = new Set(); let updates = 0, warnings = 0, falseAlarms = 0; for (const [, intent, touched, apply] of CHANGE) { apply(m); write(m); const g = graph(); const violated = RULE.filter(([, , , test]) => !test(g)); const stale = doc.filter((i) => JSON.stringify(i.oku(g)) !== i.value); if (regime === "manual") { for (const [ad, ozne] of violated) if (ozne === touched) caught.add(ad); for (const i of stale) if (i.ozne === touched) { i.value = JSON.stringify(i.oku(g)); updates += 1; } } else { for (const [ad] of violated) caught.add(ad); warnings += stale.length; if (intent === "compliant") falseAlarms += stale.length; for (const i of stale) { i.value = JSON.stringify(i.oku(g)); updates += 1; } } } const g = graph(); const actual = RULE.filter(([, , , test]) => !test(g)).map(([ad]) => ad); return { regime, doc, actual, caught, updates, warnings, falseAlarms, stale: doc.filter((i) => JSON.stringify(i.oku(g)) !== i.value).map((i) => i.ad) }; }; const RESULT = ["manual", "audit", "rule-only"].map(run); console.log(`${"regime".padEnd(10)}${"caught".padStart(9)}${"missed".padStart(18)}` + `${"updates".padStart(11)}${"snapshot warn.".padStart(16)}${"false alarms".padStart(14)}` + `${"still stale".padStart(13)}`); for (const s of RESULT) { const missed = s.actual.filter((a) => !s.caught.has(a)); console.log(`${s.regime.padEnd(10)}${`${s.caught.size}/${s.actual.length}`.padStart(9)}` + `${(missed.join(",") || "-").padStart(18)}${String(s.updates).padStart(11)}` + `${String(s.warnings).padStart(16)}${String(s.falseAlarms).padStart(14)}` + `${(s.stale.join(",") || "-").padStart(13)}`); } // Seven questions someone who comes later asks the document; each question lands on one claim. const QUESTION = [ ["How many domain modules does the system consist of?", "A1"], ["Which modules does the loan module use?", "A2"], ["How many import edges are in the graph?", "A3"], ["Who may depend on the notification module?", "K3"], ["Who may write to the loan table?", "K6"], ["Which tables does catalog own?", "A6"], ["When is the nightly maintenance window?", "D2"], ]; const g = graph(); const answer = (s, claim) => { if (RULE.some(([a]) => a === claim)) return "correct"; // a rule states intent if (UNVERIFIABLE.some(([a]) => a === claim)) return s.regime === "rule-only" ? "unanswered" : "unverifiable"; const i = s.doc.find((x) => x.ad === claim); if (!i) return "unanswered"; return JSON.stringify(i.oku(g)) === i.value ? "correct" : "wrong"; }; console.log(`\n${"question".padEnd(54)}${"claim".padEnd(7)}${"manual".padEnd(14)}${"audit".padEnd(14)}rule-only`); for (const [question, claim] of QUESTION) { console.log(`${question.padEnd(54)}${claim.padEnd(7)}` + RESULT.map((s) => answer(s, claim).padEnd(14)).join("").trimEnd()); } for (const s of RESULT) { const v = QUESTION.map(([, i]) => answer(s, i)); console.log(`${s.regime.padEnd(10)}${v.filter((x) => x === "correct").length} correct, ` + `${v.filter((x) => x === "wrong").length} wrong, ` + `${v.filter((x) => x === "unanswered").length} unanswered, ` + `${v.filter((x) => x === "unverifiable").length} unverifiable`); } ``` ``` regime caught missed updates snapshot warn. false alarms still stale manual 3/4 K6 4 0 0 A1,A3 audit 4/4 - 10 10 6 - rule-only 4/4 - 0 0 0 - question claim manual audit rule-only How many domain modules does the system consist of? A1 wrong correct unanswered Which modules does the loan module use? A2 correct correct unanswered How many import edges are in the graph? A3 wrong correct unanswered Who may depend on the notification module? K3 correct correct correct Who may write to the loan table? K6 correct correct correct Which tables does catalog own? A6 correct correct unanswered When is the nightly maintenance window? D2 unverifiable unverifiable unanswered manual 4 correct, 2 wrong, 0 unanswered, 1 unverifiable audit 6 correct, 0 wrong, 0 unanswered, 1 unverifiable rule-only 2 correct, 0 wrong, 5 unanswered, 0 unverifiable ``` Manual update catches three of the four real drifts and misses the fourth. The missed drift is K6: the new module writing to the loan table is reservation, but the claim's subject is loan, and touching reservation does not trigger a look at loan's claim. **This is manual update's blind spot — drift is invisible when the touched module and the claim's subject differ.** Its maintenance cost is the lowest: four claim updates. The generated audit catches all four of the drifts and leaves no stale claim behind. Its price is ten claim updates and ten warnings. Six of the warnings come from compliant changes, meaning they arrived while the code was still correct: a **false alarm**. The source is clear — snapshot claims go stale on every legitimate change, and the audit does not tell them apart from real drift. The rule-only document zeroes out both line items: four drifts caught, zero updates, zero false alarms. Its price is in the question table — five of the seven questions go unanswered. This regime is cheap because it says little. The question table separates the three regimes by this course's own measure. The manually updated document answers four of seven questions correctly and **two incorrectly**: it confidently states the module count and the import-edge count, and both are wrong. An unanswered question sends the reader to the code; a wrongly answered question sends them to the wrong place. Rule claims stay correct in all three regimes: a rule claim that does not hold against the code is wrong about the code, not the document. ## Summary - Documentation drift is a measurable quantity: the import graph extracted from code is compared against the document's claim; the method is borrowed from the Service Architectures course's boundary-violation measurement. - Claims are three types and behave differently: the four compliant changes dropped no rule claim, only staling snapshot claims; the four non-compliant ones dropped four rule claims. In the unupdated document, drift grows from 2 to 10, and only 2 of the 14 claims stay correct. - Manual update catches 3 of 4 drifts and costs 4 updates; its blind spot is when the touched module and the claim's subject differ. The generated audit catches all 4, at a cost of 10 updates and 10 warnings; 6 of the warnings are false alarms, and their source is the snapshot claims. - The seven questions' answers split by regime: 4 correct and 2 wrong under manual update, 6 correct under the generated audit, 2 correct and 5 unanswered under the rule-only document. ## Next Step Everything measured up to this point looked at the past or the present: a decision made, a boundary drawn, code running. There is one more thing an architect commits to in writing, and it looks at the future — how long this work will take. The next lesson measures estimation: the same set of work is estimated as a single number and as an uncertainty band, then compared against what actually happened. How the deviation is distributed, how much of it the band actually covers, and whether splitting the estimate into pieces shrinks the deviation.