--- title: 'Architectural Decisions and Documentation' source: 'https://academia.sh/en/courses/architectural-documentation' language: en updated: '2026-08-23T07:01:03+00:00' license: 'CC BY-SA 4.0' lessons: 11 outcomes: - 'Records architectural decisions together with their rationale' - 'Documents the system through different views at a fitting level of detail' - 'Turns quality attributes into measurable scenarios' --- # Architectural Decisions and Documentation Getting the decision out of memory: architecture decision records with a context, decision, and consequence structure, the written-proposal and evaluation process, trade-off analysis where alternatives are scored along quality axes, a risk register that tracks uncertainty, the separation of deliberate from inadvertent technical debt, the logical, process, development, and physical views, the layered diagram approach that changes scale from context down to code, which diagram answers which question, measurable quality attribute scenarios, keeping the document in step with the code, and size estimates carrying an uncertainty band. ## The Decision Record Leaving a trace of the decision: the structure of architecture decision records with context, decision, and consequence fields, the effect of a written-proposal culture on the evaluation process, alternatives eliminated by scoring along quality axes, a risk register that tracks uncertainty through likelihood and impact, and deliberate and inadvertent technical debt measured separately. ### Architecture Decision Records Source: https://academia.sh/en/courses/architectural-documentation/architecture-decision-records Recording a decision with context, decision, and consequence fields: the questions someone asks six months later are modeled as a question set, the same decision set is run against that set in an unrecorded and a structured recorded scheme, the answered and unanswered questions are counted, the cost of writing the record is measured in lines and minutes, and whether the superseded-decision field keeps the old decision readable is counted. The previous course measured how a decision gets made: the alternative–quality table was turned into a data structure, the weights were swept, and a weight with no recorded source turned up. All of it lives in exactly one place: the decision-maker's memory. There is nothing to hand someone who asks "why is it this way" six months later. This course's measure follows from there. A document's value is how many of the later questions it answers, how many it leaves unanswered, and the cost the record extracts. "Documentation matters" carries no decision; two schemes are run against the same question set. #### Question Set and Decision Set The through-line is the software system of a regional library network; the setup is fictional. The model has fourteen branches, an externally sourced catalog system, and loan services written in-house. An **architecture decision record** is the structured form of the decision record used to describe a design: instead of free-form description, a document with named fields — **context**, **decision**, **consequence**. Having named fields is what makes measurement possible, because every question lands on a field. The question set has ten questions and each question points to a field; the decision set has four decisions. In the unrecorded scheme, an answer can come only from the running code, the version history, or the decision-maker's memory; in the recorded scheme, every field is in the record. Whether the fields are filled (**DR1**), the source mapping, and whether the decision-maker is still reachable six months later (**DR2**) are the model's inputs. ```js // record/model.mjs — question set and decision set as a data structure; the scenario is fictional export const QUESTIONS = [ ["S1", "why is it this way", "rationale"], ["S2", "which alternative was eliminated", "alternative"], ["S3", "what was the elimination based on", "criterion"], ["S4", "what assumption was it given under", "assumption"], ["S5", "when was it given", "date"], ["S6", "who gave it", "issuer"], ["S7", "what is the known cost", "consequence"], ["S8", "is it still valid", "status"], ["S9", "did it supersede an earlier one", "supersedes"], ["S10", "where does this number come from", "threshold"], ].map(([code, text, field]) => ({ code, text, field })); // Whether each field is filled and the owner is still reachable six months later is the input (DR1, DR2). export const DECISIONS = [ { name: "loan-access", owner: false, choice: "read-only cache at the branch", rationale: "the branch network drops twice a day", alternative: "single-center, branch-copy, external-module", criterion: "weighted-score sweep", assumption: "daily volume fits in the cache", date: "Q2", issuer: "platform team", consequence: "invalidation requires maintenance", status: "valid", supersedes: "supersedes the branch-copy decision", threshold: "freshness limit 90 s" }, { name: "catalog-boundary", owner: true, choice: "access only through the mapping layer", rationale: "the catalog schema changes with every version", alternative: "direct table, view read", criterion: "calls broken per version", assumption: "the external interface stays stable", date: "Q2", issuer: "integration team", consequence: "every field is written twice", status: "valid", supersedes: "none - first decision", threshold: null }, { name: "penalty-calculation", owner: true, choice: "consolidated in a single rule engine", rationale: "the penalty rule changes a few times a year", alternative: "catalog penalty module, branch setting", criterion: "files touched per change", assumption: "the rule is the same at every branch", date: "Q3", issuer: "loan team", consequence: "the engine grows if an exception arrives", status: "valid", supersedes: "none - first decision", threshold: "delay threshold 14 days" }, { name: "return-alert", owner: false, choice: "asynchronous dispatch through a queue", rationale: "dispatch must not enter the response time", alternative: "synchronous dispatch, daily batch job", criterion: "how an outage reflects onto the loan transaction", assumption: "a one-hour delay is acceptable", date: "Q4", issuer: "loan team", consequence: "the dead-letter box must be monitored", status: "valid", supersedes: "none - first decision", threshold: "alert window 3 days" }, ]; // Which field is readable from which source in the unrecorded scheme (DR2); in the recorded scheme all of it is in the record. export const SOURCE = { code: ["status"], version: ["date", "issuer"], memory: ["rationale", "assumption", "consequence"] }; export const applicable = (d) => QUESTIONS.filter((q) => d[q.field] !== null); export function answer(d, scheme) { return applicable(d).map((q) => { let source = null; if (scheme === "recorded") source = "record"; else if (SOURCE.code.includes(q.field)) source = "code"; else if (SOURCE.version.includes(q.field)) source = "version"; else if (SOURCE.memory.includes(q.field) && d.owner) source = "memory"; return { question: q.code, source, answered: source !== null }; }); } if (import.meta.url.endsWith(process.argv[1].split("/").pop())) { const total = DECISIONS.reduce((t, d) => t + applicable(d).length, 0); console.log(`${QUESTIONS.length} questions x ${DECISIONS.length} decisions -> ${total} applicable questions`); for (const scheme of ["unrecorded", "recorded"]) { let a = 0; const sourceCount = {}; for (const d of DECISIONS) for (const r of answer(d, scheme).filter((r) => r.answered)) { a += 1; sourceCount[r.source] = (sourceCount[r.source] ?? 0) + 1; } console.log(`[${scheme}] ${a}/${total} answers, ${total - a} unanswered, source: ` + Object.entries(sourceCount).map(([s, n]) => `${s}=${n}`).join(" ")); } console.log("\ndecisions answering each question in the unrecorded scheme:"); for (const q of QUESTIONS) { const eligible = DECISIONS.filter((d) => d[q.field] !== null); const hit = eligible.filter((d) => answer(d, "unrecorded").find((r) => r.question === q.code).answered); console.log(` ${q.code.padEnd(4)} ${q.text.padEnd(36)} ${hit.length}/${eligible.length}` + `${hit.length ? "" : " <- no decision answers it"}`); } } ``` ``` 10 questions x 4 decisions -> 39 applicable questions [unrecorded] 18/39 answers, 21 unanswered, source: version=8 code=4 memory=6 [recorded] 39/39 answers, 0 unanswered, source: record=39 decisions answering each question in the unrecorded scheme: S1 why is it this way 2/4 S2 which alternative was eliminated 0/4 <- no decision answers it S3 what was the elimination based on 0/4 <- no decision answers it S4 what assumption was it given under 2/4 S5 when was it given 4/4 S6 who gave it 4/4 S7 what is the known cost 2/4 S8 is it still valid 4/4 S9 did it supersede an earlier one 0/4 <- no decision answers it S10 where does this number come from 0/3 <- no decision answers it ``` The recorded scheme's 39/39 is true by definition: if the fields are filled, every question is answered. This is not a finding; it is the zero point of the measurement scale. The unrecorded scheme answers 18 questions, and 12 of them come from the version history and the code. The six answers touching rationale, assumption, and consequence sit in the S1, S4, and S7 rows, which show 2 instead of 4 in the table: those three questions find an answer only in the two decisions where the decision-maker is still reachable. Memory standing in for a record depends on one person staying on the job. Four questions find no answer in any decision: which alternative was eliminated, what the elimination was based on, whether this decision superseded an earlier one, where this number comes from. These do not come from memory either; a criterion not written down at decision time is not remembered six months later. Fifteen of the 21 unanswered questions are these four. #### The Cost of a Record An answered question is not free. The run below actually produces the record text, counts how many lines each field takes, and multiplies by minutes per line. Two writing speeds are kept separate: header fields are looked up and written, body fields are written while thinking (**DR3**). For reading, each source's minutes per question is given (**DR4**). ```js // record/cost.mjs — the cost of writing a record and its yield per field import { QUESTIONS, DECISIONS, applicable, answer } from "./model.mjs"; const WIDTH = 76, SHORT_MIN = 0.4, LONG_MIN = 2.0; // minutes per line (DR3) const READ = { record: 2, code: 10, version: 8, memory: 25 }; // reading minutes per question (DR4) const SHORT = ["status", "date", "issuer", "supersedes"]; // one-line header fields const LONG = ["choice", "rationale", "assumption", "threshold", "alternative", "criterion", "consequence"]; const wrap = (text) => text.split(" ").reduce((s, w) => // the record text is really produced ((s.at(-1) + " " + w).trim().length > WIDTH ? s.push(w) : (s[s.length - 1] += " " + w), s), [""]); const lines = {}, minutes = {}; let totalLines = 0, totalMinutes = 0; const add = (f, n, m) => { lines[f] = (lines[f] ?? 0) + n; minutes[f] = (minutes[f] ?? 0) + n * m; totalLines += n; totalMinutes += n * m; }; for (const d of DECISIONS) { totalLines += 1; totalMinutes += SHORT_MIN; // the record's title line for (const f of SHORT) add(f, 1, SHORT_MIN); for (const f of LONG) if (d[f] !== null) add(f, 2 + wrap(d[f]).length, LONG_MIN); // blank line + heading + body } const N = DECISIONS.reduce((t, d) => t + applicable(d).length, 0); console.log(`${DECISIONS.length} records: ${totalLines} lines, ${totalMinutes.toFixed(0)} min; per record ` + `${(totalLines / DECISIONS.length).toFixed(1)} lines, ${(totalMinutes / DECISIONS.length).toFixed(0)} min`); const questionCount = Object.fromEntries(QUESTIONS.map((q) => [q.field, DECISIONS.filter((d) => d[q.field] !== null).length])); const items = [{ field: "header (4 fields)", min: SHORT.reduce((t, f) => t + minutes[f], 0), q: SHORT.reduce((t, f) => t + (questionCount[f] ?? 0), 0) }, ...LONG.filter((f) => questionCount[f]).map((f) => ({ field: f, min: minutes[f], q: questionCount[f] }))]; console.log("\nfield, writing minutes, questions it opens (sorted by yield per minute):"); let cumMin = 0, cumQ = 0; for (const r of items.sort((x, y) => y.q / y.min - x.q / x.min)) { cumMin += r.min; cumQ += r.q; console.log(` ${r.field.padEnd(19)} ${r.min.toFixed(1).padStart(5)} min ${r.q} q | ` + `cumulative ${String(cumQ).padStart(2)}/${N}, ${cumMin.toFixed(1).padStart(5)} min`); } console.log(` section answering no question in the set: ${(totalMinutes - cumMin).toFixed(1)} min (${(100 * (totalMinutes - cumMin) / totalMinutes).toFixed(0)}%)`); const readCost = (scheme, memMin = READ.memory) => DECISIONS.reduce((t, d) => t + answer(d, scheme).filter((r) => r.answered) .reduce((u, r) => u + (r.source === "memory" ? memMin : READ[r.source]), 0), 0); const answeredCount = (scheme) => DECISIONS.reduce((t, d) => t + answer(d, scheme).filter((r) => r.answered).length, 0); console.log("\nreading cost of asking the question set once:"); for (const scheme of ["unrecorded", "recorded"]) console.log(` ${scheme.padEnd(10)} ${answeredCount(scheme)} answers, ${readCost(scheme)} min, per answer ${(readCost(scheme) / answeredCount(scheme)).toFixed(1)} min`); const breakEven = (memMin) => (totalMinutes / (readCost("unrecorded", memMin) - answeredCount("unrecorded") * READ.record)).toFixed(2); console.log(`break-even (${answeredCount("unrecorded")} shared questions): if memory takes 25 min, ${breakEven(25)} rounds; if 5 min, ${breakEven(5)} rounds`); ``` ``` 4 records: 101 lines, 170 min; per record 25.3 lines, 43 min field, writing minutes, questions it opens (sorted by yield per minute): header (4 fields) 6.4 min 16 q | cumulative 16/39, 6.4 min rationale 24.0 min 4 q | cumulative 20/39, 30.4 min assumption 24.0 min 4 q | cumulative 24/39, 54.4 min threshold 18.0 min 3 q | cumulative 27/39, 72.4 min alternative 24.0 min 4 q | cumulative 31/39, 96.4 min criterion 24.0 min 4 q | cumulative 35/39, 120.4 min consequence 24.0 min 4 q | cumulative 39/39, 144.4 min section answering no question in the set: 25.6 min (15%) reading cost of asking the question set once: unrecorded 18 answers, 254 min, per answer 14.1 min recorded 39 answers, 78 min, per answer 2.0 min break-even (18 shared questions): if memory takes 25 min, 0.78 rounds; if 5 min, 1.73 rounds ``` Four records: 101 lines and 170 minutes. The real information is not in the total; it is in how the yield is distributed. The four header fields open 16 questions in 6.4 minutes; the remaining 23 questions demand 138 minutes. The status, date, issuer, and supersedes fields are single-line and call for looking up, not thinking. The first thing a team starting to keep records should write is not the rationale — it is these four fields. Fifteen percent of the total time goes to the section that answers no question in the set at all: the record's title and the section describing what the decision is. What the decision is can already be read from the code; that section makes the record readable, does not enter the measurement, and is kept short. For reading, the unrecorded scheme spends 14.1 minutes per answer, the recorded scheme 2.0. Over the 18 questions both schemes answer, the writing cost closes in 0.78 rounds if a memory answer takes 25 minutes, and in 1.73 rounds if it takes 5. The fivefold difference does not change the outcome: once the question set is asked twice, the record has paid for itself at both ends. #### Superseded Decision A record is not written once and done: when a decision changes, what happens to the record determines whether the old decision stays readable. The run below compares three versions of the same decision (**DR5**) across three record formats — one that overwrites the current record, one that overwrites and then digs the old text out of the version history, and one that keeps every version as its own record and links them with a **superseded-decision** field. The reading minutes are the model's input (**DR6**). ```js // record/superseded.mjs — superseded decision: does the old decision stay readable (DR5) const CHAIN = [ // three versions of the same decision, the model's input { version: "1", choice: "full copy at every branch", rationale: "outage tolerance was weighted heaviest", dropped: "once copy overhead was measured, the cost weight came out at 0.25", supersededBy: "2" }, { version: "2", choice: "read-only cache at the branch", rationale: "highest score under the measured weights", dropped: "volume exceeded capacity at two large branches", supersededBy: "3" }, { version: "3", choice: "copy at two branches, cache at the rest", rationale: "the branch over the threshold stayed at two", dropped: null, supersededBy: null }, ]; const QUESTIONS = [["G1", "what was being done then", "choice"], ["G2", "why was it chosen", "rationale"], ["G3", "why was it dropped", "dropped"], ["G4", "what replaced it", "supersededBy"]]; const RECOVERABLE = ["choice", "rationale"]; // fields that were written into the old file text const FORMAT = { "overwrite": null, "overwrite-dig": "dig", "supersede": "record" }; const MIN = { record: 2, dig: 12 }; // reading minutes per question (DR6) const current = CHAIN.length - 1; const read = (format, i, field) => { const source = i === current ? "record" : FORMAT[format]; return source === "record" || (source === "dig" && RECOVERABLE.includes(field)) ? source : null; }; const lineCount = (v) => Object.values(v).filter((x) => x !== null).length; for (const format of Object.keys(FORMAT)) { const kept = FORMAT[format] === "record" ? CHAIN.reduce((t, v) => t + lineCount(v), 0) : lineCount(CHAIN[current]); let answered = 0, total = 0, min = 0; const missing = []; for (const [i, v] of CHAIN.entries()) for (const [code, , field] of QUESTIONS) { if (v[field] === null) continue; total += 1; const s = read(format, i, field); if (s) { answered += 1; min += MIN[s]; } else missing.push(`${code}@${v.version}`); } console.log(`${format.padEnd(17)} kept ${String(kept).padStart(2)} lines | ` + `${answered}/${total} questions, ${min} min | unanswered: ${missing.join(" ") || "-"}`); } ``` ``` overwrite kept 3 lines | 2/10 questions, 4 min | unanswered: G1@1 G2@1 G3@1 G4@1 G1@2 G2@2 G3@2 G4@2 overwrite-dig kept 3 lines | 6/10 questions, 52 min | unanswered: G3@1 G4@1 G3@2 G4@2 supersede kept 13 lines | 10/10 questions, 20 min | unanswered: - ``` The overwrite format keeps 3 lines and answers 2 of 10 questions; the format using the superseded-decision field keeps 13 lines and answers all 10. The difference is 10 lines and 8 questions — nearly one question per line. The dig row is a finding on its own. The version history keeps the old text, so "what was being done then" and "why was it chosen" can be answered by digging: 6 answers instead of 2. The cost is 52 minutes, that is 8.7 minutes per answer, four times reading from a record. Digging does not open two questions at any cost: "why was it dropped" and "what replaced it." These two fields were never written in any version, because no one overwriting a decision writes down why they dropped it. The version history keeps what was written; it does not produce what was not. This is also where past decisions resurface: if the reason for dropping something sits nowhere, an alternative eliminated two years ago comes back as a new proposal. The job of the superseded-decision field is not to archive the old decision — it is to keep the reason it was dropped readable. #### Summary - A record's value is measured by how many questions it answers out of the question set; the same four decisions answered 18 of 39 questions in the unrecorded scheme and 39 of 39 with a structured record. - Twelve of those 18 answers come from the version history and the code; the six answers touching rationale, assumption, and consequence appeared only in the two decisions where the decision-maker was reachable. - Four questions found no answer in any decision: which alternative was eliminated, what the elimination was based on, whether it superseded an earlier one, where this number comes from — 15 of the 21 unanswered questions are these four. - Four records took 101 lines and 170 minutes; the four header fields opened 16 questions in 6.4 minutes while the remaining 23 questions demanded 138 minutes, and 15 percent of the total went to a section that answers no question. - The superseded-decision field kept 10 extra lines and answered 8 extra questions; digging through the version history raises written fields to 8.7 minutes per answer but does not open the reason for dropping something that was never written down. #### Next Step The record is what a decision looks like after it is made; everything this lesson measured started once the decision was already given. The "issuer" field in all four records holds a team name, but how that team reached the decision — who proposed it, who objected, whether the objection changed anything — does not enter the record. A decision that passes on verbal approval and one evaluated through a written proposal produce the same record. The next lesson measures that difference: the same decisions are run through two processes, and how many change direction during the evaluation round is counted. ### The Proposal and Evaluation Process Source: https://academia.sh/en/courses/architectural-documentation/proposal-and-evaluation-process Measuring how a decision gets made: the same decision set is run through verbal approval and through a written proposal plus an evaluation round, the rate of decisions that change direction under evaluation, the number of rounds, the number of roles involved, and the time elapsed are counted, a question set is applied to the record the process leaves behind to separate a correct, missing, or wrong answer, and the decisions the written process only delays are measured. The previous lesson measured how decisions get recorded, and left a team name in the "issuer" field of all four records. That field says who a decision came from, not how it came about. A decision that passes on verbal approval and one evaluated through a written proposal sent to every relevant role produce the same record; the difference between the two does not show up in the record. This lesson's question is: does the process itself change the decision, and at what cost if it does. "A written process is good" carries no decision. The same decision set is run through two schemes, and the rate of decisions that change direction under evaluation, the number of rounds, the number of roles involved, and the time elapsed are counted. #### The Same Decision, Three Processes The model has eight decisions, and behind each one stands one or two pieces of information: a fact known to someone in a particular part of the system but not to the person writing the proposal. Each piece of information has three properties: which role holds it, which evaluation round it surfaces in, and whether it can flip the decision. These three are the model's inputs (**DR7**); the scenario is fictional. The processes are separated by access to pieces of information. In verbal approval there are two roles — the proposer and the approver — so only what those two know surfaces. In the written-proposal scheme, the proposal goes to six roles. The second distinction is in the number of rounds: in one scheme a round repeats only when the decision changes, in the other at least two rounds are run regardless. The durations and round rules are also the model's inputs (**DR8**). ```js // proposal/process.mjs — the same decision set through three processes: verbal approval and two written-proposal schemes // The decisions and pieces of information are the model's input (DR7); the scenario is fictional. export const DECISIONS = [ { name: "member-data-retention", info: [ { role: "business", round: 1, flips: true, note: "the regulation requires deletion after two years" }] }, { name: "branch-reporting", info: [ { role: "business", round: 1, flips: false, note: "the branch wants its month-end report same-day" }, { role: "budget", round: 2, flips: true, note: "central reporting requires a second server" }] }, { name: "catalog-upgrade", info: [ { role: "integration", round: 1, flips: true, note: "the new version breaks four fields in the mapping layer" }, { role: "budget", round: 1, flips: false, note: "the upgrade requires a license difference" }] }, { name: "loan-limit", info: [ { role: "loan", round: 1, flips: false, note: "the limit varies by branch type" }] }, { name: "search-index", info: [ { role: "integration", round: 2, flips: false, note: "the index holds the same data as the catalog" }] }, { name: "backup-window", info: [ { role: "proposer", round: 1, flips: false, note: "load drops outside the window" }] }, { name: "card-printing", info: [ { role: "budget", round: 1, flips: false, note: "unit cost is lower with outside printing" }, { role: "business", round: 2, flips: true, note: "the card must be printed the same day as member registration" }] }, { name: "notification-channel", info: [ { role: "loan", round: 1, flips: true, note: "a third of members have already picked a channel" }] }, ]; // In verbal approval, only what the two roles in the meeting know surfaces; in the written // scheme, the proposal goes to every role. The durations and round rules are the model's input (DR8). export const PROCESS = { "verbal-approval": { role: ["proposer", "manager"], writing: 0, personMin: 30, days: 0, minRounds: 1, maxRounds: 1 }, "written-one-round": { role: ["proposer", "manager", "business", "loan", "integration", "budget"], writing: 60, personMin: 20, days: 2, minRounds: 1, maxRounds: 3 }, "written-two-round": { role: ["proposer", "manager", "business", "loan", "integration", "budget"], writing: 60, personMin: 20, days: 2, minRounds: 2, maxRounds: 3 }, }; export function run(decision, processName) { const p = PROCESS[processName]; const seen = []; let round = 0, flips = 0; while (round < p.maxRounds) { round += 1; const fresh = decision.info.filter((b) => b.round === round && p.role.includes(b.role)); seen.push(...fresh); if (fresh.some((b) => b.flips)) { flips += 1; continue; } if (round >= p.minRounds) break; } return { round, flips, personMin: p.writing + round * p.role.length * p.personMin, days: round * p.days, seen, missed: decision.info.filter((b) => !seen.includes(b)) }; } if (import.meta.url.endsWith(process.argv[1].split("/").pop())) { console.log(`${DECISIONS.length} decisions, ${DECISIONS.flatMap((d) => d.info).length} pieces of information, ` + `${DECISIONS.flatMap((d) => d.info).filter((b) => b.flips).length} of them flip the decision`); for (const name of Object.keys(PROCESS)) { const r = DECISIONS.map((d) => run(d, name)); const changed = r.map((x, i) => x.flips && DECISIONS[i].name).filter(Boolean); const missed = r.flatMap((x) => x.missed); console.log(`\n[${name}] ${PROCESS[name].role.length} roles, rounds ${PROCESS[name].minRounds}-${PROCESS[name].maxRounds}`); console.log(` changed decisions ${changed.length}/${DECISIONS.length}: ${changed.join(", ") || "-"}`); console.log(` average rounds ${(r.reduce((t, x) => t + x.round, 0) / r.length).toFixed(2)}, ` + `total ${r.reduce((t, x) => t + x.personMin, 0)} person-min, ${r.reduce((t, x) => t + x.days, 0)} days`); console.log(` information that never surfaced ${missed.length}, of which would have flipped the decision ` + `${missed.filter((b) => b.flips).length}`); } } ``` ``` 8 decisions, 11 pieces of information, 5 of them flip the decision [verbal-approval] 2 roles, rounds 1-1 changed decisions 0/8: - average rounds 1.00, total 480 person-min, 0 days information that never surfaced 10, of which would have flipped the decision 5 [written-one-round] 6 roles, rounds 1-3 changed decisions 3/8: member-data-retention, catalog-upgrade, notification-channel average rounds 1.38, total 1800 person-min, 22 days information that never surfaced 3, of which would have flipped the decision 2 [written-two-round] 6 roles, rounds 2-3 changed decisions 5/8: member-data-retention, branch-reporting, catalog-upgrade, card-printing, notification-channel average rounds 2.25, total 2640 person-min, 36 days information that never surfaced 0, of which would have flipped the decision 0 ``` In verbal approval, none of the eight decisions changes direction. The reason is not that the decisions are correct — it is that none of the five pieces of information that could flip a decision are in that room. The verbal process never sees ten pieces of information, and five of them are the ones that would flip a decision. The result a process produces is bounded by what the participating roles know; where that boundary sits is invisible once the process is over. The written-proposal scheme flips three of the same eight decisions. These three were not chosen by chance: in each of them, the information that flipped the decision sat in a role the proposal writer had no access to — in the regulation, in the mapping layer, in the member records. What the written process does is not think a decision through better; it is bring into the discussion the role that holds the information capable of flipping it. The two pieces of information the one-round scheme misses are a separate finding. In the branch-reporting and card-printing decisions, the information that came up in the first round did not flip the decision, so a second round never opened, and the two pieces of information due to surface in the second round went unseen. The scheme that runs at least two rounds flips five of the same decisions, and the missed information drops to zero. Round count is a separate variable determining whether information surfaces: reading a proposal once is not the same as reading it twice. #### Missing Answers and Wrong Answers Every process leaves a record behind, and someone arriving six months later asks that record questions. The measure here goes one step past the previous lesson's: a question can be left unanswered, but it can also be answered wrong. If a process assumes it collected every objection, it says "there was no objection" — when in fact the objection never arose because it was never asked. What counts as which state is the model's input (**DR9**): if there is information that never surfaced, the answer to "was there an objection" is wrong, and the answer to "what information was evaluated" is missing. ```js // proposal/questions.mjs — the question set applied to the three processes; an answer can be correct, missing, or wrong import { DECISIONS, PROCESS, run } from "./process.mjs"; // The rule that determines the status of an answer is the model's input (DR9). const QUESTIONS = [ ["Y1", "who saw it", () => "correct"], ["Y2", "was there an objection", (r) => (r.missed.length ? "wrong" : "correct")], ["Y3", "why did the decision go this way", (r) => (r.missed.some((b) => b.flips) ? "wrong" : "correct")], ["Y4", "what information was evaluated", (r) => (r.missed.length ? "missing" : "correct")], ["Y5", "how many rounds did it take to finalize", (r, p) => (p.writing ? "correct" : "missing")], ]; const report = {}; for (const name of Object.keys(PROCESS)) { const count = { correct: 0, missing: 0, wrong: 0 }; const questionStatus = {}; for (const d of DECISIONS) { const r = run(d, name); for (const [code, , rule] of QUESTIONS) { const status = rule(r, PROCESS[name]); count[status] += 1; questionStatus[code] = (questionStatus[code] ?? []).concat(status); } } const runs = DECISIONS.map((d) => run(d, name)); report[name] = { count, questionStatus, personMin: runs.reduce((t, x) => t + x.personMin, 0), days: runs.reduce((t, x) => t + x.days, 0), changed: runs.filter((x) => x.flips).length, idlePersonMin: runs.filter((x) => !x.flips).reduce((t, x) => t + x.personMin, 0), idleDays: runs.filter((x) => !x.flips).reduce((t, x) => t + x.days, 0) }; } const N = DECISIONS.length * QUESTIONS.length; console.log(`question set ${QUESTIONS.length} questions x ${DECISIONS.length} decisions = ${N} answers`); for (const [name, r] of Object.entries(report)) console.log(` ${name.padEnd(19)} correct ${String(r.count.correct).padStart(2)} missing ` + `${String(r.count.missing).padStart(2)} wrong ${String(r.count.wrong).padStart(2)}`); console.log("\nnot-correct answers per question (w = wrong, m = missing):"); console.log(" ".repeat(43) + Object.keys(PROCESS).map((a) => a.padStart(19)).join("")); for (const [code, text] of QUESTIONS) console.log(` ${code} ${text.padEnd(38)}` + Object.values(report).map((r) => { const w = r.questionStatus[code].filter((d) => d === "wrong").length; const m = r.questionStatus[code].filter((d) => d === "missing").length; return (w ? `${w}w` : m ? `${m}m` : "-").padStart(19); }).join("")); console.log("\nwhat the extra cost buys:"); const names = Object.keys(PROCESS); for (let i = 1; i < names.length; i++) { const a = report[names[i - 1]], b = report[names[i]]; const extraChanged = b.changed - a.changed; console.log(` ${names[i - 1]} -> ${names[i]}: +${b.personMin - a.personMin} person-min, +${b.days - a.days} days, ` + `+${extraChanged} decisions -> per flipped decision ${((b.personMin - a.personMin) / extraChanged).toFixed(0)} person-min`); } console.log("\nspent on decisions that did not change (pure delay):"); for (const [name, r] of Object.entries(report)) console.log(` ${name.padEnd(19)} ${DECISIONS.length - r.changed} decisions, ${r.idlePersonMin} person-min, ${r.idleDays} days`); ``` ``` question set 5 questions x 8 decisions = 40 answers verbal-approval correct 13 missing 15 wrong 12 written-one-round correct 32 missing 3 wrong 5 written-two-round correct 40 missing 0 wrong 0 not-correct answers per question (w = wrong, m = missing): verbal-approval written-one-round written-two-round Y1 who saw it - - - Y2 was there an objection 7w 3w - Y3 why did the decision go this way 5w 2w - Y4 what information was evaluated 7m 3m - Y5 how many rounds did it take to finalize 8m - - what the extra cost buys: verbal-approval -> written-one-round: +1320 person-min, +22 days, +3 decisions -> per flipped decision 440 person-min written-one-round -> written-two-round: +840 person-min, +14 days, +2 decisions -> per flipped decision 420 person-min spent on decisions that did not change (pure delay): verbal-approval 8 decisions, 480 person-min, 0 days written-one-round 5 decisions, 900 person-min, 10 days written-two-round 3 decisions, 900 person-min, 12 days ``` The distribution of the forty answers gives 13, 32, and 40 correct across the three processes. But the real number is not in the correct column — it is in the wrong one. Verbal approval answers twelve of the forty questions wrong; fifteen questions get no answer at all. The difference matters: an unanswered question keeps getting asked, a wrongly answered question closes. The sentence "no one objected to this decision" looks true because the objection was never asked for, and once that sentence enters the record, no one asks again. The per-question table shows where the wrong answers pile up. Y1 is correct in all three processes: who took part is known under every scheme, because participation is the process itself. Y2 and Y3 break down together with the information that never surfaced — seven and five decisions in verbal approval, three and two in the one-round written scheme. Y4 and Y5 produce missing, not wrong: which information was evaluated and how many rounds it took to finalize stay unanswered when no record was kept, but they are not invented. #### The Cost Paid and Pure Delay The cost of the written process is in the output's last two sections. Moving from verbal approval to the one-round written scheme adds 1320 person-minutes and 22 calendar days, and in return flips three decisions: 440 person-minutes per flipped decision. Making a second round mandatory adds another 840 person-minutes and 14 days and flips two more decisions: 420 person-minutes. The unit cost of the two transitions comes out nearly the same, meaning the second round does not have diminishing returns relative to the first. In this model, the return on adding rounds is at the same level as the return on adding roles. The last table is the other side of the coin. In the one-round scheme, five decisions never changed direction, and those five decisions ate 900 person-minutes and 10 calendar days. In the two-round scheme, three decisions did not change, again 900 person-minutes and 12 days. This is the cost of the decisions the process delays but does not change, and it is the measured form of the objection raised against written process. The objection is right: a substantial share of the time spent flips no decision at all. The answer to that objection is this: which decision will flip is known only after the process runs. The notification-channel decision looked like a three-line matter and flipped; the search-index decision looked contested and came out the same after two rounds. Pure delay is the price of not being able to know in advance which decision will flip. What is measurable is not eliminating the delay — it is knowing the person-minutes per flipped decision. #### Summary - Verbal approval flipped none of the eight decisions; the reason is not that the decisions were correct — it is that the five pieces of information capable of flipping a decision were not in a two-person room. Ten pieces of information were never seen. - The written-proposal scheme flipped three of the same decisions; the scheme running at least two rounds flipped five and dropped the missed information to zero — round count is a variable independent of role count. - The question set gave 13, 32, and 40 correct answers; the real distinction is in the wrong answers: verbal approval answered twelve questions wrong and fifteen missing, and a wrong answer closes the question. - Wrong answers piled up on "was there an objection" and "why did the decision go this way"; fields with no record produced missing, not wrong, answers. - The extra cost's return sits at the same level across both transitions: 440 and 420 person-minutes per flipped decision; against that, in the one-round scheme, five decisions spent 900 person-minutes and 10 days without ever changing direction. #### Next Step When an evaluation round flips a decision, what remains is the information that it "changed" — not what was traded for what. The catalog-upgrade decision flipped because of four fields in the mapping layer; in the record, that is a one-line rationale. But behind that decision stood multiple alternatives, multiple qualities, and every alternative's standing on every quality. The previous course measured how that table gets built and how the weights determine the winner. What is left open is whether the table itself makes it into the record: can a third person, looking at the table in the record, arrive at the same result. The next lesson measures that. ### Trade-Off Analysis Source: https://academia.sh/en/courses/architectural-documentation/trade-off-analysis Measuring the recorded state of the table where alternatives are scored along quality axes: whether a third person can arrive at the same result from the recorded table is counted, a missing criterion, an unwritten weight, and an unsourced score are separated as three distinct flaws, each flaw is fixed one at a time to measure the rate of reproducible decisions, and the cost of completion in minutes is computed per gained analysis. The previous lesson measured an evaluation round flipping a decision, and left behind a one-line rationale of the form "changed because of this information." Behind that line stood multiple alternatives, multiple qualities, and every alternative's standing on every quality. A **trade-off analysis** is the name of that table. How the table gets built, how the weights determine the winner, and how resilient the decision is under a sweep of the weight space were measured in the previous course. They are not repeated here. This lesson's question is different: once the analysis is done and recorded, can a third person reading that record arrive at the same result. That is the quantity being measured — **reproducibility**. #### The Recorded Table and the Third Person For an analysis to be reproducible, the record must contain four things: alternatives, criteria, scores, and weights. In practice, three flaws show up, and each breaks it a different way. **Missing criterion.** A quality that influenced the decision never entered the table at all. The table looks complete, because the missing column leaves no blank spot behind — it is not there at all. **Unwritten weight.** The criteria are written, the weights are not. The reader is forced to make an assumption, and the assumption they will make is equal weight; as measured in the previous course, equal weight is also a weight set — it just was not stated as a choice. **Unsourced score.** The score is written, but where it came from is not. This flaw does not change the winner — it blocks verification: the reader arrives at the same result, but cannot test whether the result they arrived at is correct. The model below holds five analysis records. Alongside each record sits the real information that never made it into the record, as the model's input: the missing criterion's name and scores, and the weight vector the decision actually rested on (**DR10**). The third person does not see any of this; they only read the record. The scenario is fictional. ```js // tradeoff/table.mjs — trade-off analyses as they made it into the record; the scenario is fictional // The recorded table, the missing criterion, and the actual weight vector are the model's input (DR10). export const ANALYSES = [ { name: "search-index", options: ["in-catalog", "separate-service", "external-service"], criteria: ["response-time", "operating-load", "cost"], scores: [[2, 2, 2], [5, 4, 3], [4, 5, 2]], weights: null, unsourced: 2, missing: { name: "data-freshness", scores: [5, 2, 2] }, actual: [0.20, 0.15, 0.15, 0.50], recorded: "in-catalog" }, { name: "report-generation", options: ["central", "branch", "nightly-batch"], criteria: ["freshness", "server-load", "development-time"], scores: [[4, 2, 4], [5, 4, 2], [2, 5, 5]], weights: [0.50, 0.30, 0.20], unsourced: 1, missing: null, actual: [0.50, 0.30, 0.20], recorded: "branch" }, { name: "member-identity", options: ["central-service", "branch-table", "daily-copy"], criteria: ["consistency", "outage-tolerance", "operating-load"], scores: [[5, 2, 3], [2, 5, 4], [3, 4, 2]], weights: null, unsourced: 0, missing: null, actual: [0.55, 0.25, 0.20], recorded: "central-service" }, { name: "penalty-engine", options: ["loan-service", "catalog-module", "branch-setting"], criteria: ["modifiability", "setup-load"], scores: [[5, 3], [2, 5], [3, 4]], weights: [0.60, 0.40], unsourced: 2, missing: { name: "branch-exception", scores: [2, 3, 5] }, actual: [0.35, 0.25, 0.40], recorded: "branch-setting" }, { name: "backup-path", options: ["night-window", "continuous-stream", "weekly-full"], criteria: ["data-loss-window", "operating-load", "cost"], scores: [[4, 4, 4], [5, 2, 2], [2, 5, 5]], weights: [0.45, 0.30, 0.25], unsourced: 3, missing: null, actual: [0.45, 0.30, 0.25], recorded: "night-window" }, ]; export const winner = (options, scores, w) => options .map((name, i) => ({ name, total: scores[i].reduce((t, p, j) => t + p * w[j], 0) })) .sort((a, b) => b.total - a.total)[0].name; export const equal = (n) => Array(n).fill(1 / n); // The table the reader has: no missing criterion, and an unwritten weight is assumed equal. export function asRead(a, fix = {}) { const criteria = fix.addCriterion && a.missing ? [...a.criteria, a.missing.name] : [...a.criteria]; const scores = a.options.map((_, i) => fix.addCriterion && a.missing ? [...a.scores[i], a.missing.scores[i]] : [...a.scores[i]]); let w = fix.writeWeight ? a.actual.slice(0, criteria.length) : a.weights ? a.weights.slice(0, criteria.length) : equal(criteria.length); while (w.length < criteria.length) w = [...w, 1 / criteria.length]; // default for the added criterion const s = w.reduce((t, x) => t + x, 0); return { criteria, scores, w: w.map((x) => x / s) }; } if (import.meta.url.endsWith(process.argv[1].split("/").pop())) { const k = (fn) => ANALYSES.filter(fn).length; console.log(`${ANALYSES.length} analyses; flaws: missing criterion ${k((a) => a.missing)}, ` + `unwritten weight ${k((a) => !a.weights)}, unsourced score ` + `${ANALYSES.reduce((t, a) => t + a.unsourced, 0)}\n`); console.log("reproduction with the recorded table:"); for (const a of ANALYSES) { const r = asRead(a); const flaws = [a.missing && "missing criterion", !a.weights && "unwritten weight", a.unsourced && `${a.unsourced} unsourced score`].filter(Boolean); console.log(` ${a.name.padEnd(17)} recorded ${a.recorded.padEnd(17)} read ` + `${winner(a.options, r.scores, r.w).padEnd(17)} flaws: ${flaws.join(", ") || "-"}`); } const FIXES = [["as recorded", {}], ["once weights are written", { writeWeight: true }], ["once the missing criterion is added", { addCriterion: true }], ["both", { writeWeight: true, addCriterion: true }], ["once sources are written too", { writeWeight: true, addCriterion: true, writeSource: true }]]; console.log("\neffect of each fix (reproducible / also verifiable):"); for (const [name, fix] of FIXES) { const reproducible = ANALYSES.filter((a) => { const r = asRead(a, fix); return winner(a.options, r.scores, r.w) === a.recorded; }); const verifiable = reproducible.filter((a) => fix.writeSource || a.unsourced === 0); console.log(` ${name.padEnd(36)} ${reproducible.length}/${ANALYSES.length} ${verifiable.length}/${ANALYSES.length}`); } } ``` ``` 5 analyses; flaws: missing criterion 2, unwritten weight 2, unsourced score 8 reproduction with the recorded table: search-index recorded in-catalog read separate-service flaws: missing criterion, unwritten weight, 2 unsourced score report-generation recorded branch read branch flaws: 1 unsourced score member-identity recorded central-service read branch-table flaws: unwritten weight penalty-engine recorded branch-setting read loan-service flaws: missing criterion, 2 unsourced score backup-path recorded night-window read night-window flaws: 3 unsourced score effect of each fix (reproducible / also verifiable): as recorded 2/5 0/5 once weights are written 3/5 1/5 once the missing criterion is added 3/5 0/5 both 5/5 1/5 once sources are written too 5/5 5/5 ``` Two of the five analyses are reproducible as recorded. In three, the reader arrives at a winner different from the one written in the record: the search-index record says in-catalog but the table gives separate-service, the member-identity record says central-service but the table gives branch-table. This is not the reader misunderstanding the decision — the recorded table genuinely produces that result. The decision itself is correct; the record is wrong. The fix table separates the weight of the three flaws. Writing the weights raises the number of reproducible analyses from 2 to 3; adding the missing criterion also from 2 to 3. Neither fix alone is enough, because search-index carries both flaws at once: once the weight is written, the missing column is still absent; once the column is added, the weight is still assumed equal. Once both are fixed, all five of the five analyses become reproducible. The flaws do not add — they collide. The third flaw's effect is in the second column. Even once all five analyses are reproducible, only one is verifiable: in the remaining four, the scores' source is not written, so the reader arrives at the same winner but cannot test whether the scores are correct. The gap between these two states looks small and is large. A reproducible analysis shows that a decision is **consistent**; a verifiable analysis shows that a decision is **well-founded**. The backup-path record gives the correct winner as recorded, but none of its three scores says where it came from — it is possible for all three to be wrong and the winner to come out the same regardless. #### Question Set and Completion The same five records are run against a question set. As separated in the previous lesson, an answer can be correct, missing, or wrong; the distinction matters especially here, because a table carrying a missing criterion answers "which criteria were used" **wrong**: the list is incomplete but looks complete. ```js // tradeoff/questions.mjs — the question set applied to the analysis records; the cost of completion is measured import { ANALYSES, winner, asRead } from "./table.mjs"; const reproducible = (a, fix) => { const r = asRead(a, fix); return winner(a.options, r.scores, r.w) === a.recorded; }; const QUESTIONS = [ ["O1", "which criteria were used", (a, c) => (!c && a.missing ? "wrong" : "correct")], ["O2", "what were the criteria's weights", (a, c) => (!c && !a.weights ? "missing" : "correct")], ["O3", "where did this score come from", (a, c) => (!c && a.unsourced ? "missing" : "correct")], ["O4", "does the same table give the same result", (a, c) => (reproducible(a, c ? { writeWeight: true, addCriterion: true } : {}) ? "correct" : "wrong")], ["O5", "which alternatives were evaluated", () => "correct"], ]; const state = {}; for (const complete of [false, true]) { const count = { correct: 0, missing: 0, wrong: 0 }; for (const a of ANALYSES) for (const [code, , rule] of QUESTIONS) { const status = rule(a, complete); count[status] += 1; (state[code] ??= { false: [], true: [] })[complete].push(status); } console.log(`${complete ? "after completion " : "as recorded "}: ` + `correct ${count.correct}, missing ${count.missing}, wrong ${count.wrong} ` + `(${QUESTIONS.length} questions x ${ANALYSES.length} analyses)`); } console.log("\nnot-correct answers per question as recorded (w = wrong, m = missing):"); for (const [code, text] of QUESTIONS) { const s = state[code][false]; const w = s.filter((x) => x === "wrong").length, m = s.filter((x) => x === "missing").length; console.log(` ${code} ${text.padEnd(40)} ${w ? `${w}w` : m ? `${m}m` : "-"}`); } // The completion times are the model's input (DR11). const MIN = { weight: 15, criterion: 20, source: 5 }; const count = { weight: ANALYSES.filter((a) => !a.weights).length, criterion: ANALYSES.filter((a) => a.missing).length, source: ANALYSES.reduce((t, a) => t + a.unsourced, 0) }; const total = Object.entries(count).reduce((t, [k, n]) => t + n * MIN[k], 0); console.log("\ncompletion cost:"); for (const [k, n] of Object.entries(count)) console.log(` ${k.padEnd(9)} ${n} items x ${MIN[k]} min = ${n * MIN[k]} min`); const before = ANALYSES.filter((a) => reproducible(a, {})).length; const after = ANALYSES.filter((a) => reproducible(a, { writeWeight: true, addCriterion: true })).length; console.log(` total ${total} min; reproducible ${before}/${ANALYSES.length} -> ` + `${after}/${ANALYSES.length}, per gained analysis ${(total / (after - before)).toFixed(1)} min`); ``` ``` as recorded : correct 14, missing 6, wrong 5 (5 questions x 5 analyses) after completion : correct 25, missing 0, wrong 0 (5 questions x 5 analyses) not-correct answers per question as recorded (w = wrong, m = missing): O1 which criteria were used 2w O2 what were the criteria's weights 2m O3 where did this score come from 4m O4 does the same table give the same result 3w O5 which alternatives were evaluated - completion cost: weight 2 items x 15 min = 30 min criterion 2 items x 20 min = 40 min source 8 items x 5 min = 40 min total 110 min; reproducible 2/5 -> 5/5, per gained analysis 36.7 min ``` Fourteen of the twenty-five answers are correct, six missing, five wrong. The distribution of wrong answers shows the flaws' nature: missing criterion and non-reproducibility produce wrong, unwritten weight and unsourced score produce missing. That difference determines the reader's behavior. Someone who sees that a weight is not written knows something is missing and can ask; someone looking at a missing criterion sees nothing, because the table looks complete. Completion costs 110 minutes: two weight vectors, two criterion columns, and eight score sources. At the end of that time the number of reproducible analyses rises from 2 to 5, that is, 36.7 minutes per gained analysis. The number itself depends on the model's inputs; what it carries is in the ratios. Writing the score sources takes 36 percent of the total time and does not change reproducibility at all — it only raises verifiability from one to five. Writing the weights, by contrast, touches three analyses in 30 minutes. The order that follows from this is: missing criterion first, then weight, then score source last. That order comes not from the flaws' cost but from how detectable they are. The most insidious flaw is not the cheapest one — it is the one that does not show itself. #### Summary - A trade-off analysis's recorded state is measured by whether a third person can arrive at the same winner; two of the five records were reproducible as recorded. - The three flaws break it differently: a missing criterion makes the table look complete, an unwritten weight pushes the reader into assuming equal weight, an unsourced score blocks verification without changing the winner. - Writing the weights raised the reproducible count from 2 to 3, and adding the missing criterion also from 2 to 3; the analysis carrying both flaws became reproducible only once both were fixed, and the count rose to 5. - Reproducibility and verifiability are separate measures: even once all five analyses were reproducible, only one was verifiable without the score sources written. - In the question set, missing criterion and non-reproducibility produced wrong answers, unwritten weight and unsourced score produced missing answers; completion cost 110 minutes, 36.7 minutes per gained analysis. #### Next Step Everything measured up to this point concerned a decision already made: its record, the process that produced it, and the table it rested on. In all three, the decision settles at some point and the record closes. But something stays open next to every decision: the chance that the assumption it rests on does not hold. In the search-index analysis, the data-freshness criterion was given a weight of 0.50 because freshness was assumed critical; if that assumption is wrong, the decision is wrong too, and only time reveals it. This is not a decision to record — it is an uncertainty to track. The next lesson models that uncertainty with likelihood and impact and runs it across a period, counting the realization rate of recorded risk and the risk that occurs despite never being recorded at all. ### The Risk Register Source: https://academia.sh/en/courses/architectural-documentation/risk-record Tracking uncertainty through a record: risks are modeled with likelihood and impact, a period is run with a generator whose seed is visible, the recorded likelihood estimate is compared against the measured realization rate in bands, risks that occur despite never being recorded are counted, and the effect of review frequency on the number of risks caught before they occur is measured along with its person-minute cost. The previous three lessons measured a given decision's record, the process that produced it, and the table it rested on. In all three, the decision settles at some point and the record closes. Something stays open: the chance that the assumption the decision rests on does not hold. Giving a criterion a weight of 0.50 in a trade-off analysis rests on the assumption that quality is critical; if the assumption is wrong, the decision is wrong too, and only time reveals it. This is not a decision to record — it is an uncertainty to track, and it needs its own record. How risk splits the test budget was measured in test planning; it is not repeated here. The question here concerns the record itself: what a record catches when it is kept, what escapes it, and how the frequency at which the record is reviewed changes that. #### The Risk Register and a Period A **risk register** is modeled with two fields: **likelihood** and **impact**. Likelihood is an estimate written into the record; impact is the scale of how much work it will cause once it happens. Of the fourteen risks in the model, eight are in the record and six are not — the unrecorded ones exist in the system, no one wrote them down. Every risk has two likelihoods kept separate: the estimate written into the record, and the actual likelihood that determines what is observed across the period. The second is the model's input (**DR12**); the person keeping the record does not see it — they only measure it as periods pass. Five hundred periods are run, the generator is written in-house, and the seed is visible. ```js // risk/record.mjs — a risk record is modeled with likelihood and impact, one period is run // [name, recorded estimate (null if not in the record), actual likelihood, impact]; all model input (DR12) export const RISKS = [ ["catalog-version-breakage", 0.60, 0.55, 4], ["branch-network-outage", 0.70, 0.75, 3], ["cache-freshness-overrun", 0.50, 0.15, 3], ["identity-service-outage", 0.40, 0.35, 5], ["backup-window-overrun", 0.30, 0.45, 2], ["member-data-mismatch", 0.20, 0.20, 5], ["card-printing-delay", 0.15, 0.10, 2], ["rule-engine-growth", 0.10, 0.50, 3], ["search-index-inconsistency", null, 0.40, 3], ["report-server-shortfall", null, 0.35, 4], ["notification-queue-backlog", null, 0.30, 2], ["license-renewal-delay", null, 0.25, 4], ["branch-staff-training-gap", null, 0.20, 2], ["loan-limit-exception", null, 0.15, 1], ].map(([name, estimate, actual, impact]) => ({ name, estimate, actual, impact, recorded: estimate !== null })); export const MONTHS = 12; // one period export const random = (seed) => () => { // the generator is written here, the seed is visible seed = (seed + 0x6d2b79f5) | 0; let t = Math.imul(seed ^ (seed >>> 15), 1 | seed); t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; // One period: each risk occurs at its actual likelihood; if it occurs, the month is spread evenly. export function period(rnd) { return RISKS.map((r) => { const occurred = rnd() < r.actual; return { risk: r, occurred, month: occurred ? 1 + Math.floor(rnd() * MONTHS) : null }; }); } export const BAND = (p) => (p >= 0.5 ? "high" : p >= 0.2 ? "mid" : "low"); if (import.meta.url.endsWith(process.argv[1].split("/").pop())) { const TRIALS = 500, SEED = 20240517, rnd = random(SEED); const hits = RISKS.map(() => 0); for (let i = 0; i < TRIALS; i++) period(rnd).forEach((s, j) => { if (s.occurred) hits[j] += 1; }); const measured = hits.map((h) => h / TRIALS); const recorded = RISKS.filter((r) => r.recorded); const total = hits.reduce((t, h) => t + h, 0); const unrecordedHits = RISKS.reduce((t, r, j) => t + (r.recorded ? 0 : hits[j]), 0); console.log(`${RISKS.length} risks: ${recorded.length} recorded, ${RISKS.length - recorded.length} ` + `unrecorded; ${TRIALS} periods, seed ${SEED}`); console.log(`realized risk ${total}, of which ${unrecordedHits} were unrecorded ` + `(${(100 * unrecordedHits / total).toFixed(1)}%)\n`); console.log("recorded estimate band vs. measured realization rate:"); for (const band of ["high", "mid", "low"]) { const group = RISKS.map((r, j) => ({ r, m: measured[j] })).filter((x) => x.r.recorded && BAND(x.r.estimate) === band); console.log(` ${band.padEnd(7)} ${group.length} risks, average estimate ` + `${(group.reduce((t, x) => t + x.r.estimate, 0) / group.length).toFixed(2)}, measured rate ` + `${(group.reduce((t, x) => t + x.m, 0) / group.length).toFixed(2)}`); } console.log("\nrecorded risks with the largest gap between estimate and measured rate:"); for (const x of RISKS.map((r, j) => ({ r, m: measured[j] })).filter((x) => x.r.recorded) .sort((a, b) => Math.abs(b.m - b.r.estimate) - Math.abs(a.m - a.r.estimate)).slice(0, 3)) console.log(` ${x.r.name.padEnd(28)} estimate ${x.r.estimate.toFixed(2)} measured ${x.m.toFixed(2)} ` + `gap ${(x.m - x.r.estimate > 0 ? "+" : "") + (x.m - x.r.estimate).toFixed(2)} impact ${x.r.impact}`); console.log("\nunrecorded risks that were realized (measured rate x impact):"); for (const x of RISKS.map((r, j) => ({ r, m: measured[j] })).filter((x) => !x.r.recorded) .sort((a, b) => b.m * b.r.impact - a.m * a.r.impact)) console.log(` ${x.r.name.padEnd(28)} measured ${x.m.toFixed(2)} impact ${x.r.impact} ` + `product ${(x.m * x.r.impact).toFixed(2)}`); } ``` ``` 14 risks: 8 recorded, 6 unrecorded; 500 periods, seed 20240517 realized risk 2348, of which 808 were unrecorded (34.4%) recorded estimate band vs. measured realization rate: high 3 risks, average estimate 0.60, measured rate 0.49 mid 3 risks, average estimate 0.30, measured rate 0.34 low 2 risks, average estimate 0.13, measured rate 0.30 recorded risks with the largest gap between estimate and measured rate: rule-engine-growth estimate 0.10 measured 0.50 gap +0.40 impact 3 cache-freshness-overrun estimate 0.50 measured 0.15 gap -0.35 impact 3 backup-window-overrun estimate 0.30 measured 0.45 gap +0.15 impact 2 unrecorded risks that were realized (measured rate x impact): report-server-shortfall measured 0.35 impact 4 product 1.41 search-index-inconsistency measured 0.39 impact 3 product 1.18 license-renewal-delay measured 0.24 impact 4 product 0.98 notification-queue-backlog measured 0.29 impact 2 product 0.59 branch-staff-training-gap measured 0.18 impact 2 product 0.36 loan-limit-exception measured 0.15 impact 1 product 0.15 ``` The first number gives the record's coverage: 808 of the 2348 realized risks, that is 34.4 percent, were never in the record at all. A risk register's first measurable quality is not how accurately its contents were estimated — it is how much was left out. Someone looking at an eight-item register has never seen a third of the events that actually happened. The band table gives the second number and shows the record's accuracy within itself. The high band's average estimate is 0.60, its measured rate 0.49; the low band's estimate is 0.13, its measured rate 0.30. The bands do not even preserve the ordering: risks marked low occur at a rate close to the mid band. A risk register's likelihood column, unless compared against the measured rate, is not a ranking — it is a list of impressions. The gap table shows where this breakdown comes from. rule-engine-growth was estimated at 0.10, measured at 0.50; cache-freshness-overrun was estimated at 0.50, measured at 0.15. The two gaps run in opposite directions and both are large. Someone looking at the record prepares for the second risk and runs into the first. This is the number that shows why reviewing the record is not a formality: an estimate written once and left alone does not reveal its own wrongness. The last table ranks the unrecorded risks by impact. The top two items, report-server-shortfall and search-index-inconsistency, outscore many of the recorded items in product. They are not unrecorded because they are unimportant — they are unrecorded because no one thought of them. #### Review Frequency An unrecorded risk does not enter the record on its own; someone has to notice it. Noticing happens at the moments the record is reviewed. The run below compares four frequencies over the same periods: never reviewing, once a year, once a quarter, and once a month. The probability of noticing an unrecorded risk in one review and the person-minutes of a review are the model's inputs (**DR13**). What is measured is not the number of risks noticed — it is the number of risks noticed **before** they occur. Recording a risk after it has occurred fixes the record; it does not prevent the event. ```js // risk/review.mjs — how many risks does review frequency catch before they are realized import { RISKS, MONTHS, random, period } from "./record.mjs"; const NOTICE = 0.20; // probability of noticing an unrecorded risk in one review (DR13) const PERSON_MIN = 45; // person-minutes of one review (DR13) const FREQUENCY = { never: [], yearly: [12], quarterly: [3, 6, 9, 12], monthly: [...Array(MONTHS)].map((_, i) => i + 1), }; const TRIALS = 500, SEED = 20240518; console.log(`${TRIALS} periods, seed ${SEED}; notice probability ${NOTICE}, ` + `${PERSON_MIN} person-min per review`); console.log("frequency reviews noticed caught before unrecorded realized person-min"); const result = {}; for (const [name, months] of Object.entries(FREQUENCY)) { const rnd = random(SEED); // every frequency runs on the same seed, over the same periods let noticed = 0, before = 0, missed = 0; for (let i = 0; i < TRIALS; i++) for (const s of period(rnd)) { if (s.risk.recorded) continue; let foundMonth = null; for (const m of months) if (rnd() < NOTICE) { foundMonth = m; break; } if (foundMonth !== null) noticed += 1; if (s.occurred && (foundMonth === null || foundMonth >= s.month)) missed += 1; else if (s.occurred) before += 1; } const personMin = months.length * PERSON_MIN; result[name] = { before: before / TRIALS, missed: missed / TRIALS, personMin, noticed: noticed / TRIALS }; console.log(` ${name.padEnd(10)} ${String(months.length).padStart(4)} ` + `${(noticed / TRIALS).toFixed(2).padStart(11)} ${(before / TRIALS).toFixed(2).padStart(17)} ` + `${(missed / TRIALS).toFixed(2).padStart(22)} ${String(personMin).padStart(9)}`); } console.log("\nwhat increasing frequency buys (per period):"); const names = Object.keys(FREQUENCY); for (let i = 1; i < names.length; i++) { const a = result[names[i - 1]], b = result[names[i]]; const extra = b.before - a.before; console.log(` ${names[i - 1]} -> ${names[i]}: +${(b.personMin - a.personMin)} person-min, ` + `+${extra.toFixed(2)} risks caught before -> per risk ` + `${extra > 0 ? `${((b.personMin - a.personMin) / extra).toFixed(0)} person-min` : "no return"}`); } // Question set: 14 risks x 3 questions; expected value based on how many risks are in the record by period's end const RECORDED_COUNT = RISKS.filter((r) => r.recorded).length, N = RISKS.length; console.log(`\nquestion set (${N} risks x 3 questions = ${3 * N} answers; R1 was this risk considered, ` + `R2 what likelihood was assumed, R3 did the estimate hold):`); for (const [name, months] of Object.entries(FREQUENCY)) { const inRecord = RECORDED_COUNT + result[name].noticed, outside = N - inRecord; const correct = inRecord * (months.length ? 3 : 2); const missing = outside * 2 + (months.length ? 0 : inRecord); console.log(` ${name.padEnd(10)} in record ${inRecord.toFixed(2)}/${N} risks -> correct ` + `${correct.toFixed(1)}, missing ${missing.toFixed(1)}, wrong ${outside.toFixed(1)}`); } ``` ``` 500 periods, seed 20240518; notice probability 0.2, 45 person-min per review frequency reviews noticed caught before unrecorded realized person-min never 0 0.00 0.00 1.67 0 yearly 1 1.26 0.00 1.63 45 quarterly 4 3.54 0.44 1.22 180 monthly 12 5.62 1.03 0.59 540 what increasing frequency buys (per period): never -> yearly: +45 person-min, +0.00 risks caught before -> per risk no return yearly -> quarterly: +135 person-min, +0.44 risks caught before -> per risk 310 person-min quarterly -> monthly: +360 person-min, +0.60 risks caught before -> per risk 602 person-min question set (14 risks x 3 questions = 42 answers; R1 was this risk considered, R2 what likelihood was assumed, R3 did the estimate hold): never in record 8.00/14 risks -> correct 16.0, missing 20.0, wrong 6.0 yearly in record 9.26/14 risks -> correct 27.8, missing 9.5, wrong 4.7 quarterly in record 11.54/14 risks -> correct 34.6, missing 4.9, wrong 2.5 monthly in record 13.62/14 risks -> correct 40.9, missing 0.8, wrong 0.4 ``` Yearly review notices 1.26 risks per period and catches none of them before they occur. The number being zero is not a coincidence: review happens in the twelfth month while realization is spread across all twelve, so noticing almost always comes after the event. A risk register reviewed once a year produces a correct record for the following period and does nothing for the period it is in. Quarterly frequency catches 0.44 risks before they occur, monthly 1.03; risks that stay unrecorded and occur drop from 1.67 to 0.59. What it buys, though, is a rising curve: moving to quarterly costs 310 person-minutes per risk caught early, moving to monthly costs 602. The second transition's unit cost is nearly double the first, because the risks most readily noticed are already caught at the sparser frequency. The return on increasing frequency diminishes; where to stop becomes a question of comparing the risk's impact against the person-minutes. The question set table shows the same curve from the record's readability side. Of forty-two answers, the never-reviewed record gives 16 correct, 20 missing, 6 wrong. The wrong ones are the answers to "was this risk considered": for an unrecorded risk, the answer given is "no," and that answer looks correct, because the record really is silent on it. Under monthly review, correct answers rise to 40.9. A record's value is set not by what was written into it once, but by how often it gets read again. #### Summary - 808 of the 2348 realized risks, that is 34.4 percent, were never in the record; a risk register's first measure is not the accuracy of what it holds — it is the size of what it leaves out. - The recorded bands did not preserve ordering: the high band's estimate was 0.60 against a measured rate of 0.49, the low band's estimate was 0.13 against a measured rate of 0.30. - The two large gaps run in opposite directions — rule-engine-growth was estimated at 0.10 and measured at 0.50, cache-freshness-overrun at 0.50 and measured at 0.15; an estimate written once and left alone does not reveal its own wrongness. - Yearly review noticed 1.26 risks per period and caught none of them before they occurred; quarterly caught 0.44, monthly 1.03, and risks missed dropped from 1.67 to 0.59. - The unit cost of increasing frequency rises: 310 and 602 person-minutes per risk caught early; against that, in the question set, correct answers rose from 16 to 40.9 while wrong answers dropped from 6 to 0.4. #### Next Step A risk register tracks something not yet realized: an event with a likelihood that has not happened yet. Next to it stands one more record, and that one tracks something already realized. When a decision is made, sometimes an incomplete solution is chosen deliberately — to ship faster, or because the right solution is not yet known. That gap stays in the system and produces a little extra work with every change. Sometimes the gap is not chosen deliberately, and is noticed only later. The next lesson separates these two cases, tests whether the difference between them is measurable, and counts the extra work the gap produces per change. ### Technical Debt Management Source: https://academia.sh/en/courses/architectural-documentation/technical-debt-management Separating and measuring deliberate from inadvertent debt: debt items are split into two classes, debt interest is modeled as the difference in work the same change causes in an indebted module versus a debt-free one, yearly interest per item and the rate of extra work per module are computed, the delay before inadvertent debt is noticed is measured as a function of a repeat threshold, and the interest paid unnoticed across that delay is counted. A risk register tracks something not yet realized: an event with a likelihood that has not happened yet. Next to it stands a second record, and that one tracks something already realized. When a decision is made, sometimes an incomplete solution is chosen deliberately; the gap stays in the system and produces extra work with every change. The name for this gap is **technical debt**. How technical debt gets defined as a metric was addressed in the Quality Metrics lesson; it is not repeated here. This lesson has two questions. First: once debt items are split into two classes — taken knowingly and taken without realizing it — does that split produce a measurable difference. Second: what is **debt interest**, and how is it counted. #### Two Classes and Interest **Deliberate debt** is a gap chosen knowingly at decision time: to ship faster, or because the right solution is not yet known. It can be recorded, because the person who chose it knows they chose it. **Inadvertent debt** is not chosen; it is understood only later that an assumption was wrong, or that a copy landed in two places. It cannot be recorded, because no one is aware of it while the record is being written. The way to measure interest reduces to a single question: how many more minutes does the same change take in the indebted module than in a debt-free one. This does not say the debt is "bad" — it says how bad. In the model below, five change types, their base durations, and their yearly counts are the model's inputs (**DR14**). Every debt item carries which change types it affects and how many extra minutes it adds for that type. The scenario is fictional. ```js // debt/interest.mjs — debt items are split into two classes, interest is measured as extra work per change // Change types, base durations, and yearly counts are the model's input (DR14); the scenario is fictional. export const CHANGE_TYPES = [ { type: "rule-change", base: 120, yearly: 12 }, { type: "new-field", base: 90, yearly: 8 }, { type: "new-screen", base: 240, yearly: 6 }, { type: "bug-fix", base: 60, yearly: 30 }, { type: "performance-tuning", base: 150, yearly: 4 }, ]; // extraWork: how many extra minutes the same change takes in the indebted module vs. the debt-free module export const ITEMS = [ { name: "rule-copy", class: "deliberate", module: "loan", extraWork: 35, affects: ["rule-change", "bug-fix"] }, { name: "manual-field-mapping", class: "deliberate", module: "catalog-mapping", extraWork: 45, affects: ["new-field", "bug-fix"] }, { name: "temp-membership-table", class: "deliberate", module: "membership", extraWork: 20, affects: ["new-field", "new-screen"] }, { name: "fixed-report-query", class: "deliberate", module: "reporting", extraWork: 30, affects: ["new-screen"] }, { name: "untested-penalty-branch", class: "inadvertent", module: "loan", extraWork: 40, affects: ["rule-change", "bug-fix", "performance-tuning"] }, { name: "date-format-in-two-places", class: "inadvertent", module: "catalog-mapping", extraWork: 25, affects: ["new-field", "bug-fix"] }, { name: "hidden-branch-assumption", class: "inadvertent", module: "membership", extraWork: 55, affects: ["new-screen", "bug-fix"] }, { name: "duplicated-authorization-check", class: "inadvertent", module: "reporting", extraWork: 35, affects: ["new-screen", "new-field", "bug-fix"] }, { name: "manual-cache-cleanup", class: "inadvertent", module: "loan", extraWork: 15, affects: ["performance-tuning", "bug-fix"] }, ]; export const MODULES = [...new Set(ITEMS.map((k) => k.module))]; export const yearly = (type) => CHANGE_TYPES.find((d) => d.type === type).yearly; export const itemInterest = (k) => k.extraWork * k.affects.reduce((t, type) => t + yearly(type), 0); export const baseWork = CHANGE_TYPES.reduce((t, d) => t + d.base * d.yearly, 0); // per module if (import.meta.url.endsWith(process.argv[1].split("/").pop())) { const deliberate = ITEMS.filter((k) => k.class === "deliberate"); console.log(`${ITEMS.length} debt items: ${deliberate.length} deliberate (recorded), ` + `${ITEMS.length - deliberate.length} inadvertent (unrecorded)`); console.log(`${CHANGE_TYPES.reduce((t, d) => t + d.yearly, 0)} changes a year per module, ` + `${baseWork} min in a debt-free module\n`); console.log("yearly interest per item (extra work per change x yearly change count):"); for (const k of [...ITEMS].sort((a, b) => itemInterest(b) - itemInterest(a))) console.log(` ${k.name.padEnd(31)} ${k.class.padEnd(11)} ${k.module.padEnd(15)} ` + `${String(k.extraWork).padStart(2)} min x ${String(k.affects.reduce((t, type) => t + yearly(type), 0)).padStart(2)} ` + `changes = ${String(itemInterest(k)).padStart(4)} min/yr`); console.log("\ntotal interest per class:"); for (const s of ["deliberate", "inadvertent"]) { const g = ITEMS.filter((k) => k.class === s); const f = g.reduce((t, k) => t + itemInterest(k), 0); console.log(` ${s.padEnd(11)} ${g.length} items, ${f} min/yr, per item ${(f / g.length).toFixed(0)} min`); } console.log("\nyearly work per module (base + interest):"); for (const m of MODULES) { const f = ITEMS.filter((k) => k.module === m).reduce((t, k) => t + itemInterest(k), 0); console.log(` ${m.padEnd(15)} base ${baseWork.toFixed(0)} + interest ${String(f).padStart(4)} = ` + `${(baseWork + f).toFixed(0)} min (${(100 * f / baseWork).toFixed(1)}% extra)`); } } ``` ``` 9 debt items: 4 deliberate (recorded), 5 inadvertent (unrecorded) 60 changes a year per module, 6000 min in a debt-free module yearly interest per item (extra work per change x yearly change count): hidden-branch-assumption inadvertent membership 55 min x 36 changes = 1980 min/yr untested-penalty-branch inadvertent loan 40 min x 46 changes = 1840 min/yr manual-field-mapping deliberate catalog-mapping 45 min x 38 changes = 1710 min/yr duplicated-authorization-check inadvertent reporting 35 min x 44 changes = 1540 min/yr rule-copy deliberate loan 35 min x 42 changes = 1470 min/yr date-format-in-two-places inadvertent catalog-mapping 25 min x 38 changes = 950 min/yr manual-cache-cleanup inadvertent loan 15 min x 34 changes = 510 min/yr temp-membership-table deliberate membership 20 min x 14 changes = 280 min/yr fixed-report-query deliberate reporting 30 min x 6 changes = 180 min/yr total interest per class: deliberate 4 items, 3640 min/yr, per item 910 min inadvertent 5 items, 6820 min/yr, per item 1364 min yearly work per module (base + interest): loan base 6000 + interest 3820 = 9820 min (63.7% extra) catalog-mapping base 6000 + interest 2660 = 8660 min (44.3% extra) membership base 6000 + interest 2260 = 8260 min (37.7% extra) reporting base 6000 + interest 1720 = 7720 min (28.7% extra) ``` The item table's first piece of information is in the ranking: the highest-interest item is not the one that adds the most work per change. temp-membership-table adds 20 minutes per change and costs 280 minutes a year; manual-cache-cleanup adds 15 minutes and costs 510. The difference is in how many changes the item touches. A debt item's weight is measured not by how ugly it is, but by how often it gets set off. The class table gives the second piece of information. Five inadvertent items cost 6820 minutes a year, four deliberate items 3640; 1364 minutes per item against 910. Inadvertent debt is more expensive per item. The reason is structural, not moral: deliberate debt is taken as part of a decision, and its scope is thought through when it is taken, while inadvertent debt grows without anyone watching. The measured difference is the numeric counterpart of these two states not being the same thing. The module table compares interest against base work. In the loan module, the same yearly set of changes takes 9820 minutes, against 6000 in a debt-free module — 63.7 percent extra work. In the reporting module, the same ratio is 28.7 percent. It cannot be said the two modules do different work — exactly the same set of changes was run. The entire gap between them is debt. #### The Delay Before Noticing Deliberate debt has zero delay before entering the record: it is known the moment it is chosen. Inadvertent debt is noticed at some point, and its interest keeps being paid until then. Noticing does not happen through a single painful change — it happens through the same extra work repeating: someone sees a pattern once they get stuck in the same place on two different tasks. In the model, an item is noticed once it has produced the same extra work a threshold number of times (**DR15**). This means the delay is tied not to the item's size, but to how often the changes it touches occur. ```js // debt/delay.mjs — the delay before inadvertent debt is noticed, and the interest paid unnoticed until then import { ITEMS, yearly, itemInterest } from "./interest.mjs"; // An item is noticed once the same extra work has shown up THRESHOLD times; changes are spread // evenly across the year. Both are the model's input (DR15). const THRESHOLD = [8, 24, 48]; const MONTHS = 12; const affected = (k) => k.affects.reduce((t, type) => t + yearly(type), 0); const delay = (k, threshold) => (MONTHS * threshold) / affected(k); const inadvertent = ITEMS.filter((k) => k.class === "inadvertent"); console.log(`${inadvertent.length} inadvertent items; deliberate debt has zero delay because it is written into the record`); console.log("threshold average delay unnoticed at month 12 interest paid unnoticed share of yearly interest"); const yearlyInterest = inadvertent.reduce((t, k) => t + itemInterest(k), 0); for (const threshold of THRESHOLD) { const delays = inadvertent.map((k) => delay(k, threshold)); const paid = inadvertent.reduce((t, k) => t + threshold * k.extraWork, 0); console.log(` ${String(threshold).padStart(2)} ${(delays.reduce((a, b) => a + b, 0) / delays.length).toFixed(1).padStart(13)} mo ` + `${String(delays.filter((x) => x > MONTHS).length).padStart(20)} ${String(paid).padStart(21)} min ` + `${((100 * paid / yearlyInterest).toFixed(0) + "%").padStart(15)}`); } console.log(`\nper item (threshold 24; yearly inadvertent interest ${yearlyInterest} min):`); for (const k of [...inadvertent].sort((a, b) => delay(a, 24) - delay(b, 24))) console.log(` ${k.name.padEnd(31)} ${String(affected(k)).padStart(2)} changes/yr, ` + `${delay(k, 24).toFixed(1).padStart(4)} mo until noticed, ${24 * k.extraWork} min paid until then`); // Question set: 9 items x 4 questions; based on which items are in the record by month 12 const QUESTIONS = ["B1 was this gap chosen deliberately", "B2 when did it enter", "B3 what was the payback condition", "B4 how much extra work does it cause per change"]; console.log(`\nquestion set at month 12 (${ITEMS.length} items x ${QUESTIONS.length} questions = ` + `${ITEMS.length * QUESTIONS.length} answers):`); // A deliberate item answers all four questions; a noticed inadvertent item answers B1 and B4, // B2 and B3 stay missing; a never-noticed item answers B1 wrong, the rest is missing. const DELIBERATE_COUNT = ITEMS.length - inadvertent.length; for (const threshold of THRESHOLD) { const found = inadvertent.filter((k) => delay(k, threshold) <= MONTHS).length; const outside = inadvertent.length - found; console.log(` threshold ${String(threshold).padStart(2)}: in record ${DELIBERATE_COUNT + found}/${ITEMS.length} items -> ` + `correct ${4 * DELIBERATE_COUNT + 2 * found}, missing ${2 * found + 3 * outside}, wrong ${outside}`); } ``` ``` 5 inadvertent items; deliberate debt has zero delay because it is written into the record threshold average delay unnoticed at month 12 interest paid unnoticed share of yearly interest 8 2.5 mo 0 1360 min 20% 24 7.4 mo 0 4080 min 60% 48 14.7 mo 5 8160 min 120% per item (threshold 24; yearly inadvertent interest 6820 min): untested-penalty-branch 46 changes/yr, 6.3 mo until noticed, 960 min paid until then duplicated-authorization-check 44 changes/yr, 6.5 mo until noticed, 840 min paid until then date-format-in-two-places 38 changes/yr, 7.6 mo until noticed, 600 min paid until then hidden-branch-assumption 36 changes/yr, 8.0 mo until noticed, 1320 min paid until then manual-cache-cleanup 34 changes/yr, 8.5 mo until noticed, 360 min paid until then question set at month 12 (9 items x 4 questions = 36 answers): threshold 8: in record 9/9 items -> correct 26, missing 10, wrong 0 threshold 24: in record 9/9 items -> correct 26, missing 10, wrong 0 threshold 48: in record 4/9 items -> correct 16, missing 15, wrong 5 ``` The threshold table gives the cost of the delay. Debt noticed after eight repeats enters the record after an average of 2.5 months and extracts 1360 minutes of payment until then; at twenty-four repeats, 7.4 months and 4080 minutes. A pattern requiring forty-eight repeats is not noticed at all within a year, and the interest paid rises to 120 percent of the yearly inadvertent interest — meaning no one knows what is slowing things down for more than a year. The difference across these three rows comes down to one thing: how long it takes for the extra work to be seen as a pattern. The item table shows the outcome of that link, and it runs against intuition. hidden-branch-assumption is the highest-interest item — 1980 minutes a year — but it is one of the latest to be noticed, at 8.0 months. untested-penalty-branch carries less interest and is noticed at 6.3 months. The reason is the model's rule: noticing depends not on how much each occurrence hurts, but on how many times it repeats. A debt that lands a few heavy hits stays invisible longer than one that lands frequent light ones. If the noticing rule were tied to size, the order would reverse; the measured result depends on the rule itself, and the result cannot be interpreted without the rule being written down. The question set table comes out the same at two thresholds and splits at the third. A noticed inadvertent debt item answers two of its four questions: it can be said that it was not chosen deliberately, and the extra work it causes per change can be measured. When it entered and what its payback condition was stay unanswered, because that information was never written when the item entered and cannot be produced afterward. This is the one real advantage deliberate debt has over inadvertent debt: it pays the same interest, but it answers all four questions. #### Summary - A debt item's yearly interest is the product of the work it adds per change and the number of changes it touches; the highest-interest item is not the one adding the most work per change. - Five inadvertent items cost 6820 minutes a year, four deliberate items 3640; 1364 minutes per item against 910, meaning inadvertent debt is more expensive per item. - The same yearly set of changes takes 9820 minutes in the loan module against 6000 in a debt-free module — 63.7 percent extra work; in the reporting module the same ratio is 28.7 percent. - The delay before inadvertent debt is noticed depends on the repeat threshold: 2.5 months and 1360 minutes at eight repeats, 7.4 months and 4080 minutes at twenty-four, and at forty-eight it is not noticed at all within a year, with the interest paid rising to 120 percent of the yearly inadvertent interest. - The order of noticing is not the same as the order of interest; the highest-interest item is among the latest noticed, at 8.0 months, because in this model noticing is tied to repetition, not to size. #### Next Step Everything written across this topic gathered around a single decision. The decision record holds the decision's rationale, the proposal process holds how the decision was reached, the trade-off analysis holds what was traded for what, the risk register holds what might not hold, the debt record holds what was deliberately left incomplete. Five record formats, and a measured question set for each. All five share one limit: each is the record of a single decision. Someone new to the library network's system, reading every one of these records, learns the rationale behind forty decisions but does not learn what the system looks like. Which parts exist, which depends on which, where data flows from and to, which part sits on which machine — none of these questions is answered in the record of any single decision, because none of them belongs to a single decision. Stacking the records on top of one another gives no picture either; the sum of forty records is not a map of the system. The next topic takes on that gap: it measures what a description showing the system as a whole is, how many distinct views it needs, and which question each view answers. ## Views and Models Which picture answers which question about the system: the logical, process, development, and physical views each answering separate questions, the layered diagram approach that changes scale from context to code, selecting the diagram type by question, turning quality requirements into measurable scenarios, keeping the document in step with the code, and size estimates given with an uncertainty band. ### Architectural Views Source: https://academia.sh/en/courses/architectural-documentation/architectural-views Applying a question set to the logical, process, development, and physical views: how many questions each view answers, the class of question no view answers, the repetition where two views answer at once, and how many questions are missed by settling for a single view. The previous topic turned decisions into objects: each decision's context, the alternative it ruled out, the risk it carries, and the debt it leaves behind are all written down. The regional library network's record stack can now say "the external catalog is reached through a single connector" or "notifications are written to a queue." But this stack is a record of individual decisions. Someone arriving six months later who reads through all of them learns the list of decisions, not the picture of the system. The answer to "what happens when a loan request comes in" does not sit whole inside any one record; its pieces are scattered across ten separate records. A view exists to fill that gap. A **view** is the system seen from a single axis of concern; it carries the information that is meaningful on that axis and nothing else. The common split is four: the **logical view**, the **process view**, the **development view**, and the **physical view**. This lesson defines all four not by their drawings but by **the information items they carry**, and measures them with a single question: how many of the questions someone asks later does this view answer. #### A View Is a Set of Information No diagram is drawn. The structure underneath what would be drawn is modeled: a view is a container that can carry certain kinds of information items. The logical view carries modules, responsibilities, dependencies between modules, interfaces, and data entities — but not which machine a module runs on. The process view carries running processes, call order, points of concurrency, and queues — but not the source directory structure. The development view carries the source tree, build units, team ownership, and external dependencies. The physical view carries nodes, network links, placement, and capacity. The item lists are chosen by hand (**VM1**); the boundaries might be drawn a little differently on another network. The selection itself is an input to the measurement, not its result. The question set is also a data structure (**VM2**). Twenty-four questions, of the kind someone actually asks six months later: where does a rule live, which process handles a request, which directory holds a piece of code, which node runs a service. The information items required for each question's answer are written down. The rule is a single sentence (**VM3**): **a view answers a question only if it carries every item the answer needs.** Carrying half of it is not an answer; the reader has to find the rest somewhere else. ```js // views/model.mjs — MODEL regional library network: the information items carried by // the four views and the question set someone asks six months later. Not a real // institution; item lists and questions are chosen by hand. // A view carries only the items on its list (VM1). export const VIEW = { logical: ["module", "responsibility", "module-dependency", "interface", "data-entity"], process: ["module", "interface", "process", "call-order", "concurrency", "queue"], development: ["module", "source-tree", "build-unit", "team", "external-dependency"], physical: ["node", "network-link", "placement", "capacity", "build-unit"], }; export const VIEWS = Object.keys(VIEW); // QUESTION — needs: the information items required for an answer (VM2). A view answers // a question only if it carries every needed item (VM3). export const QUESTION = [ { id: "S01", text: "which module holds the loan-period rule", needs: ["module", "responsibility"] }, { id: "S02", text: "does the penalty calculation depend on the member registry", needs: ["module", "module-dependency"] }, { id: "S03", text: "what interface does the catalog connector expose", needs: ["module", "interface"] }, { id: "S04", text: "what is the relationship between a member and a loan record", needs: ["data-entity"] }, { id: "S05", text: "what named parts exist in the system", needs: ["module"] }, { id: "S06", text: "which process handles a reservation request", needs: ["module", "process"] }, { id: "S07", text: "which process consumes the notification queue", needs: ["queue", "process"] }, { id: "S08", text: "what gets locked if two branches check out the same copy at once", needs: ["concurrency"] }, { id: "S09", text: "what order do calls go in during the checkout flow", needs: ["call-order"] }, { id: "S10", text: "is the call from the kiosk front asynchronous", needs: ["call-order", "queue"] }, { id: "S11", text: "which directory holds the penalty calculation code", needs: ["source-tree"] }, { id: "S12", text: "which team owns this module", needs: ["module", "team"] }, { id: "S13", text: "how many separate build units are there", needs: ["build-unit"] }, { id: "S14", text: "which external library does the catalog connector use", needs: ["module", "external-dependency"] }, { id: "S15", text: "which node does the loan service run on", needs: ["placement", "node"] }, { id: "S16", text: "what kind of link connects the branch front to the center", needs: ["network-link", "node"] }, { id: "S17", text: "what is the central node's capacity", needs: ["node", "capacity"] }, { id: "S18", text: "which build unit is placed on which node", needs: ["build-unit", "placement"] }, { id: "S19", text: "which node's release does the team owning the penalty calculation affect", needs: ["team", "placement"] }, { id: "S20", text: "which call in the checkout flow goes over the network", needs: ["call-order", "network-link"] }, { id: "S21", text: "which data entity resides on which node", needs: ["data-entity", "placement"] }, { id: "S22", text: "what is the responsibility of the module holding the lock", needs: ["concurrency", "responsibility"] }, { id: "S23", text: "how many processes does releasing a build unit restart", needs: ["build-unit", "process"] }, { id: "S24", text: "whose job is it when the reservation queue backs up", needs: ["queue", "team"] }, ]; // fixed order export const ITEMS = ["network-link", "interface", "call-order", "external-dependency", "node", "concurrency", "capacity", "source-tree", "queue", "module", "module-dependency", "responsibility", "process", "team", "data-entity", "build-unit", "placement"]; export const answers = (v, q) => q.needs.every((i) => VIEW[v].includes(i)); export const answeringViews = (q) => VIEWS.filter((v) => answers(v, q)); ``` #### Measurement The second file applies the question set to the four views, counts answered and unanswered questions, extracts the repetition, and runs three schemes through the same stream of changes. ```js // views/measure.mjs — the question set is applied to the four views; answered, // unanswered, repeated questions and maintenance cost are counted. import { VIEW, VIEWS, QUESTION, ITEMS, answeringViews } from "./model.mjs"; const col = (s, n) => String(s).padEnd(n); const abbr = { logical: "log", process: "prc", development: "dev", physical: "phy" }; const measured = QUESTION.map((q) => ({ ...q, a: answeringViews(q) })); console.log(col("question", 10) + col("needed item(s)", 34) + "answering view(s)"); console.log("-".repeat(72)); for (const q of measured) console.log(col(q.id, 10) + col(q.needs.join("+"), 34) + (q.a.length ? q.a.map((v) => abbr[v]).join(",") : "- none -")); const single = measured.filter((q) => q.a.length === 1).length; const multi = measured.filter((q) => q.a.length > 1); const zero = measured.filter((q) => q.a.length === 0); console.log(`\n${QUESTION.length} questions: single view ${single}, multiple views ${multi.length}, none ${zero.length}`); console.log(`the four views together answer ${QUESTION.length - zero.length}/${QUESTION.length} questions`); console.log("\nview answered missed most frequent missing item"); console.log("-".repeat(62)); for (const v of VIEWS) { const missed = measured.filter((q) => !q.a.includes(v)); const counts = {}; for (const q of missed) for (const i of q.needs.filter((x) => !VIEW[v].includes(x))) counts[i] = (counts[i] || 0) + 1; const top = Object.entries(counts).sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])).slice(0, 3); console.log(col(v, 13) + col(QUESTION.length - missed.length, 10) + col(missed.length, 11) + top.map(([i, n]) => `${i}(${n})`).join(" ")); } // ---- repetition: how many views repeat an item ---- const viewCount = (i) => VIEWS.filter((v) => VIEW[v].includes(i)).length; const totalWrites = ITEMS.reduce((t, i) => t + viewCount(i), 0); console.log(`\n${ITEMS.length} distinct information items are written ${totalWrites} times; ${totalWrites - ITEMS.length} of those writes repeat`); console.log("repeated item: " + ITEMS.filter((i) => viewCount(i) > 1).map((i) => `${i} x${viewCount(i)}`).join(", ")); console.log("repeated question: " + multi.map((q) => `${q.id}(${q.a.length})`).join(", ")); // ---- cross-mappings the unanswered questions need ---- const pairName = (a, b) => [a, b].sort().join(" <-> "); const mappings = [...new Set(zero.map((q) => pairName(q.needs[0], q.needs[1])))]; console.log(`\n${zero.length} unanswered questions need ${mappings.length} cross-mappings:`); for (const q of zero) { const owners = q.needs.map((i) => VIEWS.filter((v) => VIEW[v].includes(i)).map((v) => abbr[v]).join("/")); console.log(` ${q.id} ${col(pairName(q.needs[0], q.needs[1]), 36)}(${owners.join(" and ")})`); } // ---- maintenance: 200 changes across three schemes ---- // WEIGHT — how often each information item changes (VM4); chosen by hand const WEIGHT = { module: 10, responsibility: 8, "module-dependency": 12, interface: 9, "data-entity": 6, process: 5, "call-order": 7, concurrency: 3, queue: 4, "source-tree": 6, "build-unit": 4, team: 3, "external-dependency": 5, node: 2, "network-link": 2, placement: 3, capacity: 2, }; function prng(seed) { let s = seed >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 2 ** 32; } const rand = prng(70321); // seed is visible; the same seed gives the same sequence const totalWeight = ITEMS.reduce((t, i) => t + WEIGHT[i], 0); const pickItem = () => { let r = rand() * totalWeight; for (const i of ITEMS) if ((r -= WEIGHT[i]) < 0) return i; return ITEMS.at(-1); }; const N = 200; let singleWrites = 0, fourWrites = 0, fourRepeats = 0, mappingRefresh = 0; for (let i = 0; i < N; i++) { const item = pickItem(); if (VIEW.process.includes(item)) singleWrites++; const k = viewCount(item); fourWrites += k; fourRepeats += k - 1; mappingRefresh += mappings.filter((m) => m.split(" <-> ").includes(item)).length; } const bestSingle = VIEWS.map((v) => [v, measured.filter((q) => q.a.includes(v)).length]).sort((a, b) => b[1] - a[1])[0]; console.log(`\n${N} change events, best single view = ${bestSingle[0]}`); console.log(col("scheme", 26) + col("answers", 8) + col("writes", 8) + "writes per answer"); console.log("-".repeat(64)); const row = (name, ans, writes) => console.log(col(name, 26) + col(`${ans}/${QUESTION.length}`, 8) + col(writes, 8) + (writes / ans).toFixed(2)); row("single view", bestSingle[1], singleWrites); row("four views", QUESTION.length - zero.length, fourWrites); row("four views + mapping", QUESTION.length, fourWrites + mappingRefresh); console.log(`of the four views' ${fourWrites} writes, ${fourRepeats} write the same fact a second time`); console.log(`${mappings.length} mappings refresh ${mappingRefresh} times across ${N} changes, opening up ${zero.length} questions in return`); ``` ``` question needed item(s) answering view(s) ------------------------------------------------------------------------ S01 module+responsibility log S02 module+module-dependency log S03 module+interface log,prc S04 data-entity log S05 module log,prc,dev S06 module+process prc S07 queue+process prc S08 concurrency prc S09 call-order prc S10 call-order+queue prc S11 source-tree dev S12 module+team dev S13 build-unit dev,phy S14 module+external-dependency dev S15 placement+node phy S16 network-link+node phy S17 node+capacity phy S18 build-unit+placement phy S19 team+placement - none - S20 call-order+network-link - none - S21 data-entity+placement - none - S22 concurrency+responsibility - none - S23 build-unit+process - none - S24 queue+team - none - 24 questions: single view 15, multiple views 3, none 6 the four views together answer 18/24 questions view answered missed most frequent missing item -------------------------------------------------------------- logical 5 19 placement(4) build-unit(3) call-order(3) process 7 17 placement(4) build-unit(3) node(3) development 5 19 placement(4) call-order(3) node(3) physical 5 19 module(7) call-order(3) process(3) 17 distinct information items are written 21 times; 4 of those writes repeat repeated item: interface x2, module x3, build-unit x2 repeated question: S03(2), S05(3), S13(2) 6 unanswered questions need 6 cross-mappings: S19 placement <-> team (dev and phy) S20 call-order <-> network-link (prc and phy) S21 data-entity <-> placement (log and phy) S22 concurrency <-> responsibility (prc and log) S23 build-unit <-> process (dev/phy and prc) S24 queue <-> team (prc and dev) 200 change events, best single view = process scheme answers writes writes per answer ---------------------------------------------------------------- single view 7/24 86 12.29 four views 18/24 279 15.50 four views + mapping 24/24 396 16.50 of the four views' 279 writes, 79 write the same fact a second time 6 mappings refresh 117 times across 200 changes, opening up 6 questions in return ``` #### Answered and Unanswered The four views together answer **eighteen** of the twenty-four questions. Fifteen have their answer in a single view, three have their answer in more than one view. Six questions find no answer in any view. The unanswerable six share a common trait, and this trait is the lesson's real finding: **each one asks for two items to be combined that live in two different views.** The question of which node's release the team owning the penalty calculation affects cannot be answered without setting the development view's team information next to the physical view's placement information. Which call in the loan flow goes over the network asks for the process view's call order to be mapped onto the physical view's network link. How many processes releasing a build unit restarts is a mapping between the development and process views. This does not come from the four views being poorly chosen. Because views are split by definition along an axis of concern, **the class of question that falls between axes stays open.** No matter how well the four views are written, these six questions will not be sitting there; they need something separate — a mapping between views. The six separate mappings the six questions call for are listed at the end of the output. Counting the unanswered question is this course's most valuable measure, because someone reading the document who cannot find an answer to their question does not report it to anyone; they look inside the code, ask someone, or guess. If the count is not kept, the gap stays invisible. #### What a Single View Misses The table below shows what each view does on its own. The process view does best: it answers **seven** of the twenty-four questions and misses seventeen. The logical, development, and physical views each answer five questions and miss nineteen. The "most frequent missing item" column names the **class** of the missed questions. Used alone, the logical view most often needs placement (4 questions), call order (3), and node (3) information — that is, what it misses is the runtime and placement class. The physical view's gap is entirely different: module information is missing in seven questions. The physical view knows where something runs; it does not name what runs. The practical conclusion here is this: if a system's document consists of a single view, the questions it misses do not scatter at random; they're **always from the same class**. A document that settles for the logical view systematically leaves placement and runtime questions unanswered; a document that settles for the physical view leaves the question "what runs on this node" unanswered. #### The Cost of Repetition Three questions are answered in more than one view. At first glance this looks good — information findable from two places. The cost sits on the item side: seventeen distinct information items are written into views twenty-one times. Four writes are repeats. The module identity repeats in three views, the interface in two views, the build unit in two views. When a module's name changes, it has to change in three places; if one of the three is forgotten, two views contradict each other. The table below counts this across a stream of changes. How often items change comes from a hand-picked weight table (**VM4**); a generator seeded with 70321 produces two hundred change events. The same seed gives the same sequence. The four-view scheme asks for **279 writes** against two hundred changes; **79** of those write the same fact a second or third time. In return, eighteen questions get answered: 15.50 writes per answer. The single-view scheme settles for 86 writes but answers only seven questions: 12.29 writes per answer. Once six cross-mappings are added on top of the four, the whole set gets answered; the mappings refresh 117 times across two hundred changes and the writes per answer climb to 16.50. The way to read the ranking is this: as scope grows, cost per answer grows too, but **the growth is slow.** While the number of answers climbs from seven to twenty-four — more than triple — the writes per answer climb from 12.29 to 16.50, about a third more. The cheapness of a single view is an illusion: what's cheap is not the writing, it is that the seventeen unwritten questions have been handed off to the reader. These numbers depend on the model's inputs; if the weight table gave placement and node items a higher value, the physical view's share would grow, but the ranking would not change. #### Summary - A view is the system seen from a single axis of concern and is defined not by its drawing but by the set of information items it carries; the item lists of the logical, process, development, and physical views are distinct and overlap in only three items. - Across the twenty-four-question set, the four views together answer 18 questions; 15 questions find their answer in a single view, 3 in more than one view, 6 in none. - All six unanswered questions ask for items from two different views to be combined; this gap comes not from poorly chosen views but from their being split by axis, and it calls for six cross-mappings. - Settling for a single view, the best case is 7/24 (the process view); the missed questions are not random but from the same class — the logical view misses placement and runtime questions, the physical view misses module identity in seven questions. - Seventeen information items are written into views 21 times, 4 writes being repeats; with a generator seeded at 70321, across 200 changes the four views ask for 279 writes, 79 of them a second write of the same fact. Writes per answer are 12.29 for a single view, 15.50 for four views, and 16.50 once mappings are added. #### Next Step The four-view scheme cut the system along its **axis of concern**: what, when, whose code, where. One axis stayed uncut. In this model the logical view carries every module in a single list; someone asking how the network looks from outside and someone asking where a function sits inside a module have to look at the same list. The first reads far more than they were looking for; the second never finds the detail they wanted. The next lesson models the same system at four scale levels from context down to code, and measures: how many nodes and edges per level, at which level a question gets answered, and how many unnecessary nodes someone looking at the wrong level reads. ### The Layered Diagram Approach Source: https://academia.sh/en/courses/architectural-documentation/layered-diagram-approach Modeling the same system at the context, container, component, and code level: the number of nodes and edges per level, the level at which a question gets answered, the unnecessary nodes a reader looking at the wrong level reads, and the break-even point of keeping four levels in maintenance. The previous lesson split the system into four views along its axis of concern and counted the questions each view answered. One axis stayed uncut: **detail.** Someone asking what the regional library network exchanges with the outside and someone asking what the late-penalty formula calls both have to look at the same module list. For the first, the list is too crowded; for the second, too coarse. The layered diagram approach cuts this axis. The same system is modeled at **four scale levels**: the **context level** shows the system as a single box and its relationship with outside parties; the **container level** shows the separately running, separately deployed units inside the system; the **component level** shows the responsibility parts inside a container; the **code level** shows the functions inside a component. The levels do not show different things — they show **the same thing at a different scale**. Let the boundary between the two terms be written down. A **container** is a separately running, separately deployed unit — a front-end application, a service, a data store, a message bus. **Component** was established earlier as a responsibility part inside a module; the difference here is context: there it was a separable unit inside a codebase, here it is a separable unit **inside a container** and is not deployed on its own. #### Model No diagram is drawn. The structure underneath the drawing is modeled: nodes, edges, and the tree connecting the nodes to each other. The tree is hand-written (**VM5**), and so are the component-level dependencies (**VM6**). The code-level edges are derived from both: each component dependency is realized at the code level by a single pair of items (**VM7**), and each component's items are chained together internally (**VM8**). The seed is visible. The upper levels' edges are computed, not written: both ends of a code edge are replaced by their ancestors at the desired level, self-loops are discarded, and the remainder is deduplicated. This is called **edge promotion**. The method's correctness is a measurable claim and is tested in the output: the hand-written component edges must come back when promoted from the code edges. The question set is once again a data structure (**VM9**). Each question has a target node and an **answer level**: the shallowest level at which the target's neighbors can still be told apart. The reading cost is counted like this: the nodes read at the answer level are the target plus its neighbors. Someone reading at a deeper level has to read all the descendant nodes of the target and its neighbors. At a shallower level, the target disappears inside its own ancestor and the question **stays unanswered**. ```js // views/scale.mjs — MODEL regional library network at four scale levels: context, // container, component, code. Not a real institution; tree and component edges are hand-written. // TREE — container > component > code item (VM5) const TREE = { "web-portal": { "session-front": ["login-flow", "session-cookie", "password-reset"], "search-front": ["query-form", "result-list", "filter"], "account-front": ["loan-list", "penalty-summary"] }, "staff-app": { "counter-front": ["checkout-screen", "return-screen", "penalty-screen"], "admin-front": ["member-management", "report-screen"] }, "kiosk-front": { "kiosk-interface": ["card-read", "quick-loan"], "kiosk-cache": ["local-catalog", "local-duration"] }, "loan-service": { "loan-core": ["open-loan", "take-return", "extension"], "loan-rule": ["duration-calc", "limit-check", "late-penalty"], reservation: ["reservation-queue", "reservation-notification"] }, "member-service": { "member-registry": ["create-member", "update-member", "card-link"], "member-penalty": ["penalty-accrual", "penalty-collection"] }, "catalog-integration": { "catalog-connector": ["external-query", "record-conversion", "retry"], "catalog-cache": ["cache-key", "expiry"] }, "inventory-service": { "branch-inventory": ["copy-status", "shelf-location"], "branch-sync": ["sync-job", "conflict-resolution"] }, "notification-service": { "notification-queue": ["queue-write", "queue-read"], "notification-email": ["message-body", "send-attempt"] }, "report-service": { "report-daily": ["daily-total", "branch-breakdown"], "report-batch": ["monthly-summary", "archive-write"] }, "main-database": { "relational-schema": ["member-table", "copy-table", "loan-table", "penalty-table"] }, "event-bus": { "topic-routing": ["topic-definition", "subscription"] }, cache: { "key-value": ["write", "read", "timed-delete"] }, }; const SYSTEM = "library-network"; const EXTERNAL = ["member", "staff", "external-catalog", "email-gateway", "identity-provider"]; // COMPONENT_EDGE — component-level dependencies (VM6); chosen by hand const COMPONENT_EDGE = [ ["member", "search-front"], ["member", "session-front"], ["member", "account-front"], ["member", "kiosk-interface"], ["staff", "counter-front"], ["staff", "admin-front"], ["session-front", "identity-provider"], ["session-front", "member-registry"], ["search-front", "catalog-cache"], ["search-front", "branch-inventory"], ["account-front", "loan-core"], ["account-front", "member-penalty"], ["counter-front", "loan-core"], ["counter-front", "member-penalty"], ["counter-front", "branch-inventory"], ["admin-front", "member-registry"], ["admin-front", "report-daily"], ["kiosk-interface", "loan-core"], ["kiosk-interface", "kiosk-cache"], ["kiosk-cache", "catalog-cache"], ["loan-core", "loan-rule"], ["loan-core", "relational-schema"], ["loan-core", "topic-routing"], ["loan-core", "branch-inventory"], ["reservation", "loan-core"], ["reservation", "notification-queue"], ["reservation", "relational-schema"], ["member-registry", "relational-schema"], ["member-penalty", "member-registry"], ["member-penalty", "loan-rule"], ["member-penalty", "relational-schema"], ["catalog-connector", "external-catalog"], ["catalog-cache", "catalog-connector"], ["catalog-cache", "key-value"], ["branch-inventory", "relational-schema"], ["branch-inventory", "topic-routing"], ["branch-sync", "branch-inventory"], ["branch-sync", "topic-routing"], ["branch-sync", "catalog-connector"], ["notification-queue", "topic-routing"], ["notification-queue", "relational-schema"], ["notification-email", "notification-queue"], ["notification-email", "email-gateway"], ["report-daily", "relational-schema"], ["report-batch", "relational-schema"], ["report-batch", "branch-inventory"], ]; // ---- mappings derived from the tree ---- const CONTAINERS = Object.keys(TREE); const PARENT = { component: {}, container: {} }; // code->component, component->container for (const k of CONTAINERS) for (const b of Object.keys(TREE[k])) { PARENT.container[b] = k; for (const c of TREE[k][b]) PARENT.component[c] = b; } const COMPONENTS = Object.keys(PARENT.container); const CODE_ITEMS = Object.keys(PARENT.component); const LEVEL = ["context", "container", "component", "code"]; const isExternal = (n) => EXTERNAL.includes(n); const levelOf = (n) => (isExternal(n) ? -1 : n === SYSTEM ? 0 : CONTAINERS.includes(n) ? 1 : COMPONENTS.includes(n) ? 2 : 3); const represents = (n, L) => { let m = n; while (!isExternal(m) && levelOf(m) > L) m = levelOf(m) === 3 ? PARENT.component[m] : levelOf(m) === 2 ? PARENT.container[m] : SYSTEM; return m; }; const CONTENT = { context: [SYSTEM], container: CONTAINERS, component: COMPONENTS, code: CODE_ITEMS }; const nodesAt = (L) => [...CONTENT[LEVEL[L]], ...EXTERNAL]; const descendantsAt = (n, L) => (levelOf(n) >= L || isExternal(n) ? [represents(n, L)] : nodesAt(L).filter((m) => represents(m, levelOf(n)) === n)); // ---- code-level edges: each component edge is realized by one item pair (VM7), // each component's items are chained together internally (VM8) ---- function prng(seed) { let s = seed >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 2 ** 32; } const rand = prng(31607); // seed is visible const codeItem = (n) => (isExternal(n) ? n : TREE[PARENT.container[n]][n][Math.floor(rand() * TREE[PARENT.container[n]][n].length)]); const CODE_EDGE = COMPONENT_EDGE.map(([a, b]) => [codeItem(a), codeItem(b)]); for (const b of COMPONENTS) { const g = TREE[PARENT.container[b]][b]; for (let i = 1; i < g.length; i++) CODE_EDGE.push([g[i - 1], g[i]]); } const edgesAt = (L) => [...new Set(CODE_EDGE.map(([a, b]) => [represents(a, L), represents(b, L)]) .filter(([a, b]) => a !== b).map(([a, b]) => `${a}->${b}`))].map((s) => s.split("->")); // ---- nodes and edges per level ---- const col = (s, n) => String(s).padEnd(n); console.log(col("level", 10) + col("nodes", 8) + col("edges", 8) + "internal nodes + external parties"); console.log("-".repeat(52)); LEVEL.forEach((d, L) => console.log(col(d, 10) + col(nodesAt(L).length, 8) + col(edgesAt(L).length, 8) + `${CONTENT[d].length} + ${EXTERNAL.length}`)); const recovered = new Set(edgesAt(2).map((e) => e.join("->"))); const handWritten = COMPONENT_EDGE.map((e) => e.join("->")); console.log(`\n${handWritten.filter((e) => recovered.has(e)).length} of the ${handWritten.length} hand-written component edges come back when promoted from code edges`); // ---- question set: each question's target and answer level (VM9) ---- const QUESTION = [ { id: "D01", text: "which outside parties does the network talk to", target: SYSTEM, level: 0 }, { id: "D02", text: "is the identity provider inside or outside the system", target: "identity-provider", level: 0 }, { id: "D03", text: "which containers does the loan service talk to", target: "loan-service", level: 1 }, { id: "D04", text: "which container connects to the email gateway", target: "email-gateway", level: 1 }, { id: "D05", text: "which containers write to the main database", target: "main-database", level: 1 }, { id: "D06", text: "which containers use the event bus", target: "event-bus", level: 1 }, { id: "D07", text: "what does the loan-rule component depend on", target: "loan-rule", level: 2 }, { id: "D08", text: "which components read the catalog cache", target: "catalog-cache", level: 2 }, { id: "D09", text: "which components does the penalty calculation touch", target: "member-penalty", level: 2 }, { id: "D10", text: "which component writes to the notification queue", target: "notification-queue", level: 2 }, { id: "D11", text: "what does the late-penalty calculation call", target: "late-penalty", level: 3 }, { id: "D12", text: "which item does the duration calculation depend on", target: "duration-calc", level: 3 }, { id: "D13", text: "what does the extension function touch", target: "extension", level: 3 }, { id: "D14", text: "where is the cache key generated", target: "cache-key", level: 3 }, ]; const neighbors = (n, L) => { const t = represents(n, L); return [...new Set(edgesAt(L).filter((e) => e.includes(t)).flat())].filter((m) => m !== t); }; const nodesRead = (q, L) => (L < q.level ? null : L === q.level ? 1 + neighbors(q.target, L).length : descendantsAt(q.target, L).length + neighbors(q.target, q.level).reduce((t, m) => t + descendantsAt(m, L).length, 0)); console.log("\nquestion answer level nodes read at code level extra"); console.log("-".repeat(60)); let minimum = 0, allCode = 0; for (const q of QUESTION) { const min = nodesRead(q, q.level), code = nodesRead(q, 3); minimum += min; allCode += code; console.log(col(q.id, 10) + col(LEVEL[q.level], 14) + col(min, 14) + col(code, 15) + (code - min)); } console.log("-".repeat(60)); console.log(col("total", 24) + col(minimum, 14) + col(allCode, 15) + (allCode - minimum)); console.log("\nthe same question read at all four levels:"); console.log((col("question", 10) + LEVEL.map((d) => col(d, 10)).join("")).trimEnd()); for (const id of ["D01", "D03", "D11"]) { const q = QUESTION.find((x) => x.id === id); console.log((col(q.id, 10) + LEVEL.map((d, L) => col(nodesRead(q, L) ?? "no answer", 10)).join("")).trimEnd()); } console.log("\nanswer level distribution: " + LEVEL.map((d, L) => `${d}=${QUESTION.filter((q) => q.level === L).length}`).join(", ")); LEVEL.forEach((d, L) => { const answered = QUESTION.filter((q) => q.level <= L); const unanswered = QUESTION.length - answered.length; console.log(`${col("only " + d, 22)}answered ${col(`${answered.length}/${QUESTION.length}`, 8)}unanswered ${col(unanswered, 4)}reading ${answered.reduce((t, q) => t + nodesRead(q, L), 0)}`); }); // ---- maintenance and break-even ---- // VM10: reading a node and keeping a node current are counted as equal cost const maintFour = LEVEL.reduce((t, d, L) => t + nodesAt(L).length, 0); const maintCode = nodesAt(3).length; const gain = allCode - minimum; console.log(`\nall four levels keep ${maintFour} nodes in maintenance, code level alone keeps ${maintCode}; difference ${maintFour - maintCode}`); console.log(`one round of questions saves ${gain} unnecessary node reads (${allCode} instead of ${minimum})`); console.log(`break-even (VM10): if the question set is asked once every ${(gain / (maintFour - maintCode)).toFixed(2)} change rounds, the extra maintenance of four levels pays for itself`); ``` ``` level nodes edges internal nodes + external parties ---------------------------------------------------- context 6 5 1 + 5 container 17 30 12 + 5 component 28 46 23 + 5 code 61 79 56 + 5 46 of the 46 hand-written component edges come back when promoted from code edges question answer level nodes read at code level extra ------------------------------------------------------------ D01 context 6 61 55 D02 context 2 57 55 D03 container 9 44 35 D04 container 2 5 3 D05 container 6 29 23 D06 container 4 18 14 D07 component 3 8 5 D08 component 5 13 8 D09 component 6 17 11 D10 component 5 12 7 D11 code 3 3 0 D12 code 3 3 0 D13 code 5 5 0 D14 code 3 3 0 ------------------------------------------------------------ total 62 278 216 the same question read at all four levels: question context container component code D01 6 17 28 61 D03 no answer 9 18 44 D11 no answer no answer no answer 3 answer level distribution: context=2, container=4, component=4, code=4 only context answered 2/14 unanswered 12 reading 8 only container answered 6/14 unanswered 8 reading 51 only component answered 10/14 unanswered 4 reading 112 only code answered 14/14 unanswered 0 reading 278 all four levels keep 112 nodes in maintenance, code level alone keeps 61; difference 51 one round of questions saves 216 unnecessary node reads (278 instead of 62) break-even (VM10): if the question set is asked once every 4.24 change rounds, the extra maintenance of four levels pays for itself ``` #### The Size of the Levels The four levels' node counts are 6, 17, 28, and 61; the edge counts are 5, 30, 46, and 79. The context level consists of nothing more than one system box and five outside parties; the code level has fifty-six internal nodes. The growth between scale levels is roughly a doubling each time, and this is a deliberate design: a reader moving from one level to the next doubles the number of nodes they see at each step, not multiplies it by fifty. The promotion method's correctness is tested in the output's second line: all forty-six of the forty-six hand-written component edges come back when promoted from the code edges. The upper levels are not independent documents; they're the same graph read at a coarse resolution — as long as they're derived, they cannot contradict the level below; when they're hand-written, they can. #### At Which Level a Question Gets Answered The fourteen questions are not split equally across the four levels: two find their answer at the context level, four at the container level, four at the component level, four at the code level. The total reading cost, with every question read at its own answer level, is **62 nodes.** This is the lesson's zero point: the minimum reading needed to answer all fourteen questions. If the same fourteen questions were answered by looking only at the code level, the number of nodes read would be **278.** The difference is **216 unnecessary nodes** — nodes read that contribute nothing to the answer, read only because the right scale was not at hand. The first two questions are the most expensive: someone asking about the network's relationship with the outside reads six nodes at the context level, sixty-one at the code level. Fifty-five nodes are detail that does not change the answer. For the four questions whose answer level is code, the unnecessary-node count is zero; the layered approach gains them nothing. The gain grows with the distance between a question's answer level and the level the reader is looking at. #### The Two Directions of the Wrong Level The "same question read at all four levels" table shows the cost for a single question. D01, which asks about the network's external relationships, reads 6 nodes at the context level, 17 at the container level, 28 at the component level, 61 at the code level — the answer can be found at every level, but its cost climbs tenfold. D03, which asks about the loan service's neighbors, is **unanswered** at the context level: there's no node called the loan service at that level, it stays inside the system box. D11, which asks about the late-penalty calculation, is unanswered at all three of the first levels. Both directions produce a number, and the numbers are not symmetric. **Looking too deep** gives an answer but reads unnecessary nodes; it is a measurable, tolerable waste. **Looking too high** gives no answer at all; the reader either gives up or guesses. The table below counts this at the scheme level: a document that keeps only the context level answers two of the fourteen questions, twelve stay unanswered. The container level alone answers six, the component level alone answers ten. The code level alone answers all of them but reads 278 nodes. #### The Cost of Maintenance Keeping all four levels means keeping 112 nodes in maintenance; keeping only the code level means 61. The difference is 51 nodes, and this difference is not free: every node at the container and component level has to be updated whenever its counterpart on the code side changes. What's gained in return is 216 unnecessary node reads saved per round of questions. Because the two quantities are in different units, an assumption is needed: reading a node and keeping a node current are counted as equal cost (**VM10**). Under this assumption, the break-even point is this: if the question set is asked once every **4.24 change rounds**, the extra maintenance of the four levels pays for itself. If the question frequency falls below that — meaning nobody looks at the document while the code keeps changing — keeping four levels loses money. The number itself depends on the model's inputs; the result that actually carries over is this: the layered scheme's payoff depends on **how often it is read**, not on how good the document looks. #### Summary - The layered approach models the same system at the context, container, component, and code level; the levels show not different things but the same thing at a different scale. A container is a separately running, separately deployed unit; a component is a separable part inside a container and is not deployed on its own. - The model network's node counts are 6, 17, 28, 61; edge counts are 5, 30, 46, 79. The upper levels' edges are not written; they're promoted by replacing the code edges' ends with their ancestors — 46 of the 46 hand-written component edges come back this way. - When the fourteen questions are each read at their own answer level, 62 nodes are read in total; if all of them were answered by looking only at the code level, 278 nodes would be read, and the 216-node difference is unnecessary. - Looking too deep gives an expensive answer (61 nodes instead of 6 for D01); looking too high gives none at all: D03 is unanswered at the context level, D11 at all of the first three levels. - Four levels keep 112 nodes in maintenance, the code level alone keeps 61; if reading and maintenance are counted as equal cost (VM10), the extra maintenance pays for itself once the question set is asked every 4.24 change rounds. #### Next Step Both lessons measured a single structure made of nodes and edges: what depends on what. Every question asked was of that kind. But not all of someone's later questions take the form "what depends on what." Questions like "what order do the calls go in when a loan request comes in," "what states does a copy pass through," "which machine does this unit run on" cannot be answered by looking at a node-edge structure; a notation that does not carry order, state, and placement does not contain the answer to these questions. The next lesson separates diagram types by their function — structure-showing, sequence-showing, state-showing, deployment-showing — and measures: which type is the right one per question, what information a question is left missing when answered with the wrong type, and how many items each type has to update per code change. ### Selecting the Diagram Type by Question Source: https://academia.sh/en/courses/architectural-documentation/selecting-diagram-type-by-question Mapping the structure-showing, sequence-showing, state-showing, and deployment-showing diagram types onto a question set: the correct type per question, how many information items are left missing when a question is answered with the wrong type, and how many items each type has to update per code change. The previous two lessons measured a single structure: nodes and edges. Views split that structure along its axis of concern, the layered approach along its scale axis. Every question asked had the same shape — what depends on what. Someone asking later does not always ask in this shape. In the regional library network, someone asking "what order do the calls go in when a loan request comes in" finds no answer by looking at the dependency list: the list says who can call whom, not which one gets called first. The question "what states does a copy pass through" asks for neither dependency nor sequence; it asks for state and transition. These call for different **diagram types**. This lesson does not refer to types by a notation name. A type is defined by the information items it can carry and referred to by its function: **structure-showing**, **sequence-showing**, **state-showing**, **deployment-showing**. Two things are measured: which type answers which question, and how much it costs to keep that type current. #### Function Separates the Type A structure-showing diagram carries participants, the static link between them, the link's direction, and multiplicity. A sequence-showing diagram carries participants, messages, the order of messages, asynchrony, and conditional branches. A state-showing diagram carries states, transitions, triggers, and guard conditions. A deployment-showing diagram carries nodes, artifacts, placement, and communication paths. The interesting thing about the lists is how little they overlap: the four lists take up seventeen slots in total but contain sixteen distinct information items — only participant appears in more than one type. Types are not different drawing habits, they are **different information sets**; that is why they cannot substitute for each other. The model has three parts. The item lists each type carries are hand-written (**VM11**). So are the seven diagrams drawn for the network and the code items each one is linked to (**VM12**); because a deployment-showing diagram is linked to containers and nodes rather than code, its linked-item count is kept low. The question set is eighteen questions (**VM13**), and a type answers a question only if it carries every item the answer needs (**VM14**). ```js // views/type.mjs — MODEL regional library network: diagram types are defined by // function, mapped onto a question set, and their maintenance cost is counted. Not a // real notation or product; item lists and questions are chosen by hand. // TYPE — the information items each diagram type can carry (VM11) const TYPE = { structure: ["participant", "static-link", "direction", "multiplicity"], sequence: ["participant", "message", "sequence", "asynchrony", "conditional-branch"], state: ["state", "transition", "trigger", "guard-condition"], deployment: ["node", "artifact", "placement", "communication-path"], }; const TYPES = Object.keys(TYPE); // DIAGRAM — the diagrams drawn for the network and the code items each is linked to // (VM12). A deployment-showing diagram links to containers and nodes, not code, so its // linked-item count is kept low. const DIAGRAM = { "structure/loan-service": { type: "structure", linked: ["open-loan", "take-return", "extension", "duration-calc", "limit-check", "late-penalty", "reservation-queue", "reservation-notification"] }, "structure/data-schema": { type: "structure", linked: ["loan-table", "penalty-table", "copy-table"] }, "sequence/loan-opening": { type: "sequence", linked: ["open-loan", "duration-calc", "limit-check", "copy-status", "loan-table", "topic-definition"] }, "sequence/reservation-notification": { type: "sequence", linked: ["reservation-queue", "reservation-notification", "queue-write", "queue-read", "message-body"] }, "state/copy": { type: "state", linked: ["copy-status", "open-loan", "take-return", "reservation-queue"] }, "state/penalty": { type: "state", linked: ["penalty-accrual", "late-penalty", "penalty-table"] }, "deployment/network": { type: "deployment", linked: ["sync-job", "subscription"] }, }; // QUESTION — needs: the information items required for an answer (VM13). A type // answers a question only if it carries every needed item (VM14). const QUESTION = [ { id: "U01", text: "what items does the loan core depend on", needs: ["participant", "static-link"] }, { id: "U02", text: "which direction does the dependency go", needs: ["static-link", "direction"] }, { id: "U03", text: "how many loan records can a member have", needs: ["participant", "multiplicity"] }, { id: "U04", text: "which call comes first in the loan-opening flow", needs: ["message", "sequence"] }, { id: "U05", text: "is the notification call asynchronous", needs: ["message", "asynchrony"] }, { id: "U06", text: "where does the flow branch if the limit is exceeded", needs: ["sequence", "conditional-branch"] }, { id: "U07", text: "what states does a copy pass through", needs: ["state", "transition"] }, { id: "U08", text: "which transition does a return trigger", needs: ["transition", "trigger"] }, { id: "U09", text: "can a reserved copy be checked out directly", needs: ["transition", "guard-condition"] }, { id: "U10", text: "which node does the loan service run on", needs: ["artifact", "placement"] }, { id: "U11", text: "what kind of path connects the branch to the center", needs: ["node", "communication-path"] }, { id: "U12", text: "how many separate nodes are there", needs: ["node"] }, { id: "U13", text: "which participants exist in the loan flow", needs: ["participant"] }, { id: "U14", text: "under what condition does penalty accrual stop", needs: ["transition", "guard-condition"] }, { id: "U15", text: "which message triggers the late penalty in which state", needs: ["message", "sequence", "state"] }, { id: "U16", text: "which participant on which node sends which message", needs: ["node", "participant", "message"] }, { id: "U17", text: "in what order is the reservation notification sent", needs: ["message", "sequence"] }, { id: "U18", text: "which artifact is copied to which node", needs: ["artifact", "placement", "node"] }, ]; const col = (s, n) => String(s).padEnd(n); const missing = (t, q) => q.needs.filter((i) => !TYPE[t].includes(i)); const correctTypes = (q) => TYPES.filter((t) => missing(t, q).length === 0); // ---- type table ---- console.log(col("type", 12) + col("info items", 14) + col("diagrams", 10) + "linked items"); console.log("-".repeat(46)); for (const t of TYPES) { const d = Object.values(DIAGRAM).filter((x) => x.type === t); console.log(col(t, 12) + col(TYPE[t].length, 14) + col(d.length, 10) + d.reduce((a, x) => a + x.linked.length, 0)); } // ---- question -> type mapping and the missing information the wrong type leaves ---- console.log("\nquestion needed item(s) correct type nearest wrong type"); console.log("-".repeat(78)); let single = 0, multiple = 0, none = 0, missingTotal = 0; for (const q of QUESTION) { const c = correctTypes(q); c.length === 1 ? single++ : c.length > 1 ? multiple++ : none++; const nearest = TYPES.filter((t) => !c.includes(t)).map((t) => [t, missing(t, q).length]) .sort((a, b) => a[1] - b[1] || TYPES.indexOf(a[0]) - TYPES.indexOf(b[0]))[0]; missingTotal += nearest[1]; console.log(col(q.id, 10) + col(q.needs.join("+"), 31) + col(c.length ? c.join(",") : "- none -", 20) + `${nearest[0]} (${nearest[1]} item(s) missing)`); } console.log("-".repeat(78)); console.log(`${QUESTION.length} questions: single correct type ${single}, multiple types ${multiple}, no type ${none}`); console.log(`the four types together answer ${QUESTION.length - none}/${QUESTION.length} questions`); console.log(`had the nearest wrong type been picked, ${missingTotal} items would be missing in total (${(missingTotal / QUESTION.length).toFixed(2)} per question)`); for (const q of QUESTION.filter((x) => correctTypes(x).length === 0)) console.log(` ${q.id}: needs ${q.needs.join("+")} — ` + TYPES.map((t) => `${t} does not carry ${missing(t, q).join("/")}`).join("; ")); // ---- what each type answers on its own ---- console.log(""); for (const t of TYPES) { const y = QUESTION.filter((q) => correctTypes(q).includes(t)); console.log(`${col("only " + t, 18)}answers ${col(`${y.length}/${QUESTION.length}`, 8)}(${y.map((q) => q.id).join(" ")})`); } // ---- maintenance: 150 code changes ---- // WEIGHT — how often each code item changes (VM15); chosen by hand const WEIGHT = { "open-loan": 9, "take-return": 7, extension: 5, "duration-calc": 8, "limit-check": 6, "late-penalty": 8, "reservation-queue": 5, "reservation-notification": 4, "copy-status": 6, "sync-job": 3, "queue-write": 3, "queue-read": 3, "message-body": 4, "penalty-accrual": 5, "topic-definition": 2, subscription: 2, "loan-table": 4, "penalty-table": 3, "copy-table": 3, // code items not linked to any diagram "shelf-location": 4, "external-query": 5, "record-conversion": 4, "create-member": 4, "cache-key": 3, }; const ITEMS = Object.keys(WEIGHT); function prng(seed) { let s = seed >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 2 ** 32; } const rand = prng(20903); // seed is visible const totalWeight = ITEMS.reduce((t, i) => t + WEIGHT[i], 0); const pickItem = () => { let r = rand() * totalWeight; for (const i of ITEMS) if ((r -= WEIGHT[i]) < 0) return i; return ITEMS.at(-1); }; const N = 150; const update = Object.fromEntries(TYPES.map((t) => [t, 0])); let untouched = 0; for (let i = 0; i < N; i++) { const item = pickItem(); let touched = false; for (const d of Object.values(DIAGRAM)) if (d.linked.includes(item)) { update[d.type]++; touched = true; } if (!touched) untouched++; } console.log(`\n${N} code changes (seed 20903); ${untouched} changes touch no diagram`); console.log(col("type", 12) + col("items updated", 17) + col("answers", 8) + "maintenance per answer"); console.log("-".repeat(56)); for (const t of TYPES) { const y = QUESTION.filter((q) => correctTypes(q).includes(t)).length; console.log(col(t, 12) + col(update[t], 17) + col(y, 8) + (update[t] / y).toFixed(1)); } const totalUpdate = TYPES.reduce((a, t) => a + update[t], 0); console.log("-".repeat(56)); console.log(col("total", 12) + col(totalUpdate, 17) + col(QUESTION.length - none, 8) + (totalUpdate / (QUESTION.length - none)).toFixed(1)); ``` ``` type info items diagrams linked items ---------------------------------------------- structure 4 2 11 sequence 5 2 11 state 4 2 7 deployment 4 1 2 question needed item(s) correct type nearest wrong type ------------------------------------------------------------------------------ U01 participant+static-link structure sequence (1 item(s) missing) U02 static-link+direction structure sequence (2 item(s) missing) U03 participant+multiplicity structure sequence (1 item(s) missing) U04 message+sequence sequence structure (2 item(s) missing) U05 message+asynchrony sequence structure (2 item(s) missing) U06 sequence+conditional-branch sequence structure (2 item(s) missing) U07 state+transition state structure (2 item(s) missing) U08 transition+trigger state structure (2 item(s) missing) U09 transition+guard-condition state structure (2 item(s) missing) U10 artifact+placement deployment structure (2 item(s) missing) U11 node+communication-path deployment structure (2 item(s) missing) U12 node deployment structure (1 item(s) missing) U13 participant structure,sequence state (1 item(s) missing) U14 transition+guard-condition state structure (2 item(s) missing) U15 message+sequence+state - none - sequence (1 item(s) missing) U16 node+participant+message - none - sequence (1 item(s) missing) U17 message+sequence sequence structure (2 item(s) missing) U18 artifact+placement+node deployment structure (3 item(s) missing) ------------------------------------------------------------------------------ 18 questions: single correct type 15, multiple types 1, no type 2 the four types together answer 16/18 questions had the nearest wrong type been picked, 31 items would be missing in total (1.72 per question) U15: needs message+sequence+state — structure does not carry message/sequence/state; sequence does not carry state; state does not carry message/sequence; deployment does not carry message/sequence/state U16: needs node+participant+message — structure does not carry node/message; sequence does not carry node; state does not carry node/participant/message; deployment does not carry participant/message only structure answers 4/18 (U01 U02 U03 U13) only sequence answers 5/18 (U04 U05 U06 U13 U17) only state answers 4/18 (U07 U08 U09 U14) only deployment answers 4/18 (U10 U11 U12 U18) 150 code changes (seed 20903); 24 changes touch no diagram type items updated answers maintenance per answer -------------------------------------------------------- structure 97 4 24.3 sequence 74 5 14.8 state 59 4 14.8 deployment 11 4 2.8 -------------------------------------------------------- total 241 16 15.1 ``` #### The Correct Type per Question **Fifteen of the eighteen questions have a single correct type.** One question is answered by two types at once: U13, which asks which participants exist in the loan flow, can be answered by both the structure-showing and the sequence-showing diagram, because participant is the only item found in more than one type. Two questions are answered by no type. This is where it becomes clear that choosing a type is not a preference. When asked which call comes first in the loan-opening flow, the sequence-showing diagram is not an option, it is the **only option**; no other type carries the sequence item. In the same way, only the structure-showing diagram carries multiplicity, only the state-showing diagram carries guard condition, only the deployment-showing diagram carries communication path. When a question is asked, which diagram to draw is not up for debate; the information items the question requires determine the type. A document that settles for a single type answers between four and five questions. The sequence-showing type has the largest share (5/18), because it is the type with the longest item list. The four together answer sixteen questions. #### The Gap the Wrong Type Leaves The real measure is in the column on the right. For each question, the **nearest wrong type** that could be picked instead of the correct one is written down, along with the number of items that type does not carry. The total is 31 information items, 1.72 per question. The number matters for two reasons. First, the wrong type does not leave the question unanswered — it leaves it **half-answered**. When asked about the loan flow's first call, the structure-showing diagram lists the participants; the reader sees something, even something relevant, but the sequence information is not there. It is not obvious that the answer found is incomplete; the missing item is invisible precisely because it is missing. Second, the nearest wrong type is sometimes very near. U15 asks which message triggers the late penalty in which state; the sequence-showing diagram carries two of the three required items and only fails to carry the state item. A single item is missing, but the missing item is half the question. U16 is likewise one item away from the sequence-showing diagram: node. These two questions are answered by no type, and the reason is the same as in the previous lesson: **the question asks for the items of two types to be combined.** Tying a runtime message to a state, or tying the participant sending a message to a node, is more than any single type can carry. This is not a gap that can be closed by adding more types; every new type arrives with its own information set, and a gap remains between the sets. #### Maintenance Cost The table below measures this by tying the diagrams to the code. A hundred and fifty code changes are generated; the item selection comes from a hand-written weight table (**VM15**) and the seed 20903 is visible. A change requires one item update in every diagram linked to the code item that changed. **24 of the 150 changes touch no diagram** — part of the code does not appear in any diagram. The remaining changes generate a total of 241 diagram item updates. The difference between types is large. The structure-showing type asks for 97 updates and answers four questions: 24.3 per answer. The deployment-showing type asks for 11 updates and also answers four questions: 2.8 per answer. The ninefold gap between them comes not from drawing skill but from **how often things link to it**. A structure-showing diagram is linked directly to code items, and code is the thing that changes most often; a deployment-showing diagram is linked to nodes and containers, and those change rarely. The rule that follows is this: if a diagram's value is the number of questions it answers, its cost is the number of items updated per code change, and the ratio between the two varies ninefold from type to type. If a set of documents has to be trimmed, the place to cut is wherever the maintenance per answer is highest — in this model, the structure-showing diagrams. The numbers depend on the model's inputs; if the linked-item lists were narrowed, the cost would drop, but the questions answered would not change, because the answer comes from the type's information set, not from the linked-item list. #### Summary - Diagram types are separated not by their notation name but by the information item they carry: the structure-showing type carries static links and multiplicity, the sequence-showing type carries messages and order, the state-showing type carries transitions and guard conditions, the deployment-showing type carries nodes and placement; of sixteen distinct items, only participant is found in more than one type. - Fifteen of the eighteen questions have a single correct type, one is answered by two types, two are answered by none; the four types together answer 16/18 questions, a single type answers between 4 and 5. - The wrong type leaves a question half-answered rather than unanswered: had the nearest wrong type been picked, a total of 31 information items would be missing, 1.72 per question. In U15, the sequence-showing type carries two of three items; the one missing item is half the question. - The two questions no type can answer both ask for the items of two types to be combined; adding a type does not close this gap, it opens a new one between the new sets. - With a generator seeded at 20903, 24 of 150 code changes touch no diagram; the rest generate 241 item updates. Maintenance per answer is 24.3 for the structure-showing type, 14.8 for the sequence- and state-showing types, 2.8 for the deployment-showing type — the ninefold gap comes from how often things link to it. #### Next Step All three lessons asked questions of the same kind: what something depends on, in what order it is called, where it runs. Every one of their answers sits in the system's **structure**, and once you know where to look, a definite answer comes out. Some of someone's later questions are not of this kind. Questions like "is the network fast enough at peak hour," "does checkout stop if a branch disconnects," "how open is this system to adding a new branch" do not sit in any diagram as a node or an edge. These are not about the system's structure but about **the quality of its behavior**, and as stated they cannot be tested: "fast enough" is not a measure. The next lesson turns these questions into scenarios with stimulus, environment, response, and response-measure fields, and measures: how many scenarios have every field filled in, and how many scenarios cannot be tested because their measure was never written down. ### Quality Attribute Scenarios Source: https://academia.sh/en/courses/architectural-documentation/quality-attribute-scenarios Writing a quality requirement so it can be tested: the same twelve requirements written in plain-sentence and fielded-scenario form, counting how many of each come out testable, turning the empty fields into questions, and measuring the disagreement an unwritten threshold causes at delivery. The previous lessons split views and diagram types by question: which picture answers which is now clear. But part of what someone who comes later asks cannot be answered by a picture. Questions like "is this system fast enough," "what happens when this branch goes offline," "is this slowdown acceptable" are not closed by a box-and-arrow picture; their answer is a number. This lesson measures where that number gets written, and what happens when it is not. What a quality attribute is and which fields a quality attribute scenario has were defined in the Quality and Testing Fundamentals course; that definition is not repeated here. The question here is narrow: given a requirement list, how many of its items are **testable**. Testability is not an opinion but a criterion — a requirement is testable if a test procedure can be written from it, and a procedure needs a stimulus to trigger, an environment to set up, a response to observe, and a threshold to compare against. Which quality attributes matter for a given system is separate, taken up in the course that follows this one; no attribute list is derived here. #### The Requirement Set to Measure The block below **models** twelve quality requirements in two forms: the plain sentence and the fielded scenario. The fielded form carries only what the plain sentence says — no information that is not in the sentence gets written into a field. The model is also written to disk; the next two blocks read the same set. ```js // set.mjs — models the requirement set in two forms and measures testability import { mkdirSync, writeFileSync } from "node:fs"; // VM16: twelve quality requirements collected from the regional library network. The // fielded form carries exactly what the plain sentence says; no field is invented. const REQUIREMENT = [ ["Catalog search must respond quickly.", "a member searches the catalog", "", "a result list is returned", "", ""], ["Lending must not stop when the branch connection drops.", "the connection between the branch and the center drops", "", "lending continues", "", ""], ["The system must stay up during peak hours.", "", "peak hour", "the system keeps accepting requests", "", ""], ["A loan record must not be lost.", "", "", "the record stays readable", "", ""], ["The overdue notification must go out on time.", "the loan period expires", "", "a notification is sent", "", ""], ["The catalog bulk upload must fit inside the nightly window.", "a bulk upload starts", "the nightly maintenance window", "the upload finishes", "", ""], ["A member must be able to see their own loan history without waiting.", "the member opens the history page", "", "the history is listed", "", ""], ["Under 500 concurrent members, the loan operation must complete within 400 ms for 95% of requests.", "a member sends a loan request", "500 concurrent members", "a loan record is created", "95% of requests under 400 ms", ""], ["When the central database node is lost, the loan service must fail over to the replica within 60 seconds.", "the primary database node stops responding", "business-hours load", "the service runs on the replica node", "failover within 60 seconds", ""], ["A new branch definition must reach production in a single deployment within 1 hour.", "a branch definition is added", "a single deployment window", "the branch appears in production", "1 hour from addition to release", "branch opening calendar: two branches a month"], ["The system must support backups.", "", "", "", "", ""], ["The report screen must open within a reasonable time.", "a manager opens the report screen", "", "the report is displayed", "", ""], ]; const FIELD = ["stimulus", "environment", "response", "measure", "source"]; const SET = REQUIREMENT.map(([plain, ...value], i) => ({ no: i + 1, plain, field: Object.fromEntries(FIELD.map((f, j) => [f, value[j]])), })); mkdirSync("set", { recursive: true }); writeFileSync("set/set.mjs", `export const FIELD = ${JSON.stringify(FIELD)};\n` + `export const SET = ${JSON.stringify(SET, null, 0)};\n`); // Testability criterion: writing a test procedure requires a stimulus to trigger, an // environment to set up, a response to observe, and a threshold to compare against. The // threshold is a number carrying a unit; both forms are searched with the same pattern. const THRESHOLD = /(\d+\s*(ms|seconds?|minutes?|hours?|days?)|\d+\s*%|\d+%)/; const CONDITION = /(is lost|concurrent|deployment|window|drops|peak|load|under)/; const FOUR = ["stimulus", "environment", "response", "measure"]; const plainTestable = (r) => THRESHOLD.test(r.plain) && CONDITION.test(r.plain); const fieldedTestable = (r) => FOUR.every((f) => r.field[f] !== "") && THRESHOLD.test(r.field.measure); console.log(`${"no".padStart(2)} ${"plain form".padEnd(12)}${"fielded form (filled field)".padEnd(40)}empty field`); for (const r of SET) { const filled = FOUR.filter((f) => r.field[f] !== ""); console.log(`${String(r.no).padStart(2)} ` + `${(plainTestable(r) ? "testable" : "-").padEnd(12)}` + `${(filled.join(",") || "-").padEnd(40)}` + `${4 - filled.length}`); } const emptyFour = SET.reduce((s, r) => s + FOUR.filter((f) => r.field[f] === "").length, 0); const emptyFive = SET.reduce((s, r) => s + FIELD.filter((f) => r.field[f] === "").length, 0); const wordsPlain = SET.reduce((s, r) => s + r.plain.length, 0); const wordsFielded = SET.reduce((s, r) => s + FIELD.reduce((t, f) => t + f.length + 2 + r.field[f].length, 0), 0); console.log(`\ntestable: plain form ${SET.filter(plainTestable).length}/${SET.length}, ` + `fielded form ${SET.filter(fieldedTestable).length}/${SET.length}`); console.log(`empty field: in the four fields ${emptyFour}, including the fifth field (source) ${emptyFive}`); console.log(`writing cost (characters): plain ${wordsPlain}, fielded ${wordsFielded} ` + `(${(wordsFielded / wordsPlain).toFixed(2)}x)`); ``` ``` no plain form fielded form (filled field) empty field 1 - stimulus,response 2 2 - stimulus,response 2 3 - environment,response 2 4 - response 3 5 - stimulus,response 2 6 - stimulus,environment,response 1 7 - stimulus,response 2 8 testable stimulus,environment,response,measure 0 9 testable stimulus,environment,response,measure 0 10 testable stimulus,environment,response,measure 0 11 - - 4 12 - stimulus,response 2 testable: plain form 3/12, fielded form 3/12 empty field: in the four fields 20, including the fifth field (source) 31 writing cost (characters): plain 706, fielded 1406 (1.99x) ``` #### What the Fielded Form Changes In both forms, the number of testable requirements is three. This breaks the most common expectation loaded onto the fielded scenario: **opening fields does not manufacture information.** If the sentence carries no threshold, neither does the field; changing the form does not make a requirement testable. What does change is this: the plain form gives one piece of information for the nine untestable requirements — no number. The fielded form gives, for each one, **which field** is empty. The fourth requirement is missing three fields, the eleventh is empty in all four, the sixth is missing only the measure. This separates the size and the kind of the gap; neither shows up in the plain sentence. The empty-field count is twenty, and that number is directly a list of questions. The fielded form's writing cost is 1.99 times the plain form's: 706 characters against 1,406. That is the price; the payoff is knowing exactly where to ask each of the twenty questions. The fifth field stands apart. `source` holds where the threshold came from, and it is filled in only one of the twelve requirements. The testability criterion looks at the first four fields only, because a test procedure can be written without knowing where the threshold came from. Why the fifth field still matters is measured in this lesson's last section. #### The Cost of an Unwritten Threshold An untestable requirement does not charge a cost immediately; the cost surfaces at delivery. When a threshold is not written down, it does not disappear — it sits in each side's head as two separate thresholds. The requester thinks of one number when they say "fast," the developer another. The block below models this: each requirement gets a scale where smaller is better, a threshold per side, and a measured value for the delivered system. ```js // disagreement.mjs — reads set/set.mjs written by set.mjs; measures the disagreement an // untestable requirement causes at delivery import { SET } from "./set/set.mjs"; // VM17: every requirement has a scale where smaller is better. An untestable requirement // carries two unwritten thresholds, one for the requester and one for the developer. The // three testable requirements have a written threshold; both sides use the same number. const THRESHOLD = [ // [scale, requester's threshold, developer's threshold, delivery range] ["search response time (ms)", 300, 1200, [200, 1500]], ["loan-halt duration (min)", 5, 120, [0, 180]], ["rejected requests (per mille)", 1, 20, [0, 40]], ["lost records per year", 0, 3, [0, 6]], ["notification delay (hr)", 1, 24, [0, 36]], ["bulk upload duration (min)", 60, 240, [30, 300]], ["history page (ms)", 500, 2500, [300, 3000]], ["loan operation p95 (ms)", 400, 400, [250, 900]], ["failover (sec)", 60, 60, [20, 180]], ["release rollout (min)", 60, 60, [20, 240]], ["restore from backup (hr)", 2, 24, [1, 48]], ["report screen (sec)", 3, 20, [1, 30]], ]; // VM18: the delivered measurement is drawn from each requirement's own range with a // uniform distribution. The generator is hand-written; the seed is visible. const generator = (seed) => () => { seed = (seed + 0x6d2b79f5) | 0; let t = Math.imul(seed ^ (seed >>> 15), 1 | seed); t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; const SEED = 20802; const DELIVERY = 200; const random = generator(SEED); const result = THRESHOLD.map(() => ({ accept: 0, dispute: 0, reject: 0 })); let disputesPerDelivery = 0; for (let t = 0; t < DELIVERY; t += 1) { let n = 0; THRESHOLD.forEach(([, requester, developer, [low, high]], i) => { const measured = low + random() * (high - low); if (measured <= Math.min(requester, developer)) result[i].accept += 1; else if (measured > Math.max(requester, developer)) result[i].reject += 1; else { result[i].dispute += 1; n += 1; } }); disputesPerDelivery += n; } console.log(`seed ${SEED}, ${DELIVERY} deliveries; each row is one requirement (delivery count)`); console.log(`${"no".padStart(2)} ${"scale".padEnd(30)}${"threshold".padStart(12)}` + `${"accept".padStart(8)}${"dispute".padStart(13)}${"reject".padStart(7)}`); THRESHOLD.forEach(([scale, requester, developer], i) => { console.log(`${String(i + 1).padStart(2)} ${scale.padEnd(30)}` + `${(requester === developer ? `${requester}` : `${requester}/${developer}`).padStart(12)}` + `${String(result[i].accept).padStart(8)}${String(result[i].dispute).padStart(13)}` + `${String(result[i].reject).padStart(7)}`); }); const written = THRESHOLD.map(([, a, b], i) => (a === b ? i : -1)).filter((i) => i >= 0); const unwritten = THRESHOLD.map((_, i) => i).filter((i) => !written.includes(i)); const sum = (list) => list.reduce((s, i) => s + result[i].dispute, 0); console.log(`\n${written.length} requirements with a written threshold: ${sum(written)} disputes`); console.log(`${unwritten.length} requirements with no written threshold: ${sum(unwritten)} disputes`); console.log(`average disputes per delivery: ` + `${(disputesPerDelivery / DELIVERY).toFixed(2)} (across ${SET.length} requirements)`); ``` ``` seed 20802, 200 deliveries; each row is one requirement (delivery count) no scale threshold accept dispute reject 1 search response time (ms) 300/1200 12 142 46 2 loan-halt duration (min) 5/120 10 112 78 3 rejected requests (per mille) 1/20 3 99 98 4 lost records per year 0/3 0 90 110 5 notification delay (hr) 1/24 5 117 78 6 bulk upload duration (min) 60/240 13 148 39 7 history page (ms) 500/2500 8 158 34 8 loan operation p95 (ms) 400 45 0 155 9 failover (sec) 60 56 0 144 10 release rollout (min) 60 35 0 165 11 restore from backup (hr) 2/24 5 92 103 12 report screen (sec) 3/20 9 104 87 3 requirements with a written threshold: 0 disputes 9 requirements with no written threshold: 1062 disputes average disputes per delivery: 5.31 (across 12 requirements) ``` In the three requirements with a written threshold, the disagreement count is zero — a structural result of the model: with a single threshold, both sides look at the same measurement and reach the same conclusion. The eighth requirement is rejected in 155 of 200 deliveries; a failure, but an undisputed one. Rejection is a decision, not a disagreement. The nine requirements with no written threshold produce 1,062 disagreements in 200 deliveries — 5.31 per delivery. That means every delivery, more than five of the twelve requirements get two different answers to "was it met." The highest rate is on the seventh requirement: 158 of 200 deliveries end in disagreement. The reason shows in the scale — the gap between the two thresholds (500 to 2,500 ms) covers most of the delivery range. **The probability of disagreement grows with the gap between the two sides' implicit thresholds.** The more ambiguous the sentence, the wider the gap. That gives the lesson's cost comparison. Writing the set in fielded form costs 700 extra characters and twenty questions asked, paid once. Leaving a requirement untestable costs 5.31 disagreements per delivery, paid again at every delivery. #### The Question Set and the Fifth Field This course's measure is how many of the questions someone later asks a document, the document answers. Five questions get asked of quality requirements, and each lands on one field. ```js // question.mjs — reads set/set.mjs written by set.mjs; applies the question set to the set import { SET } from "./set/set.mjs"; // The five questions someone who comes later asks about a quality requirement; each question lands on one field. const QUESTION = [ ["When is this requirement considered satisfied?", "measure"], ["Under what condition will it be measured?", "environment"], ["What will trigger the test?", "stimulus"], ["What will be observed from the system?", "response"], ["Where did this number come from?", "source"], ]; console.log(`${"question".padEnd(48)}${"answered".padStart(11)}${"unanswered".padStart(13)}`); let answered = 0; for (const [question, field] of QUESTION) { const d = SET.filter((r) => r.field[field] !== "").length; answered += d; console.log(`${question.padEnd(48)}${`${d}/${SET.length}`.padStart(11)}` + `${String(SET.length - d).padStart(13)}`); } const total = QUESTION.length * SET.length; console.log(`total ${total} questions: ${answered} answered, ${total - answered} unanswered`); // Number of questions to ask to make a requirement testable (the fifth field is excluded: // testability only looks at the four fields). const FOUR = ["measure", "environment", "stimulus", "response"]; const missing = SET.map((r) => [r.no, FOUR.filter((f) => r.field[f] === "").length, r.plain.slice(0, 34)]).sort((a, b) => a[1] - b[1] || a[0] - b[0]); console.log(`\n${"no".padStart(2)} ${"to ask".padStart(9)} requirement`); for (const [no, n, c] of missing) console.log(`${String(no).padStart(2)} ${String(n).padStart(9)} ${c}`); console.log(`to make the set testable: ${missing.reduce((s, [, n]) => s + n, 0)} questions; ` + `closed by one question ${missing.filter(([, n]) => n === 1).length}, ` + `needing three or more ${missing.filter(([, n]) => n >= 3).length}`); ``` ``` question answered unanswered When is this requirement considered satisfied? 3/12 9 Under what condition will it be measured? 5/12 7 What will trigger the test? 9/12 3 What will be observed from the system? 11/12 1 Where did this number come from? 1/12 11 total 60 questions: 29 answered, 31 unanswered no to ask requirement 8 0 Under 500 concurrent members, the 9 0 When the central database node is 10 0 A new branch definition must reach 6 1 The catalog bulk upload must fit i 1 2 Catalog search must respond quickl 2 2 Lending must not stop when the bra 3 2 The system must stay up during pea 5 2 The overdue notification must go o 7 2 A member must be able to see their 12 2 The report screen must open within 4 3 A loan record must not be lost. 11 4 The system must support backups. to make the set testable: 20 questions; closed by one question 1, needing three or more 2 ``` 29 of the sixty questions get answered, 31 do not. The answer rate falls steadily: what to observe from the system is known for eleven of the twelve requirements, what to trigger for nine, under what condition to measure for five, when it counts as met for only three. In other words, the requirement's author can describe **what they want**, not **when it is enough**. The lowest row is the fifth field: the threshold's source is written in only one of twelve requirements. Even two of the three requirements with a written threshold do not record where the number came from. The distinction looks small but its consequence is heavy: the 400 ms threshold is testable, but six months later, when someone asks "can this threshold be raised," it cannot be answered. In the one requirement whose source is written (the branch opening calendar, two branches a month), that question is answered: if the calendar slows, the threshold can be loosened. **The measure makes the requirement testable; the threshold's source makes it changeable.** The last table shows how the twenty questions are distributed: three are already testable, one closes with a single question, six need two each, two need three or four. The eleventh requirement is not really a requirement but a heading; with all four fields empty, it is not filled by one question but by a separate piece of work. #### Summary - Testability is a criterion: a requirement is testable if a test procedure can be written from it; the procedure needs a stimulus, an environment, a response, and a unit-bearing threshold. - The same twelve requirements come out 3/12 testable in both forms — the fielded form does not manufacture information, it makes the missing information visible: 20 empty fields, i.e. 20 askable questions. - The fielded form's writing cost is 1.99x (706 versus 1,406 characters), paid once; the nine requirements with no written threshold produce 1,062 disagreements in 200 deliveries, 5.31 per delivery. - The probability of disagreement grows with the gap between the two sides' implicit thresholds; the three requirements with a written threshold have zero disagreements, and even its rejections are undisputed. - 31 of 60 applications of the five questions go unanswered; the least-answered is "where did this number come from" (11/12 unanswered) — the measure enables testing, the source enables changing it. #### Next Step When the requirement set is made measurable, a document results: fielded scenarios, thresholds, sources. That document is in step with the code on the day it is written. The next lesson measures what comes after: the code keeps changing, the document stays put, and the gap grows. This gap is found by counting, not guessing: reality extracted from a module graph is compared against the document's claim, how drift grows across a sequence of changes is tracked, and two keeping-current regimes — manual update and a generated audit — are compared by the drift caught, the maintenance cost imposed, and the false alarms produced. ### Keeping Documentation Current Source: https://academia.sh/en/courses/architectural-documentation/keeping-documentation-current Actually measuring documentation drift: comparing the reality extracted from a module import graph against the document's claim, watching the drift grow across eight changes, and comparing three keeping-current regimes by the drift they catch, the maintenance cost they impose, and the false alarms they produce. The previous lesson made quality requirements measurable and produced a document; that document was in step with the code on the day it was written. This lesson measures what comes after: the code keeps changing, the document stays put, and the gap between them grows. This gap's name is **documentation drift**, and it is found not by guessing but by counting. The measurement method is borrowed. The Service Architectures course's Boundary-Drawing Criteria lesson extracted import edges from source files and counted boundary violations. The same extraction is used here to compare the graph read from code against the document's claim; the measured quantity is different. #### Code, Document, and Audit The block below defines three things: the regional library network system's module map, a fourteen-claim document, and the audit that tests the claims against the code. The module map is a **model**; each module is written to disk as a source file, and the graph is extracted from those files — the measurement happens on real files. Splitting the claims into three is the lesson's axis: a **rule** claim states architectural intent, and if the code does not comply, there is drift; a **snapshot** claim is a count of the code's current state on that day, and it goes stale when the code changes; an **unverifiable** claim cannot be checked against the code at all. ```js // environment.mjs — defines the codebase, the document, and the audit; when run directly, // reports the starting state. The next two blocks import this file. import { mkdirSync, rmSync, writeFileSync, readdirSync, readFileSync } from "node:fs"; // VM19: the six domain modules of the regional library network system. Each module's source // carries which modules it imports, which tables it writes to, and which table it owns. export const START = { branch: { imports: [], writes: ["branch"], owns: ["branch"] }, catalog: { imports: [], writes: ["book", "copy"], owns: ["book", "copy"] }, membership: { imports: [], writes: ["member"], owns: ["member"] }, loan: { imports: ["catalog", "membership"], writes: ["loan"], owns: ["loan"] }, notification: { imports: ["membership", "loan"], writes: ["notification"], owns: ["notification"] }, fee: { imports: ["loan"], writes: ["fee"], owns: ["fee"] }, }; export const write = (m) => { // the code/ directory is rebuilt from scratch on every write rmSync("code", { recursive: true, force: true }); mkdirSync("code", { recursive: true }); for (const [name, x] of Object.entries(m)) { writeFileSync(`code/${name}.mjs`, `// owns: ${x.owns.join(", ")}\n` + x.imports.map((i) => `import { ${i}Op } from "./${i}.mjs";\n`).join("") + `export const write = (t, v) => ({ t, v });\n` + `export function ${name}Op(v) {\n` + x.writes.map((t) => ` write("${t}", v);\n`).join("") + "}\n"); } }; // The import graph is extracted from source files. The method is borrowed from the Service Architectures course's // Boundary-Drawing Criteria lesson, where it measured boundary violations; here it measures documentation drift. export const graph = () => Object.fromEntries(readdirSync("code").map((d) => { const t = readFileSync(`code/${d}`, "utf8"); return [d.replace(".mjs", ""), { imports: [...t.matchAll(/from "\.\/(\w+)\.mjs"/g)].map((x) => x[1]), writes: [...t.matchAll(/write\("(\w+)"/g)].map((x) => x[1]), owns: t.match(/^\/\/ owns: (.+)$/m)[1].split(", "), }]; })); const importerOf = (g, a) => Object.entries(g).filter(([, m]) => m.imports.includes(a)).map(([x]) => x); const writerOf = (g, t) => Object.entries(g).filter(([, m]) => m.writes.includes(t)).map(([x]) => x); // The document carries three kinds of claim. RULE: architectural intent — a violation in the code is drift. SNAPSHOT: a count // of the code's current state — it goes stale when the code changes. UNVERIFIABLE: cannot be checked against the code at all. export const RULE = [ ["K1", "membership", "Membership does not import any domain module.", (g) => g.membership.imports.length === 0], ["K2", "catalog", "Only loan may import catalog.", (g) => importerOf(g, "catalog").every((a) => a === "loan")], ["K3", "notification", "No module may depend on notification.", (g) => importerOf(g, "notification").length === 0], ["K4", "fee", "Fee may only import the loan module.", (g) => g.fee.imports.every((a) => a === "loan")], ["K5", "branch", "Branch does not import any domain module.", (g) => g.branch.imports.length === 0], ["K6", "loan", "Only loan may write to the loan table.", (g) => writerOf(g, "loan").every((a) => a === "loan")], ]; export const SNAPSHOT = [ ["A1", "system", "The system consists of six domain modules.", (g) => Object.keys(g).length], ["A2", "loan", "Loan imports the catalog and membership modules.", (g) => g.loan.imports], ["A3", "system", "The graph has five import edges.", (g) => Object.values(g).reduce((s, m) => s + m.imports.length, 0)], ["A4", "notification", "Notification imports the membership and loan modules.", (g) => g.notification.imports], ["A5", "fee", "Fee writes only to the fee table.", (g) => g.fee.writes], ["A6", "catalog", "Catalog owns the book and copy tables.", (g) => g.catalog.owns], ]; export const UNVERIFIABLE = [ ["D1", "catalog", "The catalog system is sourced externally."], ["D2", "system", "The nightly maintenance window is 02:00-04:00."], ]; // VM20: eight changes applied over the next two quarters. The intent label says whether the change // complies with an architectural rule; the label is not asserted, it is compared against the audit result. export const CHANGE = [ ["reservation module is added", "compliant", "reservation", (m) => { m.reservation = { imports: ["membership"], writes: ["reservation"], owns: ["reservation"] }; }], ["loan imports notification", "non-compliant", "loan", (m) => m.loan.imports.push("notification")], ["fee imports membership", "non-compliant", "fee", (m) => m.fee.imports.push("membership")], ["notification drops its membership import", "compliant", "notification", (m) => { m.notification.imports = m.notification.imports.filter((a) => a !== "membership"); }], ["reservation writes to the loan table", "non-compliant", "reservation", (m) => m.reservation.writes.push("loan")], ["catalog takes over the tag table", "compliant", "catalog", (m) => { m.catalog.owns.push("tag"); m.catalog.writes.push("tag"); }], ["fee writes to the penalty table", "compliant", "fee", (m) => m.fee.writes.push("penalty")], ["branch imports membership", "non-compliant", "branch", (m) => m.branch.imports.push("membership")], ]; if (process.argv[1]?.endsWith("environment.mjs")) { write(structuredClone(START)); const g = graph(); const row = [ ...RULE.map(([ad, ozne, metin, sina]) => [ad, "rule", ozne, sina(g) ? "true" : "false", metin]), ...SNAPSHOT.map(([ad, ozne, metin, oku]) => [ad, "snapshot", ozne, String(oku(g)), metin]), ...UNVERIFIABLE.map(([ad, ozne, metin]) => [ad, "unverifiable", ozne, "-", metin]), ]; console.log(`${"claim".padEnd(6)}${"type".padEnd(13)}${"subject".padEnd(13)}${"t=0".padEnd(21)}text`); for (const [a, b, c, d, e] of row) { console.log(`${a.padEnd(6)}${b.padEnd(13)}${c.padEnd(13)}${d.padEnd(21)}${e}`); } console.log(`\n${Object.keys(g).length} modules, ` + `${Object.values(g).reduce((s, x) => s + x.imports.length, 0)} import edges; the document carries ${row.length} ` + `claims (${RULE.length} rule, ${SNAPSHOT.length} snapshot, ${UNVERIFIABLE.length} unverifiable)`); console.log(`t=0 drift: 0 false rule claims, 0 stale snapshot claims ` + `(the document was written the same day as the code)`); } ``` ``` claim type subject t=0 text K1 rule membership true Membership does not import any domain module. K2 rule catalog true Only loan may import catalog. K3 rule notification true No module may depend on notification. K4 rule fee true Fee may only import the loan module. K5 rule branch true Branch does not import any domain module. K6 rule loan true Only loan may write to the loan table. A1 snapshot system 6 The system consists of six domain modules. A2 snapshot loan catalog,membership Loan imports the catalog and membership modules. A3 snapshot system 5 The graph has five import edges. A4 snapshot notification membership,loan Notification imports the membership and loan modules. A5 snapshot fee fee Fee writes only to the fee table. A6 snapshot catalog book,copy Catalog owns the book and copy tables. D1 unverifiable catalog - The catalog system is sourced externally. D2 unverifiable system - The nightly maintenance window is 02:00-04:00. 6 modules, 5 import edges; the document carries 14 claims (6 rule, 6 snapshot, 2 unverifiable) t=0 drift: 0 false rule claims, 0 stale snapshot claims (the document was written the same day as the code) ``` #### How Drift Grows The document starts with zero drift. The next block applies the eight changes in order, retests all fourteen claims after each step, and never touches the document. ```js // drift.mjs — imports environment.mjs; measures the drift of an unupdated document, step by step import { START, CHANGE, RULE, SNAPSHOT, UNVERIFIABLE, write, graph } from "./environment.mjs"; const m = structuredClone(START); write(m); const initial = graph(); const expected = Object.fromEntries(SNAPSHOT.map(([ad, , , oku]) => [ad, JSON.stringify(oku(initial))])); const driftMeasure = (g) => ({ rule: RULE.filter(([, , , test]) => !test(g)).map(([ad]) => ad), snapshot: SNAPSHOT.filter(([ad, , , oku]) => JSON.stringify(oku(g)) !== expected[ad]).map(([ad]) => ad), }); console.log(`${"t".padStart(2)} ${"change".padEnd(42)}${"intent".padEnd(15)}` + `${"claim dropped".padEnd(16)}${"false rules".padStart(12)}${"stale snapshots".padStart(16)}` + `${"drift".padStart(7)}`); let previous = []; CHANGE.forEach(([ad, intent, , apply], t) => { apply(m); write(m); const s = driftMeasure(graph()); const all = [...s.rule, ...s.snapshot]; const fresh = all.filter((a) => !previous.includes(a)); previous = all; console.log(`${String(t + 1).padStart(2)} ${ad.padEnd(42)}${intent.padEnd(15)}` + `${(fresh.join(",") || "-").padEnd(16)}${String(s.rule.length).padStart(12)}` + `${String(s.snapshot.length).padStart(16)}${String(all.length).padStart(7)}`); }); const final = driftMeasure(graph()); const total = RULE.length + SNAPSHOT.length + UNVERIFIABLE.length; console.log(`\nafter eight changes, ${final.rule.length + final.snapshot.length} of ${total} claims are ` + `wrong, ${UNVERIFIABLE.length} are unverifiable, ` + `${total - final.rule.length - final.snapshot.length - UNVERIFIABLE.length} are correct ` + `(correct ones remaining: ${RULE.filter(([a]) => !final.rule.includes(a)).map(([a]) => a).join(", ")})`); console.log(`${CHANGE.filter(([, n]) => n === "compliant").length} compliant changes ` + `dropped no rule claim; ` + `${CHANGE.filter(([, n]) => n === "non-compliant").length} non-compliant changes ` + `dropped ${final.rule.length} rule claims`); ``` ``` t change intent claim dropped false rules stale snapshots drift 1 reservation module is added compliant A1,A3 0 2 2 2 loan imports notification non-compliant K3,A2 1 3 4 3 fee imports membership non-compliant K4 2 3 5 4 notification drops its membership import compliant A4 2 4 6 5 reservation writes to the loan table non-compliant K6 3 4 7 6 catalog takes over the tag table compliant A6 3 5 8 7 fee writes to the penalty table compliant A5 3 6 9 8 branch imports membership non-compliant K5 4 6 10 after eight changes, 10 of 14 claims are wrong, 2 are unverifiable, 2 are correct (correct ones remaining: K1, K2) 4 compliant changes dropped no rule claim; 4 non-compliant changes dropped 4 rule claims ``` Drift starts at the first change and grows in one direction: 2, 4, 5, 6, 7, 8, 9, 10. By the end, ten of the fourteen claims are wrong and two are unverifiable; the two that stay correct are rule claims. The document keeps talking, but most of it no longer matches the code. The two claim types behave differently. None of the four compliant changes dropped a rule claim; all of them only staled snapshot claims. All four non-compliant changes dropped a rule claim. **A wrong rule claim is a code defect; a stale snapshot claim is a document defect.** Both show up in the same table, but one is fixed in the code, the other in the document. #### Three Keeping-Current Regimes The same eight changes now run under three regimes. **Manual update**: whoever makes the change reviews only the claims of the module they touched. **Generated audit**: after every change, every claim is tested against the code. **Rule-only document**: the document never carries a snapshot claim. ```js // regime.mjs — imports environment.mjs; compares three keeping-current regimes on the same eight changes import { START, CHANGE, RULE, SNAPSHOT, UNVERIFIABLE, write, graph } from "./environment.mjs"; // manual: whoever makes the change reviews the claims of the module they touched. // audit: after every change, every claim is automatically tested against the code. // rule-only: the document carries only rule claims; no snapshot claim is ever written. const run = (regime) => { const m = structuredClone(START); write(m); const doc = regime === "rule-only" ? [] : SNAPSHOT.map(([ad, ozne, , oku]) => ({ ad, ozne, oku, value: JSON.stringify(oku(graph())) })); const caught = new Set(); let updates = 0, warnings = 0, falseAlarms = 0; for (const [, intent, touched, apply] of CHANGE) { apply(m); write(m); const g = graph(); const violated = RULE.filter(([, , , test]) => !test(g)); const stale = doc.filter((i) => JSON.stringify(i.oku(g)) !== i.value); if (regime === "manual") { for (const [ad, ozne] of violated) if (ozne === touched) caught.add(ad); for (const i of stale) if (i.ozne === touched) { i.value = JSON.stringify(i.oku(g)); updates += 1; } } else { for (const [ad] of violated) caught.add(ad); warnings += stale.length; if (intent === "compliant") falseAlarms += stale.length; for (const i of stale) { i.value = JSON.stringify(i.oku(g)); updates += 1; } } } const g = graph(); const actual = RULE.filter(([, , , test]) => !test(g)).map(([ad]) => ad); return { regime, doc, actual, caught, updates, warnings, falseAlarms, stale: doc.filter((i) => JSON.stringify(i.oku(g)) !== i.value).map((i) => i.ad) }; }; const RESULT = ["manual", "audit", "rule-only"].map(run); console.log(`${"regime".padEnd(10)}${"caught".padStart(9)}${"missed".padStart(18)}` + `${"updates".padStart(11)}${"snapshot warn.".padStart(16)}${"false alarms".padStart(14)}` + `${"still stale".padStart(13)}`); for (const s of RESULT) { const missed = s.actual.filter((a) => !s.caught.has(a)); console.log(`${s.regime.padEnd(10)}${`${s.caught.size}/${s.actual.length}`.padStart(9)}` + `${(missed.join(",") || "-").padStart(18)}${String(s.updates).padStart(11)}` + `${String(s.warnings).padStart(16)}${String(s.falseAlarms).padStart(14)}` + `${(s.stale.join(",") || "-").padStart(13)}`); } // Seven questions someone who comes later asks the document; each question lands on one claim. const QUESTION = [ ["How many domain modules does the system consist of?", "A1"], ["Which modules does the loan module use?", "A2"], ["How many import edges are in the graph?", "A3"], ["Who may depend on the notification module?", "K3"], ["Who may write to the loan table?", "K6"], ["Which tables does catalog own?", "A6"], ["When is the nightly maintenance window?", "D2"], ]; const g = graph(); const answer = (s, claim) => { if (RULE.some(([a]) => a === claim)) return "correct"; // a rule states intent if (UNVERIFIABLE.some(([a]) => a === claim)) return s.regime === "rule-only" ? "unanswered" : "unverifiable"; const i = s.doc.find((x) => x.ad === claim); if (!i) return "unanswered"; return JSON.stringify(i.oku(g)) === i.value ? "correct" : "wrong"; }; console.log(`\n${"question".padEnd(54)}${"claim".padEnd(7)}${"manual".padEnd(14)}${"audit".padEnd(14)}rule-only`); for (const [question, claim] of QUESTION) { console.log(`${question.padEnd(54)}${claim.padEnd(7)}` + RESULT.map((s) => answer(s, claim).padEnd(14)).join("").trimEnd()); } for (const s of RESULT) { const v = QUESTION.map(([, i]) => answer(s, i)); console.log(`${s.regime.padEnd(10)}${v.filter((x) => x === "correct").length} correct, ` + `${v.filter((x) => x === "wrong").length} wrong, ` + `${v.filter((x) => x === "unanswered").length} unanswered, ` + `${v.filter((x) => x === "unverifiable").length} unverifiable`); } ``` ``` regime caught missed updates snapshot warn. false alarms still stale manual 3/4 K6 4 0 0 A1,A3 audit 4/4 - 10 10 6 - rule-only 4/4 - 0 0 0 - question claim manual audit rule-only How many domain modules does the system consist of? A1 wrong correct unanswered Which modules does the loan module use? A2 correct correct unanswered How many import edges are in the graph? A3 wrong correct unanswered Who may depend on the notification module? K3 correct correct correct Who may write to the loan table? K6 correct correct correct Which tables does catalog own? A6 correct correct unanswered When is the nightly maintenance window? D2 unverifiable unverifiable unanswered manual 4 correct, 2 wrong, 0 unanswered, 1 unverifiable audit 6 correct, 0 wrong, 0 unanswered, 1 unverifiable rule-only 2 correct, 0 wrong, 5 unanswered, 0 unverifiable ``` Manual update catches three of the four real drifts and misses the fourth. The missed drift is K6: the new module writing to the loan table is reservation, but the claim's subject is loan, and touching reservation does not trigger a look at loan's claim. **This is manual update's blind spot — drift is invisible when the touched module and the claim's subject differ.** Its maintenance cost is the lowest: four claim updates. The generated audit catches all four of the drifts and leaves no stale claim behind. Its price is ten claim updates and ten warnings. Six of the warnings come from compliant changes, meaning they arrived while the code was still correct: a **false alarm**. The source is clear — snapshot claims go stale on every legitimate change, and the audit does not tell them apart from real drift. The rule-only document zeroes out both line items: four drifts caught, zero updates, zero false alarms. Its price is in the question table — five of the seven questions go unanswered. This regime is cheap because it says little. The question table separates the three regimes by this course's own measure. The manually updated document answers four of seven questions correctly and **two incorrectly**: it confidently states the module count and the import-edge count, and both are wrong. An unanswered question sends the reader to the code; a wrongly answered question sends them to the wrong place. Rule claims stay correct in all three regimes: a rule claim that does not hold against the code is wrong about the code, not the document. #### Summary - Documentation drift is a measurable quantity: the import graph extracted from code is compared against the document's claim; the method is borrowed from the Service Architectures course's boundary-violation measurement. - Claims are three types and behave differently: the four compliant changes dropped no rule claim, only staling snapshot claims; the four non-compliant ones dropped four rule claims. In the unupdated document, drift grows from 2 to 10, and only 2 of the 14 claims stay correct. - Manual update catches 3 of 4 drifts and costs 4 updates; its blind spot is when the touched module and the claim's subject differ. The generated audit catches all 4, at a cost of 10 updates and 10 warnings; 6 of the warnings are false alarms, and their source is the snapshot claims. - The seven questions' answers split by regime: 4 correct and 2 wrong under manual update, 6 correct under the generated audit, 2 correct and 5 unanswered under the rule-only document. #### Next Step Everything measured up to this point looked at the past or the present: a decision made, a boundary drawn, code running. There is one more thing an architect commits to in writing, and it looks at the future — how long this work will take. The next lesson measures estimation: the same set of work is estimated as a single number and as an uncertainty band, then compared against what actually happened. How the deviation is distributed, how much of it the band actually covers, and whether splitting the estimate into pieces shrinks the deviation. ### Estimation and Evaluation Source: https://academia.sh/en/courses/architectural-documentation/estimation-and-evaluation Measuring work-size estimation: the same set of work estimated as a single number and as an uncertainty band, then compared against what actually happened, the deviation distribution's percentiles, the band's actual coverage rate, and how splitting the estimate into pieces changes both the deviation and the spread. The previous lesson measured how well the document's claim stayed equal to the code. Everything measured up to this point had one thing in common: it described something already past — a decision made, a boundary drawn, code running. One more thing an architect commits to in writing looks at the future instead — how long this work will take. An estimate is also a document, and this course's measure applies to it too: which question it answers, which it cannot, what its deviation is. Rough size estimation was set up in the Introduction to System Design course; the size computed there was data and request volume. The size here is work size, its unit is person-days, and its verification is different from a volume estimate's — a volume estimate is verified approximately, a work estimate is compared against what actually happened, item by item. #### Estimating in Two Forms The block below **models** and defines a quarter's work list, the band form of the estimates, and the outcome model, then runs a single outcome. The outcome model consists of two numbers, and both are visible: systematic optimism and item-specific variability. ```js // estimate.mjs — defines the work items, the band rule, and the outcome model; when run // directly, measures a single outcome. The next two blocks import this file. // VM21: the regional library network system's work list for the next quarter; estimates // in person-days. export const WORK = [ ["branch definition flow", 5], ["catalog wrapper endpoint", 8], ["loan record migration", 13], ["membership verification", 5], ["notification channel", 3], ["fee calculation rules", 8], ["reservation queue", 13], ["report screen", 5], ["failover rehearsal", 8], ["bulk upload window", 3], ["audit script", 2], ["view document", 5], ]; // The band is derived from the single number: low bound three-quarters, high bound one // and a half times. Whoever gives the estimate is saying the band will cover nine out of // ten outcomes. export const band = (t) => [Math.round(t * 0.75), Math.round(t * 1.5)]; // VM22: the outcome model. Outcome = estimate x 1.25 x exp(N(0, 0.45)). 1.25 stands for // systematic optimism (work not visible in the estimate), exp(N(0, 0.45)) stands for // item-specific independent variability. Both numbers are inputs to the model; every // result below depends on them. export const OPTIMISM = 1.25; export const SPREAD = 0.45; export const generator = (seed) => () => { // the generator is hand-written, the seed is visible seed = (seed + 0x6d2b79f5) | 0; let t = Math.imul(seed ^ (seed >>> 15), 1 | seed); t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; export const normal = (r) => { // Box-Muller: one normal value from two uniform ones const u = Math.max(r(), 1e-12); return Math.sqrt(-2 * Math.log(u)) * Math.cos(2 * Math.PI * r()); }; export const multiplier = (r) => OPTIMISM * Math.exp(SPREAD * normal(r)); if (process.argv[1]?.endsWith("estimate.mjs")) { const SEED = 60406; const r = generator(SEED); console.log(`seed ${SEED}; optimism ${OPTIMISM}, spread ${SPREAD}`); console.log(`${"work item".padEnd(26)}${"estimate".padStart(9)}${"band".padStart(10)}` + `${"actual".padStart(12)}${"ratio".padStart(7)}${"in band".padStart(10)}`); let totalEstimate = 0, totalActual = 0, within = 0; for (const [name, t] of WORK) { const [low, high] = band(t); const a = Math.round(t * multiplier(r) * 10) / 10; const inBand = a >= low && a <= high; totalEstimate += t; totalActual += a; within += inBand ? 1 : 0; console.log(`${name.padEnd(26)}${String(t).padStart(9)}${`${low}-${high}`.padStart(10)}` + `${a.toFixed(1).padStart(12)}${(a / t).toFixed(2).padStart(7)}` + `${(inBand ? "yes" : "no").padStart(10)}`); } const [pLow, pHigh] = band(totalEstimate); console.log(`total estimate ${totalEstimate} days, actual ${totalActual.toFixed(1)} days, ` + `ratio ${(totalActual / totalEstimate).toFixed(2)}`); console.log(`the band covered ${within} of ${WORK.length} items; for the project total the band ` + `is ${pLow}-${pHigh} days and the actual is ${totalActual >= pLow && totalActual <= pHigh ? "in band" : "out of band"}`); } ``` ``` seed 60406; optimism 1.25, spread 0.45 work item estimate band actual ratio in band branch definition flow 5 4-8 8.6 1.72 no catalog wrapper endpoint 8 6-12 8.0 1.00 yes loan record migration 13 10-20 12.2 0.94 yes membership verification 5 4-8 5.6 1.12 yes notification channel 3 2-5 1.5 0.50 no fee calculation rules 8 6-12 8.3 1.04 yes reservation queue 13 10-20 17.6 1.35 yes report screen 5 4-8 10.1 2.02 no failover rehearsal 8 6-12 5.0 0.63 no bulk upload window 3 2-5 2.0 0.67 yes audit script 2 2-3 3.9 1.95 no view document 5 4-8 1.2 0.24 no total estimate 78 days, actual 84.0 days, ratio 1.08 the band covered 6 of 12 items; for the project total the band is 59-117 days and the actual is in band ``` In a single run, the project total is 84 days against an estimate of 78: an eight percent deviation, inside the band. The same estimate looks much worse at the item level — six of twelve items land outside the band, two at a ratio of 1.95 and 2.02. The total's good look comes from item deviations canceling out. A single run is not evidence; the same list has to run many times. #### How the Deviation Is Distributed ```js // distribution.mjs — imports estimate.mjs; measures the deviation distribution and the band's coverage rate import { WORK, band, generator, multiplier } from "./estimate.mjs"; const SEED = 31517; const RUNS = 500; const r = generator(SEED); const totalEstimate = WORK.reduce((s, [, t]) => s + t, 0); const itemRatio = [], projectRatio = []; let itemWithin = 0, projectWithin = 0, under = 0; for (let k = 0; k < RUNS; k += 1) { let totalActual = 0; for (const [, t] of WORK) { const a = t * multiplier(r); const [low, high] = band(t); itemRatio.push(a / t); if (a >= low && a <= high) itemWithin += 1; if (a < t) under += 1; totalActual += a; } projectRatio.push(totalActual / totalEstimate); const [pLow, pHigh] = band(totalEstimate); if (totalActual >= pLow && totalActual <= pHigh) projectWithin += 1; } const percentile = (d, p) => [...d].sort((a, b) => a - b)[Math.floor((d.length - 1) * p)]; const row = (name, d) => `${name.padEnd(16)}` + [0.1, 0.25, 0.5, 0.75, 0.9].map((p) => percentile(d, p).toFixed(2).padStart(8)).join("") + `${(d.reduce((s, x) => s + x, 0) / d.length).toFixed(2).padStart(10)}`; console.log(`seed ${SEED}, ${RUNS} runs; ${itemRatio.length} item outcomes`); console.log(`${"actual/estimate".padEnd(16)}${"p10".padStart(8)}${"p25".padStart(8)}` + `${"median".padStart(8)}${"p75".padStart(8)}${"p90".padStart(8)}${"mean".padStart(10)}`); console.log(row("per item", itemRatio)); console.log(row("project total", projectRatio)); console.log(`\nband coverage: per item ${((itemWithin / itemRatio.length) * 100).toFixed(0)}%, ` + `project total ${((projectWithin / RUNS) * 100).toFixed(0)}% ` + `(the estimator said nine in ten, i.e. 90%)`); console.log(`items landing under the estimate: ${((under / itemRatio.length) * 100).toFixed(0)}%; ` + `landing over: ${(100 - (under / itemRatio.length) * 100).toFixed(0)}%`); ``` ``` seed 31517, 500 runs; 6000 item outcomes actual/estimate p10 p25 median p75 p90 mean per item 0.71 0.92 1.25 1.68 2.23 1.38 project total 1.13 1.23 1.36 1.51 1.67 1.38 band coverage: per item 54%, project total 72% (the estimator said nine in ten, i.e. 90%) items landing under the estimate: 31%; landing over: 69% ``` Three results are read separately. First, **the deviation distribution is wide and not symmetric**: per item, the range between p10 and p90 runs from 0.71 to 2.23, more than a threefold difference between the tenth-percentile ends. 69 percent of items exceed the estimate. Second, **the item median and the project total's center are not the same number**. The per-item median is 1.25, while the project total's median is 1.36 and its mean is 1.38. The reason comes from the distribution's shape: the multiplier is right-skewed, so its mean sits above its median, and once twelve items are summed, the total converges toward the mean. **Summing makes the mean visible, not the median.** Someone who looks at a single item and says "it usually holds to the estimate" hits a systematic overrun at the project total. Third, **the band's coverage rate is far below what was claimed**. The estimator said the band would cover nine in ten; the measured coverage is 54 percent per item, 72 percent at the project total. The band is not wrong, its claim is: giving a range does not tie it to a confidence level. A band's coverage is known only by comparing it against what actually happens; until then, a band differs from a single number only in form. #### Splitting the Estimate ```js // split.mjs — imports estimate.mjs; measures the effect of splitting an estimate into pieces on the deviation import { WORK, band, generator, normal, OPTIMISM, SPREAD } from "./estimate.mjs"; // VM23: when an item is split into k pieces, two things change. One, a (1 - 1/k) share of // the previously invisible work becomes visible and gets added to the estimate. Two, each // piece's variability is drawn independently, so the k pieces' outcomes average out. The // actual work comes from the same model in both cases: estimate x 1.25 x variability. const splitEstimate = (t, k) => t * (1 + (OPTIMISM - 1) * (1 - 1 / k)); const splitActual = (t, k, r) => { let total = 0; for (let i = 0; i < k; i += 1) total += (t * OPTIMISM / k) * Math.exp(SPREAD * normal(r)); return total; }; const SEED = 74113; const RUNS = 500; const totalEstimate = WORK.reduce((s, [, t]) => s + t, 0); const percentile = (d, p) => [...d].sort((a, b) => a - b)[Math.floor((d.length - 1) * p)]; console.log(`seed ${SEED}, ${RUNS} runs, ${WORK.length} items`); console.log(`${"pieces".padStart(6)}${"item estimate".padStart(14)}${"p10".padStart(7)}` + `${"median".padStart(8)}${"p90".padStart(7)}${"p90/p10".padStart(9)}` + `${"in band".padStart(10)}${"project median".padStart(16)}`); for (const k of [1, 2, 4, 8]) { const r = generator(SEED); const ratio = [], projectRatio = []; let within = 0; for (let n = 0; n < RUNS; n += 1) { let totalActual = 0, totalEst = 0; for (const [, t] of WORK) { const e = splitEstimate(t, k); const a = splitActual(t, k, r); const [low, high] = band(e); ratio.push(a / e); if (a >= low && a <= high) within += 1; totalActual += a; totalEst += e; } projectRatio.push(totalActual / totalEst); } const p10 = percentile(ratio, 0.1), p90 = percentile(ratio, 0.9); console.log(`${String(k).padStart(6)}` + `${(splitEstimate(totalEstimate, k) / totalEstimate).toFixed(3).padStart(14)}` + `${p10.toFixed(2).padStart(7)}${percentile(ratio, 0.5).toFixed(2).padStart(8)}` + `${p90.toFixed(2).padStart(7)}${(p90 / p10).toFixed(2).padStart(9)}` + `${`${((within / ratio.length) * 100).toFixed(0)}%`.padStart(10)}` + `${percentile(projectRatio, 0.5).toFixed(2).padStart(16)}`); } console.log(`\nsplitting moves two numbers in opposite directions: it grows the estimate as`); console.log(`previously invisible work becomes visible (the item estimate column), and`); console.log(`averaging independent pieces narrows the spread (the p90/p10 column).`); ``` ``` seed 74113, 500 runs, 12 items pieces item estimate p10 median p90 p90/p10 in band project median 1 1.000 0.70 1.24 2.19 3.14 54% 1.35 2 1.125 0.77 1.16 1.76 2.29 65% 1.21 4 1.188 0.83 1.13 1.52 1.82 84% 1.16 8 1.219 0.90 1.12 1.38 1.52 93% 1.13 splitting moves two numbers in opposite directions: it grows the estimate as previously invisible work becomes visible (the item estimate column), and averaging independent pieces narrows the spread (the p90/p10 column). ``` Splitting's effect comes through two separate, measurable channels. The first grows the estimate: an item split into eight pieces has an estimate 1.219 times the unsplit one's, because looking piece by piece brings a portion of previously invisible work onto the list. The second narrows the spread: independent pieces' outcomes average out and balance each other, and the p90/p10 ratio drops from 3.14 to 1.52. Together, the two channels pull the median deviation from 1.24 to 1.12, the project median from 1.35 to 1.13, and raise the band's coverage from 54 to 93 percent. Most of the gain comes in the first splits: one piece to four raises coverage by 30 points, four to eight raises it by 9. **Splitting's return grows at a diminishing rate**, and every piece then has to be estimated separately. This lesson looked for answers to a set of three questions. "How long will this work take" is answered by a single number. "How confident" is answered only by a band, but only once its coverage rate has been measured — an unmeasured band looks like it answers this question without actually answering it. The third question, "what is this estimate based on," stays unanswered in both. This is the same gap as the fourth lesson's threshold source: the number gets written down, where it came from does not. #### Summary - Work-size estimation's deviation is measured against what actually happens, item by item; in the model, the per-item p10-p90 range is 0.71-2.23 and 69 percent of items exceed the estimate. - The item median and the project total's center are separate numbers: the item median is 1.25, while the project total's median is 1.36 and its mean is 1.38 — summing makes the mean visible, not the median. - The band's coverage rate is independent of what is claimed: a band said to cover nine in ten covers 54 percent per item, 72 percent at the project total; an unmeasured band differs from a single number only in form. - Splitting works through two channels: it raises the estimate to 1.219x by adding a portion of invisible work, and it lowers the p90/p10 ratio from 3.14 to 1.52 by averaging independent pieces. - Splitting into eight pulls the median deviation from 1.24 to 1.12, the project median from 1.35 to 1.13, and carries coverage to 93 percent; the return grows at a diminishing rate, with a 9-point gain after one-to-four. #### Course Wrap-Up This course ran on one rule: **a document's value is measured by how many of the questions someone who comes later asks, it answers.** Each of the eleven lessons built its own question set, ran that set against a record, a view, a diagram, a scenario, or an estimate, and counted **the unanswered along with the answered**. | Lesson | Question set | Answered / unanswered | Maintenance cost or deviation | |---|---|---|---| | Architecture Decision Records | 39 questions asked of 4 decisions | 18/39 unrecorded, 39/39 with a structured record | 4 records, 101 lines, 170 minutes; 15% of the time went to a section answering no question | | The Proposal and Evaluation Process | question set asked of 8 decisions | correct answers 13, 32, and 40; verbal approval gave 12 wrong and 15 missing answers | 440 and 420 person-minutes per flipped decision; the one-round scheme spent 900 person-minutes without changing direction | | Trade-Off Analysis | question set asked of 5 analyses | 2/5 reproducible as recorded, 5/5 once gaps were fixed; 1 verifiable | completion cost 110 minutes, 36.7 minutes per gained analysis | | The Risk Register | 2,348 realized risks | 808 risks (34.4%) never in the record; correct answers rose from 16 to 40.9, wrong answers fell from 6 to 0.4 | 1.26 risks per period under yearly review; 602 person-minutes per risk caught early under monthly review | | Technical Debt Management | 9 debt items and a year's change set | 5 inadvertent items cost 6,820 minutes, 4 deliberate items 3,640 minutes; not noticed at all within a year at the 48-repeat threshold | the same work takes 63.7% longer in the debt-laden module; noticing delay runs from 2.5 months to 7.4 months | | Architectural Views | 24 questions | 18 answered / 6 unanswered | 279 writes across 200 changes, 79 of them repeats; 15.50 writes per answer | | The Layered Diagram Approach | 14 questions | 14 answered at their own level / D03 unanswered at the context level, D11 at the first three levels | 112 nodes in maintenance; 62 nodes read instead of 278 at a single level, the gap paid back every 4.24 rounds | | Selecting the Diagram Type by Question | 18 questions | 16 answered / 2 unanswered | 241 item updates across 150 changes; maintenance per answer between 2.8 and 24.3 | | Quality Attribute Scenarios | 5 questions asked of 12 requirements, 60 total | 29 answered / 31 unanswered | writing cost 1.99x; 9 requirements with no written threshold produce 5.31 disagreements per delivery | | Keeping Documentation Current | 7 questions | 6 correct under the generated audit / 4 correct and 2 wrong under manual update | drift grows from 2 to 10; the audit costs 10 updates and 6 false alarms | | Estimation and Evaluation | 3 questions | 2 answered / 1 unanswered ("what is this estimate based on") | item median deviation 1.25; band coverage 54%, 93% with splitting | The table shows a pattern. No lesson closed its question set completely, and that is not a flaw: the unanswered questions came from the same two classes every time — either they asked to combine information from two separate views, or they asked where a number came from. The second class ran from the start of the course to the end; "where did this number come from" in the decision record, "the threshold's source" in the quality attribute scenario, "what is this estimate based on" in estimation are three names for one gap. The second pattern is in maintenance cost: every lesson counted a document form's price, and the price never came out zero. The cost of keeping a document current is paid not at the moment of writing but at every change after; that is why a document's value is meaningful not just by how many questions it answers, but by that number's ratio to its maintenance cost. All these records, views, scenarios, and estimates stayed inside a single system: the regional library network's software system, with its own modules, boundaries, and document. Yet no system stands alone. It stands inside an institution, other systems stand alongside it, and it exchanges data with them. The next course — **Enterprise Context and Integration** — opens with this question: where does a system's boundary run once its connections to what lies outside it are taken into account.