--- title: 'Fitness Functions' source: 'https://academia.sh/en/courses/architecture-governance/fitness-functions' course: 'Quality Attributes and Governance' language: en updated: '2026-08-23T07:01:05+00:00' license: 'CC BY-SA 4.0' --- # Fitness Functions Converting an architectural rule into a runnable number: writing two fitness functions over real module files, counting caught and missed violations plus false alarms against a known leak set, sweeping the threshold end to end, the rule's line-count and file-reading cost, and the false alarm's effect of getting the rule disabled. The previous lesson wrote that five of the nine governance rules could be converted to a check, but it did not count the cost of that conversion. Conversion means this: the rule is reduced to a **number** and a **threshold**. The function that produces the number is called a **fitness function**; it reads the source, the configuration, or a graph, and returns a single quantity. The threshold is the decision itself — above this number is a violation, below it is compliant. The moment a threshold is chosen, two errors appear at once. A loose threshold lets real violations pass underneath; a tight one flags compliant modules as violations. This pair is the same phenomenon as the missed defect and the false pass in quality metrics; the difference here is that the one making the error is an architectural rule, not a test. This lesson actually writes a fitness function, runs it against a known leak set, and counts three numbers at once: caught, missed, false alarm. The through-line is the regional library network, and it is fiction. ## The Module Set to Measure The rule to be checked is this: **a module does not expose more to the outside than it needs to do its job.** As a sentence, it cannot be checked; to be checkable, "more" has to be tied to a number. The block below writes the codebase to be measured to disk: twelve modules, each exporting its own public symbols and importing others' symbols. **GV4 — the regional library network's codebase consists of twelve modules, and the number of symbols each module exports and who imports them is produced from the map below.** Reason: a fitness function must read real files; the map is the input that generates the files to be read. If the numbers change, the threshold sweep below changes with them; the function itself does not. ```js // setup.mjs — writes the regional library network's modules to disk with their exported symbols import { mkdirSync, writeFileSync } from "node:fs"; // open: number of exported symbols | consumer: which module imports which symbol indices const MODULES = { sharedFormat: { open: 9, consumer: { reporting: [0, 1, 2], loanFlow: [0, 3, 4], branchFront: [1, 5], settings: [2] } }, branchFront: { open: 11, consumer: { reporting: [0, 1, 2] } }, catalogBridge: { open: 7, consumer: { loanFlow: [0, 1, 2, 3, 4, 5, 6] } }, loanFlow: { open: 8, consumer: { branchFront: [0, 1, 2, 3], reporting: [4, 5], feeCalculation: [6] } }, feeCalculation: { open: 4, consumer: { branchFront: [0] } }, membershipRegistry: { open: 3, consumer: { branchFront: [0, 1], loanFlow: [2] } }, identityBridge: { open: 2, consumer: { branchFront: [0], membershipRegistry: [1] } }, notificationQueue: { open: 5, consumer: { loanFlow: [0] } }, reporting: { open: 6, consumer: { branchFront: [0, 1, 2, 3, 4, 5] } }, storageAccess: { open: 3, consumer: { loanFlow: [0] } }, eventLog: { open: 4, consumer: { loanFlow: [0] } }, settings: { open: 4, consumer: { branchFront: [0], loanFlow: [1], reporting: [2], feeCalculation: [3] } }, }; mkdirSync("modules", { recursive: true }); for (const name of Object.keys(MODULES)) { const imports = Object.entries(MODULES) .filter(([, m]) => m.consumer[name]) .map(([source, m]) => `import { ${m.consumer[name].map((i) => `${source}${i + 1}`).join(", ")} }` + ` from "./${source}.mjs";`); const exports = Array.from({ length: MODULES[name].open }, (_, i) => `export const ${name}${i + 1} = (x) => x;`); writeFileSync(`modules/${name}.mjs`, `${imports.join("\n")}\n${exports.join("\n")}\n`); } const edges = Object.values(MODULES).reduce((s, m) => s + Object.keys(m.consumer).length, 0); const symbols = Object.values(MODULES).reduce((s, m) => s + m.open, 0); console.log(`${Object.keys(MODULES).length} module files written; ` + `${symbols} exported symbols, ${edges} import edges`); console.log(`\n${"module".padEnd(21)}${"exported".padStart(10)} importing module`); for (const [name, m] of Object.entries(MODULES)) { console.log(`${name.padEnd(21)}${String(m.open).padStart(10)} ` + `${Object.keys(m.consumer).join(", ") || "-"}`); } ``` ``` 12 module files written; 66 exported symbols, 22 import edges module exported importing module sharedFormat 9 reporting, loanFlow, branchFront, settings branchFront 11 reporting catalogBridge 7 loanFlow loanFlow 8 branchFront, reporting, feeCalculation feeCalculation 4 branchFront membershipRegistry 3 branchFront, loanFlow identityBridge 2 branchFront, membershipRegistry notificationQueue 5 loanFlow reporting 6 branchFront storageAccess 3 loanFlow eventLog 4 loanFlow settings 4 branchFront, loanFlow, reporting, feeCalculation ``` ## The Function That Converts the Rule to a Number The same rule can be converted into two different numbers. **F1 — surface size** is the cheapest conversion: it counts the symbols a module exports and reads a single file. **F2 — dead surface ratio** is more expensive: to find how many exported symbols are never imported by any module, it reads the entire codebase. Both are candidates for the same rule; which one is better can only be said by testing against a known violation set. ```js // fitness.mjs — two fitness functions that convert an architectural rule into a number, and a threshold sweep import { readFileSync, readdirSync } from "node:fs"; const DIR = "modules"; const files = readdirSync(DIR); // >>> F1 surface size: the module's own file is enough function F1(name) { const t = readFileSync(`${DIR}/${name}.mjs`, "utf8"); return [...t.matchAll(/^export const (\w+)/gm)].length; } // <<< F1 // >>> F2 dead surface ratio: how many exported symbols are never imported function F2(name) { const open = new Set([...readFileSync(`${DIR}/${name}.mjs`, "utf8") .matchAll(/^export const (\w+)/gm)].map((m) => m[1])); const used = new Set(); for (const d of files) { for (const m of readFileSync(`${DIR}/${d}`, "utf8") .matchAll(/^import \{ ([^}]+) \} from "\.\/(\w+)\.mjs"/gm)) { if (m[2] === name) for (const s of m[1].split(", ")) used.add(s); } } return (open.size - used.size) / open.size; } // <<< F2 // GV5: the architect reviewed the twelve modules by hand and marked five as a real surface leak. branchFront and // feeCalculation carry dead symbols; notificationQueue and storageAccess expose internal detail; catalogBridge // opens all seven of its symbols to a single consumer, carrying its internal representation outward. const LEAK = new Set(["branchFront", "catalogBridge", "feeCalculation", "notificationQueue", "storageAccess"]); const names = files.map((d) => d.replace(".mjs", "")); console.log(`${"module".padEnd(21)}${"F1 surface".padStart(12)}${"F2 dead ratio".padStart(15)}${"architect label".padStart(18)}`); for (const a of names) { console.log(`${a.padEnd(21)}${String(F1(a)).padStart(12)}${F2(a).toFixed(2).padStart(15)}` + `${(LEAK.has(a) ? "leak" : "compliant").padStart(18)}`); } function sweep(fn, thresholds, label) { console.log(`\n${label.padEnd(15)}${"flagged".padStart(11)}${"caught".padStart(11)}` + `${"missed".padStart(8)}${"false alarm".padStart(14)}`); for (const e of thresholds) { const flagged = names.filter((a) => fn(a) > e); const y = flagged.filter((a) => LEAK.has(a)).length; console.log(`${`threshold > ${e}`.padEnd(15)}${String(flagged.length).padStart(11)}` + `${`${y}/${LEAK.size}`.padStart(11)}${String(LEAK.size - y).padStart(8)}` + `${String(flagged.length - y).padStart(14)}`); } } sweep(F1, [2, 3, 4, 5, 6, 7, 8, 10], "F1 surface"); sweep(F2, [0.2, 0.4, 0.5, 0.6, 0.7], "F2 dead ratio"); // the rule's own cost: lines, files read, run const source = readFileSync("fitness.mjs", "utf8").split("\n"); const lines = (m) => { const b = source.findIndex((l) => l.startsWith(`// >>> ${m}`)); return source.findIndex((l) => l.startsWith(`// <<< ${m}`)) - b - 1; }; const totalLines = names.reduce((s, a) => s + readFileSync(`${DIR}/${a}.mjs`, "utf8").split("\n").length, 0); console.log(`\n${"function".padEnd(12)}${"rule lines".padStart(13)}${"files read".padStart(13)}` + `${"lines scanned".padStart(15)}`); console.log(`${"F1".padEnd(12)}${String(lines("F1")).padStart(13)}${String(names.length).padStart(13)}` + `${String(totalLines).padStart(15)}`); console.log(`${"F2".padEnd(12)}${String(lines("F2")).padStart(13)}` + `${String(names.length * (1 + files.length)).padStart(13)}${String((1 + names.length) * totalLines).padStart(15)}`); console.log("run time depends on the environment; the quantities printed here are environment-independent " + `(F2 / F1 read ratio ${(names.length * (1 + files.length)) / names.length}x)`); ``` ``` module F1 surface F2 dead ratio architect label branchFront 11 0.73 leak catalogBridge 7 0.00 leak eventLog 4 0.75 compliant feeCalculation 4 0.75 leak identityBridge 2 0.00 compliant loanFlow 8 0.13 compliant membershipRegistry 3 0.00 compliant notificationQueue 5 0.80 leak reporting 6 0.00 compliant settings 4 0.00 compliant sharedFormat 9 0.33 compliant storageAccess 3 0.67 leak F1 surface flagged caught missed false alarm threshold > 2 11 5/5 0 6 threshold > 3 9 4/5 1 5 threshold > 4 6 3/5 2 3 threshold > 5 5 2/5 3 3 threshold > 6 4 2/5 3 2 threshold > 7 3 1/5 4 2 threshold > 8 2 1/5 4 1 threshold > 10 1 1/5 4 0 F2 dead ratio flagged caught missed false alarm threshold > 0.2 6 4/5 1 2 threshold > 0.4 5 4/5 1 1 threshold > 0.5 5 4/5 1 1 threshold > 0.6 5 4/5 1 1 threshold > 0.7 4 3/5 2 1 function rule lines files read lines scanned F1 4 12 106 F2 12 156 1378 run time depends on the environment; the quantities printed here are environment-independent (F2 / F1 read ratio 13x) ``` ## Reading the Threshold The F1 sweep shows plainly what the threshold is. At the loosest end (`> 2`), all five of the five leaks are caught and none get through — but eleven of the twelve modules get flagged, and six of those are false alarms. At the tightest end (`> 10`), false alarms drop to zero and four leaks get through. There is no threshold in between where both missing and false alarms are low at once: the `> 4` threshold catches three, misses two, and gives three false alarms. **F1 cannot separate missing from false alarms, because the number it measures is not even close to what the rule is asking** — `sharedFormat`, six of whose nine symbols are used, and `storageAccess`, two of whose three symbols are dead, sit on the same axis. The F2 sweep lands somewhere different against the same violation set. Between `> 0.4` and `> 0.6` the result does not change: four caught, one missed, one false alarm. This flatness counts as a good sign — if the threshold's exact value does not determine the outcome, the rule is not overly sensitive to the threshold. F2's one miss is `catalogBridge`, and the reason is in the definition itself: all seven of its seven symbols are imported, so its dead surface is zero; the reason the architect calls it a leak is not that symbols go unused but that all of them are opened to a single consumer. F2's one false alarm is `eventLog`: three of its four symbols are open for operational tools outside the graph, and the checker does not see those tools. **The rule is defined inside the code, but the rule itself cannot be derived from the code.** F2 sits much closer to the architect's label than F1, but both are approximations; no threshold classifies the `catalogBridge`/`eventLog` pair correctly at the same time. ## The Rule's Own Cost The last table is the rule's cost. F1 is four lines and, on a full run, reads twelve files and 106 lines. F2 is twelve lines and, on a full run, reads 156 files and 1,378 lines — because it scans the whole directory for every module, its reading is thirteen times as much. Because run time depends on the environment, the quantities printed are the environment-independent ones: files read and lines scanned. As the module count grows, F2's reading grows quadratically, F1's grows linearly. What follows from this is that the way to make the rule cheaper is to lower the read count without breaking the measure: F2 could read every file once instead of once per module and build the import map in memory. But the real cost this lesson needs to count is not the run — it is **maintenance**: because F2 parses the import syntax, every new form of import syntax breaks the rule; the only thing F1 parses is the `export` line. ## The Behavior a False Alarm Produces A false alarm is not a number, it is a behavior. When a rule produces a false alarm, the team first reviews it, then adds an exception, and eventually closes the rule down. The block below models this. ```js // circuit.mjs — the behavior a false alarm produces on the team: the rule goes offline on its third alarm import { readFileSync, readdirSync } from "node:fs"; const DIR = "modules"; const files = readdirSync(DIR); const openSymbols = (a) => [...readFileSync(`${DIR}/${a}.mjs`, "utf8") .matchAll(/^export const (\w+)/gm)].map((m) => m[1]); const F1 = (a) => openSymbols(a).length; const F2 = (a) => { const open = new Set(openSymbols(a)); const used = new Set(); for (const d of files) { for (const m of readFileSync(`${DIR}/${d}`, "utf8") .matchAll(/^import \{ ([^}]+) \} from "\.\/(\w+)\.mjs"/gm)) { if (m[2] === a) for (const s of m[1].split(", ")) used.add(s); } } return (open.size - used.size) / open.size; }; const LEAK = new Set(["branchFront", "catalogBridge", "feeCalculation", "notificationQueue", "storageAccess"]); // GV6: each of the twenty releases touches a single module, and the rule only runs on the touched module. GV7: // the team takes the rule offline after the third false alarm (disabling it instead of loosening the threshold). const RELEASES = ["settings", "branchFront", "loanFlow", "reporting", "feeCalculation", "sharedFormat", "catalogBridge", "eventLog", "storageAccess", "loanFlow", "notificationQueue", "membershipRegistry", "reporting", "branchFront", "settings", "identityBridge", "feeCalculation", "sharedFormat", "storageAccess", "catalogBridge"]; function run(fn, threshold) { let alarms = 0, active = true, caught = 0, missed = 0, closedAt = 0, missedAfterClosed = 0; RELEASES.forEach((m, i) => { if (!active) { if (LEAK.has(m)) { missed += 1; missedAfterClosed += 1; } return; } const flagged = fn(m) > threshold; if (flagged && LEAK.has(m)) caught += 1; else if (!flagged && LEAK.has(m)) missed += 1; else if (flagged) { alarms += 1; if (alarms === 3) { active = false; closedAt = i + 1; } } }); return { alarms, caught, missed, closedAt, missedAfterClosed }; } const leakyReleases = RELEASES.filter((m) => LEAK.has(m)).length; console.log(`${RELEASES.length} releases, ${leakyReleases} of them touch a leaking module`); console.log(`\n${"rule".padEnd(19)}${"false alarm".padStart(11)}${"closed at release".padStart(19)}` + `${"caught".padStart(11)}${"missed".padStart(8)}${"missed after closed".padStart(22)}`); for (const [label, fn, threshold] of [["F1 threshold > 3", F1, 3], ["F2 threshold > 0.5", F2, 0.5]]) { const r = run(fn, threshold); console.log(`${label.padEnd(19)}${String(r.alarms).padStart(11)}` + `${(r.closedAt ? String(r.closedAt) : "not closed").padStart(19)}` + `${`${r.caught}/${leakyReleases}`.padStart(11)}${String(r.missed).padStart(8)}` + `${String(r.missedAfterClosed).padStart(22)}`); } // GV8: what stands in for the rule that cannot be converted is manually reading every third symbol const symbols = files.flatMap((d) => openSymbols(d.replace(".mjs", ""))); const sample = symbols.filter((_, i) => i % 3 === 2); console.log(`\nnon-convertible rule ("a public name describes the module's job"): ` + `${sample.length} of ${symbols.length} symbols are read manually, ${symbols.length - sample.length} are not read`); ``` ``` 20 releases, 9 of them touch a leaking module rule false alarm closed at release caught missed missed after closed F1 threshold > 3 3 4 1/9 8 8 F2 threshold > 0.5 1 not closed 7/9 2 0 non-convertible rule ("a public name describes the module's job"): 22 of 66 symbols are read manually, 44 are not read ``` The result is different from what the threshold sweep alone said. When the F1 threshold was set at `> 3`, the sweep table showed it catching four of the five leaks; in the twenty-release run, though, **it closes on the fourth release** and looks at nothing for the remaining sixteen. Of the nine releases with a violation, only one is caught, eight get through, and all eight of those eight come after the rule closed. F2 gives a single false alarm in the same run, never closes, and catches seven of the nine releases with a violation. The difference does not come from the choice of threshold; it comes from the false alarm rate. From the moment a rule closes, what it measures is zero; so a rule's quality is measured not by its catch count in a single run, but by **how many releases it stays open on the team**. The rule's non-convertible part stands there by name: no function can say whether an exported name describes the module's job, and the sample that stands in for it reads 22 of the 66 symbols; 44 are never read. ## Summary - The same architectural rule was converted into two fitness functions: F1 surface size (number of exported symbols), F2 dead surface ratio (share of symbols never imported); both read real module files. - No threshold is good in F1's sweep: `> 2` catches five of five leaks and gives 6 false alarms, `> 10` zeroes out the false alarms and misses 4 leaks. - F2 does not change between `> 0.4` and `> 0.6` against the same violation set (4 caught, 1 missed, 1 false alarm), and insensitivity to the threshold is a sign of the rule's robustness; both of its errors come from the definition — it misses `catalogBridge` (all seven of its symbols are used), and gives a false alarm on `eventLog` (three of its symbols are open for tools outside the graph). - The rule's cost is 4 lines, 12 files, and 106 lines read for F1; 12 lines, 156 files, and 1,378 lines read for F2; the gap is 13-fold and grows quadratically with the module count. - In the twenty-release run, F1 closes on the fourth release at its third false alarm and misses 8 of 9 releases with a violation; F2 stays open with a single false alarm and catches 7 of them. - The non-convertible rule was written down by name ("a public name describes the module's job"), and the sample that stands in for it reads 22 of 66 symbols. ## Next Step This lesson's fitness function looked at a single module's surface and used its neighbors only while counting symbols. A portion of architectural rules, though, never shows up in a single module at all: which module may import which module lives in the codebase's **graph**, and can only be checked by reading the whole graph. The next lesson writes that rule: a layer checker is built that runs on a real import graph, and the real question is not the checker but the rule itself — the same layer rule can be written as an allowlist form or a denylist form. Both forms are run on the same graph and three numbers are compared: which gives fewer false alarms, which misses more violations, and how many places the rule has to be updated when a new module is added to the codebase.