Tension Between Quality Attributes

The previous lesson built the tree and tied every leaf to its own threshold. In that table the attributes stand independent of each other: each leaf has its own threshold, its own observation field, its own check. But an architectural decision never touches a single leaf. A caching layer that lowers search latency also raises the number of files touched in a change and the number of environment-specific calls, at the same time. A per-leaf check cannot see this, because each check only looks at its own leaf.

One attribute suppressing another is not a new observation; the tension between availability and consistency was measured in the Introduction to System Design course, in the fundamental properties topic, and is not repeated here. The question here is more general: given a list of decisions, can we count which attribute pairs conflict and which attribute is suppressed systematically. And the course’s own question continues: can this tension be turned into an executable check.

The Tension Matrix

The block below models the fictional regional library network’s fourteen measured leaves, twelve decisions proposed in one quarter, and the decisions’ effect on those leaves. A root attribute’s score is the average of its leaves’ share relative to their threshold; a decision’s impact is the score difference between before and after it is applied.

// tension.mjs — models attributes and decisions and builds the tension matrix
import { mkdirSync, writeFileSync } from "node:fs";

// QA6: the fictional regional library network's fourteen measured leaves and their starting
// values. [code, root attribute, name, unit, threshold, direction, starting value]. Model.
const LEAF = [
  ["P1", "performance", "search latency", "ms", 400, "low", 310],
  ["P2", "performance", "loan transaction latency", "ms", 600, "low", 420],
  ["P3", "performance", "hourly loan transactions", "txn", 1200, "high", 1500],
  ["E1", "availability", "monthly outage", "min", 45, "low", 22],
  ["E2", "availability", "failover", "s", 60, "low", 38],
  ["E3", "availability", "data loss window", "min", 5, "low", 2],
  ["G1", "security", "authentication coverage", "%", 100, "high", 96],
  ["G2", "security", "secret rotation interval", "day", 90, "low", 60],
  ["G3", "security", "audit record coverage", "%", 90, "high", 94],
  ["S1", "maintainability", "files touched in a change", "file", 6, "low", 4],
  ["S2", "maintainability", "dependency depth", "level", 5, "low", 3],
  ["S3", "maintainability", "test coverage", "%", 70, "high", 78],
  ["T1", "portability", "environment-specific call", "call", 12, "low", 7],
  ["T2", "portability", "edge unit install", "min", 45, "low", 30],
];

// QA7: twelve architectural decisions proposed in one quarter. Each decision gives the new value of the
// leaves it touches, relative to the starting values; a leaf it does not touch stays unchanged. Model.
const DECISION = [
  ["search cache", { P1: 180, S1: 5, S2: 4, T1: 9 }],
  ["synchronous replication", { E3: 0.5, E2: 30, P2: 560 }],
  ["authenticate every request", { G1: 100, P1: 360, P2: 470 }],
  ["secret value rotation", { G2: 30, S1: 5, T1: 10 }],
  ["audit record propagation", { G3: 99, P3: 1380, S1: 5 }],
  ["layer separation", { S2: 2, S1: 3, P2: 450 }],
  ["expanding the test suite", { S3: 88 }],
  ["local queue at the branch", { E1: 12, T2: 40, S1: 5, S2: 4 }],
  ["automatic failover", { E2: 20, E1: 18, S2: 4, G3: 90 }],
  ["per-request rate limiting", { E1: 16, P3: 1320, P1: 330 }],
  ["separating membership data", { G1: 99, S1: 6, P2: 500, T2: 38 }],
  ["moving bulk upload to nighttime", { P1: 290, P3: 1560, E3: 3 }],
];

const ROOT = [...new Set(LEAF.map((y) => y[1]))];
// Share: threshold minus value for a low-direction leaf, value minus threshold for a high-direction leaf;
// divided by the threshold to strip the unit. A root attribute's score is the average of its leaves' shares.
const share = ([, , , , threshold, direction], value) =>
  (direction === "low" ? threshold - value : value - threshold) / threshold;
const score = (value) => Object.fromEntries(ROOT.map((k) => {
  const y = LEAF.filter((n) => n[1] === k);
  return [k, y.reduce((s, n) => s + share(n, value[n[0]]), 0) / y.length];
}));

const BASELINE = Object.fromEntries(LEAF.map((y) => [y[0], y[6]]));
const baseline = score(BASELINE);
const SENSITIVITY = 0.005;   // a score change smaller than this does not count as a change

const impact = DECISION.map(([name, change]) => {
  const d = score({ ...BASELINE, ...change });
  const delta = Object.fromEntries(ROOT.map((k) => [k, d[k] - baseline[k]]));
  return { name, change, delta };
});

mkdirSync("model", { recursive: true });
writeFileSync("model/model.mjs",
  `export const LEAF = ${JSON.stringify(LEAF)};\n` +
  `export const DECISION = ${JSON.stringify(DECISION)};\n` +
  `export const BASELINE = ${JSON.stringify(BASELINE)};\n`);

const short = (k) => k.slice(0, 4);
console.log(`${"decision".padEnd(32)}${"rising".padEnd(22)}falling (measured drop)`);
for (const e of impact) {
  const up = ROOT.filter((k) => e.delta[k] > SENSITIVITY);
  const down = ROOT.filter((k) => e.delta[k] < -SENSITIVITY);
  console.log(`${e.name.padEnd(32)}${(up.map(short).join(",") || "-").padEnd(22)}` +
    `${down.map((k) => `${short(k)} ${(-e.delta[k] * 100).toFixed(1)}`).join(", ") || "-"}`);
}

// Tension matrix: row is the rising attribute, column is the falling attribute; the cell is how many decisions produce that pair.
const matrix = Object.fromEntries(ROOT.map((a) => [a, Object.fromEntries(ROOT.map((b) => [b, 0]))]));
const dropTotal = Object.fromEntries(ROOT.map((k) => [k, 0]));
const riseCount = Object.fromEntries(ROOT.map((k) => [k, 0]));
for (const e of impact) {
  const up = ROOT.filter((k) => e.delta[k] > SENSITIVITY);
  const down = ROOT.filter((k) => e.delta[k] < -SENSITIVITY);
  for (const a of up) { riseCount[a] += 1; for (const b of down) matrix[a][b] += 1; }
  for (const b of down) dropTotal[b] += -e.delta[b];
}

console.log(`\ntension matrix (row rises, column falls; cell is decision count)`);
console.log(`${"".padEnd(20)}${ROOT.map((k) => short(k).padStart(7)).join("")}`);
for (const a of ROOT)
  console.log(`${a.padEnd(20)}${ROOT.map((b) => (a === b ? "-" : String(matrix[a][b])).padStart(7)).join("")}`);

const pair = ROOT.flatMap((a) => ROOT.filter((b) => a !== b && matrix[a][b] > 0).map((b) => [a, b]));
const mutual = pair.filter(([a, b]) => matrix[b][a] > 0).length / 2;
console.log(`\nconflicting ordered pairs ${pair.length}/${ROOT.length * (ROOT.length - 1)}, ` +
  `${mutual} of them mutual (suppression runs both ways)`);
const suppresses = (k) => impact.filter((e) => e.delta[k] < -SENSITIVITY).length;
const mostSuppressed = ROOT.slice().sort((a, b) => dropTotal[b] - dropTotal[a])[0];
console.log(`most suppressed attribute: ${mostSuppressed}; ${suppresses(mostSuppressed)} decisions ` +
  `drop it for ${ROOT.filter((a) => matrix[a][mostSuppressed] > 0).length} other attributes, ` +
  `total measured drop ${(dropTotal[mostSuppressed] * 100).toFixed(1)} share points`);
for (const k of ROOT)
  if (riseCount[k] === 0)
    console.log(`attribute no decision improves: ${k}; ${suppresses(k)} of ${DECISION.length} decisions ` +
      `drop it, none raises it`);

decision                        rising                falling (measured drop)
search cache                    perf                  main 12.2, port 8.3
synchronous replication         avai                  perf 7.8
authenticate every request      secu                  perf 6.9
secret value rotation           secu                  main 5.6, port 12.5
audit record propagation        secu                  perf 3.3, main 5.6
layer separation                main                  perf 1.7
expanding the test suite        main                  -
local queue at the branch       avai                  main 12.2, port 11.1
automatic failover              avai                  secu 1.5, main 6.7
per-request rate limiting       avai                  perf 6.7
separating membership data      secu                  perf 4.4, main 11.1, port 8.9
moving bulk upload to nighttime perf                  avai 6.7

tension matrix (row rises, column falls; cell is decision count)
                       perf   avai   secu   main   port
performance               -      1      0      1      1
availability              2      -      1      2      1
security                  3      0      -      3      2
maintainability           1      0      0      -      0
portability               0      0      0      0      -

conflicting ordered pairs 11/20, 2 of them mutual (suppression runs both ways)
most suppressed attribute: maintainability; 6 decisions drop it for 3 other attributes, total measured drop 53.3 share points
attribute no decision improves: portability; 4 of 12 decisions drop it, none raises it

11 of the twenty ordered attribute pairs measure a conflict: at least one decision that raises one attribute drops the other. Two are mutual — performance with availability, performance with maintainability. A mutual pair is one where the other drops no matter which direction you move; it cannot be closed, only whose share of the drop gets chosen.

The most suppressed attribute is maintainability: six of the twelve decisions drop it, for three separate attributes, a total of 53.3 share points. Two of maintainability’s leaves — files touched in a change and dependency depth — rise on almost every decision that adds a piece. A piece gets added for performance or security; maintainability pays the cost every time.

The matrix’s last row is filled with zeros. No decision raises portability; four drop it. This does not show that portability is unimportant — it shows that no one is proposing a decision that targets it. Writing a threshold does not mean a decision defending that attribute will get proposed; the matrix is the one row that makes the missing decision visible.

One decision drops no attribute: expanding the test suite. None of the fourteen measured leaves carries this decision’s cost; an unmeasured cost shows up in the matrix as zero. The matrix is only as honest as the leaves it measures.

The Regression Check

A matrix fills a table; on its own it stops nothing. The next question is whether tension can be turned into an executable check: when a decision is applied, no root attribute’s score should drop by more than a set threshold. The block below applies this check to a known set of violations and counts what the threshold choice changes.

// regression.mjs — turns tension into an executable regression check and runs it against a
// known set of violations, then counts caught / missed / false alarm
import { LEAF, DECISION, BASELINE } from "./model/model.mjs";

// QA8: known set of violations. After the twelve decisions were applied, five were flagged in
// the records as "unacceptable regression"; the remaining seven's drop was a knowingly accepted
// trade-off. This flag is a judgment call, it does not come out of the measurement. Model.
const VIOLATION = ["search cache", "secret value rotation", "local queue at the branch",
  "separating membership data", "layer separation"];

const ROOT = [...new Set(LEAF.map((y) => y[1]))];
const CEILING = { "%": 100 };   // a leaf carrying a percentage cannot exceed 100
const share = ([, , , , threshold, direction], v) => (direction === "low" ? threshold - v : v - threshold) / threshold;
const score = (value) => Object.fromEntries(ROOT.map((k) => {
  const y = LEAF.filter((n) => n[1] === k);
  return [k, y.reduce((s, n) => s + share(n, value[n[0]]), 0) / y.length];
}));
const baseline = score(BASELINE);
const biggestDrop = ([, change]) => {
  const d = score({ ...BASELINE, ...change });
  return Math.max(0, ...ROOT.map((k) => (baseline[k] - d[k]) * 100));
};

// Check: when a decision is applied, no root attribute's score should drop by more than the threshold.
// Where to set the threshold does not come out of the measurement; the sweep below counts the options.
console.log(`${"threshold (share points)".padEnd(25)}${"fired".padStart(7)}${"caught".padStart(8)}` +
  `${"missed".padStart(8)}${"false alarm".padStart(13)}`);
for (const threshold of [2, 4, 6, 8, 10, 12]) {
  const fired = DECISION.filter((k) => biggestDrop(k) > threshold).map(([name]) => name);
  const caught = fired.filter((name) => VIOLATION.includes(name));
  console.log(`${String(threshold).padEnd(25)}${String(fired.length).padStart(7)}` +
    `${String(caught.length).padStart(8)}${String(VIOLATION.length - caught.length).padStart(8)}` +
    `${String(fired.length - caught.length).padStart(13)}`);
}

const CHOSEN = 6;
const fired = DECISION.filter((k) => biggestDrop(k) > CHOSEN).map(([name]) => name);
console.log(`\nchosen threshold ${CHOSEN} share points`);
for (const name of VIOLATION.filter((a) => !fired.includes(a)))
  console.log(`  missed: ${name} (biggest drop ` +
    `${biggestDrop(DECISION.find(([k]) => k === name)).toFixed(1)} share points, below threshold)`);
for (const name of fired.filter((a) => !VIOLATION.includes(a)))
  console.log(`  false alarm: ${name} (drop accepted in the record)`);

// Cumulative effect: decisions are applied in order, each change scaling that leaf's value relative to its own starting
// value. A single-decision check only ever looks at one decision at a time, so it cannot see the accumulation.
const end = { ...BASELINE };
for (const [, change] of DECISION)
  for (const [code, value] of Object.entries(change)) {
    const y = LEAF.find((n) => n[0] === code);
    end[code] = Math.min(end[code] * (value / BASELINE[code]), CEILING[y[3]] ?? Infinity);
  }

const exceeding = LEAF.filter((y) => share(y, end[y[0]]) < 0);
const single = DECISION.flatMap(([name, d]) =>
  LEAF.filter((y) => y[0] in d && share(y, d[y[0]]) < 0).map((y) => [name, y[0]]));
console.log(`\n(decision, leaf) pairs exceeding the threshold in a single decision ${single.length} ` +
  `(${single.map(([a, k]) => `${a}/${k}`).join(", ")}); with all twelve decisions applied together, ` +
  `leaves exceeding the threshold ${exceeding.length}, across ${[...new Set(exceeding.map((n) => n[1]))].length} attributes`);
for (const y of exceeding)
  console.log(`  ${y[0]}  ${y[2].padEnd(26)} ${BASELINE[y[0]]} -> ${end[y[0]].toFixed(1)} ` +
    `${y[3]} (threshold ${y[4]})`);

// QA9: the part that cannot be checked. The check measures the drop, not its acceptability; that is a
// judgment call. An accepted trade-off record stands in its place, and the check looks at the record instead.
const acceptedTradeOff = fired.filter((a) => !VIOLATION.includes(a));
console.log(`\ncannot be checked: whether a drop is acceptable (a judgment call). ${acceptedTradeOff.length} ` +
  `accepted trade-off records stand in its place; after the record, false alarm ${acceptedTradeOff.length} -> 0, ` +
  `caught ${fired.length - acceptedTradeOff.length}, missed ${VIOLATION.length - (fired.length - acceptedTradeOff.length)}`);
console.log(`check cost: ${LEAF.length} re-measurements per decision, ` +
  `${DECISION.length} decisions x ${DECISION.length * LEAF.length} measurements; ` +
  `run ${DECISION.length * ROOT.length + LEAF.length} comparisons; ` +
  `maintenance ${acceptedTradeOff.length} accepted trade-off records (reviewed every time a decision changes)`);

threshold (share points)   fired  caught  missed  false alarm
2                             10       4       1            6
4                             10       4       1            6
6                              9       4       1            5
8                              4       4       1            0
10                             4       4       1            0
12                             3       3       2            0

chosen threshold 6 share points
  missed: layer separation (biggest drop 1.7 share points, below threshold)
  false alarm: synchronous replication (drop accepted in the record)
  false alarm: authenticate every request (drop accepted in the record)
  false alarm: automatic failover (drop accepted in the record)
  false alarm: per-request rate limiting (drop accepted in the record)
  false alarm: moving bulk upload to nighttime (drop accepted in the record)

(decision, leaf) pairs exceeding the threshold in a single decision 1 (separating membership data/G1); with all twelve decisions applied together, leaves exceeding the threshold 4, across 3 attributes
  P2  loan transaction latency   420 -> 799.3 ms (threshold 600)
  S1  files touched in a change  4 -> 11.0 file (threshold 6)
  T1  environment-specific call  7 -> 12.9 call (threshold 12)
  T2  edge unit install          30 -> 50.7 min (threshold 45)

cannot be checked: whether a drop is acceptable (a judgment call). 5 accepted trade-off records stand in its place; after the record, false alarm 5 -> 0, caught 4, missed 1
check cost: 14 re-measurements per decision, 12 decisions x 168 measurements; run 74 comparisons; maintenance 5 accepted trade-off records (reviewed every time a decision changes)

The threshold sweep shows no single threshold finishes the job. At eight share points the check catches four of five known violations and never false-alarms; lowering it to six does not raise the catch, and false alarms climb to five. Even loosened to two, the fifth violation still gets missed. The reason is not sensitivity: that decision’s biggest drop is 1.7 share points, and what makes it count as unacceptable is not the size of the drop but where it lands — loan transactions at peak hour. The check looks at the score average, not at the distribution below that average. The missed violation is not below the check’s sensitivity; it is outside the size it looks at.

All five false alarms at the six-point threshold are knowingly accepted trade-offs. This puts a name on what cannot be checked: the check can measure the size of a drop, not its acceptability, because acceptability is a judgment call. An accepted trade-off record stands in its place — for every accepted drop, which attribute it was accepted for gets written down, and the check looks at the record. After five records, the false alarm count drops to zero. The cost is clear: the five records must be reviewed every time a decision changes, and once a record goes stale, the check goes quiet, and its silence is no longer correct.

The last measurement shows the check’s structural blindness. A single-decision check sees threshold exceedance in only one (decision, leaf) pair across the twelve decisions; when the decisions are applied together, four leaves exceed the threshold, across three separate attributes. Worse, the single exceedance the single-decision check does see closes under accumulation — a later decision carries that leaf back above the threshold. A single-decision check and a cumulative check find the opposite of each other on the same set; choosing one means the other is missed entirely. The cost has two line items: re-measuring all fourteen leaves after every decision (168 measurements) and maintaining five accepted trade-off records; the run itself, 74 comparisons, is negligible next to the measurement.

Summary

• The tension matrix counts the measured drop in other attributes from a decision that raises one; 11 of the twenty ordered pairs conflict, two are mutual.
• The most suppressed attribute is maintainability: in six decisions, for three separate attributes, a total of 53.3 share points; no decision targets it, yet it pays the cost of every addition.
• No decision raises portability, four drop it; an attribute with a written threshold only falls as long as no decision defending it gets proposed.
• No threshold in the sweep catches all five violations at once: 4/1/0 at eight points, 4/1/5 at six points (caught / missed / false alarm); the missed one is outside the size the check looks at.
• The check measures the drop, not its acceptability; five accepted trade-off records stand in its place and the false alarm count drops to zero — where the single-decision check sees one exceedance, the cumulative check finds four others.

Next Step

In the matrix, security rose with three decisions, and in return suppressed performance and maintainability with three decisions each. But in this account, security was a single score: the average of three leaves’ share. The next lesson opens up that score, because the cost of a security decision does not fit into one number. A system is made of boundaries that carry data from one place to another; how many boundaries there are, which data crosses which boundary, and how many checks stand at each boundary can all be counted. Counting them turns up two things: crossings that pass through no check at all, and the new paths a removed boundary opens.