--- title: 'Quality Attribute Scenarios' source: 'https://academia.sh/en/courses/architectural-documentation/quality-attribute-scenarios' course: 'Architectural Decisions and Documentation' language: en updated: '2026-08-23T07:01:04+00:00' license: 'CC BY-SA 4.0' --- # Quality Attribute Scenarios Writing a quality requirement so it can be tested: the same twelve requirements written in plain-sentence and fielded-scenario form, counting how many of each come out testable, turning the empty fields into questions, and measuring the disagreement an unwritten threshold causes at delivery. The previous lessons split views and diagram types by question: which picture answers which is now clear. But part of what someone who comes later asks cannot be answered by a picture. Questions like "is this system fast enough," "what happens when this branch goes offline," "is this slowdown acceptable" are not closed by a box-and-arrow picture; their answer is a number. This lesson measures where that number gets written, and what happens when it is not. What a quality attribute is and which fields a quality attribute scenario has were defined in the Quality and Testing Fundamentals course; that definition is not repeated here. The question here is narrow: given a requirement list, how many of its items are **testable**. Testability is not an opinion but a criterion — a requirement is testable if a test procedure can be written from it, and a procedure needs a stimulus to trigger, an environment to set up, a response to observe, and a threshold to compare against. Which quality attributes matter for a given system is separate, taken up in the course that follows this one; no attribute list is derived here. ## The Requirement Set to Measure The block below **models** twelve quality requirements in two forms: the plain sentence and the fielded scenario. The fielded form carries only what the plain sentence says — no information that is not in the sentence gets written into a field. The model is also written to disk; the next two blocks read the same set. ```js // set.mjs — models the requirement set in two forms and measures testability import { mkdirSync, writeFileSync } from "node:fs"; // VM16: twelve quality requirements collected from the regional library network. The // fielded form carries exactly what the plain sentence says; no field is invented. const REQUIREMENT = [ ["Catalog search must respond quickly.", "a member searches the catalog", "", "a result list is returned", "", ""], ["Lending must not stop when the branch connection drops.", "the connection between the branch and the center drops", "", "lending continues", "", ""], ["The system must stay up during peak hours.", "", "peak hour", "the system keeps accepting requests", "", ""], ["A loan record must not be lost.", "", "", "the record stays readable", "", ""], ["The overdue notification must go out on time.", "the loan period expires", "", "a notification is sent", "", ""], ["The catalog bulk upload must fit inside the nightly window.", "a bulk upload starts", "the nightly maintenance window", "the upload finishes", "", ""], ["A member must be able to see their own loan history without waiting.", "the member opens the history page", "", "the history is listed", "", ""], ["Under 500 concurrent members, the loan operation must complete within 400 ms for 95% of requests.", "a member sends a loan request", "500 concurrent members", "a loan record is created", "95% of requests under 400 ms", ""], ["When the central database node is lost, the loan service must fail over to the replica within 60 seconds.", "the primary database node stops responding", "business-hours load", "the service runs on the replica node", "failover within 60 seconds", ""], ["A new branch definition must reach production in a single deployment within 1 hour.", "a branch definition is added", "a single deployment window", "the branch appears in production", "1 hour from addition to release", "branch opening calendar: two branches a month"], ["The system must support backups.", "", "", "", "", ""], ["The report screen must open within a reasonable time.", "a manager opens the report screen", "", "the report is displayed", "", ""], ]; const FIELD = ["stimulus", "environment", "response", "measure", "source"]; const SET = REQUIREMENT.map(([plain, ...value], i) => ({ no: i + 1, plain, field: Object.fromEntries(FIELD.map((f, j) => [f, value[j]])), })); mkdirSync("set", { recursive: true }); writeFileSync("set/set.mjs", `export const FIELD = ${JSON.stringify(FIELD)};\n` + `export const SET = ${JSON.stringify(SET, null, 0)};\n`); // Testability criterion: writing a test procedure requires a stimulus to trigger, an // environment to set up, a response to observe, and a threshold to compare against. The // threshold is a number carrying a unit; both forms are searched with the same pattern. const THRESHOLD = /(\d+\s*(ms|seconds?|minutes?|hours?|days?)|\d+\s*%|\d+%)/; const CONDITION = /(is lost|concurrent|deployment|window|drops|peak|load|under)/; const FOUR = ["stimulus", "environment", "response", "measure"]; const plainTestable = (r) => THRESHOLD.test(r.plain) && CONDITION.test(r.plain); const fieldedTestable = (r) => FOUR.every((f) => r.field[f] !== "") && THRESHOLD.test(r.field.measure); console.log(`${"no".padStart(2)} ${"plain form".padEnd(12)}${"fielded form (filled field)".padEnd(40)}empty field`); for (const r of SET) { const filled = FOUR.filter((f) => r.field[f] !== ""); console.log(`${String(r.no).padStart(2)} ` + `${(plainTestable(r) ? "testable" : "-").padEnd(12)}` + `${(filled.join(",") || "-").padEnd(40)}` + `${4 - filled.length}`); } const emptyFour = SET.reduce((s, r) => s + FOUR.filter((f) => r.field[f] === "").length, 0); const emptyFive = SET.reduce((s, r) => s + FIELD.filter((f) => r.field[f] === "").length, 0); const wordsPlain = SET.reduce((s, r) => s + r.plain.length, 0); const wordsFielded = SET.reduce((s, r) => s + FIELD.reduce((t, f) => t + f.length + 2 + r.field[f].length, 0), 0); console.log(`\ntestable: plain form ${SET.filter(plainTestable).length}/${SET.length}, ` + `fielded form ${SET.filter(fieldedTestable).length}/${SET.length}`); console.log(`empty field: in the four fields ${emptyFour}, including the fifth field (source) ${emptyFive}`); console.log(`writing cost (characters): plain ${wordsPlain}, fielded ${wordsFielded} ` + `(${(wordsFielded / wordsPlain).toFixed(2)}x)`); ``` ``` no plain form fielded form (filled field) empty field 1 - stimulus,response 2 2 - stimulus,response 2 3 - environment,response 2 4 - response 3 5 - stimulus,response 2 6 - stimulus,environment,response 1 7 - stimulus,response 2 8 testable stimulus,environment,response,measure 0 9 testable stimulus,environment,response,measure 0 10 testable stimulus,environment,response,measure 0 11 - - 4 12 - stimulus,response 2 testable: plain form 3/12, fielded form 3/12 empty field: in the four fields 20, including the fifth field (source) 31 writing cost (characters): plain 706, fielded 1406 (1.99x) ``` ## What the Fielded Form Changes In both forms, the number of testable requirements is three. This breaks the most common expectation loaded onto the fielded scenario: **opening fields does not manufacture information.** If the sentence carries no threshold, neither does the field; changing the form does not make a requirement testable. What does change is this: the plain form gives one piece of information for the nine untestable requirements — no number. The fielded form gives, for each one, **which field** is empty. The fourth requirement is missing three fields, the eleventh is empty in all four, the sixth is missing only the measure. This separates the size and the kind of the gap; neither shows up in the plain sentence. The empty-field count is twenty, and that number is directly a list of questions. The fielded form's writing cost is 1.99 times the plain form's: 706 characters against 1,406. That is the price; the payoff is knowing exactly where to ask each of the twenty questions. The fifth field stands apart. `source` holds where the threshold came from, and it is filled in only one of the twelve requirements. The testability criterion looks at the first four fields only, because a test procedure can be written without knowing where the threshold came from. Why the fifth field still matters is measured in this lesson's last section. ## The Cost of an Unwritten Threshold An untestable requirement does not charge a cost immediately; the cost surfaces at delivery. When a threshold is not written down, it does not disappear — it sits in each side's head as two separate thresholds. The requester thinks of one number when they say "fast," the developer another. The block below models this: each requirement gets a scale where smaller is better, a threshold per side, and a measured value for the delivered system. ```js // disagreement.mjs — reads set/set.mjs written by set.mjs; measures the disagreement an // untestable requirement causes at delivery import { SET } from "./set/set.mjs"; // VM17: every requirement has a scale where smaller is better. An untestable requirement // carries two unwritten thresholds, one for the requester and one for the developer. The // three testable requirements have a written threshold; both sides use the same number. const THRESHOLD = [ // [scale, requester's threshold, developer's threshold, delivery range] ["search response time (ms)", 300, 1200, [200, 1500]], ["loan-halt duration (min)", 5, 120, [0, 180]], ["rejected requests (per mille)", 1, 20, [0, 40]], ["lost records per year", 0, 3, [0, 6]], ["notification delay (hr)", 1, 24, [0, 36]], ["bulk upload duration (min)", 60, 240, [30, 300]], ["history page (ms)", 500, 2500, [300, 3000]], ["loan operation p95 (ms)", 400, 400, [250, 900]], ["failover (sec)", 60, 60, [20, 180]], ["release rollout (min)", 60, 60, [20, 240]], ["restore from backup (hr)", 2, 24, [1, 48]], ["report screen (sec)", 3, 20, [1, 30]], ]; // VM18: the delivered measurement is drawn from each requirement's own range with a // uniform distribution. The generator is hand-written; the seed is visible. const generator = (seed) => () => { seed = (seed + 0x6d2b79f5) | 0; let t = Math.imul(seed ^ (seed >>> 15), 1 | seed); t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; const SEED = 20802; const DELIVERY = 200; const random = generator(SEED); const result = THRESHOLD.map(() => ({ accept: 0, dispute: 0, reject: 0 })); let disputesPerDelivery = 0; for (let t = 0; t < DELIVERY; t += 1) { let n = 0; THRESHOLD.forEach(([, requester, developer, [low, high]], i) => { const measured = low + random() * (high - low); if (measured <= Math.min(requester, developer)) result[i].accept += 1; else if (measured > Math.max(requester, developer)) result[i].reject += 1; else { result[i].dispute += 1; n += 1; } }); disputesPerDelivery += n; } console.log(`seed ${SEED}, ${DELIVERY} deliveries; each row is one requirement (delivery count)`); console.log(`${"no".padStart(2)} ${"scale".padEnd(30)}${"threshold".padStart(12)}` + `${"accept".padStart(8)}${"dispute".padStart(13)}${"reject".padStart(7)}`); THRESHOLD.forEach(([scale, requester, developer], i) => { console.log(`${String(i + 1).padStart(2)} ${scale.padEnd(30)}` + `${(requester === developer ? `${requester}` : `${requester}/${developer}`).padStart(12)}` + `${String(result[i].accept).padStart(8)}${String(result[i].dispute).padStart(13)}` + `${String(result[i].reject).padStart(7)}`); }); const written = THRESHOLD.map(([, a, b], i) => (a === b ? i : -1)).filter((i) => i >= 0); const unwritten = THRESHOLD.map((_, i) => i).filter((i) => !written.includes(i)); const sum = (list) => list.reduce((s, i) => s + result[i].dispute, 0); console.log(`\n${written.length} requirements with a written threshold: ${sum(written)} disputes`); console.log(`${unwritten.length} requirements with no written threshold: ${sum(unwritten)} disputes`); console.log(`average disputes per delivery: ` + `${(disputesPerDelivery / DELIVERY).toFixed(2)} (across ${SET.length} requirements)`); ``` ``` seed 20802, 200 deliveries; each row is one requirement (delivery count) no scale threshold accept dispute reject 1 search response time (ms) 300/1200 12 142 46 2 loan-halt duration (min) 5/120 10 112 78 3 rejected requests (per mille) 1/20 3 99 98 4 lost records per year 0/3 0 90 110 5 notification delay (hr) 1/24 5 117 78 6 bulk upload duration (min) 60/240 13 148 39 7 history page (ms) 500/2500 8 158 34 8 loan operation p95 (ms) 400 45 0 155 9 failover (sec) 60 56 0 144 10 release rollout (min) 60 35 0 165 11 restore from backup (hr) 2/24 5 92 103 12 report screen (sec) 3/20 9 104 87 3 requirements with a written threshold: 0 disputes 9 requirements with no written threshold: 1062 disputes average disputes per delivery: 5.31 (across 12 requirements) ``` In the three requirements with a written threshold, the disagreement count is zero — a structural result of the model: with a single threshold, both sides look at the same measurement and reach the same conclusion. The eighth requirement is rejected in 155 of 200 deliveries; a failure, but an undisputed one. Rejection is a decision, not a disagreement. The nine requirements with no written threshold produce 1,062 disagreements in 200 deliveries — 5.31 per delivery. That means every delivery, more than five of the twelve requirements get two different answers to "was it met." The highest rate is on the seventh requirement: 158 of 200 deliveries end in disagreement. The reason shows in the scale — the gap between the two thresholds (500 to 2,500 ms) covers most of the delivery range. **The probability of disagreement grows with the gap between the two sides' implicit thresholds.** The more ambiguous the sentence, the wider the gap. That gives the lesson's cost comparison. Writing the set in fielded form costs 700 extra characters and twenty questions asked, paid once. Leaving a requirement untestable costs 5.31 disagreements per delivery, paid again at every delivery. ## The Question Set and the Fifth Field This course's measure is how many of the questions someone later asks a document, the document answers. Five questions get asked of quality requirements, and each lands on one field. ```js // question.mjs — reads set/set.mjs written by set.mjs; applies the question set to the set import { SET } from "./set/set.mjs"; // The five questions someone who comes later asks about a quality requirement; each question lands on one field. const QUESTION = [ ["When is this requirement considered satisfied?", "measure"], ["Under what condition will it be measured?", "environment"], ["What will trigger the test?", "stimulus"], ["What will be observed from the system?", "response"], ["Where did this number come from?", "source"], ]; console.log(`${"question".padEnd(48)}${"answered".padStart(11)}${"unanswered".padStart(13)}`); let answered = 0; for (const [question, field] of QUESTION) { const d = SET.filter((r) => r.field[field] !== "").length; answered += d; console.log(`${question.padEnd(48)}${`${d}/${SET.length}`.padStart(11)}` + `${String(SET.length - d).padStart(13)}`); } const total = QUESTION.length * SET.length; console.log(`total ${total} questions: ${answered} answered, ${total - answered} unanswered`); // Number of questions to ask to make a requirement testable (the fifth field is excluded: // testability only looks at the four fields). const FOUR = ["measure", "environment", "stimulus", "response"]; const missing = SET.map((r) => [r.no, FOUR.filter((f) => r.field[f] === "").length, r.plain.slice(0, 34)]).sort((a, b) => a[1] - b[1] || a[0] - b[0]); console.log(`\n${"no".padStart(2)} ${"to ask".padStart(9)} requirement`); for (const [no, n, c] of missing) console.log(`${String(no).padStart(2)} ${String(n).padStart(9)} ${c}`); console.log(`to make the set testable: ${missing.reduce((s, [, n]) => s + n, 0)} questions; ` + `closed by one question ${missing.filter(([, n]) => n === 1).length}, ` + `needing three or more ${missing.filter(([, n]) => n >= 3).length}`); ``` ``` question answered unanswered When is this requirement considered satisfied? 3/12 9 Under what condition will it be measured? 5/12 7 What will trigger the test? 9/12 3 What will be observed from the system? 11/12 1 Where did this number come from? 1/12 11 total 60 questions: 29 answered, 31 unanswered no to ask requirement 8 0 Under 500 concurrent members, the 9 0 When the central database node is 10 0 A new branch definition must reach 6 1 The catalog bulk upload must fit i 1 2 Catalog search must respond quickl 2 2 Lending must not stop when the bra 3 2 The system must stay up during pea 5 2 The overdue notification must go o 7 2 A member must be able to see their 12 2 The report screen must open within 4 3 A loan record must not be lost. 11 4 The system must support backups. to make the set testable: 20 questions; closed by one question 1, needing three or more 2 ``` 29 of the sixty questions get answered, 31 do not. The answer rate falls steadily: what to observe from the system is known for eleven of the twelve requirements, what to trigger for nine, under what condition to measure for five, when it counts as met for only three. In other words, the requirement's author can describe **what they want**, not **when it is enough**. The lowest row is the fifth field: the threshold's source is written in only one of twelve requirements. Even two of the three requirements with a written threshold do not record where the number came from. The distinction looks small but its consequence is heavy: the 400 ms threshold is testable, but six months later, when someone asks "can this threshold be raised," it cannot be answered. In the one requirement whose source is written (the branch opening calendar, two branches a month), that question is answered: if the calendar slows, the threshold can be loosened. **The measure makes the requirement testable; the threshold's source makes it changeable.** The last table shows how the twenty questions are distributed: three are already testable, one closes with a single question, six need two each, two need three or four. The eleventh requirement is not really a requirement but a heading; with all four fields empty, it is not filled by one question but by a separate piece of work. ## Summary - Testability is a criterion: a requirement is testable if a test procedure can be written from it; the procedure needs a stimulus, an environment, a response, and a unit-bearing threshold. - The same twelve requirements come out 3/12 testable in both forms — the fielded form does not manufacture information, it makes the missing information visible: 20 empty fields, i.e. 20 askable questions. - The fielded form's writing cost is 1.99x (706 versus 1,406 characters), paid once; the nine requirements with no written threshold produce 1,062 disagreements in 200 deliveries, 5.31 per delivery. - The probability of disagreement grows with the gap between the two sides' implicit thresholds; the three requirements with a written threshold have zero disagreements, and even its rejections are undisputed. - 31 of 60 applications of the five questions go unanswered; the least-answered is "where did this number come from" (11/12 unanswered) — the measure enables testing, the source enables changing it. ## Next Step When the requirement set is made measurable, a document results: fielded scenarios, thresholds, sources. That document is in step with the code on the day it is written. The next lesson measures what comes after: the code keeps changing, the document stays put, and the gap grows. This gap is found by counting, not guessing: reality extracted from a module graph is compared against the document's claim, how drift grows across a sequence of changes is tracked, and two keeping-current regimes — manual update and a generated audit — are compared by the drift caught, the maintenance cost imposed, and the false alarms produced.