Quality Attributes

The previous lesson was about the fee calculation producing the right result. The right result is only one part of the expectations. The library is unusable if it computes the correct fee but takes minutes to produce the report, if it shows one member’s debt to another member, or if a single corrupted record makes it stop working entirely. These expectations are most often not written into the specification, and when they are, they appear as untestable sentences like “make it fast,” “make it robust.”

This lesson names those sentences and makes them testable. The criterion for testability is singular: an expectation is testable if it can be turned into a criterion whose pass or fail can be checked.

Functional Expectation and Quality Attribute

A functional requirement states what the system does: a book four days late incurs a 2-unit fee, an unknown member type is rejected. It is answered as true or false; there is no value in between.

A quality attribute states how the system does that work: how fast, how robust, how secure, how modifiable. Its answer comes in degrees and only gains meaning together with a threshold. This is why, for attributes, what’s discussed is not correctness but a budget.

The distinction is not merely a classification exercise; the two kinds of expectations are tested differently. A functional expectation can be tested with a single input–output pair. An attribute can only be tested with a measurement setup, a load, and a threshold.

Attribute Families

Attributes cannot be discussed until they are named. The commonly used families and what they mean in the library’s context are these:

Attribute family	Library question
functional suitability	Does the fee calculation conform to the rule?
performance efficiency	How long does the report take to produce?
reliability	What happens when a corrupted record arrives?
usability	In how many steps does staff see the debt?
security	Is one member’s debt visible to another?
maintainability	From how many places does the fee rule change?
portability	Does it install on a different runtime?
compatibility	Can the old record format be read?

The list is not closed; every domain has its own additions. What matters is asking which family an expectation belongs to — because the family determines how the measure gets built.

Turning an Unmeasurable Expectation into a Criterion

The sentence “make the report fast” cannot be tested, because the questions of how fast, under what load, in what environment have no answer. The form that fills this gap has four parts and is known as a quality attribute scenario:

• Stimulus: staff requests the overdue-member report.
• Environment: the library has 8,000 members and 80,000 loan records, and the system is running under ordinary load.
• Response: the report produces each member’s total late-fee debt.
• Response measure: the median of five runs does not exceed 100 milliseconds.

The fourth part is what makes the expectation testable. The budget concept set up in the Performance Budget lesson of the Frontend Quality course is used here in its domain-independent form: a criterion is written together with a number, a load, and a measurement method.

Measuring the Criterion

Once a criterion is written, it can be measured. The report is written in two forms: a version that scans all loan records for every member, and a version that walks the records once and totals them by member number.

// report.mjs — two implementations of the overdue-member report
export function generateData(memberCount, loanCount) {
  const members = Array.from({ length: memberCount }, (_, i) => ({ no: i, type: i % 3 === 0 ? 'student' : 'member' }));
  const loans = Array.from({ length: loanCount }, (_, i) => ({
    memberNo: (i * 7919) % memberCount,
    daysLate: i % 47,
  }));
  return { members, loans };
}

function fee(daysLate) {
  return Math.min(Math.max(daysLate - 3, 0) * 2, 20);
}

export function scanningReport(members, loans) {
  return members.map((m) => {
    let debt = 0;
    for (const l of loans) if (l.memberNo === m.no) debt += fee(l.daysLate);
    return { no: m.no, debt };
  });
}

export function indexedReport(members, loans) {
  const debts = new Map();
  for (const l of loans) debts.set(l.memberNo, (debts.get(l.memberNo) ?? 0) + fee(l.daysLate));
  return members.map((m) => ({ no: m.no, debt: debts.get(m.no) ?? 0 }));
}

Both versions satisfy the same functional requirement; the reports they produce are identical. Where they differ is the attribute criterion.

// measurement.mjs — measuring the response-time criterion
import { generateData, scanningReport, indexedReport } from './report.mjs';

const BUDGET_MS = 100;
const { members, loans } = generateData(8000, 80000);

function medianTime(fn, runs = 5) {
  const times = [];
  for (let i = 0; i < runs; i += 1) {
    const start = performance.now();
    fn(members, loans);
    times.push(performance.now() - start);
  }
  times.sort((a, b) => a - b);
  return times[Math.floor(runs / 2)];
}

for (const [name, fn] of [['scanning', scanningReport], ['indexed', indexedReport]]) {
  const median = medianTime(fn);
  const verdict = median < BUDGET_MS ? 'passed' : 'missed';
  console.log(`${name.padEnd(9)}: ${BUDGET_MS} ms budget — ${verdict}`);
}

scanning : 100 ms budget — missed
indexed  : 100 ms budget — passed

The output writes the verdict, not the raw time; raw time is machine-dependent and does not carry over from one lesson to another. On the machine used while preparing this text, the scanning version’s median measured approximately 560 ms and the indexed version’s approximately 3 ms — both numbers vary with the hardware and runtime version they run on. What does not change is that the criterion separates the two versions: once the budget is set, the answer to “is it fast” stops being a matter of debate.

Basing the measurement on the median of five runs rather than a single run is deliberate. The first run carries the runtime’s warm-up effect; the median leaves that spike out.

A Second Attribute: Reliability

Performance is not the only attribute, and every attribute demands its own measure. A reliability scenario is written like this: when a portion of the records arrive corrupted, the report does not stop — it skips the corrupted record and reports the skipped ratio; if the ratio exceeds five percent, the report is considered invalid.

// reliability.mjs — measuring the reliability criterion
import { generateData } from './report.mjs';

const THRESHOLD = 0.05;
const { loans } = generateData(8000, 80000);
for (let i = 0; i < loans.length; i += 33) loans[i].daysLate = -1;

let processed = 0;
let skipped = 0;
for (const l of loans) {
  if (!Number.isInteger(l.daysLate) || l.daysLate < 0) skipped += 1;
  else processed += 1;
}

const ratio = skipped / loans.length;
console.log(`processed loans: ${processed}`);
console.log(`skipped loans  : ${skipped}`);
console.log(`skipped ratio  : ${(ratio * 100).toFixed(2)} (threshold ${THRESHOLD * 100})`);
console.log(`report status  : ${ratio <= THRESHOLD ? 'valid' : 'invalid'}`);

processed loans: 77575
skipped loans  : 2425
skipped ratio  : 3.03 (threshold 5)
report status  : valid

Because this measurement does not involve time, it gives the same result on every machine. Not every attribute criterion is time-dependent: ratio, count, size, and step count are also criteria, and because they are deterministic, testing them is cheaper.

Attributes Compete with Each Other

The indexed version cut the time down, but it did so by building a map holding as many entries as there are members. Memory consumption grew. The version that skips corrupted records kept the report running, but it silently produced an incomplete total; continuity was bought at the cost of accuracy.

This is a general condition: attributes cannot be improved independently of one another. Performance and memory, security and usability, flexibility and simplicity most often pull in opposite directions. This is why attribute decisions are trade-offs and must be written down along with their rationale: which attribute was given up, and how much, for the sake of which other attribute? This is where writing a budget shows its real function. The budget settles the trade-off through a single discussion; every measurement after that is made against that decision.

Summary

• A functional expectation states what the system does, a quality attribute states how it does it; the first is answered true-false, the second in degrees.
• An attribute expectation becomes testable once it is written as stimulus, environment, response, and response measure; an expectation without a criterion cannot be tested.
• Two implementations satisfying the same function can give different results against the same budget; in the example the scanning version missed the 100 ms budget and the indexed version passed it.
• If a measurement involves time, the output should write the verdict, not the raw time; raw time is machine-dependent.
• Ratio, count, and size criteria are deterministic and cost less to test than time criteria.
• Attributes compete with each other; the budget settles the trade-off once and gives subsequent measurements a shared criterion.

Next Step

In both lessons, the defect surfaced only because someone went looking for it: because the desk records were compared, because the budget was measured. If no one had made that comparison, neither defect would have appeared. This raises a question — what does the person looking for the defect do differently? The next lesson treats testing as a mindset before it is a set of techniques: a view trying to show that the program works and a view searching for where it breaks look at the same code and find different things.