Verification and Validation

The previous lesson derived checks from the specification: whatever the written rule said, the check asked exactly that. When every check passes, the conclusion is: the library conforms to the written rule. That conclusion is narrower than it sounds. The written rule itself might not be the rule the library actually needs to enforce. In that case the program does the wrong thing flawlessly.

This lesson separates those two questions. The first asks whether the product conforms to the specification; the second asks whether the specification conforms to the need. Adding a new rule to the same library will show that the two are independent: the late fee calculation.

Two Separate Questions

Verification asks: are we building the product right? Its criterion sits one step behind the product — the specification, a design decision, an acceptance criterion. Producing the answer means comparing the product against that written criterion.

Validation asks: are we building the right thing? Its criterion sits one step behind the specification itself — the user’s work, the organization’s rule, the service’s purpose. Producing the answer means comparing the specification against real usage.

The difference between the two questions is where the criterion comes from. Verification’s criterion is a document; validation’s criterion is the world outside that document. This is why work that only verifies is self-contained: it confirms whatever the document says and never raises the possibility that the document itself is wrong.

A Wrong Program That Conforms to the Specification

A late fee rule is added to the library. The item written during analysis is this:

• K7 — The late fee is 2 units per day.
• K8 — Negative days late is rejected.

The implementation follows the item exactly.

// fee.mjs — literal implementation of the K7 rule from the specification
export const DAILY_RATE = 2;

export function lateFee(daysLate) {
  if (daysLate < 0) throw new Error('days late cannot be negative');
  return daysLate * DAILY_RATE;
}

The checks are also derived from the same item. The same arrangement as the previous lesson is used: every check is tied to a rule number.

// verification.mjs — checks derived from the written specification
import assert from 'node:assert/strict';
import { lateFee } from './fee.mjs';

const CHECKS = {
  'K7 2 units per day': () => assert.equal(lateFee(5), 10),
  'K7 no days late, no fee': () => assert.equal(lateFee(0), 0),
  'K7 fee is directly proportional to days late': () => {
    assert.equal(lateFee(10), 2 * lateFee(5));
  },
  'K8 negative days late is rejected': () => assert.throws(() => lateFee(-1)),
};

let passed = 0;
for (const [name, check] of Object.entries(CHECKS)) {
  try {
    check();
    passed += 1;
  } catch (error) {
    console.log(`failed: ${name} — ${error.message.split('\n')[0]}`);
  }
}
console.log(`verification: ${passed}/${Object.keys(CHECKS).length} checks passed`);

verification: 4/4 checks passed

Verification is complete. The rule consisted of two items, both items were checked, and both passed. The traceability measurement also comes out flawless: every written rule has a check. At this point what’s in hand is a program proven to fully conform to its specification.

Testing Against Reality

The library’s front desk had been collecting fees for years. This is validation’s criterion: not the document, but transactions that actually happened. The program recomputes those transactions and compares the result against the amount that was collected.

// validation.mjs — comparison against fees actually collected at the desk
import { lateFee } from './fee.mjs';

const DESK_RECORDS = [
  { transaction: 'A-101', daysLate: 0, collected: 0 },
  { transaction: 'A-102', daysLate: 2, collected: 0 },
  { transaction: 'A-103', daysLate: 3, collected: 0 },
  { transaction: 'A-104', daysLate: 4, collected: 2 },
  { transaction: 'A-105', daysLate: 10, collected: 14 },
  { transaction: 'A-106', daysLate: 40, collected: 20 },
];

let matched = 0;
for (const record of DESK_RECORDS) {
  const computed = lateFee(record.daysLate);
  if (computed === record.collected) matched += 1;
  else console.log(`${record.transaction}: ${record.daysLate} days — program ${computed}, desk ${record.collected}`);
}
console.log(`validation: ${matched}/${DESK_RECORDS.length} records matched`);

A-102: 2 days — program 4, desk 0
A-103: 3 days — program 6, desk 0
A-104: 4 days — program 8, desk 2
A-105: 10 days — program 20, desk 14
A-106: 40 days — program 80, desk 20
validation: 1/6 records matched

One of the six records matches. The matching record is the transaction with no days late — the case where the fee calculation never runs at all. The pattern in the five mismatched records gives away two missing rules: the first three days are free, and the fee stops at an upper limit. Both rules were being applied at the desk, but neither had made it into the specification.

The defect is not in the code. The code correctly does what was asked of it. The defect is in the analysis stage, and no amount of verification can find it — because verification uses, as its criterion, exactly the flawed document.

The Corrected Specification

Validation’s output is not a code fix but a specification fix. K7 is split into three items: the grace period is three days, every day after that costs 2 units, and the fee never exceeds 20 units.

// fee2.mjs — specification corrected after validation
export const GRACE_DAYS = 3;
export const DAILY_RATE = 2;
export const FEE_CAP = 20;

export function lateFee(daysLate) {
  if (daysLate < 0) throw new Error('days late cannot be negative');
  const billable = Math.max(daysLate - GRACE_DAYS, 0);
  return Math.min(billable * DAILY_RATE, FEE_CAP);
}

Both checks can now be applied at the same time. Verification is derived from the new items; validation uses the same desk records.

// two-checks.mjs — the corrected specification is both verified and validated
import assert from 'node:assert/strict';
import { lateFee } from './fee2.mjs';

const CHECKS = {
  'K7a first 3 days are free': () => assert.equal(lateFee(3), 0),
  'K7b 2 units for each day after': () => assert.equal(lateFee(6), 6),
  'K7c fee never exceeds 20 units': () => assert.equal(lateFee(100), 20),
  'K8 negative days late is rejected': () => assert.throws(() => lateFee(-1)),
};

let passed = 0;
for (const [name, check] of Object.entries(CHECKS)) {
  try { check(); passed += 1; } catch { console.log(`failed: ${name}`); }
}

const DESK_RECORDS = [
  { daysLate: 0, collected: 0 }, { daysLate: 2, collected: 0 },
  { daysLate: 3, collected: 0 }, { daysLate: 4, collected: 2 },
  { daysLate: 10, collected: 14 }, { daysLate: 40, collected: 20 },
];
const matched = DESK_RECORDS.filter((r) => lateFee(r.daysLate) === r.collected).length;

console.log(`verification: ${passed}/${Object.keys(CHECKS).length} checks passed`);
console.log(`validation  : ${matched}/${DESK_RECORDS.length} records matched`);

verification: 4/4 checks passed
validation  : 6/6 records matched

In the first version, the first line was also full. What distinguishes the two versions is the second line. When a quality claim is made, you should ask which line it shows.

Every Stage Has Its Own Test

Verification is not a single activity; every intermediate artifact has its own verification. The design is verified against the specification, the code against the design, the configuration against the code. This pairing of development stages with testing stages is known as the V-model: every decomposition step on the left has a corresponding integration and testing step on the right. The model is debatable as a process proposal, but the pairing it establishes is durable — every decision level gets its own criterion and its own test.

Verification’s methods do not require running the program. Review, static analysis, and type checking also look for deviation from the specification in the product. Testing is the kind of verification that runs the program.

Validation’s methods, on the other hand, require the person who will use the product, or that person’s work: acceptance testing, comparison against real transaction records, user trials, specification by example — writing agreed-upon concrete examples in place of the rule’s abstract sentence. The desk records above are the raw form of this last method.

The cost of validation arriving late is bigger than verification’s. Wrong code gets fixed; a design, data schema, interface, and training material all built on a wrong specification get redone in their entirety. In the fee calculation example, only a single function changed, but if the amounts already collected had been wrong, a retroactive correction would also have been needed. This is why the validation question is asked while the specification is being written — agreeing on a few concrete examples is a small fraction of the correction that would otherwise come later.

Summary

• Verification asks whether the product conforms to the specification, validation whether the specification conforms to the real need; the two criteria come from different places.
• Work that only verifies is self-contained: because it takes the document as its criterion, it never raises the possibility that the document is wrong.
• In the example, all four checks passed, but only one of six desk records matched; the defect was in the analysis stage, not the code.
• Validation’s output is most often a specification fix, not a code fix.
• Verification can also be done through review and static analysis; validation requires the work of the person who will use the product.
• A wrong specification costs more than wrong code, because it takes everything built on top of it down with it.

Next Step

This lesson discussed the fee calculation producing the right result. The right result is only one part of the expectations. Even if the library computes the correct fee, it is unusable if answering takes a minute, if it does not fit in memory at ten thousand records, or if it shows another member’s debt. These expectations are most often not written into the specification, and when they are, they appear as unmeasurable sentences like “make it fast.” The next lesson names these expectations and turns them into measurable criteria.