Test Levels

The previous topic established what quality is, what questions it is tested against, and how the process positions that testing. What is missing is the internal order of testing itself. The same defect can be caught by a test that exercises a single function or by one that brings up the whole system; the two do not cost the same and do not say the same thing.

This lesson separates tests by the boundary they cover. The class names — unit, integration, system, acceptance — are names for this single criterion.

What Determines the Level Is Coverage

Unit test exercises a single function or module, touching no other part. Its input is directly the function’s input, its output is directly the return value.

Integration test exercises two or more parts working together. The defect is no longer inside the parts but can be in the contract between them: a misordered argument, a mismatched unit, an unexpected null value.

System test exercises all parts together, through the real entry point. Its question is whether the result the user sees is correct.

Acceptance test overlaps with the system test in coverage; where it differs is the source of the criterion. The acceptance test’s expected result is written in the language of the party who will use the product and forms the condition of delivery. The validation question separated in the second lesson is asked at this level.

The Testing and Monitoring topic in the Frontend Quality course built the same distinction in a browser context; here the levels are treated independently of the domain.

The Same Defect at Three Levels

The library is split into three files. The rule layer computes the fee for a single delay, the aggregation layer derives a member’s debt, and the transaction layer generates the return receipt.

// fee.mjs — rule layer: the fee for a single delay
export const GRACE_DAYS = 3;
export const DAILY_FEE = 2;
export const UPPER_LIMIT = 20;

export function lateFee(lateDays) {
  return Math.max(lateDays - GRACE_DAYS, 0) * DAILY_FEE;
}

// debt.mjs — aggregation layer: the sum of a member's delays
import { lateFee } from './fee.mjs';

export function memberDebt(loans) {
  return loans.reduce((total, o) => total + lateFee(o.lateDays), 0);
}

// receipt.mjs — transaction layer: generating the return receipt
import { lateFee } from './fee.mjs';
import { memberDebt } from './debt.mjs';

export function returnReceipt(member, loans) {
  return {
    memberNo: member.no,
    items: loans.map((o) => ({ book: o.book, fee: lateFee(o.lateDays) })),
    total: memberDebt(loans),
  };
}

There is a defect in the rule layer: UPPER_LIMIT is defined but never used in the computation. The same defect is searched for at all three levels. The block below generates three test files and runs each one separately with Node’s built-in test runner. Because the duration fields in the output depend on the environment, only the result count and the assertion message are filtered through.

cat > unit.test.mjs <<'EOF'
import test from 'node:test';
import assert from 'node:assert/strict';
import { lateFee } from './fee.mjs';

test('a single delay fee does not exceed the upper limit', () => {
  const fee = lateFee(100);
  assert.ok(fee <= 20, `fee ${fee} units`);
});
EOF

cat > integration.test.mjs <<'EOF'
import test from 'node:test';
import assert from 'node:assert/strict';
import { memberDebt } from './debt.mjs';

test('the total debt of a member does not exceed the upper limit', () => {
  const debt = memberDebt([{ lateDays: 100 }, { lateDays: 30 }]);
  assert.ok(debt <= 20, `debt ${debt} units`);
});
EOF

cat > system.test.mjs <<'EOF'
import test from 'node:test';
import assert from 'node:assert/strict';
import { returnReceipt } from './receipt.mjs';

test('the return receipt total does not exceed the upper limit', () => {
  const receipt = returnReceipt({ no: 7 }, [{ book: 'K-1', lateDays: 100 }]);
  assert.ok(receipt.total <= 20, `receipt total ${receipt.total} units`);
});
EOF

for group in unit integration system; do
  node --test "$group.test.mjs" > "$group.log" 2>&1
  printf '%-12s %s | %s\n' "$group" "$(grep -E '^ℹ fail ' "$group.log")" \
    "$(grep -m1 AssertionError "$group.log" | sed 's/.*: //')"
done

unit         ℹ fail 1 | fee 194 units
integration  ℹ fail 1 | debt 248 units
system       ℹ fail 1 | receipt total 194 units

All three levels caught the defect. This does not mean the levels are redundant; what differs is the cost of catching it.

Diagnostic Distance

A test failing does not directly say where the defect is; it only says the defect is somewhere within its coverage. The wider the coverage, the larger the area in which the defect must be searched for.

// levels.mjs — the same defect caught at three levels, and the file count each covers
import assert from 'node:assert/strict';
import { lateFee } from './fee.mjs';
import { memberDebt } from './debt.mjs';
import { returnReceipt } from './receipt.mjs';

const LEVELS = [
  {
    name: 'unit', files: ['fee.mjs'],
    check: () => assert.equal(lateFee(100), 20),
  },
  {
    name: 'integration', files: ['fee.mjs', 'debt.mjs'],
    check: () => assert.equal(memberDebt([{ lateDays: 100 }, { lateDays: 2 }]), 20),
  },
  {
    name: 'system', files: ['fee.mjs', 'debt.mjs', 'receipt.mjs'],
    check: () => {
      const receipt = returnReceipt({ no: 7 }, [{ book: 'K-1', lateDays: 100 }]);
      assert.equal(receipt.total, 20);
    },
  },
];

for (const s of LEVELS) {
  let result = 'passed';
  try { s.check(); } catch { result = 'failed'; }
  console.log(`${s.name.padEnd(12)}: ${result} — spans ${s.files.length} file${s.files.length === 1 ? '' : 's'}`);
}

unit        : failed — spans 1 file
integration : failed — spans 2 files
system      : failed — spans 3 files

When a unit test fails, there is exactly one file to look at. When a system test fails, the defect could be in any of the three files, and finding out which one is a separate effort. This can be called diagnostic distance: the work between a test failing and the defect being found.

Three more costs grow along with coverage. Setup cost: a system test needs data, configuration, and an entry point. Duration: as coverage widens, the run time grows and the feedback point from the previous lesson shifts back. Fragility: any part within the coverage can fail the test, so tests with wide coverage fail more often for unrelated reasons — this is the main source of the false positive named in the fourth lesson.

Narrow coverage, in turn, has its own cost. A unit test cannot see the defect between parts: each part can work correctly on its own and still produce a wrong result once combined. The defect above was visible at every level because it lived in a single layer; contract mismatches do not behave that way.

The Distribution of Levels

The relative number of levels is a choice, and it is named after one of two extremes. In the test pyramid, the base is wide: many fast unit tests, fewer integration tests, and few system tests at the top. The rationale is cost distribution — most defects are caught at the cheap level, and the expensive level only tests the combination of parts.

The reverse arrangement is called the ice-cream cone: few unit tests, many system tests, and manual testing at the top. In this arrangement, every piece of feedback is slow and every failing test is expensive.

The pyramid is not a rule but a cost observation. The ratios change as the domain changes: a heavy computation layer widens the base, a thin interface layer shifts weight toward the upper level. The question to ask when deciding stays fixed — at which level can this defect be caught most cheaply?

Summary

• Test levels are separated not by what they test but by the boundary they cover: unit tests a single part, integration tests the combination of parts, system tests the whole.
• The acceptance test overlaps with the system test in coverage; it is separated by its criterion coming from the language of the party who will use the product.
• In the example, a defect in a single layer was caught at all three levels; the difference was not in catching it but in the cost of catching it.
• As coverage widens, diagnostic distance, setup cost, duration, and fragility all grow together; narrow coverage, in turn, cannot see the defect between parts.
• The test pyramid is not a rule but a cost observation; the deciding criterion is at which level a defect can be caught most cheaply.

Next Step

The level determines how far a test reaches. There is an independent second question: what did the person who wrote the test look at? All three tests above were derived from the specification — they read the upper-limit rule and asked about it. Had they been written by looking into the code, different questions would have come out, because the code can have paths that are never in the specification. The next lesson takes up this information-level distinction and shows that the two approaches find different kinds of defects.