---
title: 'Functional Test Types'
source: 'https://academia.sh/en/courses/testing-fundamentals/functional-test-types'
course: 'Quality and Testing Fundamentals'
language: en
updated: '2026-08-23T14:25:18+00:00'
license: 'CC BY-SA 4.0'
---

# Functional Test Types

Separating smoke, sanity, regression, confirmation, and acceptance tests by purpose, catching different defects in the same release, and the maintenance cost of the regression set.

The level and the information level determine where a test sits, not its purpose. The same
black-box unit test can be run to see whether the release is standing, or to check whether
a fixed defect has come back. The test is the same in both cases; what changes is why it is
run and what is done with its result.

This lesson names tests by their purpose. The function of the names is to build a shared
understanding of when a set is run and what happens when it fails.

## Purpose Names the Test

**Smoke test** is wide and shallow: it tries every main path once, testing none of them
deeply. Its question is one sentence — is this release worth working on? The decision when
it fails is also single: the release is rejected, and no detailed testing follows.

**Sanity test** is narrow and deep: it probes the region a specific change touches, in a
few minutes. Its question is — does this fix look reasonable? It is a pre-filter done
before running the full set.

**Regression test** checks whether behavior that used to work has broken. Its set
accumulates over time: every fixed defect leaves behind a sentinel that will warn if it
comes back.

**Confirmation test** checks that a single defect reported as fixed is actually fixed. It
is a step-by-step repeat of the defect report, and its coverage is exactly that report.

**Acceptance test** is the condition of delivery, and its criterion comes from the party
who will use the product; this is the place defined for it in the previous two lessons.

## Smoke and Regression in the Same Release

Three versions of the library are set up: a sound version, a version with a core operation
broken, and a version where a previously fixed defect has come back.

```js
// version.mjs — the library's sound version and two broken versions
export function baseLibrary() {
  return {
    findMember: (no) => (no >= 0 ? { no, type: no % 3 === 0 ? 'student' : 'member' } : null),
    lateFee: (days) => Math.min(Math.max(days - 3, 0) * 2, 20),
    lend: (member, book) => ({ memberNo: member.no, book, status: 'lent' }),
  };
}

export const VERSIONS = {
  'S0 sound': baseLibrary(),
  'S1 core broken': { ...baseLibrary(), lend: () => { throw new Error('record could not be opened'); } },
  'S2 old defect returned': { ...baseLibrary(), lateFee: (days) => Math.max(days - 3, 0) * 2 },
};
```

The smoke set tries the three main paths once each. The regression set is the sentinel of
three defect reports; each check's name carries the number of the report it comes from.

```js
// types.mjs — comparing the smoke and regression sets across three versions
import assert from 'node:assert/strict';
import { VERSIONS } from './version.mjs';

const SMOKE = {
  'a member record opens': (k) => assert.ok(k.findMember(41)),
  'a loan is issued': (k) => assert.equal(k.lend(k.findMember(41), 'K-1').status, 'lent'),
  'a fee is computed': (k) => assert.equal(typeof k.lateFee(5), 'number'),
};

const REGRESSION = {
  'H-14 fee does not exceed the upper limit': (k) => assert.equal(k.lateFee(100), 20),
  'H-22 grace period is three days': (k) => assert.equal(k.lateFee(3), 0),
  'H-31 an invalid member number returns null': (k) => assert.equal(k.findMember(-1), null),
};

function run(set, library) {
  const failed = [];
  for (const [name, check] of Object.entries(set)) {
    try { check(library); } catch { failed.push(name); }
  }
  return failed;
}

for (const [versionName, library] of Object.entries(VERSIONS)) {
  const line = [];
  for (const [setName, set] of [['smoke', SMOKE], ['regression', REGRESSION]]) {
    const failed = run(set, library);
    line.push(`${setName} ${Object.keys(set).length - failed.length}/${Object.keys(set).length}${failed.length ? ` (${failed[0]})` : ''}`);
  }
  console.log(`${versionName.padEnd(24)}: ${line.join(' | ')}`);
}
```

```
S0 sound                : smoke 3/3 | regression 3/3
S1 core broken          : smoke 2/3 (a loan is issued) | regression 3/3
S2 old defect returned  : smoke 3/3 | regression 2/3 (H-14 fee does not exceed the upper limit)
```

Two defects were caught by two separate sets. When the core operation breaks, the
regression set stays quiet, because its sentinels sit in the region of past defects; lending
a book never produced a defect there, so there is no sentinel for it. When the old defect
comes back, the smoke set stays quiet, because it only asked whether the fee computation
returned a number.

Both sets are correct in purpose, and both are incomplete. This incompleteness is not a
defect; it follows from the definition: the shallow set does not know depth, the narrow set
does not know breadth.

## Confirmation Testing Alone Is Not Enough

A fix is made for the fee defect. The person writing the fix restores the upper limit, but
drops the grace period.

```js
// sanity.mjs — the outcome of confirmation, sanity, and regression sets on the same fix
import assert from 'node:assert/strict';
import { baseLibrary } from './version.mjs';

const fixed = { ...baseLibrary(), lateFee: (days) => Math.min(days * 2, 20) };

const CONFIRMATION_TEST = {
  'H-14 the state in the report': () => assert.equal(fixed.lateFee(100), 20),
};

const SANITY = {
  'fee at 0 days is 0': () => assert.equal(fixed.lateFee(0), 0),
  'fee at 3 days is 0': () => assert.equal(fixed.lateFee(3), 0),
  'fee at 4 days is 2': () => assert.equal(fixed.lateFee(4), 2),
  'fee at 13 days is 20': () => assert.equal(fixed.lateFee(13), 20),
  'fee at 14 days is 20': () => assert.equal(fixed.lateFee(14), 20),
};

const REGRESSION = {
  'H-14 fee does not exceed the upper limit': () => assert.equal(fixed.lateFee(100), 20),
  'H-22 grace period is three days': () => assert.equal(fixed.lateFee(3), 0),
  'H-31 an invalid member number returns null': () => assert.equal(fixed.findMember(-1), null),
};

for (const [name, set] of [['confirmation', CONFIRMATION_TEST], ['sanity test ', SANITY], ['regression  ', REGRESSION]]) {
  const failed = Object.entries(set).filter(([, d]) => { try { d(); return false; } catch { return true; } });
  console.log(`${name}: ${Object.keys(set).length - failed.length}/${Object.keys(set).length} passed${failed.length ? ` — first failed: ${failed[0][0]}` : ''}`);
}
```

```
confirmation: 1/1 passed
sanity test : 3/5 passed — first failed: fee at 3 days is 0
regression  : 2/3 passed — first failed: H-22 grace period is three days
```

Confirmation testing passed: the state in the defect report now gives the correct result.
The report could be closed. The sanity test, however, showed a second breakage in the same
region within a few seconds — the fix damaged its own surroundings.

Two rules follow from this. When a defect is fixed, not only the state in the report but
also the boundary values of that rule are tested. And confirmation testing passing is not
enough to close the report; the region of the fixed defect is also probed.

## The Maintenance Cost of the Sets

The regression set grows in one direction only: every new defect adds a sentinel, and none
of them leaves on its own. Over time, the set becomes slow enough to push the feedback
point back.

There are three levers. **Regression test selection** runs only the checks that touch the
changed region; the full set is left for a point further back. **Pruning** removes the
sentinel for a rule that no longer has a counterpart — no rule, no sentinel. Pruning
**flaky tests** protects trust as well: a check that fails for unrelated reasons turns,
when it fails, into noise nobody looks at, and it lowers the value of the whole set.

The maintenance of the smoke set runs the opposite way: it must stay small. As a smoke set
grows, it stops being able to do its own job, because giving a fast answer is its only job.

## Summary

- Test types are separated by purpose: smoke is wide and shallow, sanity is narrow and
  deep, regression is the sentinel of past defects, confirmation testing is the repeat of a
  single report.
- In the example, only the smoke set caught the core breakage, and only the regression set
  caught the returning defect; the shortfall of both sets follows from their definitions.
- Confirmation testing passing shows the defect was fixed, not that the fix left its
  surroundings undamaged; in the example, the sanity test surfaced the second breakage.
- The regression set grows in one direction; selective runs, pruning checks with no
  remaining counterpart, and removing flaky tests are all parts of its maintenance.
- The maintenance of the smoke set is staying small; once it grows, it loses its function
  of giving a fast answer.

## Next Step

So far, the inputs to the checks were chosen as examples: 3 days, 4 days, 100 days. The
reason for the choice was intuition. There are systematic ways to select a finite subset
from an infinite input set, and these ways justify which inputs are tried. The next lesson
establishes these techniques: grouping inputs that behave the same into classes, probing
the edges of the classes, and counting condition combinations exhaustively.