---
title: 'Unit Test Framework'
source: 'https://academia.sh/en/courses/java-ecosystem/unit-test-framework'
course: 'Build, Testing and the Ecosystem'
language: en
updated: '2026-09-13T05:08:58+00:00'
license: 'CC BY-SA 4.0'
objectives:
    - 'Builds by hand a runner that calls setup, body and teardown in sequence'
    - 'Shows that the same assertion gives a different result before and after teardown, and that both are counted as run'
    - 'Actually counts the types the test class path carries beyond the production class path'
---

# Unit Test Framework

A small runner that calls setup, body and teardown calls in sequence is built by hand; the same assertion gives two different results when checked before and after teardown, and both are counted as run. When the same compiled test class runs against a class path missing the framework type, it measures that the test class path is a third, independent path.

The previous topic turned the runtime class path into a portable artifact and showed that
what goes into it is a decision. That decision was bounded there by two paths: the compile
class path and the runtime class path. This lesson makes the same decision for a third path.
While writing tests, one more class path sits at hand — the path the test sees — and this
path is not the same as the path production sees. Our question is: what does this third path
carry, and what happens when a type that never appears on the production class path is
missing?

The second half of the question is a separate topic, established in the M21/K02 Unit Testing
and Test-Driven Development course: test classification, assertion writing, test naming —
none of these are established again here. The one new question in this lesson is which class
path the types the test needs to run stand on. Before answering that question, we first need
to build by hand the framework itself, on top of which we will build a class path question —
the runner that calls setup, body and teardown in sequence.

These two questions stand closer together than they appear. A framework applies a sequence
that is never written in the source, over and over for every case; this repetition means a
series of calls that appears nowhere in a file's own source. For those calls to compile and
run, they must be written somewhere — and that place is not the class path production code
reads. Moving to the class path question without building the framework would mean measuring
an empty abstraction; this is why the lesson first writes the calls by hand, then counts which
type those calls need.

## The Sequence the Framework Establishes

- **TE1.** Setup and teardown are each modeled here as a `Runnable`; a real framework's test
  discovery (file scanning, name matching) is not built here — only the setup, body, teardown
  order and the way a failure is reported are modeled.
- **TE2.** Assertion failure is reported by throwing an `AssertionError`; the runner catches
  this and logs the case as "failed," and otherwise writes "passed." The teardown step runs
  under every condition — whether the body fails or not.

From the outside, what a test framework does amounts to making three calls in a fixed order:
setup first, then the body, then teardown last. An assertion is a comparison and reports its
failure with an exception; the framework catches this exception and marks the case as
"failed," and if it does not catch one, it writes "passed." Setup runs again for every case —
the state one case leaves behind does not leak into the next — and teardown runs whether the
case passes or fails; otherwise one case failing would corrupt the next one's setup. The
runner below builds these three calls and the assertion by hand.

```java
// Framework.java — a small runner that calls setup, body and teardown in sequence
public class Framework {

    static void runCase(String name, Runnable setup, Runnable body, Runnable teardown) {
        setup.run();
        try {
            body.run();
            System.out.println(pad(name) + "passed");
        } catch (AssertionError error) {
            System.out.println(pad(name) + "failed: " + error.getMessage());
        } finally {
            teardown.run();
        }
    }

    static void assertEqual(String field, int expected, int actual) {
        if (expected != actual) throw new AssertionError(field + " (expected " + expected + ", got " + actual + ")");
    }

    static String pad(String s) { return String.format("%-16s", s); }

    public static void main(String[] args) {
        int[] quantity = {0};
        runCase("store-filled",
                () -> quantity[0] = 10,
                () -> assertEqual("quantity", 10, quantity[0]),
                () -> quantity[0] = 0);

        int[] field = {0};
        Runnable setup = () -> field[0] = 5;
        Runnable teardown = () -> field[0] = 0;

        runCase("before-check", setup, () -> assertEqual("field", 5, field[0]), teardown);

        setup.run();
        runCase("after-check", () -> {}, () -> { teardown.run(); assertEqual("field", 5, field[0]); }, () -> {});
    }
}
```

```
store-filled    passed
before-check    passed
after-check     failed: field (expected 5, got 0)
```

The first case is a plain pass: setup makes `quantity` 10, the body reads and confirms it,
teardown resets it to zero. The real question is in the second and third case. Both carry the
same assertion — `field` must equal 5 — but one checks the assertion **before** teardown, the
other **after** teardown. Teardown resets `field` to zero; this is why the same assertion
passes in the first case and fails in the second. The runner itself does not know this
distinction — it only makes the three calls in sequence and catches whatever exception the
body throws. `before-check` and `after-check` each appear as one line in the output, and both
are counted as "run" in the runner's list; the difference between them can only be read in the
result column.

The conclusion that follows is this: the order the framework calls is not a guarantee, it is a
contract. The setup-body-teardown order always stays the same, but **when** the body reads —
the state setup left behind, or the state teardown left behind — is in the hands of whoever
writes it. The same assertion text measures two different truths when written in two
different positions, and the framework does not arbitrate between them; it only records which
one threw and which one did not. A case being counted as "run" does not mean that case was set
up **correctly** — it only means the three calls were made in order and a result was produced.
The test classification and assertion writing that M21/K02 established do not change here; the
only thing that changes is that where the assertion is written inside the body determines the
result.

The source of this distinction is the `finally` block: the teardown call runs independently of
the body's result, because the runner placed it inside `finally`, not outside the `try` block.
The teardown step therefore works the same way both in a passing case like `store-filled` and
a failing case like `after-check` — it resets `field` to zero. The difference is **whether**
the assertion runs before or after this reset, and this is not information the runner itself
holds — it is a decision made by whoever writes the body. The same holds for a real framework:
the order in which setup and teardown are called is documented and guaranteed, but how the
body uses that order — which moment, which state it reads — is not under the framework's
control.

## The Test Class Path: A Third Path

- **TE3.** The distinction between the compile class path and the runtime class path is
  carried over from the previous topic (Build Tools) and is not rebuilt here; the one question
  this lesson adds is what a third class path carries.
- **TE4.** "Type count" here is a count of compiled `.class` files. Each of the three sources
  below contains no inner class, so each produces exactly one `.class` file; the count
  therefore lines up with the number of sources, but in general the type count can differ from
  the number of source files.
- **TE5.** Running "without the framework type" means a real class loader searches the given
  class path for that type and does not find it — the loader here is a `URLClassLoader` and is
  built on top of the platform class loader, so it does not inherit anything from the
  application's own class path.

The runner above and the assertion helper inside it were part of the `Framework` class — but
in a real program, a helper of this kind stands **apart** from the code under test. The
measurement below actually compiles three separate classes: `Store` (production code),
`Assertion` (the framework itself, just an equality check the test needs) and `StoreTest` (the
test that uses both). `Store` compiles only on its own class path; `Assertion` compiles in a
separate directory; `StoreTest` compiles against both. It then runs against two separate class
paths: one contains all three, the other leaves `Assertion` out.

```java
// ThirdPath.java — the test class path really carries extra types the production class path does not
import java.io.*;
import java.lang.reflect.InvocationTargetException;
import java.net.*;
import java.nio.file.*;
import java.util.*;
import java.util.spi.ToolProvider;

public class ThirdPath {

    static Path write(Path dir, String name, String source) throws IOException {
        Files.createDirectories(dir);
        Path p = dir.resolve(name + ".java");
        Files.writeString(p, source);
        return p;
    }

    static boolean compile(Path output, List<Path> classPath, List<Path> sources) throws IOException {
        Files.createDirectories(output);
        List<String> arg = new ArrayList<>(List.of("-d", output.toString()));
        if (!classPath.isEmpty()) {
            StringJoiner sj = new StringJoiner(File.pathSeparator);
            for (Path p : classPath) sj.add(p.toString());
            arg.add("-classpath");
            arg.add(sj.toString());
        }
        for (Path s : sources) arg.add(s.toString());
        ToolProvider compiler = ToolProvider.findFirst("javac").orElseThrow();
        StringWriter silentOutput = new StringWriter();
        return compiler.run(new PrintWriter(silentOutput), new PrintWriter(silentOutput),
                arg.toArray(new String[0])) == 0;
    }

    static String run(List<Path> classPath, String mainClass) {
        URL[] url = new URL[classPath.size()];
        try {
            for (int i = 0; i < url.length; i++) url[i] = classPath.get(i).toUri().toURL();
            try (URLClassLoader loader = new URLClassLoader(url, ThirdPath.class.getClassLoader().getPlatformClassLoader())) {
                Class<?> cls = Class.forName(mainClass, true, loader);
                cls.getMethod("main", String[].class).invoke(null, (Object) new String[0]);
                return "ok";
            }
        } catch (InvocationTargetException e) {
            return e.getCause().getClass().getSimpleName();
        } catch (Throwable t) {
            return t.getClass().getSimpleName();
        }
    }

    static long typeCount(Path dir) throws IOException {
        try (var walk = Files.walk(dir)) {
            return walk.filter(p -> p.toString().endsWith(".class")).count();
        }
    }

    public static void main(String[] args) throws Exception {
        Path root = Files.createTempDirectory("thirdpath");
        Path source = root.resolve("source");
        Path prodDir = root.resolve("production"), frameworkDir = root.resolve("framework"), testDir = root.resolve("test");

        Path sStore = write(source, "Store",
                "public class Store {\n"
              + "  private int quantity;\n"
              + "  public Store(int initial) { quantity = initial; }\n"
              + "  public void add(int amount) { quantity += amount; }\n"
              + "  public void reduce(int amount) {\n"
              + "    if (amount > quantity) throw new IllegalStateException(\"insufficient quantity\");\n"
              + "    quantity -= amount;\n"
              + "  }\n"
              + "  public int quantity() { return quantity; }\n"
              + "}\n");
        Path sAssertion = write(source, "Assertion",
                "public class Assertion {\n"
              + "  public static void equal(int expected, int actual) {\n"
              + "    if (expected != actual) throw new AssertionError(\"expected \" + expected + \", got \" + actual);\n"
              + "  }\n"
              + "}\n");
        Path sStoreTest = write(source, "StoreTest",
                "public class StoreTest {\n"
              + "  public static void main(String[] a) {\n"
              + "    Store s = new Store(10);\n"
              + "    s.add(5);\n"
              + "    s.reduce(3);\n"
              + "    Assertion.equal(12, s.quantity());\n"
              + "    System.out.println(\"case passed\");\n"
              + "  }\n"
              + "}\n");

        compile(prodDir, List.of(), List.of(sStore));
        compile(frameworkDir, List.of(), List.of(sAssertion));
        compile(testDir, List.of(prodDir, frameworkDir), List.of(sStoreTest));

        System.out.println("type count on the production class path : " + typeCount(prodDir));
        System.out.println("type count on the test class path       : "
                + (typeCount(prodDir) + typeCount(frameworkDir) + typeCount(testDir)));

        System.out.println("full test class path run                 : " + run(List.of(prodDir, frameworkDir, testDir), "StoreTest"));
        System.out.println("without framework type (production+test) : " + run(List.of(prodDir, testDir), "StoreTest"));
    }
}
```

```
type count on the production class path : 1
type count on the test class path       : 3
case passed
full test class path run                 : ok
without framework type (production+test) : NoClassDefFoundError
```

The numbers confirm the claim. When `Store` compiles and runs on its own, the production class
path carries exactly one type. The full set needed to compile and run `StoreTest` is three
types: the production code, the framework itself, and the test itself. These two extra types —
`Assertion` and `StoreTest` — never appear anywhere in the production program; they do not
need to go into the distributed executable output, and if they do, it grows for no reason. The
test class path is therefore not a superset of the production class path, it is a **separate**
path: it carries types production does not need, and it also covers the types production does
carry.

The direction is one-sided. Compiling `StoreTest` requires putting `Store` and `Assertion` on
the class path, because the test names both of them by name in its source. Compiling `Store`,
on the other hand, needs neither `Assertion` nor `StoreTest` — the production code was written
with no knowledge that test types exist. The distinction the packaging topic established
applies here without change: if an executable output is produced from the production class
path, no type like `Assertion` or `StoreTest` ever enters it; if one does, that output is
bigger than it needs to be and carries more types than it should. The test class path being a
third path also means it must not mix into the production output — the two paths mixing
together is the most common form a leaked dependency takes.

Placing the three class paths side by side makes the ordering clear: the compile class path
asks for `Store` together with `Assertion` to compile `StoreTest`, the runtime class path
carries only `Store` and knows nothing of `Assertion`, and the test class path carries all
three at once. All three paths can point to the same `.class` files on the same disk; the
difference between them is not in the files' content, it is in which directories are given to
which command. A type being "required on the test class path" means that type was never
written in the production code, and arose only from the test's own need — and this distinction
is invisible without actually building the two paths separately.

In this lesson, `Assertion` and `StoreTest` depended on each other at compile time, but
`Assertion` being present **on the runtime class path** at run time was a separate requirement
from compiling — and the last line indeed separates the two. The next lesson carries this
observation one step further: when the type the test needs replaces a type the code under test
**already** uses, the same compiled test can give two different results across two different
runtime class paths.

The last two lines show this by actually running. When `StoreTest` runs on the full class
path — production, framework and test types together — the `Assertion.equal` call is found
and the case returns "ok." The same `StoreTest.class` file, run this time only with the
production and test directories, **without** the framework directory, has the loader find the
`Assertion` name nowhere, and it throws `NoClassDefFoundError`. The file itself did not change,
it was not recompiled; only the class path given to it narrowed. This is the most concrete
proof of the claim that "the test class path is a third path": the type the framework supplies
is required both for the test to **compile** and for it to **run**, but is required for
neither compiling nor running production.

## Summary

- The test framework was modeled here as a small runner that calls setup, body and teardown
  in sequence and catches assertion failure with an exception.
- The same assertion passed when checked before teardown and failed when checked after
  teardown; the runner counted both as "run," and the order itself did not guarantee
  correctness.
- The production class path carried a single type; the test class path carried the
  framework's and the test's own types in addition to that same type — three types in total.
- The same compiled `StoreTest.class`, run on a class path missing the framework type, gave
  `NoClassDefFoundError` without being recompiled; the file did not change, only the class
  path changed.
- Test classification and assertion writing were established in the M21/K02 Unit Testing and
  Test-Driven Development course and were not repeated here.

## Next Step

In this lesson, the types that entered the test class path were types we wrote by hand
ourselves, standing entirely apart from the code under test. But a dependency the test needs
can end up having to take the place of a type the production code **already** uses — a type
that carries the same name, provides the same signature, but behaves differently. The next
lesson runs the same compiled test once against a class path where the real type stands, and
again against another class path where a fake type carrying the same name stands, and counts
that the test gets two different results without being recompiled.