Mocking

In the previous lesson, the types that entered the test class path — the framework and the test itself — stood entirely apart from the code under test; production code never named them at all. But a dependency the test frequently substitutes takes the place of a type the production code already uses: it carries the same name, answers the same call, but behaves differently. This lesson runs the same compiled test against two separate implementations of such a type — one real, the other fake — and counts that the test gets two different results without ever being recompiled. This is a gap in the exact opposite direction from the one the previous lesson tested: there, it looked at what happens when a type is never present at all; here, it looks at what happens when a type is present in place of another type.

The test double family — mock object, stub, spy object, dummy object — was established in the M21/K02 Unit Testing and Test-Driven Development course and is not reintroduced here; which kind answers which question is written there. The one question this lesson adds sits on a different axis: which class path does the fake type stand on, and does it take the real type’s place at compile time or at run time?

This question sits in a different place from the test class path question the previous lesson established. There, the test class path carried two types the production class path never knew at all — the framework and the test itself. Here, the situation is reversed: the fake type takes the place of a dependency the test code already names. The test code wrote the name Supplier; which class file that name would resolve to was left to the class path. Same name, same call, two separate bodies — and what decides which body runs is not the source text, it is which directory is given to the class path in that particular run.

Two Implementations Carrying the Same Name

• TE6. The fake type here is a separate, hand-written class with the same name and a compatible signature. The proxy machinery that mocking libraries use to generate bytecode at run time is not built here — the behavior produces the same result with a hand-written class.
• TE7. The test source looks up the name Supplier only through reflection (Class.forName); its source is not bound to any implementation at compile time. This is why the test compiles once and runs unchanged across different runtime class paths.

The behavior under test is a store-fill call: a type named Supplier takes a requested quantity and returns how much of it was fulfilled. Behind this call in production code stands a real supply process — it is not modeled here, only the call itself and the number it returns matter. A test going to the real supply process every time it runs this call would be both slow and unstable; this is why the test needs another type that can take the real implementation’s place, carrying the same name and the same call shape. We write three separate implementations. The production implementation fulfills the entire requested quantity. The fake implementation fulfills one unit less on every call and keeps how many times it was called in a static field — the spy object’s call record is carried by this field here. The third implementation carries the same name but changes its first parameter’s type from int to long; this is used afterward to show what a signature mismatch does at run time. The test is bound to none of the three implementations at compile time — it only looks up the name Supplier and the fill(int) signature through reflection.

// TwoRuns.java — the same compiled test gives a different result when the run class path swaps the real type for a same-named fake type
import java.io.*;
import java.lang.reflect.InvocationTargetException;
import java.net.*;
import java.nio.file.*;
import java.util.*;
import java.util.spi.ToolProvider;

public class TwoRuns {

    static Path write(Path dir, String name, String source) throws IOException {
        Files.createDirectories(dir);
        Path p = dir.resolve(name + ".java");
        Files.writeString(p, source);
        return p;
    }

    static boolean compile(Path output, List<Path> classPath, List<Path> sources) throws IOException {
        Files.createDirectories(output);
        List<String> arg = new ArrayList<>(List.of("-d", output.toString()));
        if (!classPath.isEmpty()) {
            StringJoiner sj = new StringJoiner(File.pathSeparator);
            for (Path p : classPath) sj.add(p.toString());
            arg.add("-classpath");
            arg.add(sj.toString());
        }
        for (Path k : sources) arg.add(k.toString());
        ToolProvider compiler = ToolProvider.findFirst("javac").orElseThrow();
        StringWriter silentOutput = new StringWriter();
        return compiler.run(new PrintWriter(silentOutput), new PrintWriter(silentOutput),
                arg.toArray(new String[0])) == 0;
    }

    static String run(List<Path> classPath, String mainClass) {
        URL[] url = new URL[classPath.size()];
        try {
            for (int i = 0; i < url.length; i++) url[i] = classPath.get(i).toUri().toURL();
            try (URLClassLoader loader = new URLClassLoader(url, TwoRuns.class.getClassLoader().getPlatformClassLoader())) {
                Class<?> cls = Class.forName(mainClass, true, loader);
                cls.getMethod("main", String[].class).invoke(null, (Object) new String[0]);
                return "ok";
            }
        } catch (InvocationTargetException e) {
            return e.getCause().getClass().getSimpleName();
        } catch (Throwable t) {
            return t.getClass().getSimpleName();
        }
    }

    public static void main(String[] args) throws Exception {
        Path root = Files.createTempDirectory("tworuns");
        Path source = root.resolve("source");
        Path testDir = root.resolve("test"), prodDir = root.resolve("production"),
             fakeDir = root.resolve("fake"), brokenDir = root.resolve("broken");

        Path sTest = write(source, "SupplierTest",
                "import java.lang.reflect.Field;\n"
              + "import java.lang.reflect.Method;\n"
              + "public class SupplierTest {\n"
              + "  public static void main(String[] a) throws Exception {\n"
              + "    Class<?> c = Class.forName(\"Supplier\");\n"
              + "    Object obj = c.getDeclaredConstructor().newInstance();\n"
              + "    Method m = c.getMethod(\"fill\", int.class);\n"
              + "    int[] requests = {5, 3, 8};\n"
              + "    int total = 0;\n"
              + "    for (int request : requests) total += (int) m.invoke(obj, request);\n"
              + "    int expected = 5 + 3 + 8;\n"
              + "    System.out.println(\"expected    : \" + expected);\n"
              + "    System.out.println(\"actual      : \" + total);\n"
              + "    System.out.println(\"assertion   : \" + (total == expected));\n"
              + "    try {\n"
              + "      Field f = c.getField(\"callCount\");\n"
              + "      System.out.println(\"call count  : \" + f.getInt(null));\n"
              + "    } catch (NoSuchFieldException e) {\n"
              + "      System.out.println(\"call count  : not tracked\");\n"
              + "    }\n"
              + "  }\n"
              + "}\n");

        Path sProd = write(source.resolve("production"), "Supplier",
                "public class Supplier {\n"
              + "  public int fill(int requested) { return requested; }\n"
              + "}\n");
        Path sFake = write(source.resolve("fake"), "Supplier",
                "public class Supplier {\n"
              + "  public static int callCount = 0;\n"
              + "  public int fill(int requested) { callCount++; return requested - 1; }\n"
              + "}\n");
        Path sBroken = write(source.resolve("broken"), "Supplier",
                "public class Supplier {\n"
              + "  public int fill(long requested) { return (int) requested; }\n"
              + "}\n");

        compile(testDir, List.of(), List.of(sTest));
        compile(prodDir, List.of(), List.of(sProd));
        compile(fakeDir, List.of(), List.of(sFake));
        compile(brokenDir, List.of(), List.of(sBroken));

        System.out.println("-- with the real type --");
        System.out.println("run result: " + run(List.of(testDir, prodDir), "SupplierTest"));
        System.out.println("-- with the fake type (test not recompiled) --");
        System.out.println("run result: " + run(List.of(testDir, fakeDir), "SupplierTest"));
        System.out.println("-- with the signature-changed type --");
        System.out.println("run result: " + run(List.of(testDir, brokenDir), "SupplierTest"));
    }
}

-- with the real type --
expected    : 16
actual      : 16
assertion   : true
call count  : not tracked
run result: ok
-- with the fake type (test not recompiled) --
expected    : 16
actual      : 13
assertion   : false
call count  : 3
run result: ok
-- with the signature-changed type --
run result: NoSuchMethodException

SupplierTest compiles once (testDir) and runs three times with a different second class path component each time: first prodDir, then fakeDir, finally brokenDir. The only difference is the second element of the list given to the run call; the test’s own .class file carries exactly the same bytecode in all three.

Compiling SupplierTest needs no implementation at all — the name Supplier appears in its source only as a string constant, not as an imported type. This sharpens even further the distinction the first lesson established between the compile class path and the runtime class path: here, the test compiles needing no compile class path component at all, yet needs a different runtime class path component for each of its three separate runs. A dependency invisible at compile time becomes mandatory at run time — this is the test class path’s counterpart to the “compiled is not proof” reading the previous topic established.

Real mocking libraries usually make this swap interface-based: the code under test binds to an interface, the fake type implements that interface, and the compiler checks that the two are compatible. The path chosen here is more bare — direct reflection — because the lesson has to produce the same result without writing a third-party framework name. The result is the same with the interface-based approach too: which implementation gets used is still left to the runtime class path, and the compiler only checks that one implementation exists — it does not check how many there are or which one gets chosen.

Silent Divergence: No Error, Different Result

• TE8. The call record is kept here as a static field (callCount). Because each run uses its own URLClassLoader, this field starts from zero on every run; the counter never gets mixed up between two runs.

The first run, executed with the real type, passes as expected: the sum of the three requests is 16, the returned total is also 16, the assertion is true. The second run runs the same test file, with the same bytecode, only making the second class path component fakeDir. The fake implementation fulfills one unit less than requested on every call; the sum of the three calls therefore drops to 13 and the assertion line prints false. No exception is thrown here, there is no compile error — the run call returns “ok” in both cases. The only difference is in the content of one line: the assertion field drops from true to false. This is the course’s cleanest example of the silent defect family: both runs complete successfully, both are counted as “ran,” but the result they produce is not the same.

The call record the spy object keeps confirms this distinction. The production implementation has no field named callCount — the test catches this and writes “not tracked.” In the fake implementation, that same field carries the value 3: fill was called three times, and all three incremented the record. This number carries proof on its own, without ever looking at the test’s return value — the fake type really was called three times, no fewer and no more. The assertion field says what was returned, callCount says how many times it was called; the two answer separate questions, and both can be read together in the same run.

The difference these two runs produce shows what a test suite saying “passed” does not guarantee. The suite run with the real implementation passes, the suite run with the fake implementation fails — both carry the same test file, the same assertion. If a delivery pipeline runs its tests only on a class path where the fake implementation stands, it never sees a regression in the production implementation; the tests keep passing because none of them ever ran with the real type. This is the most direct counterpart to the course’s measuring question in the test topic: a test “passing” cannot be read without looking at which class path it ran on.

When the Signature Diverges, the Defect Falls to Run Time

• TE9. The signature comparison here looks only at the first parameter’s type (getMethod("fill", int.class)); a real reflection lookup matches the name, the parameter count and the parameter types together, and does not look at the return type.
• TE10. Mocking libraries’ verification (verify) call is not built here; the signal the spy object produces in this lesson is only a counter that gets read, not a self-verifying object.

The third run runs the same SupplierTest.class, this time on a class path where an implementation stands whose fill‘s first parameter is long, not int. Class.forName ("Supplier") finds the class — the name is the same, the class really exists — but the call c.getMethod("fill", int.class) cannot find a method with exactly this signature, and throws NoSuchMethodException. This exception exits the test’s own main uncaught, and the run helper catches it and returns its name: run result: NoSuchMethodException.

The distinction here falls into a different defect class than the previous two runs. The run with the fake type was silent — both sides produced a valid result, the difference carried only in a number. The run with the diverged signature, on the other hand, fails at run time: the program stops with an exception. But it is worth noting what this exception actually says — the NoSuchMethodException message says which class was searched for which method, but it does not say why that method is no longer there. The test text itself did not change at all; the cause of the defect is visible only in the changed implementation’s source, not in the test file. When a test depends this heavily on which type stands on the runtime class path, the signature mismatch is caught not at compile time but at run time, and all that is left in hand at that moment is the name of an exception class.

The order of these three runs is deliberate: the real type, then a compatible fake type, and finally an incompatible type. The difference between them is how closely the fake type resembles the real type. Because the compatible fake type’s signature matches the real type’s signature exactly, it can take its place with the test never knowing — the defect is carried only by a number in the result. The incompatible type, even though its name is the same, has a diverged signature, so the test’s reflection call cannot find it at all — the defect is no longer a number, it is an exception. The difference between these two extremes shows the real risk of mocking: keeping a fake type current requires keeping not just its name but also its signature matching the real type, and this match is not checked by the compiler, because the two are never present together in the same compilation.

All three runs together sample the same three defect classes: the run with the real type is trouble-free, the run with the fake type silently gives a different result, and the run with the diverged signature fails with an exception. None of them was caught during compilation — all three were a decision invisible on the compile class path surfacing on the runtime class path. A test suite’s reliability therefore depends not only on which assertions it writes, but also on which class path those assertions run against.

Summary

• The test double family was established in M21/K02; the one question this lesson adds is which class path the fake type stands on.
• The same compiled test ran without being recompiled against real and fake implementations carrying the same name and gave two different results; both completed without error.
• The call counter the fake implementation kept carried the spy object’s signal through a field the production implementation does not have.
• A run with an implementation whose signature had changed gave NoSuchMethodException; this exception failed at run time and its cause did not appear in the test text.
• Two types carrying the same name can take each other’s place only as long as their signatures match; once the signature diverges, the defect surfaces at run time, not at compile time.

Next Step

In this lesson, a single test body ran against three separate class paths, but the same three numbers — 5, 3, 8 — were used in every run. Multiplying a test body across several data sets opens a separate question: same body, same assertion, different inputs — how many separate runs, how many separate failures, and how does the record capture which data row a failure came from? The next lesson counts this by running a single body repeatedly over a data set.