---
title: 'Testing Frameworks'
source: 'https://academia.sh/en/courses/c-systems-programming/testing-frameworks'
course: 'Systems Programming with C'
language: en
updated: '2026-09-26T03:02:44+00:00'
license: 'CC BY-SA 4.0'
objectives:
    - 'Builds a test table, assertion macro, and pass/fail count in C'
    - "Measures why process isolation keeps one test's change from reaching the next test"
    - 'Shows that process isolation cuts off memory state but not filesystem state'
---

# Testing Frameworks

Measures why a change made inside a forked child running a test never leaks into the next test, and why that isolation stops short of covering filesystem state.

Being correctly bound to a library doesn't show that the code using it works correctly.
Testing this itself carries a visibility problem: if a test sees the state left behind by
another test, you can no longer tell which test actually failed. A small test framework solves
this problem by running each test in its own copy.

## The Test Table and Isolated Running

The core of a test framework is a table of function pointers, a single assertion macro, and a
pass/fail count. When an assertion fails, the test function returns 0; if it finishes with
nothing left to assert, it returns 1.

```c
// framework.h — declaration of the test table, assertion macro, and shared state
#ifndef FRAMEWORK_H
#define FRAMEWORK_H

typedef int (*TestFunction)(void);

struct TestEntry {
    const char *name;
    TestFunction run;
};

#define ASSERT(condition) do { if (!(condition)) return 0; } while (0)

extern int shared_state;            /* to observe leakage between tests */
extern int mock_call_count;         /* how many times the mock dependency was called */
extern int fixture_setup_count;     /* how many times the fixture was set up before a test */
extern int fixture_teardown_count;  /* how many times the fixture was torn down after a test */

int test_set_state(void);
int test_state_is_zero(void);
int test_see_fixture_value(void);
int test_write_to_file(void);
int test_read_from_file(void);
int mock_dependency(int input);
int stub_dependency(int input);
int do_real_work(int input, int (*dependency)(int));
int fixture_setup(void);
int fixture_teardown(void);

#endif
```

```c
// framework.c — test bodies, mock dependency, and two file-based tests
#include <stdio.h>
#include "framework.h"

int shared_state = 0;
int mock_call_count = 0;

int test_set_state(void) {
    shared_state = 9;
    return 1;
}

int test_state_is_zero(void) {
    ASSERT(shared_state == 0);
    return 1;
}

int test_see_fixture_value(void) {
    ASSERT(shared_state == 5);   /* fixture_setup must have brought this to 5 */
    return 1;
}

int test_write_to_file(void) {
    FILE *f = fopen("trace.dat", "w");
    if (!f) return 0;
    fputc('1', f);
    fclose(f);
    return 1;
}

int test_read_from_file(void) {
    FILE *f = fopen("trace.dat", "r");
    if (!f) return 0;
    int value = fgetc(f);
    fclose(f);
    return value == '1';
}

int mock_dependency(int input) {
    mock_call_count++;
    return input * 2;
}

int fixture_setup_count = 0;
int fixture_teardown_count = 0;

int stub_dependency(int input) {
    (void)input;              /* the stub's answer is fixed, it never reads the input at all */
    return 42;                 /* a predetermined answer that never changes */
}

int do_real_work(int input, int (*dependency)(int)) {
    return dependency(input) + 1;
}

int fixture_setup(void) {
    fixture_setup_count++;
    shared_state = 5;    /* the known starting state the fixture sets up */
    return 1;
}

int fixture_teardown(void) {
    fixture_teardown_count++;
    shared_state = 0;    /* leave the state clean for the next test */
    return 1;
}
```

**PI5.** The test table is shown with only two entries; the mechanism generalizes to any
number of entries, because running them in order is nothing more than a loop over the table.

The helper that runs each test in its own child uses the forking contract set up in the
previous module: the buffer is flushed, the child runs the test and reports the result through
its exit status, and the parent only reads that status.

**PI6.** The fixture is shown here with a single pair of functions — one setup, one teardown;
in a real framework the fixture can vary from test to test, but the requirement of being a pair
(one teardown for every setup) doesn't change.

If there is common preparation that has to run before and after the test itself, pulling that
preparation out into a separate **fixture** (a setup/teardown pair) instead of writing it into
the body of every test one by one keeps the same preparation from being written over and over,
and, more importantly, guarantees that the teardown always runs regardless of the test's
result. A fixture's contract is this: the setup runs exactly once before the test, the teardown
runs exactly once after the test — whether the test passes or fails, even if the test returns
early at an assertion. The `fixture_setup`/`fixture_teardown` pair below models this contract:
`fixture_setup` brings `shared_state` to a known starting value (5) and increments the setup
count; `fixture_teardown` resets the state and increments the teardown count. These two counts
are the way to verify that the fixture is really set up once and torn down once for every test.

A stub serves a different purpose than a mock, and this lesson keeps the two apart. A **stub**
(`stub_dependency`) returns a fixed, predetermined answer no matter what input it's given, and
never records that it was called at all — its only purpose is to stand in for the real
dependency so the rest of the test can run. A **mock** (`mock_dependency`), on top of that,
makes its own calling pattern observable; in this lesson it does that with a call count. The
distinction comes down to this: a stub prepares the test's **input**, a mock becomes part of
the test's **verification**. If a test only wants to see that `do_real_work` produces the right
number, a stub is enough and simpler; if the test also wants to verify how many times the
dependency was called, a mock is needed. Mixing the two up and using a mock everywhere piles an
unnecessary observation burden onto tests that never ask about call counts; using a stub
everywhere leaves blind the tests that need to verify calling behavior.

```c
// measurement.c — isolated running, unisolated leakage, and the filesystem boundary in one run
#define _POSIX_C_SOURCE 200809L
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include "framework.h"

static int run_isolated(TestFunction func) {
    fflush(stdout);   /* otherwise the child would also carry the parent's unflushed buffer */
    pid_t pid = fork();
    if (pid == 0) {
        _exit(func() ? 0 : 1);
    }
    int status = 0;
    waitpid(pid, &status, 0);
    return WIFEXITED(status) && WEXITSTATUS(status) == 0;
}

/* run_with_fixture: setup and teardown in the parent, the test itself in an isolated child */
static int run_with_fixture(TestFunction func) {
    fixture_setup();
    int result = run_isolated(func);
    fixture_teardown();
    return result;
}

int main(void) {
    printf("isolated set state: %s\n",
           run_isolated(test_set_state) ? "passed" : "failed");
    printf("isolated state is zero: %s\n",
           run_isolated(test_state_is_zero) ? "passed" : "failed");

    /* the same two tests, this time called directly in the same process, unisolated */
    test_set_state();
    printf("unisolated state is zero: %s\n",
           test_state_is_zero() ? "passed" : "failed");

    int result = do_real_work(3, mock_dependency);
    printf("mock dependency call count: %d\n", mock_call_count);
    printf("do_real_work result: %d\n", result);

    int stub_result = do_real_work(3, stub_dependency);
    printf("do_real_work result with stub: %d\n", stub_result);

    printf("fixtured see-value test: %s\n",
           run_with_fixture(test_see_fixture_value) ? "passed" : "failed");
    printf("fixture setup count: %d\n", fixture_setup_count);
    printf("fixture teardown count: %d\n", fixture_teardown_count);
    printf("shared_state in parent after fixture: %d\n", shared_state);

    printf("isolated write to file: %s\n",
           run_isolated(test_write_to_file) ? "passed" : "failed");
    printf("isolated read from file: %s\n",
           run_isolated(test_read_from_file) ? "passed" : "failed");
    return 0;
}
```

```text
isolated set state: passed
isolated state is zero: passed
unisolated state is zero: failed
mock dependency call count: 1
do_real_work result: 7
do_real_work result with stub: 43
fixtured see-value test: passed
fixture setup count: 1
fixture teardown count: 1
shared_state in parent after fixture: 0
isolated write to file: passed
isolated read from file: passed
```

The first two lines show two tests that use the same `shared_state` variable running in
separate children: `test_set_state` sets `shared_state` to 9 in its own child, but when that
child exits, the change disappears along with it. When `test_state_is_zero` runs in its own
child, it inherits a copy of the parent's `shared_state`, still 0, and the assertion passes.
The third line runs the same two tests unisolated: both run in a single process, on the real
`shared_state`, and the 9 left behind by the first directly affects the second, so the
assertion fails — this is exactly the leakage that process isolation prevents.

The dangerous side of this leak is that the result depends on the **order** in the test file.
If the same two tests had been written with `test_state_is_zero` running first, the leak would
never show up, and both tests would pass even in their unisolated form — because
`shared_state` wouldn't have been pulled to 9 yet. This is the most deceptive property of an
unisolated test suite: the suite looks completely green in one order, and turns red in the same
suite once the tests are reordered or a new test steps in between, even though no test's own
logic has changed. The source of the fault is not in any single test but in the hidden
dependency between them — and that dependency stays invisible to code review, because the two
tests never call each other, they only share the same variable. The real payoff of process
isolation shows up here: because every test runs in its own child, the result stays the same
no matter what order the tests are written in — order stops being a variable that affects the
correctness of a test.

`mock_dependency` is a function pointer that stands in for the real dependency; `do_real_work`
calls it without changing its own logic, and the call count is measured through
`mock_call_count`. This count, just like `shared_state`, lives in a single process's memory —
if a test read it inside a forked child, the count that child sees would also be its own copy
and wouldn't make it back to the parent; the only thing that returns to the parent is whether
the test passed.

What the mock doesn't change here is `do_real_work`'s own logic: the function still calls the
pointer it's given, only the body that pointer points to has been swapped from the real
dependency for a body that keeps a count. This splits what the test measures in two — the
correctness of `do_real_work`'s own arithmetic on one side, and how many times and in what
order the dependency it calls was invoked on the other. The second question can't be tested
with the real dependency, because there's no way to observe the real dependency's own call
count from outside; the mock is the only tool that makes this observation possible.

The result obtained with `stub_dependency` is 43 — the 1 that `do_real_work` adds on top of
the stub, which returns a fixed 42 no matter the input. This line says nothing about the call
count, because the stub doesn't count anything to begin with; it only shows that
`do_real_work`'s own arithmetic correctly processes the value the dependency returns. Compared
against the result of 7 obtained with the mock, the two tests test the same function for two
different questions: one, "does it produce the right answer", the other, "does it call the
dependency the right number of times".

**PI7.** The `fixture_setup`/`fixture_teardown` pair here runs in the parent process,
immediately before and immediately after the call to `run_isolated`; the test itself still
runs in its own child, the fixture is not moved into the child.

The `run_with_fixture` line confirms the fixture's contract with three numbers. The setup
count is 1, the teardown count is 1 — that is, both ran exactly once, no more and no less. The
test itself passed the `shared_state == 5` assertion, because `fixture_setup` set this value in
the parent **before** the test's child, and the child inherited this value through the fork.
The last line shows that `shared_state` in the main process is back to 0 after
`fixture_teardown` runs — the fixture's teardown happens in the main process after the test's
child has already exited, because the setup and teardown never fork, they run directly in the
parent. This shows why the fixture has to stay **outside** the isolated run: the fixture is
common preparation repeated before and after every test, it doesn't need to be moved into a
separate child each time the way the test itself does — if it were moved, the effect of the
setup (shared_state being 5) could never reach the test's child at all, because every child
inherits only the state at the moment of its own fork.

## The Bounding Measurement

The last two lines show where process isolation stops. `test_write_to_file` writes a value to
the `trace.dat` file in its own child, and the child exits; the memory state disappears along
with it. But `test_read_from_file`, even though it runs in an entirely separate child, can open
the same file and read the value the previous test wrote, and the assertion passes. Forking
only copies memory; the filesystem is an object that lives outside the process and isn't part
of the area fork isolates. If two tests touch the same file, they can affect each other even
though there is isolation between them at the memory level.

This boundary isn't limited to files; every shared resource that lives outside the process
carries the same problem. A network socket, a lock file, an environment variable kept by the
operating system itself, or a shared-memory region — none of these are inside the memory area
fork copies, so none of them get automatically isolated by forking. Knowing this distinction
while writing a test suite has a practical consequence: running tests in separate processes
doesn't **guarantee** that the tests are independent of each other — it only guarantees
independence at the memory level. Tests that touch a resource outside the process need an
extra precaution: either the resource itself is reset before every test, or the tests are
designed to never share the resource at all — for instance, by having every test generate its
own temporary file name.

Another side of the filesystem boundary is whether the test gets **cleaned up**. The
`trace.dat` file stays on disk after this run; on the next run, `test_write_to_file` writes
over a file that already exists, which doesn't change the result in this example because the
function reopens and rewrites the file every time, but if a different test used the same name
for another purpose, the content left over from the old run would be the first value the new
test reads. Memory doesn't have this problem, because every run starts its own process from
scratch and the previous run's memory is reclaimed by the operating system; the filesystem, on
the other hand, keeps sitting there unchanged even after the process has ended. This is why the
contract for tests that use files is one step broader than for tests that use memory: a clean
starting point has to be guaranteed not only between tests but also **between runs** — the
fixture can step in here too, by adding deleting the file to the teardown step.

This extended fixture could have been built by having `fixture_teardown` not only reset
`shared_state` but also call `remove("trace.dat")`; this step wasn't added here, because the
goal was to show that the filesystem falls outside fork, not how file cleanup should be done.
The general rule is this: a fixture only tears down the resource it itself set up — if
`fixture_setup` touches `shared_state`, then `fixture_teardown` only reverts that; if there
were another fixture that created a file, its teardown would likewise be limited to deleting
its own file. Changing, at the teardown stage, a resource the setup never touched widens the
fixture's contract and makes it harder to track which test is responsible for which resource.

## Summary

- A test framework consists of a table of function pointers, an assertion macro, and a
  pass/fail count; each entry is a row in the table.
- Running each test by forking it into a separate child keeps the change the test leaves in
  memory from reaching the next test — the class this lesson measures is **copy**.
- When the same tests run unisolated, in a single process, the first one's change leaks into
  the second, and the run order can change the result.
- Turning a dependency into a mock through a function pointer measures the call count without
  changing the real dependency; a stub, without counting calls, prepares the test's input by
  returning a fixed answer — the two serve different purposes.
- A fixture separates setup and teardown from the test itself and guarantees the teardown
  always runs regardless of the test's result; it runs in the parent process, it isn't moved
  into the test's child.
- Process isolation cuts off memory, not the filesystem; two tests touching the same file
  aren't separated from each other by forking. This boundary isn't limited to files — every
  resource outside the process, such as a socket, a lock, or an environment variable, is
  equally outside isolation.

## Next Step

When a test fails, you need to see what deviated from expectation, and at which step. The next
lesson looks at how stopping a running program and stepping through it makes the intermediate
values in its memory visible.