Testing

The previous lesson measured the trace an error leaves in the tree. Debugging is finding out afterward that something went wrong; testing is writing down in advance what should go right and checking it at every change. This lesson writes the expected result of an update as a list of records and compares it with the actual list.

The course’s question: what does this update do to the node in the view tree? The answers are kept, updated, mounted, and unmounted; the sum of everything other than kept is the commands. The rule is in the core file.

// fate.mjs — classifies what an update does to each node of the view tree
export const FATES = ["kept", "updated", "mounted", "unmounted"];

export function el(type, props = {}, children = [], key = null) {
  return { type, props, children, key };
}

function segment(node) {
  return node.key === null ? node.type : `${node.type}[${node.key}]`;
}

function identity(node, index) {
  return node.key === null ? `#${index}` : `key:${node.key}`;
}

function sameProps(a, b) {
  const names = [...new Set([...Object.keys(a), ...Object.keys(b)])].sort();
  return names.every((n) => JSON.stringify(a[n]) === JSON.stringify(b[n]));
}

function whole(node, path, fate, ops) {
  const here = `${path}/${segment(node)}`;
  ops.push({ fate, path: here });
  node.children.forEach((child) => whole(child, here, fate, ops));
}

export function reconcile(prev, next, path = "", ops = []) {
  if (prev === null) {
    whole(next, path, "mounted", ops);
    return ops;
  }
  if (next === null) {
    whole(prev, path, "unmounted", ops);
    return ops;
  }
  if (prev.type !== next.type || prev.key !== next.key) {
    whole(prev, path, "unmounted", ops);
    whole(next, path, "mounted", ops);
    return ops;
  }
  const here = `${path}/${segment(next)}`;
  ops.push({ fate: sameProps(prev.props, next.props) ? "kept" : "updated", path: here });
  const before = new Map(prev.children.map((child, i) => [identity(child, i), child]));
  const matched = new Set();
  next.children.forEach((child, i) => {
    const id = identity(child, i);
    matched.add(id);
    reconcile(before.has(id) ? before.get(id) : null, child, here, ops);
  });
  prev.children.forEach((child, i) => {
    if (!matched.has(identity(child, i))) reconcile(child, null, here, ops);
  });
  return ops;
}

export function printOps(ops) {
  ops.forEach((op) => console.log(`${op.fate}: ${op.path}`));
}

export function countFates(ops) {
  for (const name of FATES) {
    console.log(`${name}: ${ops.filter((op) => op.fate === name).length}`);
  }
  console.log(`commands: ${ops.filter((op) => op.fate !== "kept").length}`);
}

The model to be tested is a counter screen: a text and a button. Pressing the button increments the counter by one, and the text shows the new number. renderCounter(count) builds the screen. The test helper check takes a name, the expected records, and the old and new trees. The expected records are the fates other than kept, written in the form fate: path; check converts the actual records to the same form, compares them, and writes PASS or FAIL.

// counter.mjs — turns a counter screen into a view tree
import { el } from "./fate.mjs";

export function renderCounter(count) {
  return el("Counter", {}, [
    el("Text", { value: count }),
    el("Button", { label: "Increase" }),
  ]);
}

// check.mjs — compares the expected list of records with the actual list of records
import { reconcile } from "./fate.mjs";

export function check(name, expected, before, after) {
  const actual = reconcile(before, after)
    .filter((op) => op.fate !== "kept")
    .map((op) => `${op.fate}: ${op.path}`);
  const passed = JSON.stringify(actual) === JSON.stringify(expected);
  console.log(`${passed ? "PASS" : "FAIL"}: ${name}`);
  if (!passed) {
    console.log(`expected: ${expected.join(" | ") || "empty"}`);
    console.log(`actual: ${actual.join(" | ") || "empty"}`);
  }
}

renderCounter produces a tree from the state. check discards the kept records, because the question of an update is what changed; counting what did not change makes the test brittle. For this reason, the expected list writes only what changed.

Pressing the Button

In the first test, the user presses the button. The counter goes from zero to one. Expected: only the text node is updated, and the button and the root stay kept.

// main.mjs
import { renderCounter } from "./counter.mjs";
import { check } from "./check.mjs";

check("pressing the button updates the text", ["updated: /Counter/Text"], renderCounter(0), renderCounter(1));

PASS: pressing the button updates the text

The test passed. The actual update gave a single record: /Counter/Text updated. Because the button’s label did not change, the button stayed kept and did not enter the expected list.

Unchanged Update

In the second test, nothing changes: the tree is built again with the same counter value. The expected list is empty. A re-render is not a command; this test fixes that as a rule.

// main.mjs
import { renderCounter } from "./counter.mjs";
import { check } from "./check.mjs";

check("same value gives no command", [], renderCounter(3), renderCounter(3));

PASS: same value gives no command

An empty list is also an expectation. The test fixes that an unnecessary update sends no command to the tree; if someone later adds to the counter a prop that changes at every render, this test fails.

Failing Test

In the third test, the expectation is wrong: the author thought the button was updated too. The actual update changes only the text. The test fails and writes the two lists one under the other.

// main.mjs
import { renderCounter } from "./counter.mjs";
import { check } from "./check.mjs";

const expected = ["updated: /Counter/Text", "updated: /Counter/Button"];
check("wrong expectation", expected, renderCounter(0), renderCounter(1));

FAIL: wrong expectation
expected: updated: /Counter/Text | updated: /Counter/Button
actual: updated: /Counter/Text

The output of the failing test says where the error is: there is one extra record in the expected list. This is a finding in both directions. Either the expectation is wrong, or the application kept the button when it should have updated it. The test does not decide which; it only shows the difference.

Expectation When a Node Changes

So far only props have changed. A real screen may put a warning in place of the button when a limit is reached. In this case the test’s expectation moves to another kind of fate: the button is unmounted, the warning is mounted. In the measurement below, the two trees are built directly with el, and the expected list carries three records.

// main.mjs
import { el } from "./fate.mjs";
import { check } from "./check.mjs";

const before = el("Counter", {}, [el("Text", { value: 9 }), el("Button", { label: "Increase" })]);
const after = el("Counter", {}, [el("Text", { value: 10 }), el("Limit", { text: "Limit reached" })]);
const expected = ["updated: /Counter/Text", "unmounted: /Counter/Button", "mounted: /Counter/Limit"];
check("at the limit the button gives way to a warning", expected, before, after);

PASS: at the limit the button gives way to a warning

The test passed, and the order of the three records matched the expected one. Two things are seen here. The first is that, even though the expected list writes only what changed, an update can have more than one fate: the text was updated, the button was unmounted, the warning was mounted. The second is that check compares the lists in order. If the same three records had been written in a different order, the test would have failed even though the application was correct. The order of the records is a consequence of the rule; the person writing the test learns the order from the rule and does not guess it.

This measurement is a counterexample: the earlier tests had a single record each, and could have given the impression that “a test checks one record”. A test checks a list of records; the list can be as long as one record or as long as three.

Common Misconceptions

The first misconception is that a passing test proves the application is correct. A passing test says only that the written expectation holds. If the expectation is incomplete or wrong, the application stays wrong and the test still passes. In the third test above, the situation was reversed: the expectation was wrong, and the test blamed the expectation, not the application.

The second misconception is that writing everything makes a test sturdier. This is why check discards the kept records. A test that writes the fate of every node in the tree into the list fails the moment a new node is added to the screen, even though the added node may have nothing to do with the behavior being checked. A sturdy test writes the records that the behavior it checks leaves, and stays silent about the rest.

The third misconception is that a tree test also verifies the native side. A tree test checks what the JavaScript side leaves in the tree; it does not check what appears on the screen. This distinction becomes concrete in the lesson’s bounding measurement.

Component Test and End-to-End Test

The tests above look at the tree: a component test builds a screen, triggers an event, and checks how the tree changed. It runs fast and needs no real phone, because there are no native views; there is only the tree. The snippet below shows the source form of such a test; the model does not run it.

// Platform code: Counter test
test("pressing the button updates the text", () => {
  render(<Counter />);
  fireEvent.press(screen.getByText("Increase"));
  expect(screen.getByText("1")).toBeTruthy();
});

An end-to-end test, on the other hand, opens the real application on a real screen, really touches the button, and looks at the screen. It also sees what does not show up in the tree: the pixels drawn by the native side, the keyboard opening, a permission dialog. It is slow and brittle, so there are few of them. A screen is described in prose; the model looks at the tree, and a screen is not a tree.

Reading the Source Snippet Line by Line

The component test’s source snippet is five lines, and each line has a counterpart in the model.

• render(<Counter />) builds the screen into a test tree. In the model, this corresponds to producing the first tree with renderCounter(0). No real native view is mounted.
• fireEvent.press(screen.getByText("Increase")) first finds in the tree the button whose label is “Increase”, then sends it a press event. There are no events in the model; the result of the press is represented by the call renderCounter(1).
• expect(screen.getByText("1")).toBeTruthy() says that, after the event, a node carrying the text “1” is found in the tree. Its counterpart in the model is the /Counter/Text node coming out updated.

The difference between them is important. The source snippet checks the result by content: is the text “1” in the tree? The model checks by fate: was the text node updated? The query does not tell whether the node was updated in place or unmounted and mounted again; both leave “1” in the tree. A test that wants to check the fate should save the node’s identity in addition to the query. The rule side of this lesson makes that distinction visible.

The Name of a Test

The name of a test is a short sentence and says what is being tested: “pressing the button updates the text” names a behavior, “test 1” does not. A good name lets the reader understand what broke without opening the code when the test fails. For this reason, the failing test in this lesson carries the name “wrong expectation”: the name says from the start that the error is in the expectation, not in the application. The name also honestly limits what the test does not test: a test that says “updates the text” says nothing about the button’s label, and another test takes that on.

How to Write an Expectation

There is an order to writing the expected list, and the order protects against copying the list from the application. First, what the update promises the user is written in prose: “when the button is pressed, the number goes up by one.” Then it is thought through which nodes this promise should touch in the tree: the node showing the number changes, the button and the root do not. Only after that is the list written and check run.

Reversing the order, running check first and pasting the actual list that comes out as the expected one makes the test pass at once, but tests nothing. Such a test freezes the application’s behavior of today; if today’s behavior is wrong, it freezes the wrongness too. A failing test is therefore not harmless but valuable: the difference between the expectation and the actual result is evidence that a person thought about the two things separately.

Which Test to Write Where

All three measurements were at the tree level. In practice, which question is asked with which test divides as follows.

• The question of which node an event changes is at the tree level. A component test answers it quickly and stably; it can run at every change.
• Whether a node keeps its identity is also a tree question, but it does not show up with a query. The difference between updated and unmounted is caught only with a tool that measures fates, or with a test that saves the node’s identity.
• What appears on the screen, whether the keyboard opens, whether a permission dialog appears are outside the tree. These are the job of the end-to-end test.

An end-to-end test is slow; a real application is opened at every touch. For this reason, it is kept to a small number of tests that cover the most important paths. Tests at the tree level make up the majority. This distribution rests on the fact that most errors can be caught in the tree and that an error in the tree is found cheaply.

The Bounding Measurement

The last measurement shows what a test cannot see. There is no text color in the model; a detail that is not in the tree is not in the rule either. Two screens built with the same counter value produce the same tree, whatever their colors. The measurement below compares these two screens and filters to the text node.

// main.mjs
import { reconcile, printOps } from "./fate.mjs";
import { renderCounter } from "./counter.mjs";

const before = renderCounter(1);
const after = renderCounter(1);
const ops = reconcile(before, after);
printOps(ops.filter((op) => op.path === "/Counter/Text"));

kept: /Counter/Text

The focus node is /Counter/Text. The two screens are the same as trees, and the text node came out kept. A test cannot know a difference that does not enter the tree, such as the text color: the rule looks at the prop, not at what is on the pixel. A test that wants to check that the color is right must carry the color into the tree as a prop, or look at the screen with an end-to-end test.

When the counter goes to one, the same filter gives /Counter/Text updated (the first measurement). This is the lesson’s fate: a test that tests an update looks for the thing being verified on the path of the updated node.

PP5. The model builds this fate’s rule; the model is not the framework itself. In the model, a test is a comparison of lists of records, and it discards the kept records. A real component test queries a text, an accessibility label, or a role; the query finds a node of the tree, but does not say whether the node it found was updated or mounted anew. The expectations measured in the model do not correspond one to one with the queries of a real test.

Summary

• This lesson’s fate: updated. When the button was pressed, the /Counter/Text node was updated; the button and the root stayed kept.
• The expected list of records writes the fates other than kept; check compares it with the actual list and gives PASS or FAIL.
• The expectation of an unchanged update is the empty list; a re-render is not a command.
• An expectation is written first in prose and then as a list of records; pasting the actual list as the expected one tests nothing.
• A component test looks at the tree; a detail that does not enter the tree, for example the text color, is seen only by an end-to-end test.

Next Step

Tests look at the tree but do not go down to the native layer beneath it. The next lesson, Writing Native Modules, measures what an update does in the tree when platform-specific code is written behind a node of the tree, and it builds a bridge to the course’s last lesson, Build and Release.