Skip to content
academia.sh

Lesson 23 / 27

End-to-End Tests

The risk covered by tests that run in a real browser, modeling and verifying the flow as a state machine, decisions about waiting and test data isolation, and why these tests are kept few in number.

Contents

A component test is limited to the component’s own tree. The measurement list coming from the server, the address updating, the form being submitted, and the list refreshing all lie outside that boundary. While the parts work correctly one by one, it is common for the flow to break somewhere: a module has dropped out of the build output, a routing rule has changed, the field name the server returns differs from what the client expects.

An end-to-end test closes this gap. It starts the application in a real browser, sends real input events, and verifies what the user sees.

The Risk Covered

A browser driver runs the test: a program that launches the browser under control, navigates to an address, produces click and key events, and can read the document and the accessibility tree. From the application’s point of view, the events the driver produces are real user events.

This setup covers a set of risks no other test type can see. Whether the build output actually works — code that passes in development mode can break in the production build because tree shaking dropped an export. Routing and deep linking. The data contract between server and client. Session and cookie behavior. Whether security headers are actually enforced: the policy written in the Content Security Policy lesson is tested only once a real browser actually rejects a script.

Its cost is just as large. An end-to-end test takes hundreds of times as long as a component test. When it fails, it does not say which layer is at fault; diagnosis requires separate work. And because it depends on the outside world, it is exposed to flakiness — the fifth lesson will enumerate the reasons for that.

Modeling the Flow First

The efficient way to write an end-to-end test is to think of the flow not as a list of steps but as a state machine. The same concept used for the parser in the Web Fundamentals and HTML course is applied here to the interface flow: a finite number of states, with transitions between them defined by events.

The measurement entry flow of the North Slope interface carries the following states: page idle, list loading, list ready, form open, submitting, list error.

// flow.mjs — the measurement entry flow's state machine
export const TRANSITIONS = {
  idle: { 'page-opened': 'listLoading' },
  listLoading: { 'list-arrived': 'listReady', 'request-failed': 'listError' },
  listReady: { 'new-measurement': 'formOpen', 'filter-changed': 'listLoading' },
  formOpen: { 'submit': 'submitting', 'cancel': 'listReady' },
  submitting: { 'record-accepted': 'listLoading', 'validation-error': 'formOpen' },
  listError: {},
};

export function runScenario(transitions, start, steps) {
  let state = start;
  const trace = [state];
  for (const step of steps) {
    const next = transitions[state]?.[step];
    if (next === undefined) {
      throw new Error(`no '${step}' step defined in state '${state}'`);
    }
    state = next;
    trace.push(state);
  }
  return { state, trace };
}

export function reachable(transitions, start) {
  const seen = new Set([start]);
  const queue = [start];
  while (queue.length > 0) {
    const current = queue.shift();
    for (const target of Object.values(transitions[current] ?? {})) {
      if (seen.has(target)) continue;
      seen.add(target);
      queue.push(target);
    }
  }
  return seen;
}

export function unrecoverable(transitions, safeState) {
  return Object.keys(transitions).filter(
    (s) => s !== safeState && !reachable(transitions, s).has(safeState),
  );
}

The reachable function is a direct application of breadth-first search from the Data Structures course: the transition table is a graph, states are nodes, events are edges.

Once the model is built, two kinds of verification can be written. The first is scenario verification: a specific sequence of steps must produce the expected state trace. The second is model verification: propositions about the table itself that must stay true across every scenario.

// flow.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { TRANSITIONS, runScenario, reachable, unrecoverable } from './flow.mjs';

test('happy path: measurement is entered and the list refreshes', () => {
  const { trace } = runScenario(TRANSITIONS, 'idle', [
    'page-opened', 'list-arrived', 'new-measurement', 'submit', 'record-accepted', 'list-arrived',
  ]);
  assert.deepEqual(trace, [
    'idle', 'listLoading', 'listReady', 'formOpen',
    'submitting', 'listLoading', 'listReady',
  ]);
});

test('an invalid measurement stays in the form', () => {
  const { state } = runScenario(TRANSITIONS, 'idle', [
    'page-opened', 'list-arrived', 'new-measurement', 'submit', 'validation-error',
  ]);
  assert.equal(state, 'formOpen');
});

test('an undefined step stops with a clear error', () => {
  assert.throws(
    () => runScenario(TRANSITIONS, 'idle', ['page-opened', 'submit']),
    /no 'submit' step defined in state 'listLoading'/,
  );
});

test('every state is reachable from the start', () => {
  const seen = reachable(TRANSITIONS, 'idle');
  assert.deepEqual([...seen].sort(), Object.keys(TRANSITIONS).sort());
});

test('every state can return to listReady', () => {
  assert.deepEqual(unrecoverable(TRANSITIONS, 'listReady'), []);
});
node --test flow.test.mjs

The stack-trace lines inside the failure body have been removed from the output below.

✔ happy path: measurement is entered and the list refreshes (0.495709ms)
✔ an invalid measurement stays in the form (0.054958ms)
✔ an undefined step stops with a clear error (0.132709ms)
✔ every state is reachable from the start (0.065583ms)
✖ every state can return to listReady (0.638792ms)
ℹ tests 5
ℹ pass 4
ℹ fail 1
✖ every state can return to listReady
  AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
  + actual - expected
  + [
  +   'listError'
  + ]
  - []

The model found a design gap before a single line of interface code ran: once the state reached by a failed list request, there is no way back. The user gets stuck on the error screen. The fix is to add an edge to the transition table:

// flow.mjs — fixed version of the TRANSITIONS table; only the last line changed
export const TRANSITIONS = {
  idle: { 'page-opened': 'listLoading' },
  listLoading: { 'list-arrived': 'listReady', 'request-failed': 'listError' },
  listReady: { 'new-measurement': 'formOpen', 'filter-changed': 'listLoading' },
  formOpen: { 'submit': 'submitting', 'cancel': 'listReady' },
  submitting: { 'record-accepted': 'listLoading', 'validation-error': 'formOpen' },
  listError: { 'retry': 'listLoading' },
};
✔ happy path: measurement is entered and the list refreshes (0.566708ms)
✔ an invalid measurement stays in the form (0.060625ms)
✔ an undefined step stops with a clear error (0.136541ms)
✔ every state is reachable from the start (0.077917ms)
✔ every state can return to listReady (0.386875ms)
ℹ tests 5
ℹ suites 0
ℹ pass 5
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 34.57375

The model’s value runs in two directions. While writing tests, it shows which scenarios are actually different — the happy path and the validation-error path use different transitions, so they are two separate tests. And a transition missing from the table is a button missing from the interface.

Steps, Waiting, and Data

Every step of a test running in the browser consists of two parts: a user action and that action’s observable result. If the next action begins before the result is verified, the test starts running faster than the application and becomes dependent on timing.

Waiting is therefore not done for a fixed duration. A fixed wait is wrong in two ways: it is not enough on a slow run, and it is wasted on a fast one. The correct form is a conditional wait — “wait until at least one row appears in the measurement table, up to this much time”. The condition itself is built with the criterion from the previous lesson: an element found by role and name.

Test data isolation is the second rule. Two tests working on a shared account see each other’s data; while one adds a measurement, the other’s count verification breaks. Each run sets up its own station and its own measurements, then cleans them up at the end.

The sign-in flow is a separate decision. Running authentication through the interface in every test adds a few seconds to each one and breaks the whole suite when the sign-in screen changes. Setting up the session directly — placing the token or cookie without going through the interface — removes that cost. The sign-in flow itself is tested separately, once.

On the network side, there are two options. Connecting to the real backend catches contract mismatches but is slow and depends on the backend’s state. Stubbing the responses makes the test fast and deterministic but cannot see contract drift. The common solution is to split the two: a few flows against the real backend, the rest with stubbed responses.

How Much End-to-End

The shape of the test suite is a budget question. Unit and component tests give feedback within seconds and point to the location of the defect; end-to-end tests take minutes and say only “something broke somewhere”. This is why the majority sits at the bottom and the minority at the top.

The selection criterion is not coverage percentage but the cost of failure. In the North Slope interface, three flows meet this criterion: the measurement list loading and being filtered, a new measurement being entered and saved, and recovery when the list request fails. Everything else is tested at a cheaper layer whenever it can be.

Summary

  • An end-to-end test covers the build output, routing, the data contract, the session, and security headers together; no lower layer can see this set.
  • Modeling the flow as a state machine shows which scenarios are different; reachability and recoverability can be verified on the transition table with breadth-first search.
  • The model reported that there was no way back from the error state before any interface code ran; a missing transition corresponds to a missing button in the interface.
  • Every step consists of an action and an observable result; waiting is done by condition, not by a fixed duration.
  • Each run sets up its own data, establishes the session without going through the interface, and tests part of the flows against the real backend and the rest with stubbed responses.

Next Step

An end-to-end test verifies that the flow works: the row appeared, the record was accepted, recovery from the error screen happened. What it does not verify is how those rows look. When a style rule is applied to the wrong container, a spacing value doubles, or a button becomes invisible, the flow tests keep passing — because the element still sits in the tree and can still be clicked. The next lesson covers the method that closes this gap: regression testing that compares two images pixel by pixel.

To keep your progress and take notes, Log in

My notes

Log in to take notes.

Start typing to search.

↑↓ Esc navigate · open · close