Retry and Backoff

The previous lesson treated the failed event as a single gate: an error arrived, the view moved to the error state, the user was shown a “try again” button. But some errors should not be shown to the user at all. An observer connecting to the North Slope station from the mountain has a connection that drops and recovers within seconds; a request retried two hundred milliseconds later succeeds, and the user sees nothing in between.

This lesson’s question is: which error gets retried automatically, how long to wait, how many times to try, and when to give up. Answering all four wrong produces the same outcome — an already-strained server gets strained further by its own clients.

Which Error Gets Retried

The decision looks at two pieces of information: the error’s class and the request’s method.

The error class was established in the REST Client lesson. In HTTP errors, the server’s decision is known: a 4xx is about the request itself, and repeating the same request produces the same response. There are two exceptions — 429 reports a rate limit, 5xx the server’s current state; both can change over time.

In network errors, whether the server received the request is unknown. The response may have been lost on the way back; in that case the server has already processed the request. So for a network error, the decision falls to the method. The safe methods defined in the How the Internet Works course do not change state, and repeating them is harmless. A request that submits a measurement record, on the other hand, can produce two records if processed twice; it cannot be retried automatically.

There is a way around this limit: an idempotency key generated by the client is attached to the request, and the server does not count a second request with the same key as a new record. Sending the key makes a state-changing request retryable too. This is a contract the client and server must agree on together; it cannot be set up unilaterally.

Parse errors and cancellation are never retried under any circumstance. The first is the program’s own bug, the second is already a decision to give up.

Computing the Wait Duration

Retrying immediately does not help: the problem needs time to clear. A fixed interval is not enough either; if the server cannot get back up, it keeps taking the same load at the same frequency. Exponential backoff grows the wait duration by a multiplier on every attempt; recovery is fast if the problem is short, and the client backs off if it is long.

Exponential growth alone is unbounded, so a cap is set. The real missing piece is something else. When a server goes down, every client connected to it gets an error at the same moment; if they all apply the same formula, they all retry at the same moment. The instant the server comes back up, it meets a synchronized wave and goes back down. Jitter spreads out that wave by adding randomness to the wait duration.

// backoff.mjs — wait durations and jitter's effect on distribution
// Deterministic pseudo-random generator: the same seed gives the same sequence.
function makeRng(seed) {
  let a = seed >>> 0;
  return () => {
    a = (a + 0x6d2b79f5) >>> 0;
    let t = Math.imul(a ^ (a >>> 15), 1 | a);
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

const BASE = 200, MULTIPLIER = 2, CAP = 5000;
const exponential = (n) => Math.min(CAP, BASE * MULTIPLIER ** n);

const random = makeRng(20260114);
const fullJitter = (n) => Math.round(random() * exponential(n));

// Decorrelated jitter: the next wait is chosen from a range tied to the previous one.
let previous = BASE;
const decorrelatedJitter = () => {
  previous = Math.min(CAP, Math.round(BASE + random() * (previous * 3 - BASE)));
  return previous;
};

console.log("attempt  fixed   exponential  exp+cap  full jitter  decorrelated");
for (let n = 0; n < 6; n++) {
  console.log(
    String(n + 1).padEnd(9) +
    String(BASE).padEnd(8) +
    String(BASE * MULTIPLIER ** n).padEnd(13) +
    String(exponential(n)).padEnd(9) +
    String(fullJitter(n)).padEnd(13) +
    String(decorrelatedJitter()));
}

// --- Wake-up distribution of clients that dropped at the same time ----------
const CLIENTS = 24, ATTEMPT = 3;   // third attempt's wait: exponential(2) = 800 ms
const bucket = (ms) => Math.floor(ms / 200) * 200;

const count = (list) => {
  const m = new Map();
  for (const v of list) m.set(v, (m.get(v) ?? 0) + 1);
  return [...m.entries()].sort((a, b) => a[0] - b[0])
    .map(([k, s]) => `${String(k).padStart(4)}ms:${"#".repeat(s)}`);
};

const noJitter = Array.from({ length: CLIENTS }, () => bucket(exponential(ATTEMPT - 1)));
const rng2 = makeRng(20260114);
const withJitter = Array.from({ length: CLIENTS },
  () => bucket(Math.round(rng2() * exponential(ATTEMPT - 1))));

console.log("\nno jitter (24 clients, 3rd attempt):");
count(noJitter).forEach((s) => console.log("  " + s));
console.log("with jitter (24 clients, 3rd attempt):");
count(withJitter).forEach((s) => console.log("  " + s));

attempt  fixed   exponential  exp+cap  full jitter  decorrelated
1        200     200          200      70           384
2        200     400          400      296          1034
3        200     800          800      780          2121
4        200     1600         1600     303          1933
5        200     3200         3200     480          3986
6        200     6400         5000     957          5000

no jitter (24 clients, 3rd attempt):
   800ms:########################
with jitter (24 clients, 3rd attempt):
     0ms:######
   200ms:#####
   400ms:#######
   600ms:######

The two histograms show the difference plainly. In the no-jitter calculation, all twenty-four clients pile up on a single instant; in the jittered calculation, the same clients spread across four separate intervals. The peak load the server sees drops to a quarter.

The table’s two jitter forms make different trade-offs. Full jitter picks the wait from anywhere between zero and the exponential value — the widest spread, but some attempts land very early. Decorrelated jitter picks the next wait from a range tied to the previous one; growth is smoother and less jumpy, reaching the cap by the sixth attempt in the table. The choice depends on the service’s recovery time: full jitter suits a fast-recovering service, decorrelated jitter a slow-recovering one.

Note that the randomness is built with a deterministic generator. In production the generator is unseeded; here the seed is held fixed so the output is the same on every run and the table can be verified.

Building the Loop

The calculation alone is not enough; when the loop stops and how much it honors what the server says also need to be defined.

// retry-loop.mjs — retry loop running against a local server
import http from "node:http";

let measurementRequests = 0;
const server = http.createServer((req, res) => {
  const path = new URL(req.url, "http://127.0.0.1").pathname;
  if (path === "/measurements" && req.method === "GET") {
    measurementRequests += 1;
    if (measurementRequests <= 2) {                // the first two requests fail
      res.writeHead(503, { "content-type": "application/json" });
      return res.end(JSON.stringify({ code: "unavailable" }));
    }
    res.writeHead(200, { "content-type": "application/json" });
    return res.end(JSON.stringify([{ id: "m-114", value: -4.2 }]));
  }
  if (path === "/measurements" && req.method === "POST") {
    res.writeHead(422, { "content-type": "application/json" });
    return res.end(JSON.stringify({ code: "validation" }));
  }
  if (path === "/report") {
    res.writeHead(429, { "content-type": "application/json", "retry-after": "2" });
    return res.end(JSON.stringify({ code: "rate_limit" }));
  }
  res.writeHead(404, { "content-type": "application/json" });
  res.end(JSON.stringify({ code: "no_route" }));
});
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
const BASE = `http://127.0.0.1:${server.address().port}`;

// --- Wait calculation (deterministic) ---------------------------------------
function makeRng(seed) {
  let a = seed >>> 0;
  return () => {
    a = (a + 0x6d2b79f5) >>> 0;
    let t = Math.imul(a ^ (a >>> 15), 1 | a);
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}
const random = makeRng(20260114);
const wait = (n) => Math.round(random() * Math.min(5000, 200 * 2 ** n));

// --- Policy: which outcome is retried ---------------------------------------
const SAFE_METHODS = new Set(["GET", "HEAD", "OPTIONS"]);
function isRetryable(outcome, method) {
  if (outcome.type === "network") return SAFE_METHODS.has(method);  // unknown if it arrived
  if (outcome.type === "http") return outcome.status === 429 || outcome.status >= 500;
  return false;                                                     // parse error, cancellation
}

// --- Loop ---------------------------------------------------------------
async function retry(path, { method = "GET", maxAttempts = 4, sleep }) {
  const waits = [];
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    let outcome;
    try {
      const response = await fetch(BASE + path, { method });
      const body = await response.json();
      outcome = response.ok
        ? { type: "success", data: body }
        : { type: "http", status: response.status,
            serverSuggestion: Number(response.headers.get("retry-after") ?? 0) * 1000 };
    } catch {
      outcome = { type: "network" };
    }
    if (outcome.type === "success") return { outcome, attempt: attempt + 1, waits };
    if (!isRetryable(outcome, method)) return { outcome, attempt: attempt + 1, waits };
    if (attempt === maxAttempts - 1) return { outcome, attempt: attempt + 1, waits };

    // If the server's suggestion is longer than the computed wait, the suggestion wins.
    const w = Math.max(wait(attempt), outcome.serverSuggestion ?? 0);
    waits.push(w);
    await sleep(w);
  }
}

// The test records durations instead of actually waiting; a real
// application would have a real timer here.
const recorded = [];
const fakeSleep = (ms) => { recorded.push(ms); return Promise.resolve(); };

const log = (name, r) => console.log(
  name.padEnd(24),
  `attempt=${r.attempt}`.padEnd(11),
  `outcome=${r.outcome.type}${r.outcome.status ? "/" + r.outcome.status : ""}`.padEnd(17),
  `waits=[${r.waits.join(", ")}]`);

log("503, then success", await retry("/measurements", { sleep: fakeSleep }));
log("422 (not retried)", await retry("/measurements", { method: "POST", sleep: fakeSleep }));
log("429 (Retry-After: 2)", await retry("/report", { maxAttempts: 3, sleep: fakeSleep }));
log("404 (not retried)", await retry("/missing", { sleep: fakeSleep }));

console.log("measurement requests reaching the server:", measurementRequests);
console.log("all recorded waits                      :", recorded.join(", "));

server.close();

503, then success        attempt=3   outcome=success   waits=[70, 184]
422 (not retried)        attempt=1   outcome=http/422  waits=[]
429 (Retry-After: 2)     attempt=3   outcome=http/429  waits=[2000, 2000]
404 (not retried)        attempt=1   outcome=http/404  waits=[]
measurement requests reaching the server: 3
all recorded waits                      : 70, 184, 2000, 2000

The first row shows the expected behavior: the server returned 503 twice, the loop waited twice, and data arrived on the third attempt. The user saw none of this; the view was still in the loading state.

The second and fourth rows show the policy’s discriminating power. The validation error and the not-found response both finished in a single attempt, with no wait at all, since retrying would not change the outcome. The most common retry mistake is feeding every error into the loop: when the user enters a bad value, finding out takes three times as long.

The third row shows the server’s suggestion winning. The computed wait was a few hundred milliseconds; because the server said two seconds, two seconds were waited. The server knows its own recovery time better than the client does, so its suggestion is used not as a floor and not as a ceiling but as a lower bound. If the suggestion is unreasonably long, giving up on the attempt — telling the user the situation — beats waiting.

The loop has three limits. An attempt count ceiling prevents an infinite loop. A total budget limits the combined duration of all attempts; a user pressing a button and waiting eight seconds is worse than seeing an error. Third, user-triggered requests and background requests have separate policies: attempt count is kept low and duration short when a user is waiting, and the reverse holds for a background refresh.

Circuit Breaker

Retrying is a single request’s problem. If the server is entirely unreachable, every request runs its own attempts, and the application both keeps the user waiting and loads the server with requests that will never succeed. A circuit breaker recognizes this situation at a level above: once consecutive failures cross a threshold, it stops sending the request at all.

// circuit-breaker.mjs — not sending the request at all after consecutive failures
const THRESHOLD = 3, COOLDOWN = 5000;   // 3 consecutive failures; retry allowed after 5000 ms

function circuitBreaker() {
  let state = "closed", consecutiveFailures = 0, openedAt = 0;

  return {
    allow(now) {
      if (state === "open" && now - openedAt >= COOLDOWN) state = "halfOpen";
      return { allowed: state !== "open", state };
    },
    report(success, now) {
      if (success) { state = "closed"; consecutiveFailures = 0; return state; }
      consecutiveFailures += 1;
      if (state === "halfOpen" || consecutiveFailures >= THRESHOLD) {
        state = "open"; openedAt = now; consecutiveFailures = 0;
      }
      return state;
    },
  };
}

// Logical clock: advances step by step instead of real time.
const breaker = circuitBreaker();
const EVENTS = [
  [    0, false], [  200, false], [  600, false],   // the third failure opens the circuit
  [  900, false], [ 1500, false],                   // during cooldown: the request does not go out
  [ 6000, false],                                    // half-open attempt, fails
  [ 8000, false],                                    // back in cooldown
  [11500, true ],                                    // half-open attempt, succeeds
  [11800, true ],
];

let sent = 0, blocked = 0;
console.log("time    allow  prev state    result        new state");
for (const [now, success] of EVENTS) {
  const { allowed, state } = breaker.allow(now);
  if (!allowed) {
    blocked += 1;
    console.log(String(now).padEnd(8) + "no     " + state.padEnd(14) +
                "not sent".padEnd(13) + state);
    continue;
  }
  sent += 1;
  const next = breaker.report(success, now);
  console.log(String(now).padEnd(8) + "yes    " + state.padEnd(14) +
              (success ? "success" : "failure").padEnd(13) + next);
}
console.log(`sent: ${sent}  blocked: ${blocked}`);

time    allow  prev state    result        new state
0       yes    closed        failure      closed
200     yes    closed        failure      closed
600     yes    closed        failure      open
900     no     open          not sent     open
1500    no     open          not sent     open
6000    yes    halfOpen      failure      open
8000    no     open          not sent     open
11500   yes    halfOpen      success      closed
11800   yes    closed        success      closed
sent: 6  blocked: 3

The three states and their transitions can be read from the output. In the closed state requests pass through; a third consecutive failure moves the circuit to open. In the open state no request is sent — the call fails immediately and without touching the network — three of the nine events never reached the server. Once the cooldown elapses, the circuit becomes half-open and allows exactly one trial request. That trial’s outcome is decisive: on failure the circuit opens again and the cooldown restarts (the behavior seen at the 6000 mark), on success it closes and the counter resets.

While the circuit is open, the call site gets no response but does get an error. This error should be kept distinct from a network error: the user is told “the service is not responding right now,” and since the “try again” button will not do anything until the cooldown elapses anyway, reporting that duration is meaningful.

The circuit breaker’s scope is also a design decision. A single breaker covering the whole application lets one endpoint’s failure stop every screen. In practice a breaker is usually set up per service or endpoint group, so the station list stays readable while the measurement-writing path is broken.

Summary

• The retry decision looks at error class and method: 429 and 5xx are always retried, a network error only for safe methods; validation errors, parse errors, and cancellations are never retried.
• A state-changing request is retryable only if an idempotency key generated by the client has been agreed on with the server.
• Exponential backoff grows the wait on every attempt and is bounded by a cap; jitter keeps clients that failed at the same moment from coming back at the same moment.
• Full jitter has the widest spread, decorrelated jitter smooths the growth; the choice depends on the service’s recovery time.
• The server’s reported wait suggestion is compared with the computed duration and the larger one is applied; a total duration budget limits things alongside the attempt count.
• A circuit breaker stops sending the request at all after consecutive failures; it moves between open, half-open, and closed states and is set up per endpoint group.

Next Step

The Data Access topic is complete here: data is fetched, stored, updated as a stream, unexpected states are reduced to a finite set, and network instability is absorbed before it reaches the user. All of this flow was one-directional — from server to screen. The next topic sets up the reverse direction: data the user supplies. The measurement entry form asks all at once where a field’s value is held, when to validate it, how to report an error, and what to show during submission. The first lesson opens with the first of these questions: where does a field’s value live — in the document, or in the application’s state?