Timeout Budgets

The retry policy is settled: try count, wait between tries, jitter width. One number stays open and decides more than the policy itself — how long the caller waits before giving up. Without it, the backoff grows toward open space, not a limit.

The Application Layer and Service Interaction course established this duration as a budget: the end-to-end threshold splits across the chain’s steps, the remaining time passes with the call, and the retry count multiplies it. That computation is an input here, not retold. This lesson goes one step further: when a chain step slows down, where does the budget run out, how does the share split across sub-calls, and how much work does the dependency still do after the caller gives up.

Failure Mode

The chain comes from this course’s shared structure. The tracking query enters at the gateway and calls two arms in parallel. The delivery-operations arm reads a single record from its own store; the billing arm reads two records — first the tariff, then the shipment’s size — and computes the fee from both. The Application Layer course counted four step calls per external request; once the billing arm’s store step splits into two consecutive reads, the count climbs to five.

AY10 — the tariff store slows down. It does not go down, answer incorrectly, or suffer a network partition; only its response time climbs to tens of times its usual six-millisecond median. Duration 12 minutes, frequency once a month. Rationale: the slowdown starts with a maintenance window or a query-plan change and lasts until noticed and rolled back; twelve minutes sits a little above K01’s ten-minute recovery assumption. Its sensitivity is below, not added to K01’s table.

Slowdown is harder than being down. A down dependency rejects the connection immediately, and the caller learns this in under a millisecond. One that is slowing down rejects nothing, it just makes the caller wait — and with no limit set, that wait is paid for with the caller’s resources. The budget is the only mechanism deciding how much of it is accepted.

Splitting the Remaining Share

The arm’s share is fixed: K01’s 200-millisecond read threshold minus the Application Layer course’s U3 assumption of a 20-millisecond gateway share leaves 180 milliseconds, which the billing arm must distribute across two consecutive reads. Three policies exist.

The greedy split gives the entire remainder to each call: the first read can wait up to 180 milliseconds, the second gets whatever is left. The equal split halves the share. The weighted split allocates it by the typical durations’ ratio — tariff 6 milliseconds, size 12, so one to two.

All three look nearly identical on a failure-free day, since both reads stay well under their share. The difference appears the moment one read exceeds it.

// budget/chain.mjs — MODEL of the tracking query chain. Logical clock: step durations are
// read from a cycle, real time is not measured; no service, network, or store is set up.
export const K01 = { threshold: 200, readPeak: 416.67, edgePeak: 513.89, responseBytes: 480 };
export const GATEWAY_SHARE = 20;              // K03 service-design/05, assumption U3
export const TYPICAL = { delivery: 8, tariff: 6, size: 12 };   // model parameter (ms)

const D = (body, tail) => [...body, ...body, tail];   // a long tail once in sixteen
export const REGIME = {                       // sub-call duration (ms), sixteen-step cycle
  "failure-free": {
    delivery: D([8, 9, 8, 10, 8, 9, 8, 10], 40),
    tariff: D([6, 7, 6, 8, 6, 7, 6, 8], 80),
    size: D([12, 13, 12, 14, 12, 13, 12, 14], 100),
  },
  "tariff slowdown": {                        // AY10: only the tariff store slows down
    delivery: D([8, 9, 8, 10, 8, 9, 8, 10], 40),
    tariff: D([150, 400, 220, 400, 180, 300, 260, 400], 400),
    size: D([12, 13, 12, 14, 12, 13, 12, 14], 100),
  },
};

// Splitting the remaining share for the arm across two consecutive sub-calls.
export function shares(policy, remaining) {
  if (policy === "greedy") return [remaining, remaining];
  if (policy === "equal") return [remaining / 2, remaining / 2];
  const t = TYPICAL.tariff + TYPICAL.size;    // weighted: ratio of the typical durations
  return [(remaining * TYPICAL.tariff) / t, (remaining * TYPICAL.size) / t];
}

export function run({ regime, policy, cancel, n = 48 }) {
  const R = K01.threshold - GATEWAY_SHARE;    // remaining share for the arm
  const s = { answered: 0, dropped: 0, wastedWork: 0, totalDuration: 0,
    failed: { delivery: 0, tariff: 0, size: 0 }, endToEnd: [] };
  const cursor = { delivery: 0, tariff: 0, size: 0 };
  const duration = (name) => REGIME[regime][name][cursor[name]++ % REGIME[regime][name].length];

  const call = (name, limit) => {             // one sub-call, timeout `limit`
    if (limit <= 0) return { ok: false, elapsed: 0 };
    const d = duration(name);
    if (d <= limit) return { ok: true, elapsed: d };
    if (cancel === false) s.wastedWork += d - limit;   // caller gave up, dependency keeps working
    return { ok: false, elapsed: limit };
  };

  for (let i = 0; i < n; i += 1) {
    const a = call("delivery", R);            // delivery-ops arm: one store call
    const p = shares(policy, R);              // billing arm: two consecutive records
    const b1 = call("tariff", Math.min(p[0], R));
    const b2 = call("size", Math.min(p[1], R - b1.elapsed));
    for (const [name, r] of [["delivery", a], ["tariff", b1], ["size", b2]]) {
      if (r.ok === false) s.failed[name] += 1;
    }
    const elapsed = GATEWAY_SHARE + Math.max(a.elapsed, b1.elapsed + b2.elapsed);
    s.endToEnd.push(elapsed);
    s.totalDuration += elapsed;
    if (a.ok && b1.ok && b2.ok && elapsed <= K01.threshold) s.answered += 1; else s.dropped += 1;
  }
  const sorted = [...s.endToEnd].sort((x, y) => x - y);
  return { ...s, median: sorted[n >> 1], worst: sorted[n - 1], average: s.totalDuration / n, n };
}

Both reads are attempted on every request; the billing arm decides only after adding both outcomes. This carries the measurement: if the first read eats the budget, the second read experiences this as its own failure.

// budget/run.mjs — comparing the three split policies over two days
import { K01, GATEWAY_SHARE, TYPICAL, shares, run } from "./chain.mjs";

const R = K01.threshold - GATEWAY_SHARE;
const POLICY = ["greedy", "equal", "weighted"];
const CALLS = 5;                       // sub-calls per external request: 2 arms + 1 + 2 records
const DEADLINE_BYTES = 8;              // field carrying the remaining time
const slot = (ms) => (K01.readPeak * ms) / 1000;   // L = lambda x W

console.log(`end-to-end threshold ${K01.threshold} ms (K01), gateway share ${GATEWAY_SHARE} ms (K03/U3), remaining for the arm ${R} ms`);
console.log(`typical durations: delivery ${TYPICAL.delivery}, tariff ${TYPICAL.tariff}, size ${TYPICAL.size} ms`);
console.log(`${"policy".padEnd(11)}${"tariff share(ms)".padStart(17)}${"size share(ms)".padStart(15)}`);
for (const p of POLICY) {
  const [a, b] = shares(p, R);
  console.log(`${p.padEnd(11)}${a.toFixed(1).padStart(17)}${b.toFixed(1).padStart(15)}`);
}

for (const regime of ["failure-free", "tariff slowdown"]) {
  console.log(`\n-- ${regime} (48 external requests, no cancellation signal) --`);
  console.log(`${"policy".padEnd(11)}${"answered".padStart(9)}${"dropped".padStart(8)}${"blame:tariff".padStart(13)}` +
    `${"blame:size".padStart(11)}${"median".padStart(8)}${"worst".padStart(8)}${"avg.".padStart(7)}` +
    `${"held slots".padStart(13)}${"wasted work(ms)".padStart(16)}`);
  for (const p of POLICY) {
    const r = run({ regime, policy: p, cancel: false });
    console.log(`${p.padEnd(11)}${String(r.answered).padStart(9)}${String(r.dropped).padStart(8)}` +
      `${String(r.failed.tariff).padStart(13)}${String(r.failed.size).padStart(11)}` +
      `${String(r.median).padStart(8)}${String(r.worst).padStart(8)}${r.average.toFixed(1).padStart(7)}` +
      `${slot(r.average).toFixed(1).padStart(13)}${String(r.wastedWork).padStart(16)}`);
  }
}

console.log(`\n-- tariff slowdown: slots the system keeps busy (peak read ${K01.readPeak}/s) --`);
console.log(`${"policy".padEnd(11)}${"caller".padStart(9)}${"abandoned".padStart(12)}` +
  `${"total(no cancel)".padStart(17)}${"total(cancel)".padStart(16)}`);
for (const p of POLICY) {
  const r = run({ regime: "tariff slowdown", policy: p, cancel: false });
  const c = slot(r.average), t = slot(r.wastedWork / r.n);
  console.log(`${p.padEnd(11)}${c.toFixed(1).padStart(9)}${t.toFixed(1).padStart(12)}` +
    `${(c + t).toFixed(1).padStart(17)}${c.toFixed(1).padStart(16)}`);
}

const bytes = K01.edgePeak * CALLS * DEADLINE_BYTES;
const a = run({ regime: "failure-free", policy: "greedy", cancel: false });
console.log(`\n-- failure-free day's cost (computed value) --`);
console.log(`deadline field ${DEADLINE_BYTES} B x ${CALLS} sub-calls = ${CALLS * DEADLINE_BYTES} B/request,` +
  ` ${((CALLS * DEADLINE_BYTES) / K01.responseBytes * 100).toFixed(1)}% of the tracking response (${K01.responseBytes} B)`);
console.log(`at ${K01.edgePeak} req/s peak edge -> ${bytes.toFixed(2)} B/s = ${(bytes / 1024).toFixed(2)} KiB/s`);
for (const p of ["equal", "weighted"]) {
  const r = run({ regime: "failure-free", policy: p, cancel: false });
  const diff = r.dropped - a.dropped;
  console.log(`${p.padEnd(11)}: ${diff}/${r.n} more dropped requests than greedy` +
    ` = ${((diff / r.n) * 100).toFixed(2)}% -> ${((K01.readPeak * diff) / r.n).toFixed(2)} req/s at peak read`);
}

console.log(`\n-- AY10's sensitivity: how long the tariff store stays slow --`);
console.log(`${"duration(min)".padStart(14)}${"a month".padStart(8)}${"minutes/mo".padStart(11)}${"of failure share(28.2 min)".padStart(27)}`);
for (const [min, times] of [[12, 1], [12, 2], [24, 1]]) {
  console.log(`${String(min).padStart(14)}${String(times).padStart(8)}${String(min * times).padStart(11)}` +
    `${`${(((min * times) / 28.2) * 100).toFixed(1)}%`.padStart(27)}`);
}

end-to-end threshold 200 ms (K01), gateway share 20 ms (K03/U3), remaining for the arm 180 ms
typical durations: delivery 8, tariff 6, size 12 ms
policy      tariff share(ms) size share(ms)
greedy                 180.0          180.0
equal                   90.0           90.0
weighted                60.0          120.0

-- failure-free (48 external requests, no cancellation signal) --
policy      answered dropped blame:tariff blame:size  median   worst   avg.   held slots wasted work(ms)
greedy            48       0            0          0      40     200   46.2         19.2               0
equal             46       2            0          2      40     190   45.8         19.1              20
weighted          46       2            2          0      40     180   45.3         18.9              40

-- tariff slowdown (48 external requests, no cancellation signal) --
policy      answered dropped blame:tariff blame:size  median   worst   avg.   held slots wasted work(ms)
greedy             6      42           36         42     200     200  197.8         82.4            5540
equal              0      48           48          2     123     200  126.0         52.5            9700
weighted           0      48           48          0      93     180   96.4         40.2           11120

-- tariff slowdown: slots the system keeps busy (peak read 416.67/s) --
policy        caller   abandoned total(no cancel)   total(cancel)
greedy          82.4        48.1            130.5            82.4
equal           52.5        84.2            136.7            52.5
weighted        40.2        96.5            136.7            40.2

-- failure-free day's cost (computed value) --
deadline field 8 B x 5 sub-calls = 40 B/request, 8.3% of the tracking response (480 B)
at 513.89 req/s peak edge -> 20555.60 B/s = 20.07 KiB/s
equal      : 2/48 more dropped requests than greedy = 4.17% -> 17.36 req/s at peak read
weighted   : 2/48 more dropped requests than greedy = 4.17% -> 17.36 req/s at peak read

-- AY10's sensitivity: how long the tariff store stays slow --
 duration(min) a month minutes/mo of failure share(28.2 min)
            12       1         12                      42.6%
            12       2         24                      85.1%
            24       1         24                      85.1%

All these numbers belong to the computed value class: they are counted over a deterministic sequence of durations. The time unit is a logical clock, not a measured latency; the run gives the same output independent of the machine.

Failure-Free Day’s Cost

The budget carries two costs even on a failure-free day.

The first is the field itself: passing the remaining time with the call adds an eight-byte field to every sub-call, 40 bytes per external request across five — 8.3 percent of K01’s 480-byte tracking response, and 20.07 KiB/s of internal traffic at the peak edge rate. Small but not zero, and it grows linearly as the chain deepens.

The second is costlier: a tight split cuts off healthy requests. In the failure-free regime greedy answers forty-eight of forty-eight requests; equal and weighted answer forty-six, dropping two each. These are false positives on a day with no failure at all — the model’s long tail, arriving once in sixteen, exceeds the step’s share: 4.17 percent, 17.36 requests/s at peak read. Which step gets cut off depends on the policy: under equal it is size (share 90, tail 100); under weighted, tariff (share 60, tail 80).

Choosing the share by the step’s tail lowers this cost but does not zero it. Splitting the budget is a decision to cut a tail, and the cut tail is always made of real requests.

What Changes on a Failing Day

The second table runs under AY10 and the three policies produce three distinct behaviors.

Greedy rescues six requests. The tariff returns in 150 or 180 milliseconds on some rounds; with a 180-millisecond share these succeed, leaving enough for the size read. Under equal and weighted, the same reads are cut off at 90 and 60 milliseconds, and forty-eight of forty-eight requests drop — the cost of a tight split continues on a failing day too.

The trade-off shows up in end-to-end duration. Under greedy the median is 200 milliseconds: every dropped request eats the whole budget. Under equal it is 123, under weighted 93 — the caller learns the same failure in under half the time. Its resource counterpart is held slots: by queueing theory’s $L = \lambda W$ relation, at K01’s peak read rate of 416.67 requests/s, greedy keeps 82.4 concurrent caller slots busy, weighted 40.2. The relation is not this course’s finding; it comes from the background jobs topic and is used here as a converter.

The blame columns hide the failure mode. Under greedy, the tariff fails thirty-six times, the size forty-two. The size’s regime never changed; all forty-two failures come from the tariff eating the budget. Under weighted, the size fails zero times. The difference is not a counting difference but a diagnostic one: greedy hides the slowing step behind the healthy one, and the name showing up most in the timeout log is not where the failure happened.

This is spread through the budget. The steps look isolated — separate stores, separate records — but they share a single resource, time. Fixing the share is the name for dividing up that shared resource.

Abandoned Work

The last column flips the table read so far. When the caller gives up, the tariff store does not stop; it finishes the read it started. This is work nobody waits for, and the model counts it as wasted work: 5540 milliseconds under greedy, 11120 under weighted. A tight split doubles abandoned work — giving up earlier leaves a longer remainder ahead of the dependency.

The third table settles it by adding both sides. With no cancellation signal, total busy slots are 130.5 under greedy, 136.7 under equal and weighted — nearly the same: splitting only moves the load from caller to dependency, nothing else. A tight split’s gain shows up only with a cancellation signal — the caller tells the dependency it has given up, and the read stops. Then abandoned work zeroes out and total slots become 82.4 against 40.2.

This is the deciding sentence: deadline propagation is not complete without a cancellation signal. Passing the remaining time with the call says when the caller will give up; the signal says that it has. Without the first, the budget cannot be split; without the second, splitting only relocates cost.

The last table is AY10’s sensitivity. A single twelve-minute slowdown spends 42.6 percent of K01’s monthly 28.2-minute failure share. Twice a month, or once for twenty-four minutes, the share climbs to 85.1 percent, leaving 4.2 minutes for the rest of the month. Splitting the budget does not bring these minutes back — requests still drop — but it fixes how many resources a dropped request occupies and confirms which step the failure shows up in.

Summary

• The arm’s remaining share (200 ms threshold minus 20 ms gateway share = 180 ms) splits three ways: greedy 180/180, equal 90/90, weighted 60/120.
• The failure-free day’s cost has two items: a deadline field adding 40 bytes per external request (8.3 percent of the response, 20.07 KiB/s at peak edge), and a tight split cutting off valid requests — 2 in 48, 4.17 percent, 17.36 requests/s at peak read.
• Under AY10, greedy rescues six requests but keeps the median at 200 ms; weighted rescues none but brings the median to 93 ms, dropping held slots from 82.4 to 40.2.
• Under greedy, the healthy step (size) appears to fail forty-two times, under weighted zero times; the split sets not just the resource but the diagnosis of the failure.
• A tight split raises abandoned work from 5540 to 11120 ms; with no cancellation signal, total busy slots run 130.5 against 136.7 — no gain. With one, the total becomes 82.4 against 40.2 — a gain only with cancellation.
• A single twelve-minute slowdown spends 42.6 percent of K01’s monthly 28.2-minute failure share; twice a month, 85.1 percent.

Next Step

The budget decided how long a single request would wait and distributed that decision across the chain. One assumption stayed silent: the caller is able to make the call for every request. Facing a slowing dependency, the budget cuts the request off, reclaims the resource, and shows the failure at the right step — but does not stop the next request from arriving. Requests keep coming at the same rate, each spending the same budget on the same slow step. The problem’s other face is on the write path: status events from the carrier flow in at a fixed rate, and if the consumer feeding the projection falls behind, work piles up somewhere the budget never touches. The next lesson takes up that pile and rounds the question around: how does the system tell the party feeding it work that it can no longer keep up, how far does that notice spread, and where does work collect if it is never made.