On-Call Rotation

The signal was generated, filtered, and confirmed. The metric was collected, the log was parsed, the dashboard was built, the alert rule was tied to a threshold. What stands at the end of the chain is not a gauge, but a person whose phone rings at 02:40.

The on-call rotation is the schedule that determines in advance which call goes to whom, and at what moment. The M20/K05 Operations Handover lesson compared full handover with shared on-call and counted the cost of the handover; here the on-call format is a given, and what is measured is the rotation’s own cost: on-call weeks per person per year, night wake-ups, and the recovery window.

Schedule, Coverage, and the Question Set

The example is once more the course’s fictional regional measurement network: the water-meter reading collector, verifier, billing, and work-order services. Both the network and the rotation are fictional; there is no real schedule or call log.

IM1: Four services are on a single team’s rotation; the nightly batch job starts at 02:00 and runs for two hours. IM2: The year is 52 weeks; the rotation hands off in order, a shift lasts 1 or 2 weeks, and the team is the first n people of an eight-person pool. IM3: A call arriving between 00:00–06:00 wakes the on-call person; one arriving during the day does not. IM4: The recovery window for a night wake-up is 4 hours; each additional wake-up in the same shift grows the window by 2 hours, and the accumulation resets at shift handoff. IM5: If the on-call person cannot touch the call’s service, the first person down the list who covers it is woken as a backup (their own 4-hour window), and the first step starts at 35 minutes instead of 10. IM6: The band is at most 13 on-call weeks, 12 night wake-ups, and 60 recovery hours per person per year.

// oncall/rotation.mjs — the on-call rotation model of the fictional regional measurement network
// (model): there is no real schedule, call log, or person; the year is a data structure and randomness comes from our own generator.

export const WEEK = 52, DAY = 7, HOUR = 24, YEAR = WEEK * DAY;
export const SERVICE = ["reading-collector", "verifier", "billing", "work-order"];
export const NIGHT_END = 6, BATCH_START = 2;  // calls between 00-06 wake; the nightly batch starts at 02
export const RECOVERY = 4, ACCUMULATION = 2; // first wake-up is 4 hours, each extra one in the same shift adds +2 hours
export const BACKUP_WINDOW = 4;           // the backup is woken too: their own 4-hour window
export const FIRST_STEP = 10, BACKUP_STEP = 35;   // delay before the first step starts (minutes)
export const THRESHOLD = { weeks: 13, wakeups: 12, recovery: 60 };   // annual bands per person

// daily call probability, and the share of calls falling into the nightly batch job window
export const RATE = { "reading-collector": 0.16, verifier: 0.09, billing: 0.06, "work-order": 0.05 };
export const BATCH_SHARE = { "reading-collector": 0.40, verifier: 0.35, billing: 0.30, "work-order": 0.10 };

// Coverage: the pool's eight people and the services each can touch. The first three people cover all four services.
export const COVERAGE = `
  k1  reading-collector verifier billing
  k2  verifier work-order
  k3  reading-collector billing work-order
  k4  reading-collector verifier
  k5  billing work-order
  k6  reading-collector work-order
  k7  verifier billing
  k8  reading-collector verifier billing work-order`
  .trim().split("\n").map((r) => { const [name, ...s] = r.trim().split(/\s+/); return { name, s }; });

// shift-register generator; the seed is visible, the sequence is the same run to run
export function generator(seed) {
  let x = seed >>> 0 || 1;
  return () => { x ^= x << 13; x >>>= 0; x ^= x >>> 17; x ^= x << 5; x >>>= 0;
    return x / 4294967296; };
}

// batchHour: the call set is identical for the same seed; only the hour of calls tied to the batch shifts.
export function calls(seed, batchHour = BATCH_START) {
  const r = generator(seed), list = [];
  for (let d = 0; d < YEAR; d++)
    for (const s of SERVICE) {
      if (r() >= RATE[s]) continue;
      const batch = r() < BATCH_SHARE[s];
      const hour = batch ? batchHour + Math.floor(r() * 2) : Math.floor(r() * HOUR);
      list.push({ day: d, hour, service: s, batch, night: hour < NIGHT_END });
    }
  return list;
}

// rotation: n people, shift length v weeks. In the "unscheduled" counter-run there is no
// schedule; night calls fall on the whole team. run(): the annual per-person rotation/wake-up/recovery counter.
export function run(callList, n, v, noSchedule = false) {
  const people = COVERAGE.slice(0, n).map((k) => ({ ...k, weeks: 0, wakeups: 0, backups: 0,
    recovery: 0, shift: -1, consecutive: 0 }));
  let offCoverage = 0, delay = 0;

  for (let h = 0; h < WEEK; h++) people[Math.floor(h / v) % n].weeks++;
  for (const c of callList) {
    const shift = Math.floor(c.day / (DAY * v));
    const i = shift % n, onCall = people[i];
    const inCoverage = noSchedule ? people.some((k) => k.s.includes(c.service))
      : onCall.s.includes(c.service);
    if (!inCoverage) offCoverage++;
    delay += inCoverage ? FIRST_STEP : BACKUP_STEP;
    if (!c.night) continue;
    if (noSchedule) {                                  // no rotation: night calls wake everyone
      for (const k of people) { k.wakeups++; k.recovery += RECOVERY; }
      continue;
    }
    if (onCall.shift !== shift) { onCall.shift = shift; onCall.consecutive = 0; }
    onCall.wakeups++;
    onCall.recovery += RECOVERY + ACCUMULATION * onCall.consecutive++;
    if (inCoverage) continue;
    for (let off = 1; off < n; off++) {                   // off coverage: the backup is woken too
      const y = people[(i + off) % n];
      if (!y.s.includes(c.service)) continue;
      y.wakeups++; y.backups++; y.recovery += BACKUP_WINDOW;
      break;
    }
  }
  const max = (f) => Math.max(...people.map(f));
  const avg = (f) => people.reduce((a, k) => a + f(k), 0) / n;
  const withinBand = (h, u, t) => h <= THRESHOLD.weeks && u <= THRESHOLD.wakeups && t <= THRESHOLD.recovery;
  return { n, v, people, offCoverage,
    weeks: avg((k) => k.weeks), wakeups: avg((k) => k.wakeups), maxWakeups: max((k) => k.wakeups),
    recovery: avg((k) => k.recovery), maxRecovery: max((k) => k.recovery),
    backups: people.reduce((a, k) => a + k.backups, 0), delay: delay / callList.length,
    avgBand: withinBand(avg((k) => k.weeks), avg((k) => k.wakeups), avg((k) => k.recovery)),
    personBand: people.every((k) => withinBand(k.weeks, k.wakeups, k.recovery)) };
}

// oncall/measure.mjs — the rotation's own cost: a sweep over team size and shift length.
import { calls, run, COVERAGE, THRESHOLD, WEEK, BACKUP_WINDOW } from "./rotation.mjs";

const SEED = 20260615;
const write = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const f1 = (x) => x.toFixed(1);
const yn = (b) => (b ? "yes" : "no");

const C = calls(SEED);
const night = C.filter((c) => c.night);
console.log(`seed ${SEED}: ${C.length} calls in ${WEEK} weeks, ${night.length} of them at night ` +
  `(${night.filter((c) => c.batch).length} of them in the nightly batch job window)`);

const S = [-8, -13, 13, 16, 18, 10, 13];
console.log("\n1. on-call rotation sweep (annual per person; avg / most loaded person)");
write(S, "people", "shift", "on-call wks", "wake-ups", "recovery hours", "avg band", "person band");
const sweep = [];
for (const n of [3, 4, 5, 6, 8])
  for (const v of [1, 2]) {
    const R = run(C, n, v);
    sweep.push(R);
    write(S, n, `${v} wk`, f1(R.weeks), `${f1(R.wakeups)} / ${R.maxWakeups}`,
      `${f1(R.recovery)} / ${f1(R.maxRecovery)}`, yn(R.avgBand), yn(R.personBand));
  }
const D = run(C, 5, 1, true);
write(S, 5, "no rotation", "-", `${f1(D.wakeups)} / ${D.maxWakeups}`,
  `${f1(D.recovery)} / ${f1(D.maxRecovery)}`, yn(D.avgBand), yn(D.personBand));
console.log(`band: on-call <= ${THRESHOLD.weeks} wks, wake-ups <= ${THRESHOLD.wakeups}, ` +
  `recovery <= ${THRESHOLD.recovery} hours`);

const Q = [-69, 20, 8];
console.log("\n2. the question set the on-call rotation answers");
write(Q, "question", "data", "answer");
const QUESTION = [
  ["who is on call at 02:40 at night", "on-call schedule", true],
  ["when does the rotation hand off", "on-call schedule", true],
  ["can the on-call person touch this service", "coverage schedule", true],
  ["if not, who is called, and when", "coverage schedule", true],
  ["how many times has the on-call person woken in the last thirty days", "wake-up log", true],
  ["which step does the call close with", "-", false],
  ["how many hours are unproductive after a wake-up", "-", false],
  ["can this rotation be sustained for a year", "-", false]];
for (const [q, d, y] of QUESTION) write(Q, q, d, y ? "yes" : "no");
console.log(`${QUESTION.filter((x) => x[2]).length} of ${QUESTION.length} questions are answered ` +
  `from the schedule, ${QUESTION.filter((x) => !x[2]).length} remain unanswered`);

const K = [-8, 14, 18, 22, 16];
console.log("\n3. the cost of an off-coverage call (1-week shift)");
write(K, "people", "off coverage", "backup wake-ups", "backup's share (hrs)", "first step min");
for (const R of sweep.filter((r) => r.v === 1))
  write(K, R.n, `${R.offCoverage} (${((R.offCoverage / C.length) * 100).toFixed(0)}%)`, R.backups,
    f1((R.backups * BACKUP_WINDOW) / R.n), f1(R.delay));

const H = [-8, 10, 13, 14, 17];
console.log("\n4. load distribution in a five-person team (1-wk / 2-wk shift)");
write(H, "person", "coverage", "on-call wks", "wake-ups", "recovery hours");
const [A, B] = [run(C, 5, 1), run(C, 5, 2)];
for (let i = 0; i < 5; i++)
  write(H, COVERAGE[i].name, COVERAGE[i].s.length, A.people[i].weeks,
    `${A.people[i].wakeups} / ${B.people[i].wakeups}`,
    `${f1(A.people[i].recovery)} / ${f1(B.people[i].recovery)}`);

const C9 = calls(SEED, 9);                      // same call set, batch job at 09:00
const T = [-8, 18, 20, 11, 10];
console.log(`\n5. if the nightly batch job runs at 09:00 instead of 02:00 (1-week shift)`);
console.log(`night calls ${night.length} -> ${C9.filter((c) => c.night).length}`);
write(T, "people", "wake-ups 02 / 09", "recovery 02 / 09", "band 02", "band 09");
for (const n of [3, 4, 5, 6, 8]) {
  const [a, b] = [run(C, n, 1), run(C9, n, 1)];
  write(T, n, `${f1(a.wakeups)} / ${f1(b.wakeups)}`, `${f1(a.recovery)} / ${f1(b.recovery)}`,
    yn(a.avgBand), yn(b.avgBand));
}

seed 20260615: 136 calls in 52 weeks, 61 of them at night (41 of them in the nightly batch job window)

1. on-call rotation sweep (annual per person; avg / most loaded person)
people  shift          on-call wks        wake-ups    recovery hours  avg band  person band
3       1 wk                  17.3       25.7 / 29     118.7 / 138.0        no           no
3       2 wk                  17.3       28.7 / 38     154.7 / 180.0        no           no
4       1 wk                  13.0       19.8 / 25      91.0 / 110.0        no           no
4       2 wk                  13.0       19.3 / 27     107.0 / 146.0        no           no
5       1 wk                  10.4       16.2 / 20       74.4 / 96.0        no           no
5       2 wk                  10.4       16.6 / 22      90.4 / 124.0        no           no
6       1 wk                   8.7       13.3 / 16       61.3 / 82.0        no           no
6       2 wk                   8.7       14.5 / 20       78.0 / 96.0        no           no
8       1 wk                   6.5       10.1 / 14       46.5 / 66.0       yes           no
8       2 wk                   6.5       11.3 / 16       60.0 / 96.0       yes           no
5       no rotation              -       61.0 / 61     244.0 / 244.0        no           no
band: on-call <= 13 wks, wake-ups <= 12, recovery <= 60 hours

2. the question set the on-call rotation answers
question                                                                             data  answer
who is on call at 02:40 at night                                         on-call schedule     yes
when does the rotation hand off                                          on-call schedule     yes
can the on-call person touch this service                               coverage schedule     yes
if not, who is called, and when                                         coverage schedule     yes
how many times has the on-call person woken in the last thirty days           wake-up log     yes
which step does the call close with                                                     -      no
how many hours are unproductive after a wake-up                                         -      no
can this rotation be sustained for a year                                               -      no
5 of 8 questions are answered from the schedule, 3 remain unanswered

3. the cost of an off-coverage call (1-week shift)
people    off coverage   backup wake-ups  backup's share (hrs)  first step min
3             42 (31%)                16                  21.3            17.7
4             42 (31%)                18                  18.0            17.7
5             50 (37%)                20                  16.0            19.2
6             44 (32%)                19                  12.7            18.1
8             45 (33%)                20                  10.0            18.3

4. load distribution in a five-person team (1-wk / 2-wk shift)
person    coverage  on-call wks      wake-ups   recovery hours
k1               3           11       20 / 22     96.0 / 124.0
k2               2           11       18 / 14      84.0 / 90.0
k3               3           10       20 / 21     86.0 / 112.0
k4               2           10       14 / 14      66.0 / 68.0
k5               2           10        9 / 12      40.0 / 58.0

5. if the nightly batch job runs at 09:00 instead of 02:00 (1-week shift)
night calls 61 -> 20
people    wake-ups 02 / 09    recovery 02 / 09    band 02   band 09
3               25.7 / 9.3        118.7 / 38.7         no        no
4               19.8 / 6.5         91.0 / 27.0         no       yes
5               16.2 / 5.0         74.4 / 20.8         no       yes
6               13.3 / 4.7         61.3 / 19.3         no       yes
8               10.1 / 3.0         46.5 / 12.5        yes       yes

The numbers are of the measurement class; their inputs are the IM1–IM6 assumptions. The schedule-free counter-run sits in the last row of the first table: with no rotation, every night call wakes the whole team, and wake-ups per person climb to 61 with recovery reaching 244 hours. The on-call rotation’s first job is not to reduce the load, but to dump it on one person — the same call set gives 16.2 wake-ups per person with five people.

What Happens As the Team Shrinks

On-call weeks fall out directly from team size: 17.3 weeks with three people, that is, a third of the year. Wake-ups and recovery degrade faster than that, because the backup for an off-coverage call also comes from the same small pool — 25.7 wake-ups and 118.7 hours with three people, versus 10.1 and 46.5 with eight.

Shift length does not change the total, but it does change the accumulation. In a two-week shift, recovery rises from 118.7 to 154.7 hours with three people, and from 61.3 to 78.0 with six; the wake-up count stays nearly constant. The reason is singular: a long shift stacks wake-ups onto a single person, and each extra wake-up opens the window by two more hours. The same number of wake-ups is cheaper when it is spread out.

Is the Threshold in the Average, or in the Person

Looking at the average, the rotation clears the band at eight people; at six people, recovery at 61.3 sits outside the 60-hour limit and wake-ups at 13.3 sit above 12. Looking at the person, no rotation clears the band: even at eight people, the most loaded person carries 14 wake-ups and 66 recovery hours.

The fourth table shows why. In the five-person team, k1 wakes 20 times and k5 wakes 9 times; recovery spreads between 96.0 and 40.0 hours, even though on-call weeks are 10 and 11. The difference comes from coverage: a person who covers three services wakes both on their own shift and as a backup on someone else’s. The average on-call load says nothing about the person carrying the heaviest share of it.

The Three Questions Left Unanswered

The schedule and the coverage schedule together answer five of the eight questions, and all five take the form of who and when. The three left unanswered fall into the same gap: which step will close the call, how many hours of productivity a wake-up takes away, and whether the rotation can be sustained for a year.

When “which step” goes unanswered, the call falls outside coverage: in the third table this share sits between 31% and 37%, and it does not fall as the team grows, because coverage is a property of the person, not of the pool. The bill has two line items — starting the first step rises from 10 minutes to 17.7–19.2 minutes, and a second person is woken between 16 and 20 times a year; that second wake-up costs 16.0 recovery hours per person with five people, more than a fifth of the 74.4 hours there. The recovery window itself is absent from every operational record: it is an assumption in the model, not something measured in the field, so the rotation’s cost never gets logged, and the arrangement becomes visible only once a person leaves.

Who Produces the Wake-Up

The fifth table runs the same call set with a single change: the nightly batch job starts at 09:00 instead of 02:00. The call count does not change, night calls fall from 61 to 20, and the band is cleared not at eight people but at four; at five people, wake-ups drop from 16.2 to 5.0 and recovery from 74.4 to 20.8 hours. The three-person team still does not clear the band even after this — 9.3 wake-ups is inside the band, 17.3 on-call weeks is outside it. The on-call rotation looks like a scheduling question; its bill is written in the job calendar.

Summary

• 61 of the year’s 136 calls arrive at night; without a schedule, the per-person load is 61 wake-ups and 244 recovery hours. With a schedule, this load drops to 17.3 on-call weeks, 25.7 wake-ups, and 118.7 recovery hours with three people, and to 6.5 / 10.1 / 46.5 with eight.
• Extending the shift to two weeks does not change the wake-up count, but it grows recovery: 118.7 → 154.7 hours with three people, 61.3 → 78.0 with six.
• By the average, the band is cleared at eight people; by the person, it is cleared under no rotation — with five people on the same schedule, 20 and 9 wake-ups, and 96.0 and 40.0 recovery hours, stand side by side.
• Five of eight questions are answered, three remain unanswered; the “which step” gap drops 31%–37% of calls outside coverage and produces 16–20 second wake-ups a year.
• Moving the nightly batch job to 09:00 drops night calls from 61 to 20 and pulls the threshold from eight people to four.

Next Step

This lesson assumed that every call was worth waking a person for. The next lesson labels a known set of incidents with a severity rule and counts misclassification in both directions: the unnecessary wake-up of over-escalation, the delayed response of under-escalation. The escalation chain is swept over the same set as well — how many steps, how many minutes per step, and how many incidents go unreached by anyone.