Service Objects

The previous lesson measured keeping six replicas up, and in measuring it made something visible: a replica set guarantees that there will be six pods, not which six. In the fourteen-round run ten pods were created, and in the rollout six were deleted and six created. Every new pod is a new pod; the previous one’s address went with it. When the verifier wants to send a request to the reading collector, who will it connect to?

The object that fills this gap is the service object. What it holds is not a piece of software; it is a name and the endpoint list behind that name. The list is derived from the replica set and updates itself as pods come and go. In M16/K06, service was a separately deployed and separately owned piece of software; the service object here is a name standing in front of that software’s replicas — the same word, a different object. This lesson models the name and the list with node and counts both; no real cluster runs.

CO13 — a service object holds an endpoint list tied to a name, and the list is derived from the pods in a replica set. CO14 — resolving the name to a list adds one round to a request, and the result can be cached for a lifetime. CO15 — requests are distributed round-robin over the endpoint list. CO16 — a new pod becomes ready two rounds later; without a readiness check it enters the list in the round it is created. CO17 — a request to an endpoint that is not ready or has been dropped from the declaration fails. CO18 — the run lasts twenty-four rounds, twelve requests arrive per round, and pod churn is produced by a seeded generator.

The List That Changes, the Name That Does Not

The run below applies the same churn schedule twice. The only thing that changes is where the endpoint list is derived from: all pods, or only the ready ones.

// service.mjs — service object: the name stays fixed while the endpoint list changes (MODEL).
export const SEED = 20260803, ROUNDS = 24, REQUESTS = 12, N = 6, READY = 2, CHURN = 0.45;
// The generator is hand-written; 32-bit integer operations, seed is visible.
export const make = (t) => () =>
  { t ^= t << 13; t ^= t >>> 17; t ^= t << 5; return (t >>> 0) / 2 ** 32; };

// The churn schedule is generated once; both runs lose the same pods in the same rounds.
const rnd = make(SEED);
export const SCHEDULE = Array.from({ length: ROUNDS },
  () => (rnd() < CHURN ? Math.floor(rnd() * N) : null));

export const SERVICE_NAME = 'reading-collector.measurement';  // the one thing that never changes

// The replica set holds N pods every round; churned slot gets a new pod (CO13).
export function run(check) {
  let counter = N, name = SERVICE_NAME;
  const slot = Array.from({ length: N }, (_, i) => ({ name: `pod-${i}`, readyAt: 0 }));
  let requests = 0, failed = 0, churns = 0, nameChanges = 0, listChanges = 0, cursor = 0;
  let previous = '';
  const endpointCounts = [], load = [];
  for (let t = 0; t < ROUNDS; t++) {
    const j = SCHEDULE[t];
    if (j !== null) { slot[j] = { name: `pod-${counter++}`, readyAt: t + READY }; churns++; }
    // With the readiness check the list only draws from ready pods; without it, from all of them.
    const endpoints = check ? slot.filter((k) => k.readyAt <= t) : [...slot];
    const signature = endpoints.map((k) => k.name).sort().join(',');
    if (t && signature !== previous) listChanges++;
    if (name !== SERVICE_NAME) nameChanges++;                  // no path ever touches the name
    previous = signature;
    endpointCounts.push(endpoints.length);
    load.push(endpoints.length ? Math.ceil(REQUESTS / endpoints.length) : REQUESTS);
    for (let i = 0; i < REQUESTS; i++) {
      requests++;
      if (!endpoints.length) { failed++; continue; }
      const target = endpoints[cursor++ % endpoints.length];   // CO15: round-robin distribution
      if (target.readyAt > t) failed++;                        // CO17
    }
  }
  return { requests, failed, churns, nameChanges, listChanges,
    endpointMin: Math.min(...endpointCounts), endpointMax: Math.max(...endpointCounts), loadMax: Math.max(...load) };
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const s = (x, n) => String(x).padStart(n);
  console.log(`seed ${SEED}, ${ROUNDS} rounds, ${REQUESTS} requests per round, ${N} replicas,`
    + ` readiness ${READY} rounds, service name '${SERVICE_NAME}'`);
  console.log(`${'check'.padEnd(14)}${s('requests', 9)}${s('failed', 8)}${s('rate', 8)}`
    + `${s('endpoint count', 16)}${s('max per endpoint', 19)}${s('list changes', 15)}`
    + `${s('name changes', 14)}`);
  for (const [label, d] of [['off', false], ['readiness', true]]) {
    const r = run(d);
    console.log(`${label.padEnd(14)}${s(r.requests, 9)}${s(r.failed, 8)}`
      + `${s(`%${(100 * r.failed / r.requests).toFixed(1)}`, 8)}`
      + `${s(`${r.endpointMin}-${r.endpointMax}`, 16)}${s(r.loadMax, 19)}${s(r.listChanges, 15)}`
      + `${s(r.nameChanges, 14)}`);
  }
  console.log(`\n${ROUNDS} rounds, ${run(true).churns} pods came and went;`
    + ` the caller used the same name throughout`);
}

seed 20260803, 24 rounds, 12 requests per round, 6 replicas, readiness 2 rounds, service name 'reading-collector.measurement'
check          requests  failed    rate  endpoint count   max per endpoint   list changes  name changes
off                 288      44   %15.3             6-6                  2             10             0
readiness           288       0    %0.0             4-6                  3             13             0

24 rounds, 11 pods came and went; the caller used the same name throughout

The last line is this lesson’s whole rationale. In twenty-four rounds eleven pods came and went; the endpoint list changed thirteen times; the name changed zero times. When the count of change and the count of constancy stand side by side, what a service object does becomes visible: it moves a source of volatility to a place the caller never sees. The only thing written in the verifier’s code is the name reading-collector.measurement, and that name is unaffected by any event in the replica set.

The list itself is not declared either, it is derived. The only thing written to the service object is which pods count as belonging to this name; which pods exist at any given moment comes from a measurement. This distinction is the same one that ran through the previous two lessons’ information items: the desired replica count is a declaration and does not age, the endpoint list is a measurement and has to be regenerated every round. Whether an object carries a declaration or a measurement determines how often it can be wrong.

The failed-request column, though, says the name is not enough. Without the readiness check the endpoint list always has six pods — the count is right, the content is wrong. A newly created pod cannot serve for two rounds, yet it sits in the list, and round-robin distribution sends it a request in turn. The result is forty-four of two hundred eighty-eight requests, that is, one in every seven. When the readiness check is turned on this number drops to zero; the list only shows pods that can serve.

The cost sits in the two columns right next to it. In the run with the check, the endpoint count drops to four at times and the maximum load on one endpoint rises from two to three. The check does not save requests, it redistributes them: a request that would have gone to a not-ready pod is added to the ready ones instead. The list-change count rising by three over ten is due to the same cause — a pod now enters and leaves the list twice: it enters when ready, not when created, and leaves when it drops. A stable name is bought at the cost of the list underneath it changing more often.

The source of the zero is not luck but a separation. The name sits in an object distinct from the replica set; no path that creates a pod, deletes a pod, or counts readiness ever touches that object. Without the service object, this zero would be a thirteen instead: every change in the endpoint list would mean the caller’s own address list had to change too, and keeping that list current would be a component in the verifier’s own code. The name exists precisely to remove that component.

The Cost of Resolving the Name

The name is fixed, the list is variable. The caller has to turn the name into a list before sending a request, and CO14 says this costs one round. It is possible to pay this round on every request, and it is possible to pay it once and keep the result; the space between the two is measured below.

// resolution.mjs — whether the name is resolved fresh every time or read from a cache (MODEL).
import { SCHEDULE, N, READY, ROUNDS, REQUESTS } from './service.mjs';

// Same churn schedule; each round's live set and ready-endpoint list are extracted once.
function timeline() {
  let counter = N;
  const slot = Array.from({ length: N }, (_, i) => ({ name: `pod-${i}`, readyAt: 0 }));
  const alive = [], readyList = [];
  for (let t = 0; t < ROUNDS; t++) {
    const j = SCHEDULE[t];
    if (j !== null) slot[j] = { name: `pod-${counter++}`, readyAt: t + READY };
    alive.push(new Set(slot.map((k) => k.name)));
    readyList.push(slot.filter((k) => k.readyAt <= t).map((k) => k.name));
  }
  return { alive, readyList };
}

// CO14: every resolution adds one round to the request; the result may be cached for a lifetime.
function cache(lifetime) {
  const { alive, readyList } = timeline();
  let resolves = 0, requests = 0, stale = 0, cursor = 0, list = [], last = -Infinity;
  for (let t = 0; t < ROUNDS; t++) {
    for (let i = 0; i < REQUESTS; i++) {
      if (lifetime === 0 || t - last >= lifetime) { list = readyList[t]; last = t; resolves++; }
      requests++;
      const target = list[cursor++ % list.length];
      if (!alive[t].has(target)) stale++;                // request to an endpoint dropped from the declaration
    }
  }
  return { resolves, requests, stale };
}

const s = (x, n) => String(x).padStart(n);
console.log(`${ROUNDS} rounds x ${REQUESTS} requests = ${ROUNDS * REQUESTS} requests`);
console.log(`${s('cache lifetime', 14)}${s('resolves', 11)}${s('extra rounds', 14)}`
  + `${s('extra rounds per request', 26)}${s('stale-endpoint requests', 25)}${s('rate', 8)}`);
for (const lifetime of [0, 2, 4, 8, ROUNDS]) {
  const r = cache(lifetime);
  console.log(`${s(lifetime === 0 ? 'none' : `${lifetime} rounds`, 14)}${s(r.resolves, 11)}`
    + `${s(r.resolves, 14)}${s((r.resolves / r.requests).toFixed(2), 26)}${s(r.stale, 25)}`
    + `${s(`%${(100 * r.stale / r.requests).toFixed(1)}`, 8)}`);
}

24 rounds x 12 requests = 288 requests
cache lifetime   resolves  extra rounds  extra rounds per request  stale-endpoint requests    rate
          none        288           288                      1.00                        0    %0.0
      2 rounds         12            12                      0.04                       12    %4.2
      4 rounds          6             6                      0.02                       35   %12.2
      8 rounds          3             3                      0.01                       72   %25.0
     24 rounds          1             1                      0.00                      216   %75.0

The table should be read from both ends. At the top there is no caching at all: every request resolves the name again, two hundred eighty-eight requests pay two hundred eighty-eight extra rounds, and stale-endpoint requests are zero. At the bottom the name is resolved once: extra rounds drop to one, and in exchange two hundred sixteen requests — three quarters of them — go to pods that no longer exist. The three rows in between are the curve between these two ends, and the curve is steep: when the lifetime goes from two to four, extra rounds drop to six and stale requests rise from twelve to thirty-five.

What this table says is this course’s measurement axis itself. The endpoint list is an information item, and every information item has two prices: the cost of acquiring it and the cost of it going stale. At zero lifetime the first price is paid, in the last row the second. The seventy-five-percent share of stale-endpoint requests arises in a cluster where no component has failed — it is only that the list in the caller’s hands is twenty-four rounds behind the cluster’s actual state.

The middle of the curve also says something. A two-round lifetime cuts resolution load from two hundred eighty-eight to twelve, that is, to a twenty-fourth, while keeping stale requests at four and a half percent. A four-round lifetime only halves resolution again, while tripling stale requests. At neither end do gain and loss grow at the same rate: the first step gains a lot and pays little, the later steps gain little and pay a lot. This asymmetry shows that the lifetime is a number to be chosen, and that neither end is correct.

The Devolved Decision and What It Does Not Know

The devolved decision here is clear: which pod a request goes to. This decision used to be written in the caller’s configuration; now it is read from the service object’s list. The decision rests on three information items: which pods exist, which are ready, when the list was read. The first comes free from the replica set. The second requires a check, and its cost is written in the first table: the maximum load per endpoint rises from two to three. The third also costs a round, and it sits in the second table’s first column.

What the system does not know is when the list goes stale. The model resolves this approximately with a lifetime number; the lifetime is an estimate of how often pods churn. If churn is more frequent than the estimate, stale-endpoint requests rise; if it is sparser, unnecessary resolution rounds are paid. In this run, with eleven churns over twenty-four rounds, even a two-round lifetime produces twelve stale requests; had the churn frequency been half as much, the same lifetime would have been cheaper. The source of the wrong decision here is not an arithmetic error, it is not knowing how long a number stays true.

When the three results are counted together, the table comes out like this. In the run with the check and no caching, all two hundred eighty-eight routing decisions are correct. In the run without the check, forty-four are wrong, and the source of the error is a single missing information item: whether the pod is ready. Failure to decide never appeared in this run, because the ready-endpoint count dropped to four at the lowest and never to zero. If churn frequency rose and every pod ended up waiting to become ready at the same time, the list would go empty, and the service object would hold a name with nowhere to route to; in that case the decision is neither right nor wrong, it is not made at all.

Summary

• A service object is not a piece of software, it is a name and the endpoint list behind it; the list is derived from the replica set. M16/K06’s service was a separately deployed piece of software; the object here is the name in front of that piece’s replicas.
• In twenty-four rounds eleven pods came and went and the endpoint list changed thirteen times; the name changed zero times. The only thing written in the caller’s code is the name.
• Without a readiness check, forty-four of two hundred eighty-eight requests went to a not-ready pod; with the check on, this number dropped to zero.
• The check’s cost is not lost requests but shifted load: the endpoint count drops to four, the maximum load on one endpoint rises from two to three, and the list changes thirteen times instead of ten.
• Name resolution adds one round to a request; with no caching at all, two hundred eighty-eight extra rounds are paid, and with the name resolved once and kept, extra rounds drop to one but seventy-five percent of requests go to endpoints that no longer exist.
• These two tables show an information item’s two prices: the cost of acquiring it and the cost of it going stale; choosing the lifetime is an estimate of how often pods churn.

Next Step

Every request in this lesson came from inside the cluster: the verifier called the reading collector in the same cluster. Both name resolution and round-robin distribution ran inside that boundary. Yet a request to the regional measurement network’s billing interface arrives from outside, from a client that knows nothing about the cluster. How many layers does that request have to cross, and how many rules have to be written at each layer, before it reaches a service object? The next lesson counts the layers of a single incoming external request: the round each layer adds, the relationship between rule count and service count for path-based versus name-based routing, and how many endpoints and how many paths the externally exposed surface consists of.