Indicator, Objective, and Agreement

The previous topic joined the three signals in a single event: which process a request passed through, where it waited, where it was lost, all now readable from a single correlation id. The one thing missing is the sentence that says whether that record is good or bad. Telemetry produces observation, not judgment. Knowing that the loan service answered a request in 380 milliseconds does not say whether that request counts as successful.

This lesson lays the first stone of the decision layer. Three concepts that are often confused with one another are defined separately, then three separate indicator definitions are derived from the same telemetry, and the lesson counts how many different numbers the same thirty days can be described with.

Three Concepts, Three Separate Sentences

A service level indicator is a measured ratio. It is computed from telemetry and carries no commitment by itself: “what percentage of the loan system’s requests are good” is an indicator.

A service level objective is the internal commitment placed on top of that indicator, and it has three parts: a definition, a window, a number. “Over a thirty-day window, the request-based indicator does not drop below 99.5%” is an objective. No one is paid anything in return, but what the team does when the objective is missed is written down in advance.

A service level agreement is the commitment given to the outside world, and it carries a penalty: if it is violated, a refund, a membership extension, or some other cost follows. An agreement is always looser than the objective, because there has to be time left to fix things after the objective is missed.

The question that separates the three is this: the indicator answers “what is being measured”, the objective answers “when does someone intervene”, and the agreement answers “what is paid if it is violated”. All three draw on the same telemetry. That is why, when the indicator’s definition changes, the meaning of the other two changes with it — and that is exactly what this lesson measures.

The Thirty-Day Log

The measurement comes from the split loan system’s request log. The generator below writes thirty days of requests across five services using seeded integer arithmetic; four degradation windows and one scheduled maintenance window are placed on known schedules.

// slo/telemetry.mjs — the split loan system's 30-day request log. Generated with seeded
// integer arithmetic. Record format: [minute, service, status, duration_ms, member, type].
export const MIN = 30 * 1440, MEMBER = 4000, SEED = 20260731;
export const SERVICE = ["catalog", "membership", "loan", "notification", "fee"];
const BASE = [40, 25, 60, 30, 45];
// SD1 (assumption): requests per minute follow the daily rhythm (40 between 08-22, 10 outside it).
// SD2 (assumption): four incidents [day, hour, duration min, service (-1 all), mode, severity (per mille)].
export const EVENT = [
  [4, 8, 90, 2, "error", 60], [11, 14, 240, 0, "slow", 1000],
  [19, 2, 30, -1, "error", 600], [26, 20, 120, 3, "error", 250],
];
// SD3 (assumption): the scheduled maintenance window is day 15's 03:00-04:00 slot.
export const MAINTENANCE = [15 * 1440 + 180, 60];

export function records() {
  let s = SEED;
  const rand = (n) => Math.floor(((s = (s * 1664525 + 1013904223) >>> 0) / 2 ** 32) * n);
  const out = [];
  for (let t = 0; t < MIN; t += 1) {
    const hour = Math.floor(t / 60) % 24;
    const maintenance = t >= MAINTENANCE[0] && t < MAINTENANCE[0] + MAINTENANCE[1];
    for (let i = 0, n = hour >= 8 && hour < 22 ? 40 : 10; i < n; i += 1) {
      const sv = rand(5);
      let status = 200, duration = BASE[sv] + rand(60);
      if (rand(1000) < 8) status = 404;
      if (rand(1000) < 1) status = 500;
      if (rand(1000) === 0) duration += 400 + rand(600);
      for (const [d, h, dr, sd, mode, sev] of EVENT) {
        const start = d * 1440 + h * 60;
        if (t < start || t >= start + dr || (sd >= 0 && sd !== sv)) continue;
        if (mode === "error" && rand(1000) < sev) status = 500;
        if (mode === "slow") duration += 300 + rand(sev);
      }
      if (maintenance) status = 503;
      out.push([t, SERVICE[sv], status, duration, rand(MEMBER), rand(50) === 0 ? "crawler" : "human"]);
    }
  }
  return out;
}

if (import.meta.filename === process.argv[1]) {
  const R = records();
  const count = (f) => R.filter(f).length;
  console.log(`seed ${SEED}, ${MIN} minutes, ${R.length} requests, ${MEMBER} members`);
  console.log(`status 5xx ${count((r) => r[2] >= 500)}, status 4xx ${count((r) => r[2] >= 400 && r[2] < 500)}, ` +
    `duration > 400 ms ${count((r) => r[3] > 400)}, crawler ${count((r) => r[5] === "crawler")}`);
}

seed 20260731, 43200 minutes, 1188000 requests, 4000 members
status 5xx 2214, status 4xx 9457, duration > 400 ms 3087, crawler 23868

Three Indicators from the Same Telemetry

The definition of a bad request is shared by all three indicators: a request is bad if its status code is 400 or above, or its duration exceeds 400 milliseconds. What changes is what plays the numerator and what plays the denominator.

• Request-based: good request count divided by total request count.
• Minute-based: a minute is good if its bad ratio is under 5%; the indicator is good-minute count divided by populated-minute count.
• Member-based: a member-day is good if at most 1% of that member’s requests that day are bad; the indicator is the thirty-day average of the daily good-member ratio.

Four filters are layered on top of this. A filter states what the indicator does not count: crawler requests, requests with a client error, and the scheduled maintenance window are excluded in turn.

// slo/indicator.mjs — three indicator definitions and four filters from the same telemetry.
// Bad request: status 400 and above, or duration above 400 ms. The three definitions' results
// are compared against the 99.5% objective.
import { records, MIN, MEMBER, MAINTENANCE } from "./telemetry.mjs";
const ALL = records(), OBJECTIVE = 0.995, pct = (x) => `%${(x * 100).toFixed(3)}`;
const bad = (r) => r[2] >= 400 || r[3] > 400;

const requestBased = (rows) => 1 - rows.filter(bad).length / rows.length;

function minuteBased(rows) {                  // one minute: good if the bad ratio is under 5%
  const total = new Int32Array(MIN), badC = new Int32Array(MIN);
  for (const r of rows) { total[r[0]] += 1; if (bad(r)) badC[r[0]] += 1; }
  let good = 0, filled = 0;
  for (let t = 0; t < MIN; t += 1) if (total[t] > 0) { filled += 1; if (badC[t] <= total[t] * 0.05) good += 1; }
  return good / filled;
}

function memberBased(rows) {                  // one member-day: good if the bad ratio is under 1%
  let sum = 0, reqSum = 0, dayCount = 0;
  for (let d = 0; d < 30; d += 1) {
    const total = new Int32Array(MEMBER), badC = new Int32Array(MEMBER);
    for (const r of rows) {
      if (Math.floor(r[0] / 1440) !== d) continue;
      total[r[4]] += 1; if (bad(r)) badC[r[4]] += 1;
    }
    let active = 0, good = 0, n = 0;
    for (let u = 0; u < MEMBER; u += 1) if (total[u] > 0) {
      active += 1; n += total[u]; if (badC[u] <= total[u] * 0.01) good += 1;
    }
    sum += good / active; reqSum += n / active; dayCount += 1;
  }
  return [sum / dayCount, reqSum / dayCount];
}

const FILTER = [
  ["raw", () => true],
  ["crawler excluded", (r) => r[5] !== "crawler"],
  ["+ client error excluded", (r) => r[5] !== "crawler" && (r[2] < 400 || r[2] >= 500)],
  ["+ maintenance excluded", (r) => r[5] !== "crawler" && (r[2] < 400 || r[2] >= 500)
    && (r[0] < MAINTENANCE[0] || r[0] >= MAINTENANCE[0] + MAINTENANCE[1])],
];

console.log(`${"filter".padEnd(24)}${"requests".padStart(9)}${"excluded".padStart(9)}` +
  `${"request-based".padStart(15)}${"minute-based".padStart(16)}${"member-based".padStart(13)}` +
  `${"over objective".padStart(15)}`);
let memberReq = 0;
for (const [name, f] of FILTER) {
  const rows = ALL.filter(f), [mb, n] = memberBased(rows);
  const u = [requestBased(rows), minuteBased(rows), mb];
  memberReq = n;
  console.log(`${name.padEnd(24)}${String(rows.length).padStart(9)}` +
    `${String(ALL.length - rows.length).padStart(9)}` +
    `${u.map((v, i) => pct(v).padStart([15, 16, 13][i])).join("")}` +
    `${`${u.filter((v) => v >= OBJECTIVE).length}/3`.padStart(15)}`);
}
console.log(`objective ${pct(OBJECTIVE)}; same 30 days, same ${ALL.length} records, 12 numbers`);
console.log(`average requests per member per day ${memberReq.toFixed(1)}: ` +
  `the 1% member threshold allows ${Math.floor(memberReq * 0.01)} bad requests`);

filter                   requests excluded  request-based    minute-based member-based over objective
raw                       1188000        0        %98.760         %94.655      %88.763            0/3
crawler excluded          1164132    23868        %98.761         %93.074      %88.980            0/3
+ client error excluded   1154873    33127        %99.553         %98.123      %96.089            1/3
+ maintenance excluded    1154280    33720        %99.604         %98.259      %96.540            1/3
objective %99.500; same 30 days, same 1188000 records, 12 numbers
average requests per member per day 9.6: the 1% member threshold allows 0 bad requests

What the Definition Counts

Without a single record changing, the same thirty days produced twelve numbers, spread between 88.763% and 99.604% — a range of 10.8 points. Only two of the twelve numbers clear the 99.5% objective. The sentence “we hit the objective” cannot be verified until the definition is written down.

There is no relationship between how many records are excluded and how much the indicator moves. Crawler requests excluded 23,868 records and moved the request-based indicator from 98.760% to 98.761%; client errors excluded 9,259 records and jumped the same indicator by 0.79 points. What decides the outcome is not how many records are excluded, but whether they were bad. The maintenance window’s 593 records added 0.05 points — a small number, but once the distance to the objective narrows to 0.05 points, whether these records are excluded decides the result.

Excluding crawler requests lowered the minute-based indicator (from 94.655% to 93.074%). Since the excluded requests were not bad, this looks backwards; the reason lies in the denominator. In a minute with forty requests, two bad requests make exactly 5%, and the minute counts as good. Once one crawler request is excluded, the denominator becomes 39, the same two requests make 5.1%, and the minute turns bad. Excluding good requests distorts a ratio-based indicator: a filter raises the indicator when it excludes bad requests, and lowers it when it excludes good ones.

The member-based indicator is the strictest of the three: where the request-based one says 99.553%, it says 96.089%. The source of the strictness is not the threshold but the denominator. An average of 9.6 requests falls to each member per day, and a 1% threshold applied to that denominator allows zero bad requests; in practice, the definition becomes “the member sees no bad request that day at all”. The three definitions measure three separate faces of the same event: the request-based one measures its volume, the minute-based one its duration, the member-based one its spread.

The Gap Between Objective and Agreement

An objective cannot be measured unless it is written together with its definition. “99.5%” by itself does not say which of the twelve numbers above is being looked at, and depending on which number is looked at, the same month can be both hit and missed. The objective’s three written parts — definition, window, number — must all be present at once.

An agreement demands one more thing: the other party must be able to compute the same number. An agreement cannot be written on a member-based indicator, because a member does not see their own member-days. This is why an agreement uses the coarsest and most verifiable definition: request-based, crawler and client error excluded, maintenance window excluded. If the loan system sets the agreement at 99.0% and the objective at 99.5%, the 0.5-point gap between them is the intervention zone; when the objective is missed, no penalty has come due yet, and there is still room left in the month to fix it.

The counterpart of this distinction in code is the FILTER array. Every exclusion sentence in the agreement’s text is a row in that array: four filter levels, three exclusion rules. If what the text says and what the array holds drift apart, the committed number and the measured number diverge — if the maintenance window’s 593 records are not excluded in the text, the agreement looks violated even though the measurement is working correctly.

Summary

• An indicator is the measured ratio, an objective is the internal commitment placed on that ratio, and an agreement is the external commitment that carries a penalty; an agreement is always looser than the objective.
• The same thirty-day log of 1,188,000 records produced twelve numbers from three definitions and four filters: between 88.763% and 99.604%, and only two of them clear the 99.5% objective.
• The effect of an exclusion is measured not by how many records it excludes but by whether those records were bad: 23,868 crawler records moved the indicator by 0.001 points, 9,259 client errors moved it by 0.79 points.
• Excluding good requests lowers the minute-based indicator, because it shrinks the denominator: from 94.655% to 93.074%.
• With 9.6 requests falling to each member per day, the member-based indicator turns the 1% threshold into zero tolerance; the three definitions measure the same event’s volume, duration, and spread.
• The agreement’s exclusion list turns into code; if the filter array and the agreement text drift apart, the measured number and the committed number drift apart with them.

Next Step

All three definitions here were built on a single “bad request” measure: status code or duration. But in the loan system, that is not the only way a request can go bad. The catalog service can slow down without ever returning an error, the notification service can fill its queue while running error-free, and the fee service receiving no requests at all can be a failure signal on its own. All of this is written in the same telemetry, but an indicator that looks at a single ratio cannot see part of it. The next lesson computes four separate signals from the same records and counts what each signal alone fails to catch.