Pipeline Runners

The previous lesson brought pipeline duration down from 4,092 seconds to 2,273 seconds and showed the limit stopping at 203 seconds. Two silent assumptions sat under that measurement: the cache stayed in place between runs, and runners were always ready. Neither is free, and both come down to the same decision — where the pipeline runs.

There are two types to choose from. A managed runner is a machine set aside per run and torn down when the work is done; every run starts on a clean disk. A self-hosted runner is a machine that stays up, one you own; its disk persists between runs. The difference between them is not a preference, it is four separate columns: queue wait, the item paid per run, the cache’s persistence, and maintenance load.

PD31. Both types are a model set up with node; they are referred to as types, and no product or vendor name appears. PD32. Run durations are taken from the previous lesson: the managed type is 203 seconds because it starts cold on every run, the self-hosted type is 163 seconds because it keeps its cache. PD33. Arrival times are produced by a self-written generator, seed 4831; the fictional workday forms three peaks, because the fictional measurement network’s team sends its commits in the morning, in the afternoon, and at day’s close. PD34. Costs are in units; no currency is named. PD35. The measurement covers a single day; machine failure, spare capacity, and rerunning a run that finished failed do not enter the model. PD36. The gate account carries forward the previous lesson’s eight-defect set and twelve-run sequence.

Two Types, the Same Pipeline

In the model, a type is defined by four numbers: how many concurrent slots it has, how many seconds a run takes, how the run is billed, and how many maintenance items it carries.

// runner.mjs — two runner types and a day's queue (model)
export const SEED = 4831;
export const generator = (t) => () => ((t = (t * 48271) % 2147483647) / 2147483647);

// duration: three-slot run duration measured in 05 (seconds). managed starts cold every run
// (203 s), self-hosted keeps its cache and runs warm (163 s). hitRate: the rate measured in 05.
export const TYPE = {
  managed: { slots: 8, duration: 203, hitRate: 0, perMinuteCost: 0.42, fixedDaily: 0, lineItems: 1, maintenance: [] },
  selfHosted: {
    slots: 2, duration: 163, hitRate: 0.479, perMinuteCost: 0, fixedDaily: 2 * 11 + (6 * 28) / 30, lineItems: 2,
    maintenance: ["OS patching", "disk filling up", "runner version", "network and certificates", "capacity/hardware"],
  },
};

// Fictional workday: runs land in three peaks (10:00, 14:00, 17:00) under a triangular distribution.
export const arrivals = (n, rnd) => Array.from({ length: n }, () => {
  const m = [10, 14, 17][Math.floor(rnd() * 3)] * 3600;
  return Math.round(m + (rnd() + rnd() - 1) * 1200); // ±20 minutes
}).sort((a, b) => a - b);

// Slot schedule: every arrival goes to the earliest-free slot; wait = free moment - arrival moment.
export const day = (n, type, rnd) => {
  const free = Array(type.slots).fill(0), wait = [];
  for (const a of arrivals(n, rnd)) {
    const i = free.indexOf(Math.min(...free));
    const w = Math.max(0, free[i] - a);
    wait.push(w); free[i] = a + w + type.duration;
  }
  return {
    avgWait: wait.reduce((a, b) => a + b, 0) / n,
    longest: Math.max(...wait),
    waiting: wait.filter((w) => w > 0).length,
    cost: type.fixedDaily + (n * type.duration * type.perMinuteCost) / 60,
  };
};

The self-hosted type’s fixed daily cost sums two items: the daily cost of two machines and the day’s share of six hours of monthly maintenance labor. In the managed type, the fixed cost is zero and the only item is run duration. This is the real difference between the two types — one charges per run, the other per time.

Slot count sits on the same axis. In the self-hosted type, a slot is a purchased machine, and the count is capped at two; in the managed type, eight slots is a ceiling the model declares, not a machine. This ceiling itself is counted as a line item — eight concurrent runs are not free, where that number comes from is declared in the model, and it is a matter of negotiation in a real setup.

One Day’s Queue

Same pipeline, same arrival times, two types. It runs at six separate intensities: from 10 runs a day to 80.

// threshold.mjs — the same pipeline on two runner types: wait, cost, threshold
import { TYPE, SEED, generator, day } from "./runner.mjs";

const min = (s) => (s / 60).toFixed(1);
console.log(`seed ${SEED}; slots: managed ${TYPE.managed.slots}, self-hosted ${TYPE.selfHosted.slots}`);
console.log("\n              managed (cold, 203 s)          self-hosted (warm, 163 s)");
console.log("runs/day    wait   longest  waiting   cost    wait   longest  waiting   cost");
for (const n of [10, 20, 30, 40, 60, 80]) {
  const y = day(n, TYPE.managed, generator(SEED)), b = day(n, TYPE.selfHosted, generator(SEED));
  console.log(String(n).padStart(7) + [min(y.avgWait) + " m", min(y.longest) + " m", `${y.waiting}/${n}`,
    y.cost.toFixed(1), min(b.avgWait) + " m", min(b.longest) + " m", `${b.waiting}/${n}`, b.cost.toFixed(1)]
    .map((h, i) => h.padStart(i === 4 ? 12 : 9)).join(""));
}

let threshold = 0;
while (day(++threshold, TYPE.managed, generator(SEED)).cost <= day(threshold, TYPE.selfHosted, generator(SEED)).cost);
const y = day(threshold, TYPE.managed, generator(SEED)), b = day(threshold, TYPE.selfHosted, generator(SEED));
console.log(`\nthreshold: at ${threshold} runs/day managed rises to ${y.cost.toFixed(1)} units, ` +
  `self-hosted stays flat at ${b.cost.toFixed(1)} units.`);
console.log(`cost line items per run: managed ${TYPE.managed.lineItems} (runner-minutes), ` +
  `self-hosted ${TYPE.selfHosted.lineItems} (machine + maintenance labor)`);
console.log(`maintenance line items: managed ${TYPE.managed.maintenance.length}, self-hosted ` +
  `${TYPE.selfHosted.maintenance.length} — ${TYPE.selfHosted.maintenance.join(", ")}`);

// Gate account: in 05 the cache had missed two defects (D4, D6). A runner that starts clean
// runs every gate on every run, so those two defects do not escape.
const seq = 12, diff = TYPE.managed.duration - TYPE.selfHosted.duration;
console.log(`\ncache hit rate: managed ${TYPE.managed.hitRate * 100}%, self-hosted ${(TYPE.selfHosted.hitRate * 100).toFixed(1)}%`);
console.log(`gate account (${seq}-run sequence): managed 8/8 defects, self-hosted 6/8 (D4, D6 stale hit)`);
console.log(`cost of the two defects: ${diff} s per run, ${seq * diff} s over the sequence ` +
  `(${min(seq * diff)} m); at 30 runs/day ${min(30 * diff)} m and ` +
  `${(day(30, TYPE.managed, generator(SEED)).cost - day(30, TYPE.selfHosted, generator(SEED)).cost).toFixed(1)} unit difference`);

// Managed type + remote cache store: the hit rate comes back, transfer time and storage cost are added.
const TRANSFER = 24, STORAGE = 3;
const remote = { ...TYPE.managed, duration: TYPE.selfHosted.duration + TRANSFER, fixedDaily: STORAGE };
console.log(`\nmanaged type + remote cache store: run ${remote.duration} s, at 30 runs/day ` +
  `${day(30, remote, generator(SEED)).cost.toFixed(1)} units — but the stale hit comes back, so defects 6/8`);

seed 4831; slots: managed 8, self-hosted 2

              managed (cold, 203 s)          self-hosted (warm, 163 s)
runs/day    wait   longest  waiting   cost    wait   longest  waiting   cost
     10    0.0 m    0.0 m     0/10     14.2       0.1 m    0.6 m     1/10     27.6
     20    0.0 m    0.0 m     0/20     28.4       0.1 m    1.1 m     2/20     27.6
     30    0.0 m    0.0 m     0/30     42.6       1.0 m    5.3 m    12/30     27.6
     40    0.0 m    0.0 m     0/40     56.8       1.5 m    8.0 m    21/40     27.6
     60    0.0 m    1.0 m     2/60     85.3       4.3 m   15.1 m    44/60     27.6
     80    0.1 m    1.0 m     8/80    113.7       6.6 m   20.9 m    63/80     27.6

threshold: at 20 runs/day managed rises to 28.4 units, self-hosted stays flat at 27.6 units.
cost line items per run: managed 1 (runner-minutes), self-hosted 2 (machine + maintenance labor)
maintenance line items: managed 0, self-hosted 5 — OS patching, disk filling up, runner version, network and certificates, capacity/hardware

cache hit rate: managed 0%, self-hosted 47.9%
gate account (12-run sequence): managed 8/8 defects, self-hosted 6/8 (D4, D6 stale hit)
cost of the two defects: 40 s per run, 480 s over the sequence (8.0 m); at 30 runs/day 20.0 m and 15.0 unit difference

managed type + remote cache store: run 187 s, at 30 runs/day 42.3 units — but the stale hit comes back, so defects 6/8

Threshold and Queue

If the cost column is read alone, the decision looks clear: the threshold is 20 runs a day. Below it, the managed type is cheaper — 14.2 units against 27.6 units at 10 runs a day. Above it, the gap widens fast: 113.7 units against 27.6 at 80 runs a day, 4.1 times. The side paying a fixed cost gains as run count grows, because what it pays for is not the run, but time.

When the same numbers are read per run, the curve becomes visible. In the managed type, cost per run is 1.42 units at every intensity and never changes — 14.2/10 and 113.7/80 give the same number. In the self-hosted type, cost per run drops from 2.76 at 10 runs, to 1.38 at 20, to 0.35 units at 80. This also separates the behavior the two types produce on a team: under a fixed cost, running one more run is close to free, and nobody counts runs; under per-run billing, every extra run has a price, and unnecessary runs show up in the account. This same measure will feed into trigger design in a later lesson.

The queue column reverses the decision. The two-slot self-hosted type starts clogging at 30 runs a day: 12 of 30 runs wait, the average wait is 1 minute, the longest wait is 5.3 minutes. At 80 runs a day, 63 runs wait, averaging 6.6 minutes with the longest wait at 20.9 minutes. The eight-slot managed type, on the same arrivals, makes only 8 of 80 runs wait, and the longest wait is 1 minute. The total wait is striking: at 80 runs, 6.6 × 80 = 528 minutes, more than a full workday of waiting, against a total of 5 minutes for the managed type.

It is also worth counting that the queue does not come from the daily average. At 30 runs a day, total work is 30 × 163 = 4,890 seconds; two slots over a nine-hour day offer 64,800 seconds of capacity, that is, 8 percent utilization. Yet 12 runs wait. The reason is the shape of the arrivals: runs land in three peaks, each within a twenty-minute window, and at the center of a peak both slots are full at once. If the capacity decision is made against the daily total, the queue never shows up; what the decision needs to look at is the busiest twenty minutes.

This wait feeds directly into the gate account. The “time a gate adds,” which the previous lesson measured, was only pipeline duration; the queue is added in front of it. At 80 runs a day on the self-hosted type, a change’s feedback averages 163 + 396 seconds — two and a half times the run duration in waiting. The effort spent shortening the pipeline by 40 seconds becomes invisible next to what is lost in the queue.

The queue can be closed with money, and the arithmetic is simple. If four machines are kept instead of two, the fixed cost rises to 4 × 11 + 5.6 = 49.6 units, still less than half of the 113.7 units at 80 runs a day. The threshold thus stops being a single number: the decision sits at the intersection of run count and an acceptable wait time.

The Gate Value of a Clean Start

The cache column is where the two types are really split apart. The self-hosted runner keeps the 47.9 percent hit rate the previous lesson measured; the managed runner starts clean on every run, and its hit rate is zero. The 40-second-per-run difference comes from here.

The gate account shows the other side of this difference. In the previous lesson, two defects (D4, D6) escaped, because when an undeclared input changed, the gate in question came from the cache and never ran at all. On a runner with no cache, that cannot happen: every gate runs on every run, and in the twelve-run sequence, eight of eight defects are caught. The two defects cost 40 seconds per run, 480 seconds over the sequence, 20 minutes and 15 units at 30 runs a day.

A middle path can be tried: the managed type can run by restoring its cache from a remote store. The measurement counts this option too — the run drops to 187 seconds, costing 42.3 units at 30 runs a day. The gain is not as large as advertised: the drop from 203 to 187 seconds is paid for with transfer time and storage cost, so the daily cost falls only from 42.6 to 42.3. More importantly, the stale hit comes back — the defect count reverts from 8/8 to 6/8. The managed type’s two-defect edge was not born from its speed at all; it was born exactly from its lack of a cache.

When these three columns come together, the decision rule does not collapse into one sentence, but it does fall into an order. If run count is below the threshold, the managed type is both cheaper and ahead in the gate account; the choice is not up for debate. If it is clearly above the threshold, the money points to the self-hosted type, but two items still have to be paid back: the extra slots needed to close the queue, and the defect the stale hit misses. The second cannot be closed with money; the way to close it is where the previous lesson pointed — declaring gates’ inputs completely. In the twelve-run sequence, that cost 54 seconds; catching the same two defects by changing runner type instead costs 480 seconds. The same defect, a nine-times more expensive fix.

What the measure leaves out is named. On the managed side: moments when the concurrency ceiling is exceeded, data transfer fees, the pipeline stopping entirely during a provider outage, the pipeline’s secret values being resolved outside it, and the pipeline definition being adapted when the runner version changes. On the self-hosted side: the initial setup effort, spare machines, days lost to failure, delayed security patches, the electricity and floor-space cost of idle capacity, and capacity-planning error. Both sides share two uncounted items: the context loss of the person waiting in the queue, and rerunning a run that finished failed. Five maintenance items are listed, but they are priced only for six hours of labor a month; a month where maintenance is skipped never shows up in this column.

Summary

• The threshold is 20 runs a day: below it the managed type is cheaper (14.2 against 27.6 units at 10 runs), above it the self-hosted type is cheaper (27.6 against 113.7 units at 80 runs, 4.1 times).
• The threshold does not decide alone. At 30 runs a day, the two-slot self-hosted type makes 12 runs wait; at 80 runs, 63 runs wait, averaging 6.6 minutes with a longest of 20.9. On the same arrivals, the eight-slot managed type makes only 8 runs wait, up to 1 minute.
• The queue is written in front of the time a gate adds: at 80 runs a day, the wait is two and a half times the 163-second run. Moving to four machines brings the fixed cost to 49.6 units and closes the queue with money.
• The cost items sit on different axes: one item for the managed type (runner-minutes, per run), two items for the self-hosted type (machine and maintenance labor, per time), plus five maintenance items.
• A clean-starting runner zeroes the cache hit rate (47.9 percent against 0) and catches 8/8 defects in the twelve-run sequence; the self-hosted type stays at 6/8. The two defects cost 40 seconds per run, 20 minutes and 15 units at 30 runs a day.
• The remote cache store middle path brings the run down to 187 seconds but brings the stale hit back, and the defect count reverts to 6/8; the managed type’s edge comes not from its speed but from its lack of a cache.

Next Step

In this measurement the pipeline always ran the same pipeline — eight stages, the same dependencies, the same gates. One question went unasked: where does this definition live, and who changes it? Raising the slot count from two to four, removing a gate, or splitting the integration test all touch the pipeline’s definition. If the definition lives as something clicked through in an interface, who made the change, when, and why is written down nowhere; and when the pipeline breaks, there is no prior state to go back to. If the definition lives as a file in the repository, all of that exists — but a file per service for five services means the same lines repeated five times. The next lesson defines the schema itself and counts two forms: the traceability of a change, the fraction of pipeline changes that go through without review, the rollback step when the pipeline itself breaks, and the line cost of duplicating the definition.