Feedback Loops

The previous lesson counted how the budget rule saw the incident only after it happened, and why the measured effect stayed small because of that. How late the signal arrives has come up somewhere in every lesson of this topic: 7.0 steps when observation scope was split, 61.0 steps for context dropped at the boundary, six steps back for a drift found at production verification. But the source of the delay — the loop itself — was never counted directly.

A feedback loop is the path from the moment an error is born to the moment it reaches the role that can fix it. A flow does not have a single loop; it has several, each with its own length and its own coverage. Detection delay was measured in the Process, Team and Delivery course and the Testing Process and Automation Infrastructure course; what is measured here is not that delay but the loop itself: how many there are, how long each one runs, which error class it covers, and what it costs to move a class to an earlier loop.

Building the Loops

The example is again the fictional regional measurement network; the network and the loops are both fictional. CF32: there are four loops — writing, integration, staging, production; each has a length, and the lengths are cumulative, because the signal only returns to the development role at that link in the chain. CF33: there are eight error classes and 120 errors are born in the period; the counts per class are fixed. CF34: every loop has the classes it covers and a catch rate for each class; the production loop catches every class, because that is where the person using the error finds it. CF35: randomness is written with a generator, the seed is 20260806; one die per loop is rolled in advance for every error, so an error’s luck does not change across move runs. CF36: 120 errors are distributed to 60 changes in order; a change’s lead time is the staging cumulative plus the loop cumulative of every one of its errors caught before production. CF37: moving an error class to an earlier loop adds a check to that loop: the loop’s length grows, the cumulative of the later loops shifts too, and the class is caught at that loop with a lower rate.

The loops, the classes, and the move are a process-internal model; there is no real pipeline or error record.

// loop/loop.mjs — the model of the feedback loops (model): there is no real pipeline or error
// record, minutes are a data structure. Randomness is written with a generator, the seed is visible.

export const N_CHANGES = 60, SEED = 20260806;
export const LOOP = [["writing", 2], ["integration", 28], ["staging", 210],
  ["production", 4080]];                           // name and its increase over the previous loop

// error class: how many are born in the period
export const CLASS = { syntax: 24, type: 18, "unit-logic": 22, integration: 14,
  "data-format": 12, configuration: 12, scheduling: 10, resource: 8 };

// coverage: loop -> class -> catch rate. The production loop catches every class.
export const COVERAGE = {
  writing: { syntax: 1.00, type: 0.90, "unit-logic": 0.20 },
  integration: { type: 1.00, "unit-logic": 0.85, integration: 0.35 },
  staging: { "unit-logic": 0.60, integration: 0.85, "data-format": 0.55,
    configuration: 0.35, scheduling: 0.20, resource: 0.15 },
  production: Object.fromEntries(Object.keys(CLASS).map((s) => [s, 1.00])),
};

export function rng(seed) {                        // linear congruential generator
  let x = seed >>> 0;
  return () => ((x = (x * 1664525 + 1013904223) >>> 0) / 2 ** 32);
}

// move: shifts an error class to an earlier loop; that loop's length grows by the extra.
export function run(moves = []) {
  const increase = LOOP.map(([, a]) => a);
  const coverage = Object.fromEntries(LOOP.map(([ad]) => [ad, { ...COVERAGE[ad] }]));
  for (const { cls, loop, rate, extra } of moves) {
    coverage[loop][cls] = rate;
    increase[LOOP.findIndex(([ad]) => ad === loop)] += extra;
  }
  const cumulative = increase.map((_, i) => increase.slice(0, i + 1).reduce((s, v) => s + v, 0));
  const roll = rng(SEED);
  const ERR = Object.entries(CLASS).flatMap(([cls, n]) =>
    Array.from({ length: n }, () => ({ cls, rolls: LOOP.map(() => roll()) })));
  ERR.forEach((h, i) => { h.change = i % N_CHANGES; });   // errors are assigned to changes in order

  for (const h of ERR) {                            // which loop catches the error
    h.loop = null;
    for (let i = 0; i < LOOP.length; i++) {
      const o = coverage[LOOP[i][0]][h.cls] ?? 0;
      if (h.rolls[i] < o) { h.loop = i; h.delay = cumulative[i]; break; }
    }
  }
  const CHANGE = Array.from({ length: N_CHANGES }, (_, i) => {
    const errors = ERR.filter((h) => h.change === i);
    const early = errors.filter((h) => h.loop < LOOP.length - 1);
    return { i, errors, escaped: errors.filter((h) => h.loop === LOOP.length - 1).length,
      lead: cumulative[LOOP.length - 2] + early.reduce((s, h) => s + cumulative[h.loop], 0) };
  });
  return { ERR, CHANGE, cumulative, increase, coverage };
}

export const avg = (a) => (a.length ? (a.reduce((s, v) => s + v, 0) / a.length).toFixed(1) : "-");

export function measure(moves = []) {
  const { ERR, CHANGE, cumulative, increase, coverage } = run(moves);
  const escaped = ERR.filter((h) => h.loop === LOOP.length - 1);
  const leads = CHANGE.map((d) => d.lead);
  return { ERR, CHANGE, cumulative, increase, coverage,
    totalDelay: ERR.reduce((s, h) => s + h.delay, 0),
    avgDelay: avg(ERR.map((h) => h.delay)), escaped: escaped.length,
    leadTime: avg(leads),
    deployFrequency: (1440 / (leads.reduce((s, v) => s + v, 0) / CHANGE.length)).toFixed(2),
    changeFailureRate: (100 * CHANGE.filter((d) => d.escaped > 0).length / N_CHANGES).toFixed(0) + "%",
    recovery: cumulative[LOOP.length - 1] };
}

// loop/measure.mjs — how many loops there are, how long they run, which error class they cover; the cost of moving one.
import { LOOP, CLASS, COVERAGE, SEED, measure, avg } from "./loop.mjs";

const print = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const T = measure();
const total = Object.values(CLASS).reduce((s, v) => s + v, 0);

console.log(`${LOOP.length} feedback loops, ${Object.keys(CLASS).length} error classes, ` +
  `${total} errors, seed ${SEED}`);

const A = [-15, 10, 12, 9, 11, 12];
console.log("\n1. the loops: length and coverage");
print(A, "loop", "length", "cumulative", "classes", "caught", "caught %");
LOOP.forEach(([ad], i) => {
  const g = T.ERR.filter((h) => h.loop === i);
  print(A, ad, T.increase[i], T.cumulative[i], Object.keys(COVERAGE[ad]).length, g.length,
    (100 * g.length / total).toFixed(0) + "%");
});

const B = [-14, 9, 18, 14, 23];
console.log("\n2. which loop an error class falls into");
print(B, "class", "errors", "first covered by", "avg. delay", "escaped to production");
for (const [s, n] of Object.entries(CLASS)) {
  const g = T.ERR.filter((h) => h.cls === s);
  const first = LOOP.findIndex(([ad]) => (COVERAGE[ad][s] ?? 0) > 0);
  print(B, s, n, LOOP[first][0], avg(g.map((h) => h.delay)),
    g.filter((h) => h.loop === LOOP.length - 1).length);
}

const K = {
  "baseline": [],
  "data-format to integration": [{ cls: "data-format", loop: "integration",
    rate: 0.45, extra: 12 }],
  "configuration to integration": [{ cls: "configuration", loop: "integration",
    rate: 0.30, extra: 8 }],
  "scheduling to integration": [{ cls: "scheduling", loop: "integration",
    rate: 0.10, extra: 20 }],
  "all three at once": [{ cls: "data-format", loop: "integration", rate: 0.45, extra: 12 },
    { cls: "configuration", loop: "integration", rate: 0.30, extra: 8 },
    { cls: "scheduling", loop: "integration", rate: 0.10, extra: 20 }],
};
const R = Object.fromEntries(Object.entries(K).map(([ad, t]) => [ad, measure(t)]));

const C = [-30, 13, 16, 14, 23];
console.log("\n3. the cost of moving an error class to an earlier loop");
print(C, "run", "integration", "total delay", "avg. delay", "escaped to production");
for (const [ad, r] of Object.entries(R))
  print(C, ad, r.increase[1], r.totalDelay, r.avgDelay, r.escaped);

const E = [-27, 17, 18, 17];
console.log("\n4. what happened to the moved class itself (baseline values in parentheses)");
print(E, "class (added time)", "caught early", "avg. delay", "still escapes");
for (const [ad, t] of Object.entries(K).slice(1, 4)) {
  const s = t[0].cls, r = R[ad], g = r.ERR.filter((h) => h.cls === s);
  const b = T.ERR.filter((h) => h.cls === s), last = LOOP.length - 1;
  print(E, `${s} (+${t[0].extra} min)`, `${g.filter((h) => h.loop === 1).length}/${g.length}`,
    `${avg(g.map((h) => h.delay))} (${avg(b.map((h) => h.delay))})`,
    `${g.filter((h) => h.loop === last).length} (${b.filter((h) => h.loop === last).length})`);
}

const F = [-30, 12, 12, 16, 12];
console.log("\n5. four delivery metrics");
print(F, "run", "deploy", "lead time", "change failure", "recovery");
print(F, "", "(chg./day)", "(minutes)", "(change)", "(minutes)");
for (const [ad, r] of Object.entries(R))
  print(F, ad, r.deployFrequency, r.leadTime, r.changeFailureRate, r.recovery);

4 feedback loops, 8 error classes, 120 errors, seed 20260806

1. the loops: length and coverage
loop               length  cumulative  classes     caught    caught %
writing                 2           2        3         42         35%
integration            28          30        3         21         18%
staging               210         240        6         25         21%
production           4080        4320        8         32         27%

2. which loop an error class falls into
class            errors  first covered by    avg. delay  escaped to production
syntax               24           writing           2.0                      0
type                 18           writing           5.1                      0
unit-logic           22           writing         241.5                      1
integration          14       integration         792.9                      2
data-format          12           staging        2280.0                      6
configuration        12           staging        3300.0                      9
scheduling           10           staging        3096.0                      7
resource              8           staging        3810.0                      7

3. the cost of moving an error class to an earlier loop
run                             integration     total delay    avg. delay  escaped to production
baseline                                 28          144954        1208.0                     32
data-format to integration               40          128520        1071.0                     28
configuration to integration             36          136578        1138.2                     30
scheduling to integration                48          142224        1185.2                     31
all three at once                        68          117414         978.5                     25

4. what happened to the moved class itself (baseline values in parentheses)
class (added time)              caught early        avg. delay    still escapes
data-format (+12 min)                   5/12    844.5 (2280.0)            2 (6)
configuration (+8 min)                  4/12   2558.0 (3300.0)            7 (9)
scheduling (+20 min)                    1/10   2687.0 (3096.0)            6 (7)

5. four delivery metrics
run                                 deploy   lead time  change failure    recovery
                                (chg./day)   (minutes)        (change)   (minutes)
baseline                              4.09       351.9             53%        4320
data-format to integration            3.87       372.4             47%        4332
configuration to integration          4.00       360.3             50%        4328
scheduling to integration             3.71       388.1             52%        4340
all three at once                     3.43       420.2             42%        4360

The numbers are of the measurement kind; their inputs are the assumptions above.

Four Loops, Four Lengths

The first table lays the loops side by side. The lengths are 2, 28, 210, and 4080 minutes; cumulatively 2, 30, 240, and 4320. The ratio between the two ends is 2160. The distribution of catches, however, is not even: of 120 errors, 42 are caught while writing (35%), 21 at integration (18%), 25 at staging (21%), and 32, that is a quarter of them, fall into none of the first three loops.

The classes column explains why. The writing loop covers three classes, integration covers three, staging covers six; the production loop covers all eight, because there is no filtering rate there. If an error class is not in any early loop’s coverage, the loops’ lengths stop mattering at all.

Total delay is 144954 minutes, and 138240 of that — thirty-two times 4320 — comes from errors that escape to production alone. A quarter of the errors carry 95% of the delay.

Where Each Class Is Found

The second table breaks the same number down by class, and the average delay stretches from 2.0 to 3810.0 minutes. Syntax and type errors finish in the first two loops (2.0 and 5.1 minutes) and none of them escape to production. Unit-logic is in the coverage of three separate loops, its average is 241.5 minutes, and only one of its 22 errors escapes.

Of the 32 escaped errors, 29 come from the four classes first covered at staging: configuration 9, scheduling 7, resource 7, data format 6. These four classes’ catch rates at staging are 0.35, 0.20, 0.15, and 0.55; in other words, staging is not a loop for these classes, it is a sieve. The later a class is covered, the lower the catch rate where it is covered — because if a class is covered late, it depends on something that does not show up in the earlier environment.

The Cost of a Move and What It Misses

The third table measures three moves. Moving the data-format class to the integration loop adds 12 minutes to that loop; total delay falls from 144954 to 128520 minutes (an 11% decrease), and errors escaping to production fall from 32 to 28. The fourth table shows the counterpart of this in the class itself: only five of the twelve errors are caught early, the class’s average delay falls from 2280.0 to 844.5 minutes, and two errors still escape (six at baseline).

The scheduling class is the limit of this decision. Moving it to the integration loop demands 20 minutes — the most expensive of the three moves — and gains only one of ten errors; the number that escapes falls from 7 to 6, and total delay drops by only 2730 minutes. The batch window does not exist in the integration environment, so the rate is 0.10. The most expensive move is the one that gains the least, and the reason is written in the rate.

The fifth table gives the four metrics. All three moves carry the same sign: the change failure rate improves (from 53% to 47%, 50%, and 52%), and the other three get worse. When all three are applied together, the integration loop grows from 28 to 68 minutes; escaped errors fall from 32 to 25, the change failure rate falls to 42%, but deployment frequency falls from 4.09 to 3.43 changes/day, lead time climbs from 351.9 to 420.2 minutes, and time to restore goes from 4320 to 4360 minutes. Lengthening the loop is a bill charged to every change that passes through it.

Where the Difference Hides

In this lesson, the difference hides in loop coverage, and its number is this: four of the eight classes are not covered in either of the first two loops at all, and one is never covered above a 0.20 rate in any early loop. The declared part is the loop lengths — 2, 30, 240, 4320 minutes; anyone can read them. The undeclared part is the coverage: which loop catches which class at which rate is only visible in the run.

How many steps it takes the signal to reach whom also reads from here: a syntax error reaches the development role in 2.0 minutes, a resource error in 3810.0 minutes. Same role, same flow, the same change within the same flow — a 1905-fold difference comes from the error class alone.

Summary

• The four loops’ cumulative lengths are 2, 30, 240, and 4320 minutes; of 120 errors, 42 are caught while writing, 21 at integration, 25 at staging, 32 fall into none of them.
• Of the 32 escaped errors, 29 come from the four classes first covered at staging (configuration 9, scheduling 7, resource 7, data format 6), and those classes’ catch rates at staging range from 0.15 to 0.55.
• The average delay per class spans from 2.0 to 3810.0 minutes; the difference does not come from loop length but from which loop covers the class.
• Moving data format to integration gains 5 of 12 errors for 12 minutes and cuts total delay by 11%; moving scheduling gains only 1 of 10 errors for 20 minutes.
• The three moves together bring escaped errors from 32 to 25 and the change failure rate from 53% to 42%, while carrying deployment frequency from 4.09 to 3.43 and lead time from 351.9 to 420.2 minutes.

Next Step

The loops are built, and how many steps it takes the signal to reach whom has been counted: 7.0 steps in observation scope, 61.0 steps for context dropped at the boundary, 2.0 to 3810.0 minutes in loop coverage. All six lessons measured the same thing — where the difference hides.

But all of these measurements rested on a silent assumption: that the same software is the same software in every environment. An error not being caught at staging and surfacing in production instead was written, in the model above, as a rate — 0.35, 0.20, 0.15. Why that rate is not one was never asked. What runs on the developer’s machine and what runs in production are not the same thing; there is a difference between them, and this topic never even named that difference. The next topic starts exactly there: what environments are, what difference is hidden at every link in the chain, and what it costs to skip a link.