Continuous Integration

The previous course left a cluster behind: the desired state is written down somewhere, a reconciler keeps applying it, and the actual state is pulled back whenever it drifts. The question left standing at its close was — who changes that desired state, on what evidence, and through which check? This course builds that path. Every step that stands on the route from a developer’s machine to production, and either lets a change through or does not, is a gate, and the gate is this course’s unit of measure.

The first gate on the path is the oldest one: a change entering the main branch. Continuous integration is every developer returning their work to the main branch at frequent intervals and running an automated check on every return. “Frequent” is not a number but an axis, and this lesson scans that axis. What gets scanned is the merge interval: how long a branch lives apart from the main branch. As the interval grows, two things grow together — the number of conflicting files and the errors born while resolving the conflict. As the interval shrinks, something else grows — the number of runs and runner time. This is where frequency stops being free. The feedback delay measured in the Process Models topic is not remeasured here; what this lesson measures is the conflict itself.

The Repository Model

The measurement runs on the repository of the regional measurement network: the service that collects water-meter readings, the service that verifies the readings, and the billing and work-order services. This network is fictional; the repository below is a model too, and no real repository was scanned. Durations are model minutes and are independent of the machine the measurement runs on.

PD1 — the repository is 120 files split across four services; eight developers work twenty workdays, and a day is 480 model minutes. Each service’s first three files are interface files: they carry another service’s expectations.

PD2 — a developer touches 0.5 files per hour. Total work is independent of the interval: every scan row has 640 file touches, and the only thing that changes is how many merges this work is split into.

PD3 — one fifth of the files are hot and are touched eight times as often; a developer prefers their own service 75 percent of the time. The generator is self-written and the seed is visible.

PD4 — resolution per conflicting file is 20 model minutes. The main branch keeps advancing while resolution runs; if a new merge touches the same files during that time, a new round opens, up to five rounds.

PD5 — the probability that resolving a conflicting file gives birth to an error is 0.09. Finding and fixing, later on, either an error born in resolution or an interface mismatch the gate missed costs 180 model minutes.

PD6 — one pipeline run spends 14 model minutes of runner time; two percent of runs flake red and stop a clean change for nothing, demanding one more run.

// repo.mjs — model of the fictional regional measurement network's repository: service, file, developer
// Seeded generator: 32-bit linear congruential map; no external library is used.
export const generator = (seed) => () => {
  seed = (seed * 1664525 + 1013904223) >>> 0;
  return seed / 4294967296;
};

// PD1: four services, files per service; the first three files are interface files.
export const services = [
  { name: 'collector', files: 38 },
  { name: 'verifier', files: 26 },
  { name: 'billing', files: 31 },
  { name: 'work-order', files: 25 },
];

export const files = services.flatMap((s, si) =>
  Array.from({ length: s.files }, (_, i) => ({
    name: `${s.name}/${i}`, service: si, interface: i < 3,
    // PD3: one fifth of the files are hot and touched eight times as often.
    weight: i % 5 === 0 ? 8 : 1,
  })));

// Weighted file pick: a developer prefers their own service 75 percent of the time.
export const pickFile = (r, developer) => {
  const own = r() < 0.75;
  const pool = own ? files.filter((f) => f.service === developer % 4) : files;
  const total = pool.reduce((a, f) => a + f.weight, 0);
  let x = r() * total;
  for (const f of pool) { x -= f.weight; if (x <= 0) return f; }
  return pool[pool.length - 1];
};

export const DEVELOPERS = 8;
export const DAY = 20, DAY_MIN = 480;          // PD1: twenty workdays, 480 model minutes per day
export const RATE = 0.5;                       // PD2: 0.5 files touched per hour
export const RESOLVE_MIN = 20;                 // PD4: resolution time per conflicting file
export const ERROR_PROBABILITY = 0.09;         // PD5: probability an error is born during resolution
export const LATER_MIN = 180;                  // PD5: cost of a later-found error
export const FLAKY = 0.02;                     // PD6: two percent of runs falsely hold
export const RUN_MIN = 14;                     // PD6: runner time of one pipeline run

Scanning the Interval

The scan runs the same work for six intervals: two hours, four hours, one day, two days, four days, and ten workdays. Every row starts with the same seed, so what is being compared is the interval itself.

// interval.mjs — conflict and gate cost when the merge interval is scanned
import * as R from './repo.mjs';

const simulate = (hours, seed = 7391) => {
  const r = R.generator(seed);                     // every row starts with the same seed
  const T = hours * 60, horizon = R.DAY * R.DAY_MIN, n = R.RATE * hours;
  const trunk = [], events = [];                    // trunk: merged files {t, file, ...}
  for (let dev = 0; dev < R.DEVELOPERS; dev++) {
    for (let k = 0; k < horizon / T; k++) events.push({ dev, start: k * T + dev * (T / R.DEVELOPERS) });
  }
  events.sort((a, b) => a.start - b.start);
  let merges = 0, conflicts = 0, rounds = 0, errors = 0, semantic = 0, falseHold = 0, runs = 0, resolveMin = 0;

  for (const e of events) {
    const touched = new Set(), ifaceServices = new Set();
    while (touched.size < n) {                      // each branch touches n files
      const f = R.pickFile(r, e.dev);
      if (touched.has(f.name)) continue;
      touched.add(f.name);
      if (f.interface) ifaceServices.add(f.service);
    }
    merges++;
    let t = e.start + T, window = e.start, step = 0;
    while (true) {
      const rivals = trunk.filter((g) => g.t > window && g.t <= t && g.dev !== e.dev);
      const c = new Set(rivals.filter((g) => touched.has(g.file)).map((g) => g.file)).size;
      runs++;
      if (r() < R.FLAKY) { falseHold++; runs++; }    // flaky red: stops a clean change for nothing
      if (step === 0) {                              // interface mismatch with no textual conflict
        semantic += new Set(rivals.filter((g) => g.interface && ifaceServices.has(g.service) && !touched.has(g.file))
          .map((g) => g.file)).size;
      }
      if (c === 0 || step >= 5) break;
      conflicts += c; rounds++;
      for (let i = 0; i < c; i++) if (r() < R.ERROR_PROBABILITY) errors++;
      const duration = c * R.RESOLVE_MIN;
      resolveMin += duration; window = t; t += duration; step++;
    }
    for (const name of touched) {
      const f = R.files.find((x) => x.name === name);
      trunk.push({ t, file: name, dev: e.dev, service: f.service, interface: f.interface });
    }
  }
  const laterCost = (errors + semantic) * R.LATER_MIN;
  return { hours, n, merges, laterCost, conflicts, rounds, errors, semantic, falseHold, runs,
    resolveMin, runMin: runs * R.RUN_MIN, touches: trunk.length };
};

const pad = (x, w) => String(x).padStart(w);
const rows = [2, 4, 8, 16, 32, 80].map((h) => simulate(h));

const f2 = (x, w) => String(x.toFixed(2)).padStart(w);
console.log(`${'interval'.padEnd(9)}${pad('branch files', 14)}${pad('merges', 12)}${pad('conflicting', 13)}${pad('conflict/mrg', 14)}${pad('round/mrg', 11)}${pad('resolve errs', 14)}${pad('iface missed', 16)}`);
for (const x of rows) {
  console.log(`${(x.hours + ' hr').padEnd(9)}${pad(x.n, 14)}${pad(x.merges, 12)}${pad(x.conflicts, 13)}`
    + `${f2(x.conflicts / x.merges, 14)}${f2(x.rounds / x.merges, 11)}${pad(x.errors, 14)}${pad(x.semantic, 16)}`);
}
console.log(`\n${'interval'.padEnd(9)}${pad('runs', 7)}${pad('runner min', 15)}${pad('resolve min', 13)}${pad('late-found min', 16)}${pad('total min', 11)}${pad('false hold', 14)}`);
for (const x of rows) {
  console.log(`${(x.hours + ' hr').padEnd(9)}${pad(x.runs, 7)}${pad(x.runMin, 15)}${pad(x.resolveMin, 13)}${pad(x.laterCost, 16)}`
    + `${pad(x.runMin + x.resolveMin + x.laterCost, 11)}${pad(x.falseHold, 14)}`);
}
const best = rows.reduce((a, x) =>
  (x.runMin + x.resolveMin + x.laterCost < a.runMin + a.resolveMin + a.laterCost ? x : a));
console.log(`\ntouched files in every row ${rows.map((x) => x.touches).join('/')} (work is constant); `
  + `for seed 7391 the lowest total is in the ${best.hours}-hour interval`);
const at = (seed) => [2, 4, 8, 16, 32, 80].map((h) => simulate(h, seed))
  .reduce((a, x) => (x.runMin + x.resolveMin + x.laterCost < a.runMin + a.resolveMin + a.laterCost ? x : a));
console.log('other seeds: ' + [20260801, 4517, 66049, 131].map((s) => `${s} -> ${at(s).hours} hr`).join(', '));

interval   branch files      merges  conflicting  conflict/mrg  round/mrg  resolve errs    iface missed
2 hr                  1         640           74          0.12       0.12            12               3
4 hr                  2         320           90          0.28       0.27             4              10
8 hr                  4         160          205          1.28       0.81            22              17
16 hr                 8          80          320          4.00       0.97            31              26
32 hr                16          40          484         12.10       1.52            41              32
80 hr                40          16          654         40.88       1.63            55              43

interval    runs     runner min  resolve min  late-found min  total min    false hold
2 hr         731          10234         1480            2700      14414            17
4 hr         407           5698         1800            2520      10018             2
8 hr         297           4158         4100            7020      15278             8
16 hr        163           2282         6400           10260      18942             5
32 hr        103           1442         9680           13140      24262             2
80 hr         43            602        13080           17640      31322             1

touched files in every row 640/640/640/640/640/640 (work is constant); for seed 7391 the lowest total is in the 4-hour interval
other seeds: 20260801 -> 4 hr, 4517 -> 8 hr, 66049 -> 4 hr, 131 -> 4 hr

Conflict Does Not Grow Linearly

The first table’s real column is conflict/mrg. When branch lifetime grows forty-fold from two hours, conflicting files per merge climb from 0.12 to 40.88 — a factor of 354. The growth is not proportional to files per branch, because conflict is the intersection of two sets: both the branch’s own set and the total work landing on the main branch during that time grow. Two long-lived branches both raise the odds of touching the same hot files.

The round/mrg column does not, at first glance, keep pace with this growth: it climbs from 0.12 to 1.63 and saturates there. The reason is inside the measurement — a resolution round opens when the main branch moves while resolution is running. When branches are long-lived, merges are sparse, so the main branch usually does not move during resolution and one round is enough. The cost of a long branch does not show up in the round count; it shows up in round length: resolution time per merge is 2.3 model minutes on the two-hour branch and 818 model minutes on the ten-workday branch.

Errors born in resolution also grow with the interval: from 12 to 55. These are defects the merge itself produces; nobody wrote wrong code, they were born while two correct changes were reconciled by hand. The value of 4 in the four-hour row sits below its neighbors; because these numbers are small, that is seed noise, not a trend.

Frequency’s Own Cost

The second table gives the reverse direction. In the two-hour interval the pipeline runs 731 times and spends 10,234 model minutes of runner time; in the ten-workday interval, 43 runs and 602 minutes. Frequent merging is seventeen times more expensive to the machine. The number that has to be read alongside that column is false hold: a flaky red is a fixed probability per run, so as runs grow, false stops grow with them — 17 against 1. On a team that merges frequently, the number of changes the gate unfairly stops is seventeen times that of a team that merges rarely, and the urge to skip the gate is born right there.

The total column combines three items: runner time, resolution time, and the cost of late-found errors. The curve rises from both ends, and the lowest value sits at four hours, 10,018 minutes. Four of five seeds pick the same interval; one picks eight hours. The result is not “the more often, the better”: in the two-hour interval, runner time and false holds together inflate the total by more than 4,000 minutes; in the ten-workday interval, resolution and late finding push it up to 30,000 minutes. The optimum of the interval sits where the cost of a run intersects the cost of a conflict; as a run gets cheaper, the optimum shifts toward more frequent merging, and that is the real reason later lessons work to shorten pipeline time.

The Merge Gate’s Three Numbers

This gate is the entry gate to the main branch, and it is written with three numbers. What it holds: in the four-hour interval, 90 conflicting files become visible at merge time. All of this belongs only to this gate — no later gate in the pipeline can see a textual conflict. The build gate can only report a badly resolved merge as “broken”; it cannot say which two changes collided. The conflict information exists at merge time, then it is gone.

What it misses: interface mismatches. These are cases where two developers touch different files — no textual conflict — but one’s expectation diverges from what the other delivers. 10 of these pass through the gate in the four-hour interval, 43 in the ten-workday interval. Errors born in resolution also pass through the gate, because the gate does not check the resolution itself. What it falsely holds: 2 clean changes in the four-hour interval, 17 in the two-hour interval. The time it adds: 14 minutes of runner time per run and 20 minutes of human time per conflicting file; 23 minutes per change in the four-hour interval.

The gate cannot be defended unless these four numbers are read together. An evaluation that looks only at runner time finds frequent merging expensive and grows the interval; one that looks only at conflict count wants to shrink the interval to zero. Both are wrong, because the gate’s value is the difference between what it holds and what it misses, and that difference depends on the interval. The same gate holds 74 conflicts and misses 3 mismatches in the two-hour interval, and holds 654 while missing 43 in the ten-workday interval: holding more does not mean holding better — most of what it holds, the interval itself produced.

The two items it misses say this gate is not enough on its own, and that is the reason for the rest of the pipeline: the contract test is what will catch the interface mismatch, and the test gate is what will catch the error born in resolution. The order of the gates and what each one catches is the subject of the course’s later lessons.

Summary

• When the merge interval grew from two hours to ten workdays, conflicting files per merge rose from 0.12 to 40.88, errors born in resolution from 12 to 55, and interface mismatches the gate missed from 3 to 43.
• The resolution-round count saturated at 1.63; the cost of a long branch is not in the round count but in round length — resolution time per merge rose from 2.3 minutes to 818 model minutes.
• Frequency’s own cost was counted: the two-hour interval takes 731 runs and 10,234 runner minutes, the ten-workday interval takes 43 runs and 602 minutes.
• False holds are proportional to run count: 17 against 1; the urge to skip the gate is born from this number.
• When the three items are added together, the lowest value sits in the four-hour interval (10,018 minutes), and four of five seeds point to the same place; as a run gets cheaper, the optimum shifts toward merging more often.

Next Step

This lesson’s gate sits at entry to the main branch, and its decision is the answer to a single question: should this change get in? The question of what happens after the main branch turns green was never touched. Is a green main branch something that can be put into production, or only something that is a candidate for production? The two answers build two different arrangements, and the difference between them collects into a single gate: the human approval standing between a green run and production. The next lesson runs the same pipeline in two modes and counts the real defects that approval catches, together with the time it holds up.