Patch and Change Management

The previous lesson treated remediation as a one-dimensional operation: person-hours are spent on an item and the item closes. In reality every remediation is a change, and a change touches the production environment. Closing an item fast means applying it under-tested; testing it fully keeps the item open longer. This lesson measures that trade-off on a single decision variable: the patch window — the time between the day an advisory is published and the day the patch is applied.

The fleet-side mechanics of the window — the maintenance calendar, the machine that misses its cycle, point-days — were measured in the Endpoint and System Hardening course and are taken here as input. The question here is at program scale, and it sits between two curves: as the window shortens, the number of open findings caught before the exploit becomes accessible rises, and that same narrowing lowers test coverage and produces unplanned outages. The two curves are drawn on the same axis, and the point where they cross is calculated.

• SP7. The forty advisories, the affected-asset counts and the day the exploit becomes accessible belong to the fictional regional measurement network and are a model. No real patch is downloaded.
• SP8. Test coverage grows linearly with window length and completes at twenty-one days; the error rate grows with the square of the missing coverage. In reality the curve is jumpy.
• SP9. The risk unit (ru) is fictional; it is not a real currency. One unplanned outage was counted as 3,600 ru. This is an assumption, and the second block measures what happens when it changes.
• SP10. The day the exploit becomes accessible is known in the model. In reality it is estimated; the last block measures the cost of that estimate being wrong.
• SP11. The approval gate’s error multiplier is fixed, and the gate itself introduces no defects.
• SP12. Faulty patches, rollbacks and staged rollout were not modeled; all three are held constant.

// patch/model.mjs — MODEL of window length: fictional advisory set, the day the
// exploit becomes accessible, test coverage and the change failure rate.
// No real patch is downloaded; fleet mechanics are taken as input.
export const SEED = 20260811, ANNOUNCEMENT_COUNT = 40, ROLLOUT = 6;
export const FULL_TEST = 21, ERROR_BASE = 0.01, ERROR_RISE = 0.34;
export const OUTAGE_RU = 3600;             // cost of one unplanned outage (risk units, fictional)
const generator = (t) => { let s = t >>> 0;
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };

export const ANNOUNCEMENT = (() => {
  const r = generator(SEED);
  return Array.from({ length: ANNOUNCEMENT_COUNT }, (_, i) => ({
    no: i + 1,
    score: +(4 + r() * 5).toFixed(1),
    affected: 20 + Math.round(r() * 280),
    // the day the exploit becomes usable: some are ready on the advisory day
    exploit: Math.round(Math.max(0, -6 + r() * 46)),
  }));
})();

export const coverage = (W) => Math.min(1, W / FULL_TEST);
export const errorRate = (W) => ERROR_BASE + ERROR_RISE * (1 - coverage(W)) ** 2;
export const outages = (W) => Math.round(ANNOUNCEMENT_COUNT * ROLLOUT * errorRate(W));
export const caught = (W) => ANNOUNCEMENT.filter((d) => W <= d.exploit).length;
export const exposure = (W) => ANNOUNCEMENT.reduce((a, d) =>
  a + d.score * d.affected * Math.max(0, W - d.exploit), 0);
export const total = (W) => exposure(W) + outages(W) * OUTAGE_RU;

// Regime: a function that assigns a window to each advisory. Outage count is computed per advisory.
export const REGIME = { 'no gate': { delay: 0, multiplier: 1.00 },
  'light approval': { delay: 2, multiplier: 0.60 }, 'full approval': { delay: 5, multiplier: 0.35 } };
export function measure(window, regimeName = 'no gate') {
  const d = REGIME[regimeName]; let exp = 0, out = 0, caughtCount = 0;
  for (const x of ANNOUNCEMENT) {
    const W = Math.max(window(x), d.delay);
    exp += x.score * x.affected * Math.max(0, W - x.exploit);
    out += ROLLOUT * errorRate(W) * d.multiplier;
    if (W <= x.exploit) caughtCount += 1;
  }
  return { exposure: exp, outages: Math.round(out), caught: caughtCount,
    total: exp + Math.round(out) * OUTAGE_RU };
}
export const bestSingle = (regimeName) => { let b = 1;
  for (let W = 1; W <= 40; W += 1)
    if (measure(() => W, regimeName).total < measure(() => b, regimeName).total) b = W;
  return b; };

Where the Two Curves Cross

// patch/window.mjs — window length sweep: caught open findings, exposure, outages and
// the gain and cost of shortening by one day
import { ANNOUNCEMENT, ROLLOUT, OUTAGE_RU, coverage, errorRate, measure } from './model.mjs';

const y = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd());
const single = (W) => measure(() => W);

console.log(`${ANNOUNCEMENT.length} announcements, ${ROLLOUT} rollout events per announcement; ` +
  `one unplanned outage costs ${OUTAGE_RU} risk units (fictional)`);
const A = [-9, 9, 11, 12, 10, 11, 11, 18];
y(A, 'window', 'test', 'caught', 'exposure', 'outage', 'outage', 'total', '1-day cut');
y(A, 'days', 'coverage', 'open', 'ru', 'events', 'ru', 'ru', 'gain - cost');
for (let W = 5; W <= 15; W += 1) {
  const r = single(W), previous = single(W - 1);
  const gain = r.exposure - previous.exposure;                 // gain of shortening by 1 day
  const cost = (previous.outages - r.outages) * OUTAGE_RU;     // cost of shortening by 1 day
  y(A, W, coverage(W).toFixed(2), r.caught, Math.round(r.exposure), r.outages,
    r.outages * OUTAGE_RU, Math.round(r.total),
    `${Math.round(gain)} - ${Math.round(cost)}`);
}

const bestAt = (k) => { let b = 1; const t = (W) => single(W).exposure + single(W).outages * k;
  for (let W = 1; W <= 40; W += 1) if (t(W) < t(b)) b = W; return b; };
const cut = bestAt(OUTAGE_RU);
const step = (W) => `${W} -> ${W - 1}: gain ` +
  `${Math.round(single(W).exposure - single(W - 1).exposure)} ru, cost ` +
  `${(single(W - 1).outages - single(W).outages) * OUTAGE_RU} ru`;
console.log(`\ncrossing at day ${cut}: ${step(cut + 1)} | ${step(cut)}`);
console.log('best window as outage cost changes:');
for (const k of [900, 3600, 14000]) {
  const W = bestAt(k);
  console.log(`  ${String(k).padStart(5)} ru/outage -> ${String(W).padStart(2)} days; ` +
    `caught ${single(W).caught}/${ANNOUNCEMENT.length}, error rate ${errorRate(W).toFixed(3)}`);
}

40 announcements, 6 rollout events per announcement; one unplanned outage costs 3600 risk units (fictional)
window        test     caught    exposure    outage     outage      total         1-day cut
days      coverage       open          ru    events         ru         ru       gain - cost
5             0.24         30       48357        50     180000     228357     10800 - 21600
6             0.29         28       60960        44     158400     219360     12602 - 21600
7             0.33         27       74746        39     140400     215146     13787 - 18000
8             0.38         27       88533        34     122400     210933     13787 - 18000
9             0.43         26      103237        29     104400     207637     14704 - 18000
10            0.48         25      119524        25      90000     209524     16286 - 14400
11            0.52         24      137399        21      75600     212999     17876 - 14400
12            0.57         23      156171        17      61200     217371     18772 - 14400
13            0.62         23      174942        14      50400     225342     18771 - 10800
14            0.67         23      193714        11      39600     233314     18772 - 10800
15            0.71         23      212485         9      32400     244885      18772 - 7200

crossing at day 9: 10 -> 9: gain 16286 ru, cost 14400 ru | 9 -> 8: gain 14704 ru, cost 18000 ru
best window as outage cost changes:
    900 ru/outage ->  1 days; caught 34/40, error rate 0.318
   3600 ru/outage ->  9 days; caught 26/40, error rate 0.121
  14000 ru/outage -> 17 days; caught 22/40, error rate 0.022

Two columns in the table move in opposite directions. As the window stretches from five days to fifteen, exposure climbs from 48,357 risk units to 212,485, while outage events drop from 50 to 9. Neither carries the decision alone: a program that watches only exposure shrinks the window to one day and ends up above fifty unplanned outages a year; a program that watches only outages stretches the window and never sees this.

The last column makes the decision. Dropping from ten to nine gains 16,286 ru of exposure and adds 14,400 ru of outage cost — the gain is larger, so the cut is made. Dropping from nine to eight gains 14,704 ru but costs 18,000 ru — this step is reversed. The two curves cross at day nine, and the total takes its lowest value there (207,637 ru).

The caught-open column carries a separate warning. It holds steady at 23 from day twelve through day fifteen, but exposure rises thirty-six percent over that same span. The caught-open count is a discrete metric and loses sensitivity as the window lengthens; exposure is continuous and counts every day. The question “how many open findings did we close in time” is the least informative form of the same data.

The bottom block shows where the window comes from. When the outage cost is counted as 900 ru, the best window is one day; at 14,000 ru it is seventeen days — a seventeen-fold difference for the same forty advisories. Window length is not a property of the finding; it is a consequence of the outage’s cost. When two organizations look at the same set of advisories and choose different windows, that is not an inconsistency — as long as the cost is written down. If it is not written down, the window is argued as a bare number, and the argument never closes.

Approval Gate and the Two-Path Window

// patch/regime.mjs — change regime: the cost of the approval gate, the two-path window
// and the cost of a classification error
import { ANNOUNCEMENT, ROLLOUT, REGIME, measure, bestSingle } from './model.mjs';

const y = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd());
const APPROVAL_HOURS = { 'no gate': 0, 'light approval': 1.5, 'full approval': 5 };   // approval hours per rollout event
const EVENTS = ANNOUNCEMENT.length * ROLLOUT;

const A = [-16, 9, 9, 9, 11, 9, 11, 12];
console.log('1. three change regimes, each at its own best window');
y(A, 'regime', 'delay', 'error', 'window', 'caught', 'outage', 'total', 'approval');
y(A, '', 'days', 'factor', 'days', 'open', 'events', 'ru', 'hours');
const rows = [];
for (const name of Object.keys(REGIME)) {
  const W = Math.max(bestSingle(name), REGIME[name].delay);
  const r = measure(() => W, name), hours = EVENTS * APPROVAL_HOURS[name];
  rows.push({ name, W, r, hours });
  y(A, name, REGIME[name].delay, REGIME[name].multiplier.toFixed(2), W, r.caught, r.outages,
    Math.round(r.total), hours);
}

console.log('\n2. risk reduction per unit of effort when moving up one regime');
for (let i = 1; i < rows.length; i += 1) {
  const d = rows[i - 1].r.total - rows[i].r.total, s = rows[i].hours - rows[i - 1].hours;
  console.log(`   ${rows[i - 1].name} -> ${rows[i].name}: ${Math.round(d)} ru / ${s} hours = ` +
    `${(d / s).toFixed(0)} ru/hour`);
}

// Two-path regime: advisories whose exploit is ready early get a short window, the rest a long one.
let best = null;
for (let threshold = 2; threshold <= 20; threshold += 1)
  for (let s = 1; s <= 12; s += 1)
    for (let l = s; l <= 30; l += 1) {
      const r = measure((x) => (x.exploit <= threshold ? s : l));
      if (!best || r.total < best.r.total) best = { threshold, s, l, r };
    }

const B = [-38, 11, 10, 11, 11];
console.log('\n3. two-path regime instead of a single window (no gate)');
y(B, 'regime', 'caught', 'outage', 'total ru');
const t9 = measure(() => bestSingle('no gate'));
y(B, `single window ${bestSingle('no gate')} days`, t9.caught, t9.outages, Math.round(t9.total));
y(B, `two paths: <=${best.threshold} days -> ${best.s}, other -> ${best.l}`, best.r.caught, best.r.outages,
  Math.round(best.r.total));

// Classification error: the n advisories closest to the threshold are placed on the flipped path.
const sorted = [...ANNOUNCEMENT].sort((a, b) =>
  Math.abs(a.exploit - best.threshold) - Math.abs(b.exploit - best.threshold) || a.no - b.no);
console.log('\n4. if the decision separating the two paths is wrong');
y(B, 'advisories placed on the flipped path', 'caught', 'outage', 'total ru');
for (const n of [0, 4, 12]) {
  const flipped = new Set(sorted.slice(0, n).map((x) => x.no));
  const r = measure((x) => ((x.exploit <= best.threshold) !== flipped.has(x.no) ? best.s : best.l));
  y(B, `${n} advisories`, r.caught, r.outages, Math.round(r.total));
}

1. three change regimes, each at its own best window
regime              delay    error   window     caught   outage      total    approval
                     days   factor     days       open   events         ru       hours
no gate                 0     1.00        9         26       29     207637           0
light approval          2     0.60        6         28       26     154560         360
full approval           5     0.35        5         30       17     109557        1200

2. risk reduction per unit of effort when moving up one regime
   no gate -> light approval: 53078 ru / 360 hours = 147 ru/hour
   light approval -> full approval: 45002 ru / 840 hours = 54 ru/hour

3. two-path regime instead of a single window (no gate)
regime                                     caught    outage   total ru
single window 9 days                           26        29     207637
two paths: <=11 days -> 3, other -> 19         30        28     130159

4. if the decision separating the two paths is wrong
advisories placed on the flipped path      caught    outage   total ru
0 advisories                                   30        28     130159
4 advisories                                   26        22     155943
12 advisories                                  23        19     199971

The first table reads in an unexpected direction. The full-approval gate adds a five-day delay to every change, yet its best window is shorter than the no-gate regime’s (5 versus 9), and it catches more open findings (30 versus 26). The reason is in the second column: because the gate multiplies the error rate by 0.35, the outage cost of shortening by one day falls, so shortening stays profitable for longer. The approval gate’s cost is paid in hours, not in days.

The second block counts those hours. Moving from no gate to light approval gains 53,078 ru and costs 360 hours: 147 ru per hour. Moving from light approval to full approval gains 45,002 ru and costs 840 hours: 54 ru/hour. A ranking based on total risk puts full approval first; a ranking based on risk reduction per unit of effort stops at light approval. The ranking changes when the metric changes, and which one is right depends on where else those 840 hours could be used — the remediation-capacity gap from the previous lesson is one place those hours could go.

The third block shows a gain that costs no hours. Splitting the advisories in two — a three-day window for those whose exploit is ready within eleven days, a nineteen-day window for the rest — drops the total from 207,637 to 130,159 ru. The gain comes from the sorting itself: the expensive testing cost of a short window is paid only for the advisories that need it.

The fourth block measures what that gain depends on. When the four advisories closest to the threshold are placed on the flipped path, the total rises to 155,943 — a third of the gain is gone. When twelve advisories are misclassified, it settles at 199,971, and the gain over the single window is only ten percent. A second thing happens in the same rows: outage events drop from 28 to 19. A program that watches only the unplanned-outage count reads this deterioration as an improvement.

Summary

• As the window stretches from five days to fifteen, exposure climbs from 48,357 ru to 212,485, and outage events drop from 50 to 9; because the two columns move in opposite directions, neither carries the decision alone.
• The two curves cross at day nine: dropping from 10 to 9 gains 16,286 ru at a cost of 14,400 ru; dropping from 9 to 8 gains 14,704 ru at a cost of 18,000 ru. The total is lowest there, at 207,637 ru.
• The caught-open count holds steady between days 12 and 15 while exposure rises 36 percent: a discrete metric loses sensitivity as the window lengthens.
• The best window shifts along with the outage’s cost: 1 day at 900 ru, 17 days at 14,000 ru. The window is a consequence of the cost, not of the finding.
• The full-approval gate adds a five-day delay, yet its best window is shorter than the no-gate regime’s (5 versus 9) and it catches more open findings (30 versus 26); its cost is 1,200 approval hours.
• By total risk, full approval leads; by gain per hour, light approval leads (147 versus 54 ru/hour). The two-path window drops the total to 130,159, but a twelve-advisory classification error gives back nine-tenths of that gain, and it does so while lowering the outage count.

Next Step

Both lessons worked on a queue and a calendar: items entered the queue, windows were tuned, and in both cases what changed was a configuration. A configuration stays changed once it is changed. The next lesson’s subject is the one item that does not stay where it is tuned: human behavior. A training session raises a metric, then the metric drifts back to its old value within weeks, and the program has to spend the same effort again. What must be measured is not how good the training was, but in which week the gain decays — and because that decay does not run at the same speed in every group, which group gets the scarce training hours is once again a ranking question.