Auto Scaling Groups

The previous lesson reduced instance launch to a single definition and left two things the template does not write: how many instances there will be and when they will be terminated. An auto scaling group takes over both of these questions. The group is bound to a template, a minimum and maximum instance count is given, and two separate loops begin changing the instance count on their own.

The first loop is the scaling policy: it looks at a demand metric and adds or removes instances. M22/K03 measured horizontal scaling and the oscillation born from metric lag; that scan is not repeated here. The second loop is the health check: it asks the instances at regular intervals, terminates the ones that do not respond, and launches a new one in their place. This lesson’s measure is not the two loops’ behavior taken separately, it is their combined behavior — because both change the same number, the group’s size, and each can mistake the other’s action for demand.

CC25. The group is the read layer of the regional measurement network — the fictional software that collects readings from water meters; the demand curve is fictional and identical on every run. CC26. One round is one minute, the period is 240 rounds, and capacity per instance is 100 requests/round. CC27. Startup takes 2 rounds; the previous lessons’ 48.9 seconds rounds up to two rounds at this scale. CC28. The transient check failure probability is 0.08; the generator is self-written and the seed 20261001 is visible. CC29. Permanent breakage occurs at rounds 40, 95, and 170; the moments are fictional and identical on every run. CC30. Scaling counts only ready instances, and health replacement does not know about cooldown.

Two Loops in the Same Group

When a health check terminates an instance, the group shrinks. The scaling policy looks at the group at that moment and finds the ready instance count lower than needed — the replacement instance is still booting. The policy mistakes this for a demand increase and adds an instance. When the booting instances become ready, the group is larger than needed, and this time the policy shrinks it.

// measurement-network/group.mjs — the combined run of an auto scaling group and health checks (model)

export const generator = (seed) => () =>
  (seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648;

export const ROUNDS = 240, CAPACITY = 100, STARTUP = 2;   // rounds, requests per instance, startup rounds
const TRANSIENT = 0.08;                                    // transient health-check failure probability
const BREAK_ROUNDS = [40, 95, 170];                         // rounds at which permanent breakage occurs (fictional)

// The demand curve is fictional and identical on every run; randomness is only in the health check.
export const demand = (t) =>
  900 + 380 * Math.sin((2 * Math.PI * t) / 120) + (t >= 150 && t < 190 ? 520 : 0);
export const needed = (t) => Math.ceil(demand(t) / CAPACITY);

export const run = ({ threshold, cooldown, seed = 20261001 }) => {
  const r = generator(seed);
  let group = Array.from({ length: needed(0) }, () => ({ ready: true, remaining: 0, streak: 0, broken: false }));
  const m = { min: 99, max: 0, concurrent: 0, ratio: 0, underCapacity: 0, started: 0,
    wrong: 0, brokenRounds: 0, scaling: 0, replaced: 0 };
  let lastAction = -cooldown;
  for (let t = 0; t < ROUNDS; t++) {
    const g = needed(t);
    if (BREAK_ROUNDS.includes(t)) {
      const victim = group.find((o) => o.ready && !o.broken);
      if (victim) victim.broken = true;
    }
    const survivors = [];                               // health check (does not know cooldown)
    for (const o of group) {
      if (!o.ready) { survivors.push(o); continue; }
      if (o.broken) m.brokenRounds++;
      const passed = !o.broken && r() >= TRANSIENT;
      o.streak = passed ? 0 : o.streak + 1;
      if (o.streak >= threshold) {
        m.replaced++; if (!o.broken) m.wrong++;
        survivors.push({ ready: false, remaining: STARTUP, streak: 0, broken: false });
        m.started++;
      } else survivors.push(o);
    }
    group = survivors;
    const readyCount = group.filter((o) => o.ready).length;  // scaling only counts ready instances
    if (t - lastAction >= cooldown) {
      if (readyCount < g) {
        for (let i = 0; i < g - readyCount; i++)
          group.push({ ready: false, remaining: STARTUP, streak: 0, broken: false });
        m.started += g - readyCount; m.scaling++; lastAction = t;
      } else if (readyCount > g + 1) {
        for (let i = 0; i < readyCount - g - 1; i++) group.splice(group.findIndex((o) => o.ready), 1);
        m.scaling++; lastAction = t;
      }
    }
    for (const o of group) if (!o.ready && --o.remaining <= 0) o.ready = true;
    const booting = group.filter((o) => !o.ready).length;
    m.min = Math.min(m.min, group.length); m.max = Math.max(m.max, group.length);
    m.concurrent = Math.max(m.concurrent, booting);
    m.ratio = Math.max(m.ratio, booting / group.length);
    if (group.filter((o) => o.ready).length < g) m.underCapacity++;
  }
  return m;
};

if (import.meta.url === `file://${process.argv[1]}`) {
  const bad = run({ threshold: 2, cooldown: 0 }), good = run({ threshold: 5, cooldown: 8 });
  console.log(`${ROUNDS} rounds, ${CAPACITY} requests per instance, ${STARTUP} startup rounds; seed 20261001`);
  console.log(`needed instances range between ${Math.min(...Array.from({ length: ROUNDS }, (_, t) => needed(t)))}` +
    ` and ${Math.max(...Array.from({ length: ROUNDS }, (_, t) => needed(t)))}\n`);
  console.log("pair".padEnd(25) + "group min-max".padEnd(15) + "swing".padEnd(13) +
    "concurrent change".padEnd(20) + "under-capacity rounds".padEnd(23) + "instances started");
  for (const [name, m] of [["threshold 2, cooldown 0", bad], ["threshold 5, cooldown 8", good]])
    console.log(name.padEnd(25) + `${m.min}-${m.max}`.padEnd(15) + String(m.max - m.min).padEnd(13) +
      `${m.concurrent} (${Math.round(100 * m.ratio)}%)`.padEnd(20) + `${m.underCapacity}/${ROUNDS}`.padEnd(23) +
      String(m.started));
  for (const [name, m] of [["threshold 2, cooldown 0", bad], ["threshold 5, cooldown 8", good]])
    console.log(`${name}: ${m.replaced} health replacements (${m.wrong} wrong), ` +
      `${m.scaling} scaling actions, broken instance stayed in traffic for ${m.brokenRounds} rounds`);
}

240 rounds, 100 requests per instance, 2 startup rounds; seed 20261001
needed instances range between 6 and 18

pair                     group min-max  swing        concurrent change   under-capacity rounds  instances started
threshold 2, cooldown 0  7-24           17           6 (26%)             10/240                 46
threshold 5, cooldown 8  7-18           11           5 (28%)             46/240                 19
threshold 2, cooldown 0: 17 health replacements (14 wrong), 40 scaling actions, broken instance stayed in traffic for 5 rounds
threshold 5, cooldown 8: 2 health replacements (0 wrong), 22 scaling actions, broken instance stayed in traffic for 11 rounds

The clearest number is the upper bound of group size. Even though demand never requires more than 18 instances in any round, the first pair pushes the group to 24 instances. The extra six instances do not come from demand; they come from the policy filling the gap that health replacements open. The second pair never exceeds 18 on the same demand curve.

The second number is less visible but heavier: in the first pair, 26 percent of the group was changing at the same time — one in four instances was booting, that is, neither old nor new. When the healthy instance count swings at this rate, the metric starts measuring the result of the policy’s own action.

The rows below separate the source. The first pair made 17 health replacements, and 14 of them were wrong — those instances were not broken, they only failed two consecutive checks. The same pair produced 40 scaling actions, the second pair 22. Every wrong replacement sends the policy a false demand signal.

But the second pair is not free. Under-capacity rounds rise from 10 to 46: because the cooldown is long, the policy responds late to a real demand increase. And a broken instance stays in traffic for 11 rounds instead of 5; because the threshold is high, an instance that is really broken is also caught late. Each pair reduces one error and enlarges the other.

Scanning Threshold against Cooldown

The two endpoints showed the direction; the real question is what the pairs in between do. The scan below runs three thresholds against three cooldown durations together — nine pairs, the same demand curve, the same seed.

// measurement-network/group-scan.mjs — combined scan of health threshold and cooldown duration (model)
import { ROUNDS, run } from "./group.mjs";

const THRESHOLDS = [2, 3, 5], COOLDOWNS = [0, 3, 8], HOURS = ROUNDS / 60;   // one round is one minute (fictional)

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log(`${ROUNDS} rounds = ${HOURS} hours; every row was run with the same demand curve and the same seed\n`);
  console.log("threshold".padEnd(11) + "cooldown".padEnd(10) + "group min-max".padEnd(15) +
    "swing".padEnd(13) + "concurrent".padEnd(13) + "under capacity".padEnd(16) +
    "wrong changes".padEnd(17) + "broken rounds".padEnd(15) + "started per hour");
  const all = [];
  for (const e of THRESHOLDS)
    for (const s of COOLDOWNS) {
      const m = run({ threshold: e, cooldown: s });
      all.push({ e, s, m });
      console.log(String(e).padEnd(11) + String(s).padEnd(10) + `${m.min}-${m.max}`.padEnd(15) +
        String(m.max - m.min).padEnd(13) + String(m.concurrent).padEnd(13) +
        `${m.underCapacity}/${ROUNDS}`.padEnd(16) + String(m.wrong).padEnd(17) +
        String(m.brokenRounds).padEnd(15) + (m.started / HOURS).toFixed(1));
    }
  const mostChange = all.reduce((a, b) => (b.m.started > a.m.started ? b : a));
  const leastChange = all.reduce((a, b) => (b.m.started < a.m.started ? b : a));
  const longestBroken = all.reduce((a, b) => (b.m.brokenRounds > a.m.brokenRounds ? b : a));
  console.log(`\npair producing constant change: threshold ${mostChange.e}, cooldown ${mostChange.s} — ` +
    `${(mostChange.m.started / HOURS).toFixed(1)} instances/hour, ${mostChange.m.wrong} wrong replacements`);
  console.log(`pair producing the least change: threshold ${leastChange.e}, cooldown ${leastChange.s} — ` +
    `${(leastChange.m.started / HOURS).toFixed(1)} instances/hour, ${leastChange.m.underCapacity} under-capacity rounds`);
  console.log(`pair carrying a broken instance the longest: threshold ${longestBroken.e}, cooldown ${longestBroken.s} — ` +
    `${longestBroken.m.brokenRounds} rounds in traffic`);
}

240 rounds = 4 hours; every row was run with the same demand curve and the same seed

threshold  cooldown  group min-max  swing        concurrent   under capacity  wrong changes    broken rounds  started per hour
2          0         7-24           17           6            10/240          14               5              11.5
2          3         7-18           11           5            13/240          12               6              8.5
2          8         7-18           11           4            42/240          13               6              8.5
3          0         7-23           16           5            8/240           0                8              6.3
3          3         7-18           11           5            14/240          0                7              5.0
3          8         7-18           11           5            47/240          1                8              5.3
5          0         7-23           16           5            7/240           0                12             6.0
5          3         7-18           11           5            13/240          0                12             4.8
5          8         7-18           11           5            46/240          0                11             4.8

pair producing constant change: threshold 2, cooldown 0 — 11.5 instances/hour, 14 wrong replacements
pair producing the least change: threshold 5, cooldown 3 — 4.8 instances/hour, 13 under-capacity rounds
pair carrying a broken instance the longest: threshold 5, cooldown 0 — 12 rounds in traffic

Each column tracks a single setting, and the four go in separate directions. Wrong replacements track only the threshold: 12–14 at threshold 2, 0–1 at threshold 3, 0 at threshold 5. Broken rounds also track only the threshold, but in the opposite direction: they climb from 5–6 to 11–12. The threshold is a direct trade-off between two errors — killing a healthy instance and keeping a broken one in traffic.

Cooldown duration governs two other columns. Swing tracks only cooldown: 16–17 at cooldown 0, 11 at cooldowns 3 and 8. Zero cooldown lets the policy act every round, and the group mistakes its own action’s result for the metric. Under-capacity rounds also track only cooldown, and in the opposite direction: they climb from 7–10 to 42–47. A long cooldown cuts the swing but delays the response to a real demand increase.

The pairs at the extremes are read by name. The worst is threshold 2, cooldown 0: it launches 11.5 instances an hour, pushes the group to 24, makes 14 wrong replacements. What this pair produces is not scaling, it is constant change; the group never settles. The calmest is threshold 5, cooldown 3 — 4.8 instances an hour — but it keeps a broken instance in traffic for 12 rounds. The middle pair (threshold 3, cooldown 3) gives nearly the same calm at 5.0 instances an hour and catches a broken instance in 7 rounds.

The Cost of Constant Change

The visible cost of constant change is startup: every launched instance is paid for across two rounds and does no work at all. The invisible cost is what those instances terminate.

// measurement-network/group-ledger.mjs — the cost of constant change and the group's decision ledger (model)
import { ROUNDS, STARTUP, run } from "./group.mjs";

const HOURS = ROUNDS / 60, UNIT = 1.0;      // persistent capacity unit (fictional; input from M23/K01)
const PAIRS = [[2, 0], [3, 3], [5, 3]];

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log("pair".padEnd(25) + "started per day".padEnd(18) + "spent booting".padEnd(16) +
    "daily idle unit".padEnd(18) + "under capacity".padEnd(16) + "broken rounds");
  for (const [e, s] of PAIRS) {
    const m = run({ threshold: e, cooldown: s }), daily = (m.started / HOURS) * 24;
    const idle = (daily * STARTUP) / 60;                // instance-hours: paid while booting but doing no work
    console.log(`threshold ${e}, cooldown ${s}`.padEnd(25) + daily.toFixed(0).padEnd(18) +
      `${idle.toFixed(1)} instance-h`.padEnd(16) + (idle * UNIT).toFixed(1).padEnd(18) +
      `${m.underCapacity}/${ROUNDS}`.padEnd(16) + String(m.brokenRounds));
  }
  const KEPT = ["minimum and maximum instances", "scaling metric and its thresholds", "cooldown duration",
    "health check threshold and interval", "which instance is chosen when scaling in",
    "the template version the group is bound to"];
  const HANDED_OFF = ["the health check call's own latency",
    "the latency of applying a scaling action", "whether the requested capacity is available at that moment"];
  const IRREVERSIBLE = ["the work and local disk of a terminated instance",
    "the unhealthy mark — the instance is not reverted, it is replaced",
    "capacity that has been scaled in — reclaiming it demands a new startup and cooldown round"];
  console.log(`\ndecision ledger: ${KEPT.length} items kept, ${HANDED_OFF.length} items with the provider`);
  console.log(`  with the provider: ${HANDED_OFF.join("; ")}`);
  for (const g of IRREVERSIBLE) console.log(`  cannot be undone — ${g}`);
}

pair                     started per day   spent booting   daily idle unit   under capacity  broken rounds
threshold 2, cooldown 0  276               9.2 instance-h  9.2               10/240          5
threshold 3, cooldown 3  120               4.0 instance-h  4.0               14/240          7
threshold 5, cooldown 3  114               3.8 instance-h  3.8               13/240          12

decision ledger: 6 items kept, 3 items with the provider
  with the provider: the health check call's own latency; the latency of applying a scaling action; whether the requested capacity is available at that moment
  cannot be undone — the work and local disk of a terminated instance
  cannot be undone — the unhealthy mark — the instance is not reverted, it is replaced
  cannot be undone — capacity that has been scaled in — reclaiming it demands a new startup and cooldown round

The bad pair launches 276 instances a day; at the good setting, the same group does the same job with 120. The 156-instance gap boots, becomes ready, and is terminated shortly after. The 9.2 instance-hours spent booting is capacity that costs 9.2 fictional units a day and does no work — nothing that catches the eye of anyone looking at the bill.

The real cost is in the three items that cannot be undone. Every terminated instance takes its work and local disk with it; in the bad pair, this happens 156 extra times a day, and most of these are terminations of healthy instances. The unhealthy mark cannot be reversed: if an instance is counted unhealthy because it failed two checks, it is not undone even if it responds on the third check — because it no longer exists. Capacity that has been scaled in cannot be undone either; reclaiming it demands a new startup and a cooldown round.

In the group’s decision ledger, six of nine items are kept, three are with the provider. The kept items are the settings this lesson scanned, and all of them can be chosen by number. The three that stay with the provider are insidious: the health check call’s own latency effectively shifts the threshold, the latency of applying a scaling action adds to the cooldown duration, and the requested capacity not being available at that moment quietly fails the policy.

Summary

• Two loops change the same number: health replacement shrinks the group, the policy mistakes this for demand and grows it — even though demand never exceeds 18 instances, the bad pair pushes the group to 24.
• In the bad pair, 26 percent of the group is changing at the same time; at this rate, neither the metric nor an observation tells the group’s actual state at that moment.
• Threshold is a direct trade-off between two errors: 12–14 wrong replacements at threshold 2, 0 at threshold 5 — but a broken instance stays in traffic for 11–12 rounds instead of 5–6.
• Cooldown duration governs two other columns: swing drops from 16–17 to 11, under-capacity rounds climb from 7–10 to 42–47.
• The pair producing constant change is threshold 2, cooldown 0: 11.5 instances an hour, 276 launches a day, 9.2 instance-hours of idle capacity; threshold 3, cooldown 3 does the same job with 120 launches a day.
• Six of nine items are kept, three are with the provider; the three items that cannot be undone are a terminated instance’s work, the unhealthy mark, and capacity that has been scaled in.

Next Step

This lesson handed the group off to two loops, and in exchange left six setting items: minimum and maximum instances, the metric, thresholds, cooldown, health interval, template version. The handoff is real — no one launches an instance manually anymore — but the number of kept items has not dropped, only its kind has changed. Instead of choosing an instance, a policy is chosen.

The handoff can go one step further: never seeing the machine at all. M22/K02 and M22/K03 measured containers and orchestration; that mechanics is not repeated here. The next lesson compares running the same workload on a self-operated cluster against a managed service and asks a single question: of the node size, version, patch, network plugin, and scaling decisions, how many stay with you at each handoff level, what does this number become under task-based execution, and which item becomes irreversible at each level?