Resource Sizing

The previous lesson measured that alerts catch unexpected rises but never question expected spending: as the baseline edges toward the budget, no alert fires — there is no event, only resources built larger than needed.

The example sizing was measured in M23/K02 and is not repeated here. This lesson’s axis is the bill side: which resource can be downsized is drawn from usage history, each candidate’s savings and risk counted on the same line.

Usage History and the Tier Ladder

The example is the fictional regional measurement network: five teams, three environments, a rented machine, network, store, function, and edge layer. Resources, tiers, and amounts are fictional; the currency is fictional too and is called unit, corresponding to no real price list.

CC19: there are ten resources and five types; every type has a tier ladder, each tier a capacity unit paired with a monthly price. CC20: the usage history is 30 days × 24 hours, built from a daily pattern, a weekend dip, noise, and sparse peaks. CC21: a downsizing candidate is the smallest tier that still suffices once a safety margin is added on top of p95 demand. CC22: risk is two numbers — how many hours the new tier fell short in the past, and how far short at the worst moment. CC23: whether stepping a tier down requires a disruption depends on the type. CC24: an idle resource’s usage ratio is undefined; it never shows up in any occupancy report.

The ladder, history, and candidates are a process-internal model; there is no real measurement record.

// cost/capacity.mjs — tier ladder and usage history model: no real measurement record. Tiers are
// [capacity unit, monthly unit] pairs; a 30-day x 24-hour history is derived from a generator,
// the seed is visible, the currency is fictional and is called `unit`. The example sizing was
// measured in M23/K02; what is measured here is the savings and risk of stepping a tier down.

export const SEED = 8814;
export const DAYS = 30, HOURS = DAYS * 24;

export const LADDER = {
  "rented machine": [[1, 14], [2, 27], [4, 52], [8, 101], [16, 198]],
  "block storage":  [[100, 7], [250, 17], [500, 33], [1000, 64], [2000, 126]],
  "function memory": [[1, 3], [2, 6], [4, 11], [8, 21]],
  "network gateway": [[1, 22], [2, 41], [4, 79]],
  "edge node":      [[1, 9], [2, 17], [4, 32]],
};

// does stepping the tier down require a disruption, and why
export const DISRUPTION = {
  "rented machine": ["yes", "restart"],
  "block storage":  ["yes", "data move"],
  "function memory": ["no", "next call"],
  "network gateway": ["no", "tier switch"],
  "edge node":      ["yes", "node swap"],
};

// name, type, current tier index, team, base demand, peak multiplier, peak probability
export const RESOURCE = [
  ["meter-receiver-1", "rented machine", 3, "meter-reading", 2.10, 2.1, 0.012, 13],
  ["meter-receiver-2", "rented machine", 3, "meter-reading", 3.40, 1.6, 0.020, 13],
  ["validator",        "rented machine", 2, "validation",   1.15, 2.4, 0.008, 11],
  ["bill-builder",     "rented machine", 4, "billing",      4.20, 2.8, 0.006, 14],
  ["report-engine",    "rented machine", 3, "reporting",    1.05, 5.2, 0.004, 12],
  ["raw-measurement-disk", "block storage", 3, "meter-reading", 340,  1.1, 0.010, 15],
  ["archive-disk",     "block storage", 4, "reporting",     610,  1.0, 0.000, 11],
  ["event-function",   "function memory", 3, "validation",  1.80, 2.0, 0.015, 11],
  ["north-gateway",    "network gateway", 2, "meter-reading", 1.30, 2.2, 0.011, 12],
  ["field-node",       "edge node",     2, "field",         0.75, 2.6, 0.007, 11],
];

// resources invisible to any usage measurement: usage ratio is undefined
// name, count, unit monthly price, why idle, does deleting it need a disruption
export const IDLE = [
  ["unattached block storage", 4, 17, "machine deleted, storage remained", "no"],
  ["unused static address", 6, 4, "migration finished, address never released", "no"],
  ["old snapshot", 23, 2.4, "no retention rule", "no"],
];

const PATTERN = [0.42, 0.38, 0.36, 0.35, 0.38, 0.48, 0.66, 0.84, 0.98, 1.06, 1.10, 1.12,
  1.08, 1.09, 1.11, 1.07, 1.00, 0.92, 0.84, 0.76, 0.68, 0.60, 0.52, 0.46];

export const history = (k) => {
  const [, , , , base, peak, p] = k;
  let s = (SEED + k[7] * 7919) >>> 0;        // seed tied to a fixed field, not the name
  const r = () => { s = (Math.imul(s, 1664525) + 1013904223) >>> 0; return s / 4294967296; };
  const v = [];
  for (let t = 0; t < HOURS; t++) {
    const weekend = (Math.floor(t / 24) % 7) >= 5 ? 0.74 : 1.0;
    let x = base * PATTERN[t % 24] * weekend * (0.92 + r() * 0.16);
    if (r() < p) x *= peak;
    v.push(x);
  }
  return v;
};

export const percentile = (v, q) => {
  const s = [...v].sort((a, b) => a - b);
  return s[Math.min(s.length - 1, Math.floor(q * s.length))];
};
export const sum = (v) => v.reduce((a, b) => a + b, 0);

// cost/scale.mjs — downsizing candidates are drawn from usage history and each candidate's *savings and risk*
// are counted together: how many hours the proposed tier fell short in the past, how much it fell short in the
// worst moment. Those that require a disruption are separated out, and idle resources are counted on their own.
import { RESOURCE, LADDER, DISRUPTION, IDLE, HOURS, SEED, history, percentile,
  sum } from "./capacity.mjs";

const write = (w, ...s) => console.log(s.map((v, i) =>
  (w[i] < 0 ? String(v).padEnd(-w[i]) : String(v).padStart(w[i]))).join(""));
const t0 = process.hrtime.bigint();

const H = RESOURCE.map(history);
const tierCapacity = (t, i) => LADDER[t][i][0], tierUnit = (t, i) => LADDER[t][i][1];
const T = sum(RESOURCE.map((k) => tierUnit(k[1], k[2])));
const IDLE_TOTAL = sum(IDLE.map(([, n, b]) => n * b));
console.log(`seed ${SEED}: ${RESOURCE.length} resources, ${HOURS} hours history, ` +
  `monthly ${T} units + idle ${IDLE_TOTAL.toFixed(1)} units (fictional currency)`);

console.log("\n1. usage history: how full is the current tier");
const A = [-22, -19, 10, 15, 9, 9, 10];
write(A, "resource", "type", "capacity", "monthly units", "p95", "peak", "p95 full");
for (let i = 0; i < RESOURCE.length; i++) {
  const k = RESOURCE[i], c = tierCapacity(k[1], k[2]), v = H[i];
  write(A, k[0], k[1], c, tierUnit(k[1], k[2]), percentile(v, 0.95).toFixed(2),
    Math.max(...v).toFixed(2), `${(100 * percentile(v, 0.95) / c).toFixed(0)}%`);
}
const occupancy = RESOURCE.map((k, i) => percentile(H[i], 0.95) / tierCapacity(k[1], k[2]));
console.log(`  p95 occupancy average ${(100 * sum(occupancy) / occupancy.length).toFixed(1)}%; ` +
  `below 50% ${occupancy.filter((x) => x < 0.5).length} resources`);

// margin: safety margin added on top of p95. Proposed tier = smallest tier that still covers it.
const candidate = (margin) => RESOURCE.map((k, i) => {
  const [name, type, cur] = k, v = H[i], target = percentile(v, 0.95) * (1 + margin);
  let proposed = cur;
  for (let j = 0; j < LADDER[type].length; j++)
    if (LADDER[type][j][0] >= target) { proposed = Math.min(cur, j); break; }
  const c = tierCapacity(type, proposed);
  const short = v.filter((x) => x > c).length;
  const shortfall = Math.max(0, (Math.max(...v) - c) / c);
  return { name, type, cur, proposed, savings: tierUnit(type, cur) - tierUnit(type, proposed),
    short, shortfall, drop: cur - proposed, remaining: proposed };
});

console.log("\n2. downsizing candidates (p95 + 20% safety margin): savings and risk together");
const B = [-22, 9, 8, 7, 9, 11, 10, 22];
write(B, "resource", "tier", "new", "drop", "savings", "short hrs.", "worst", "disruption");
const CAND = candidate(0.20);
let disrupt = 0, noDisrupt = 0;
for (const a of CAND) {
  if (a.drop === 0) continue;
  const [needs, reason] = DISRUPTION[a.type];
  if (needs === "yes") disrupt += a.savings; else noDisrupt += a.savings;
  write(B, a.name, tierCapacity(a.type, a.cur), tierCapacity(a.type, a.proposed), a.drop, a.savings,
    a.short, `${(100 * a.shortfall).toFixed(1)}%`, `${needs} (${reason})`);
}
const down = CAND.filter((a) => a.drop > 0);
const tt = sum(CAND.map((a) => a.savings));
console.log(`  ${down.length}/${RESOURCE.length} resources are candidates, savings ${tt} units/month ` +
  `(${(100 * tt / T).toFixed(1)}%)`);
console.log(`  disruptive ${disrupt} units (${(100 * disrupt / tt).toFixed(1)}%), ` +
  `non-disruptive ${noDisrupt} units`);
const risky = CAND.filter((a) => a.drop > 0 && a.short > 0);
console.log(`  ${risky.length} candidates have a past peak that exceeds the new tier: total ` +
  `${sum(risky.map((a) => a.short))} hours, worst shortfall ` +
  `${(100 * Math.max(...CAND.map((a) => a.shortfall))).toFixed(1)}%`);
console.log(`  tier steps remaining on the ladder: ${sum(down.map((a) => a.remaining))}, ` +
  `${down.filter((a) => a.remaining === 0).length} resources at the bottom tier`);

console.log("\n3. safety-margin scan: savings and short hours move together");
const C = [8, 12, 11, 11, 12, 12, 11];
write(C, "margin", "candidates", "savings", "% of bill", "short hrs.", "% of hours", "worst");
for (const p of [0, 0.20, 0.35, 0.60, 1.00]) {
  const a = candidate(p), i2 = a.filter((x) => x.drop > 0), s = sum(a.map((x) => x.savings));
  const y = sum(a.map((x) => x.short)), e = Math.max(...a.map((x) => x.shortfall));
  write(C, `${(100 * p).toFixed(0)}%`, i2.length, s, `${(100 * s / T).toFixed(1)}%`, y,
    `${(100 * y / (HOURS * RESOURCE.length)).toFixed(2)}%`, `${(100 * e).toFixed(1)}%`);
}
console.log(`  short hours stay under 1% of ${HOURS} x ${RESOURCE.length} hours across the whole scan:`);
console.log(`  what decides it is not the count, it is the worst-moment shortfall`);

console.log("\n4. idle resources: invisible in any usage-ratio report");
const D = [-26, 8, 12, 15, 45, 12];
write(D, "item", "count", "unit price", "monthly units", "why idle", "disruption");
for (const [name, n, b, why, needs] of IDLE)
  write(D, name, n, b, (n * b).toFixed(1), why, needs);
write(D, "total", sum(IDLE.map((x) => x[1])), "-", IDLE_TOTAL.toFixed(1), "-", "no");
console.log(`  idle ${IDLE_TOTAL.toFixed(1)} units/month, ${(100 * IDLE_TOTAL / T).toFixed(1)}% ` +
  `of the used bill; all of it is deleted with no disruption`);
console.log(`  and it hands over ${(100 * IDLE_TOTAL / tt).toFixed(0)}% of the downsizing ` +
  `savings on its own`);

const ms = Number(process.hrtime.bigint() - t0) / 1e6;   // raw measurement is not printed, only the band is checked
console.log(`\nfour tables within the 2000 ms band: ${ms < 2000 ? "yes" : "no"}`);

seed 8814: 10 resources, 720 hours history, monthly 875 units + idle 147.2 units (fictional currency)

1. usage history: how full is the current tier
resource              type                 capacity  monthly units      p95     peak  p95 full
meter-receiver-1      rented machine              8            101     2.42     5.13       30%
meter-receiver-2      rented machine              8            101     3.92     6.33       49%
validator             rented machine              4             52     1.31     3.15       33%
bill-builder          rented machine             16            198     4.78    13.85       30%
report-engine         rented machine              8            101     1.19     5.37       15%
raw-measurement-disk  block storage            1000             64   387.62   438.71       39%
archive-disk          block storage            2000            126   694.29   737.55       35%
event-function        function memory             8             21     2.06     4.11       26%
north-gateway         network gateway             4             79     1.48     2.81       37%
field-node            edge node                   4             32     0.86     2.23       21%
  p95 occupancy average 31.4%; below 50% 10 resources

2. downsizing candidates (p95 + 20% safety margin): savings and risk together
resource                   tier     new   drop  savings short hrs.     worst            disruption
meter-receiver-1              8       4      1       49          6     28.3%         yes (restart)
validator                     4       2      1       25          3     57.5%         yes (restart)
bill-builder                 16       8      1       97          5     73.1%         yes (restart)
report-engine                 8       2      2       74          2    168.4%         yes (restart)
raw-measurement-disk       1000     500      1       31          0      0.0%       yes (data move)
archive-disk               2000    1000      1       62          0      0.0%       yes (data move)
event-function                8       4      1       10          1      2.7%        no (next call)
north-gateway                 4       2      1       38          1     40.6%      no (tier switch)
field-node                    4       2      1       15          2     11.3%       yes (node swap)
  9/10 resources are candidates, savings 401 units/month (45.8%)
  disruptive 353 units (88.0%), non-disruptive 48 units
  7 candidates have a past peak that exceeds the new tier: total 20 hours, worst shortfall 168.4%
  tier steps remaining on the ladder: 16, 0 resources at the bottom tier

3. safety-margin scan: savings and short hours move together
  margin  candidates    savings  % of bill  short hrs.  % of hours      worst
      0%          10        458      52.3%          47       0.65%     168.4%
     20%           9        401      45.8%          20       0.28%     168.4%
     35%           8        370      42.3%          20       0.28%     168.4%
     60%           5        245      28.0%          16       0.22%     168.4%
    100%           2         64       7.3%           3       0.04%      34.2%
  short hours stay under 1% of 720 x 10 hours across the whole scan:
  what decides it is not the count, it is the worst-moment shortfall

4. idle resources: invisible in any usage-ratio report
item                         count  unit price  monthly units                                     why idle  disruption
unattached block storage         4          17           68.0            machine deleted, storage remained          no
unused static address            6           4           24.0   migration finished, address never released          no
old snapshot                    23         2.4           55.2                            no retention rule          no
total                           33           -          147.2                                            -          no
  idle 147.2 units/month, 16.8% of the used bill; all of it is deleted with no disruption
  and it hands over 37% of the downsizing savings on its own

four tables within the 2000 ms band: yes

The numbers belong to the measurement class; their inputs are the assumptions above.

Occupancy Is Low, Savings Look Large

The first table divides ten resources’ p95 demand by current capacity. Average occupancy is 31.4%, and all ten resources sit under half. The extreme case, the report engine, sits on an eight-unit tier with p95 demand of 1.19 — 15% of capacity.

The second table turns this into money. With a 20% safety margin over p95, nine resources become candidates and 401 units drop off the monthly 875-unit bill: 45.8%. The one exception is meter-receiver-2: its p95 fills half its capacity, so one tier down falls short — the two columns to the right change this figure.

Two Numbers Sit Beside the Savings

The first column counts risk: how many hours the new tier fell short in the past. On seven candidates this is above zero, totaling 20 hours — three in a thousand of the 7,200-hour history. The second number keeps this from being negligible. The capacity missing at the worst moment is 168.4% for the report engine: a two-unit new tier runs more than half short against a 5.37-unit peak. Bill-builder falls 73.1% short, validator 57.5%. At the peak, service queues or gets rejected; the measure saying “two hours” does not soften that.

The second column writes out disruption item by item. The machine tier needs a restart, block storage needs a data move, an edge node needs a node swap; only function memory and the network gateway step down without one. 353 units (88.0%) of the savings require a disruption, the non-disruptive share is 48 units.

Together, the two columns narrow the table: 38 of the non-disruptive 48 units is the network gateway, worst shortfall 40.6%. What is left — non-disruptive, no notable shortfall — is the event function, 10 units, 2.5% of the 401-unit promise, free of cost.

The disks show the reverse: the two disk candidates carry zero risk, but their 93-unit savings require a data-move disruption.

What the Safety Margin Buys

The third table scans the margin. At zero, savings are 458 units (52.3%), short hours are 47. At 20%, short hours drop to 20 and savings lose only 57 units: the curve’s most efficient point. At 60%, savings fall to 245 units and short hours drop only to 16 — 156 units paid for 4 hours gained.

The worst shortfall, by contrast, does not change at all up to a 100% margin (168.4%), because the margin is added to p95, not the peak. A rare, large peak’s risk cannot be bought with a safety margin.

The Item Nothing Touches

The fourth table counts the item the occupancy report never shows. Four unattached block storage volumes, six unused static addresses, and twenty-three old snapshots with no retention rule run 147.2 units a month — 16.8% of the used bill. They never enter the candidate list: their usage ratios are undefined, there is no demand to divide.

The comparison is the payoff: the whole 147.2 units is deleted without disruption and carries no peak risk, while downsizing’s free-of-cost share is only 10 units. The idle resource hands over 37% of the savings on its own.

Summary

• All ten resources sit under half full at p95 (average 31.4%, lowest 15%); a p95 + 20% margin makes nine resources candidates, and 401 of the monthly 875 units (45.8%) shows up as savings.
• 353 units (88.0%) of the savings require a disruption; 38 of the non-disruptive 48 units is a candidate with a worst-moment shortfall of 40.6%. The share that is both non-disruptive and risk-free is 10 units, 2.5% of the savings. Risk and disruption are independent: the two disk candidates produce no shortfall, yet both need a data-move disruption.
• The safety margin buys short hours, it cannot buy the worst shortfall: as the margin climbs from 0% to 100%, savings drop from 458 to 64 units, but the worst shortfall stays at 168.4% — the margin is added to p95, not the peak.
• An idle resource runs 147.2 units (16.8%) and never shows up in any occupancy report; it is deleted without disruption and hands over 37% of the savings on its own.

Next Step

Every number in this lesson points downward: what can be downsized, what can be deleted. A number points toward growth too, and it never shows up on the bill. The next lesson applies the scaling plan to a limit table: how many units remain before each limit, and which limit fills first.