Virtual Machine Instances

The previous course made the rental decision and chose its geography: which layer was handed off, which region was chosen, and what egress cost held were counted. But the most concrete thing being rented — the machine the process runs on — was never addressed. When the region is chosen, not even a single processor core has been allocated yet.

This lesson rents that machine. A virtual machine instance is the unit of compute the provider carves out of a physical server and offers by type; the instance family states the ratio in which processor and memory arrive, and the size states how many multiples of that ratio are taken. The handed-off decision stops here: which physical machine it is, which processor generation, what the neighbor is doing all stay with the provider. The kept decision is family and size — and when these two items do not fit the shape of the workload, the cost is a choice paid every hour.

CC1. The fleet, that of the regional measurement network — the fictional software that collects readings from water meters — consists of four workloads; peak demands are fictional. CC2. There are three instance families, and they give 2, 4, and 8 GB of memory per vCPU; prices are in a fictional unit, not a real currency. CC3. Size steps are multiples of 2, and the ratio stays constant within a family. CC4. On every instance, the operating system and the measurement agent hold 0.6 vCPU of fixed overhead. CC5. The collector layer’s effective demand is 84 vCPU, and it is held constant through the scan. CC6. The minutes in the decision ledger are fictional; changes proceed with two instances at a time.

The Shape of the Workload versus the Shape of the Family

A workload has two dimensions: how much processor, how much memory. An instance family does not sell these two separately; it packages them at a fixed ratio and sells that. When the match ratio does not hold, the dimension that is not needed gets taken too and sits as idle resource. The model below maps four profiles to three families.

// measurement-network/matching.mjs — maps workload profiles to instance families (model; not a product)

// Family: memory (GB) per vCPU and fictional unit per vCPU-hour.
export const FAMILY = {
  "compute-heavy": { gb: 2, unit: 1.0 },
  "balanced": { gb: 4, unit: 1.3 },
  "memory-heavy": { gb: 8, unit: 1.9 },
};
export const SIZE = [2, 4, 8, 16, 32, 64];          // vCPU steps within a family

// Workload: [name, peak vCPU, peak GB, hours open per day, instance count].
export const WORKLOADS = [
  ["read collector", 12, 10, 24, 7], ["meter registry", 5, 46, 24, 3],
  ["api endpoint", 7, 26, 24, 5], ["night report", 26, 38, 2, 2],
];

export const place = ([name, cpu, gb, hours, count], familyName) => {
  const f = FAMILY[familyName];
  const i = SIZE.findIndex((s) => s >= cpu && s * f.gb >= gb);
  const size = SIZE[i], memory = size * f.gb, lowerSize = SIZE[i - 1] ?? 0;
  return { name, familyName, size, memory, count, hours,
    idleCpu: size - cpu, idleGb: memory - gb,
    bottleneck: lowerSize < cpu ? (lowerSize * f.gb < gb ? "both" : "processor") : "memory",
    daily: size * f.unit * hours * count };
};

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log("workload / family".padEnd(24) + "instance".padEnd(15) + "idle vCPU".padEnd(12) +
    "idle GB".padEnd(12) + "bottleneck".padEnd(12) + "daily unit");
  const cheapest = {};
  for (const job of WORKLOADS) {
    console.log(`${job[0]} (${job[1]} vCPU, ${job[2]} GB, ${job[4]} instances, ${job[3]} h/day)`);
    let best = null;
    for (const name of Object.keys(FAMILY)) {
      const fit = place(job, name);
      if (!best || fit.daily < best.daily) best = fit;
      console.log("  " + name.padEnd(22) + `${fit.size}/${fit.memory} GB`.padEnd(15) +
        `${fit.idleCpu} (${Math.round(100 * fit.idleCpu / fit.size)}%)`.padEnd(12) +
        `${fit.idleGb} (${Math.round(100 * fit.idleGb / fit.memory)}%)`.padEnd(12) +
        fit.bottleneck.padEnd(12) + fit.daily.toFixed(1));
    }
    cheapest[job[0]] = best;
  }
  console.log("\nfleet daily cost — standardizing on one family versus choosing per workload:");
  console.log("policy".padEnd(28) + "daily unit".padEnd(15) + "difference".padEnd(12) +
    "idle vCPU".padEnd(12) + "idle GB");
  const mixed = Object.values(cheapest);
  const sum = (l) => [l.reduce((t, fit) => t + fit.daily, 0),
    l.reduce((t, fit) => t + fit.idleCpu * fit.count, 0), l.reduce((t, fit) => t + fit.idleGb * fit.count, 0)];
  const [mixedCost] = sum(mixed);
  for (const name of Object.keys(FAMILY)) {
    const [cost, idleCpu, idleGb] = sum(WORKLOADS.map((job) => place(job, name)));
    console.log(`only ${name}`.padEnd(28) + cost.toFixed(1).padEnd(15) +
      `+${Math.round(100 * (cost - mixedCost) / mixedCost)}%`.padEnd(12) + String(idleCpu).padEnd(12) + String(idleGb));
  }
  const [cost, idleCpu, idleGb] = sum(mixed);
  console.log("per-workload choice".padEnd(28) + cost.toFixed(1).padEnd(15) + "—".padEnd(12) +
    String(idleCpu).padEnd(12) + String(idleGb));
}

workload / family       instance       idle vCPU   idle GB     bottleneck  daily unit
read collector (12 vCPU, 10 GB, 7 instances, 24 h/day)
  compute-heavy         16/32 GB       4 (25%)     22 (69%)    processor   2688.0
  balanced              16/64 GB       4 (25%)     54 (84%)    processor   3494.4
  memory-heavy          16/128 GB      4 (25%)     118 (92%)   processor   5107.2
meter registry (5 vCPU, 46 GB, 3 instances, 24 h/day)
  compute-heavy         32/64 GB       27 (84%)    18 (28%)    memory      2304.0
  balanced              16/64 GB       11 (69%)    18 (28%)    memory      1497.6
  memory-heavy          8/64 GB        3 (38%)     18 (28%)    both        1094.4
api endpoint (7 vCPU, 26 GB, 5 instances, 24 h/day)
  compute-heavy         16/32 GB       9 (56%)     6 (19%)     memory      1920.0
  balanced              8/32 GB        1 (13%)     6 (19%)     both        1248.0
  memory-heavy          8/64 GB        1 (13%)     38 (59%)    processor   1824.0
night report (26 vCPU, 38 GB, 2 instances, 2 h/day)
  compute-heavy         32/64 GB       6 (19%)     26 (41%)    both        128.0
  balanced              32/128 GB      6 (19%)     90 (70%)    processor   166.4
  memory-heavy          32/256 GB      6 (19%)     218 (85%)   processor   243.2

fleet daily cost — standardizing on one family versus choosing per workload:
policy                      daily unit     difference  idle vCPU   idle GB
only compute-heavy          7040.0         +36%        166         290
only balanced               6406.4         +24%        78          642
only memory-heavy           8268.8         +60%        54          1506
per-workload choice         5158.4         —           54          290

The bottleneck column is the only column that says the family was chosen wrong. meter registry is a memory-heavy job: in the compute-heavy family, getting 46 GB requires taking 32 vCPU, and 27 vCPU — 84 percent of the processor — sits idle. The same job fits an 8-vCPU instance in the memory-heavy family; idle processor drops from 27 to 3, and the daily cost falls from 2,304 to 1,094. The reverse holds too: read collector is processor-heavy, and placed in the memory-heavy family, 118 GB of memory — 92 percent of memory — sits idle.

Idle resource is a cost but not the only cost. In the api endpoint row, the bottleneck column reads “both”: one size step down falls short on both processor and memory at once. That is the sign the family is right for that match; when both dimensions are at the limit at the same time, the shape of the package taken matches the shape of the job. A single dimension reading “processor” or “memory” instead of “both” shows that one dimension of that instance was taken larger than needed.

The lower table turns the decision into a policy. Standardizing the fleet on a single family simplifies management; its cost shows up in three numbers. Even the closest single family costs 24 percent more per day and raises idle memory from 290 GB to 642 GB. The worst single family costs 60 percent more. When the decision is made separately four times for four workloads, 5,158.4 fictional units are paid per day; reduced to a single family, 6,406.4 units are paid; the 1,248-unit gap is standardization’s daily price.

How Many Pieces to Split the Same Demand Into

After the family is chosen, the second decision follows: whether the same total capacity comes from a small number of large instances or a large number of small ones. This decision does not show up in the size row, because total vCPU is roughly the same at both ends. The difference arises in two other places: in each instance’s own fixed overhead, and in the share lost when one instance drops.

// measurement-network/sizing.mjs — scan of splitting the same demand across different instance sizes (model)
import { FAMILY, SIZE } from "./matching.mjs";

const DEMAND = 84;                // effective vCPU demand of the collector layer (fictional)
const OVERHEAD = 0.6;             // OS + measurement agent per instance (vCPU)
const HOURS = 24, FAMILY_NAME = "balanced";

export const scan = (size) => {
  const effective = size - OVERHEAD, count = Math.ceil(DEMAND / effective);
  const provisioned = count * size, overhead = count * OVERHEAD;
  const daily = provisioned * FAMILY[FAMILY_NAME].unit * HOURS;
  return { size, count, provisioned, overhead, rounding: provisioned - DEMAND - overhead,
    idle: provisioned - DEMAND, shortfall: Math.max(0, DEMAND - (count - 1) * effective),
    daily, spare: size * FAMILY[FAMILY_NAME].unit * HOURS };
};

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log(`demand ${DEMAND} effective vCPU, ${OVERHEAD} vCPU fixed overhead per instance, ${FAMILY_NAME} family\n`);
  console.log("size".padEnd(9) + "instances".padEnd(11) + "provisioned".padEnd(13) + "fixed overhead".padEnd(16) +
    "rounding".padEnd(11) + "idle".padEnd(9) + "one failure".padEnd(13) + "N+1 cost");
  for (const s of SIZE) {
    const t = scan(s);
    console.log(`${s} vCPU`.padEnd(9) + String(t.count).padEnd(11) + `${t.provisioned}`.padEnd(13) +
      t.overhead.toFixed(1).padEnd(16) + t.rounding.toFixed(1).padEnd(11) +
      `${Math.round(100 * t.idle / t.provisioned)}%`.padEnd(9) +
      `${(100 * t.shortfall / DEMAND).toFixed(1)}%`.padEnd(13) +
      `+${(100 * t.spare / t.daily).toFixed(1)}%`);
  }
  const lowest = SIZE.map(scan).reduce((a, b) => (b.idle < a.idle ? b : a));
  console.log(`\nlowest idle: ${lowest.size} vCPU (${lowest.idle.toFixed(1)} vCPU idle, ` +
    `${lowest.count} instances); there, one failure exposes ${(100 * lowest.shortfall / DEMAND).toFixed(1)}% of demand`);
}

demand 84 effective vCPU, 0.6 vCPU fixed overhead per instance, balanced family

size     instances  provisioned  fixed overhead  rounding   idle     one failure  N+1 cost
2 vCPU   61         122          36.6            1.4        31%      0.0%         +1.6%
4 vCPU   25         100          15.0            1.0        16%      2.9%         +4.0%
8 vCPU   12         96           7.2             4.8        13%      3.1%         +8.3%
16 vCPU  6          96           3.6             8.4        13%      8.3%         +16.7%
32 vCPU  3          96           1.8             10.2       13%      25.2%        +33.3%
64 vCPU  2          128          1.2             42.8       34%      24.5%        +50.0%

lowest idle: 8 vCPU (12.0 vCPU idle, 12 instances); there, one failure exposes 3.1% of demand

The two waste items move in opposite directions. Fixed overhead — each instance’s operating system and measurement agent — holds at 36.6 vCPU on the 2-vCPU instance and drops to 1.2 vCPU on the 64-vCPU instance; on the small instance, nearly a third of the rented capacity is spent doing no work. Rounding waste goes the other way: because demand does not divide evenly into the instance size, 42.8 extra vCPU is taken on the large instance. Total idle is therefore not a flat curve; it falls to its lowest point between 8 and 32 vCPU and rises at both ends.

The failure column, however, is one-directional, and it is the one that actually steers the decision. In the 61-instance layout, one instance dropping exposes 0 percent of demand; in the three-instance layout of 32-vCPU instances, the same failure exposes 25 percent of demand. The last column is the price of fixing that: adding one spare instance raises the daily cost by 1.6 percent in the 61-instance layout and by 33.3 percent in the three-instance layout. The point that minimizes idle resource and the point that minimizes failure exposure are not the same place; sizing is choosing a spot between these two numbers, not searching for one correct size.

On an Instance, Which Decision Sits Where

Once both decisions are made, the third question follows: if they turn out wrong, can they be undone? Some of the items on an instance change while it runs, some require stopping the instance, and some are given at launch time and never change again.

// measurement-network/decision-ledger.mjs — which decision on an instance sits where, and which cannot be undone (model)
import { WORKLOADS } from "./matching.mjs";

const FLEET = WORKLOADS.reduce((t, job) => t + job[4], 0);   // 17 instances
const WINDOW = 2, ROUNDS = Math.ceil(FLEET / WINDOW);         // instances pulled out at once

// Kept item: [name, reversibility, change path, minutes per instance]
const KEPT = [
  ["instance family", "yes", "stop-modify-start", 4],
  ["instance size", "yes", "stop-modify-start", 4],
  ["root disk size", "one-way", "grows, never shrinks", 2],
  ["processor architecture", "no", "image rebuild + re-creation", 35],
  ["subnet placement", "no", "re-creation", 35],
  ["root disk encryption", "no", "re-creation", 35],
];
const PROVIDER = ["physical machine", "processor generation", "oversubscription ratio",
  "neighboring workload", "maintenance window timing"];

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log(`${FLEET}-instance fleet; changes proceed ${WINDOW} instances at a time, over ${ROUNDS} rounds\n`);
  console.log("kept item".padEnd(26) + "reversible".padEnd(17) +
    "change path".padEnd(40) + "fleet wall-clock hours");
  for (const [name, reversible, path, min] of KEPT)
    console.log(name.padEnd(26) + reversible.padEnd(17) + path.padEnd(40) + `${((ROUNDS * min) / 60).toFixed(1)} h`);
  const irreversible = KEPT.filter((x) => x[1] === "no");
  console.log(`\n${PROVIDER.length} items stay with the provider: ${PROVIDER.join(", ")}`);
  console.log(`irreversible items: ${irreversible.length}; if all three change together the fleet ` +
    `renews in ${(ROUNDS * Math.max(...irreversible.map((x) => x[3])) / 60).toFixed(1)} hours, ` +
    `a wrong family choice is fixed in ${(ROUNDS * 4 / 60).toFixed(1)} hours`);
}

17-instance fleet; changes proceed 2 instances at a time, over 9 rounds

kept item                 reversible       change path                             fleet wall-clock hours
instance family           yes              stop-modify-start                       0.6 h
instance size             yes              stop-modify-start                       0.6 h
root disk size            one-way          grows, never shrinks                    0.3 h
processor architecture    no               image rebuild + re-creation             5.3 h
subnet placement          no               re-creation                             5.3 h
root disk encryption      no               re-creation                             5.3 h

5 items stay with the provider: physical machine, processor generation, oversubscription ratio, neighboring workload, maintenance window timing
irreversible items: 3; if all three change together the fleet renews in 5.3 hours, a wrong family choice is fixed in 0.6 hours

Six of the eleven items are kept, five are with the provider. For the ones that stay with the provider, the only thing you can do is measure: two same-sized instances running at different speeds is a difference in neighboring workload or processor generation, not a decision but an observation.

The six kept items split into three groups by reversibility. Family and size are reversible items: a wrong family choice is fixed in 0.6 hours across a 17-instance fleet, and it can be revisited every month as idle resource is counted. Root disk size is one-way — it grows, never shrinks; a disk taken larger than needed is paid for the instance’s entire life. The three irreversible items — processor architecture, the subnet the instance is placed in, and whether the root disk is encrypted — are given at launch time, and when a change is wanted the instance does not change, it is re-created: the fleet renews in 5.3 hours, nine times as long as a family change. These three are chosen on day one, and fixing them gets more expensive as the fleet grows; work that takes 5.3 hours at 17 instances takes more than two days at 170 instances.

Summary

• When the family match does not hold, the cost is paid every hour: meter registry holds 27 vCPU (84 percent of the processor) idle in the compute-heavy family; in the right family, the same job runs with 3 vCPU idle and the daily cost falls from 2,304 to 1,094 fictional units.
• Standardizing on a single family costs 24 percent more per day in the best case and 60 percent more in the worst case; a separate choice for four workloads costs 5,158.4 units, the closest single family 6,406.4 units.
• In sizing, the two waste items move in opposite directions: fixed overhead is 36.6 vCPU on the 2-vCPU instance and 1.2 vCPU on the 64-vCPU instance; rounding waste climbs from 1.4 to 42.8 vCPU, and total idle is lowest in the 8–32 vCPU range.
• Failure exposure is one-directional: one instance dropping exposes 0 percent of demand in the 61-instance layout and 25 percent in the three-instance layout; the cost of a spare instance climbs from 1.6 percent to 33.3 percent.
• Six of the eleven items are kept, five are with the provider; the three irreversible items are processor architecture, subnet placement, and root disk encryption — changing any of the three renews the fleet in 5.3 hours, a family change in 0.6 hours.

Next Step

Every number in this lesson measured what happens after an instance opens: how many vCPU sit idle, how much a failure takes away, how many hours changing an item costs. Startup itself passed in a single line — “re-creation, 35 minutes” — and what happens inside those 35 minutes was never asked.

An instance does not open with an empty disk; it opens from a machine image, and the more ready-made work sits inside that image, the shorter the startup. The next lesson scans the balance between what is embedded in the image and what is done at startup, and asks: as each item is embedded in the image, how many seconds does startup shorten by, how many days does the image’s freshness drop by, and how many calls to how many external dependencies does the work remaining at startup make?