Resource Requests and Limits

The objects running on the cluster have been declared: a pod was defined, its copies were turned into a replica set, it was placed inside a namespace, and that namespace’s total resources were shared out with a quota. Through all of this, one question was never asked: which pod lands on which node. A quota sets a ceiling; it does not place. The replica count says how many there will be, not where.

Placement is the first mechanical decision handed from a human to the system, and it rests on two numbers. These two numbers, sitting side by side in the pod declaration, do their work in separate places: the resource request in the scheduler’s ledger, the limit in the node’s runtime. How many nodes are needed is not asked here — that calculation, as the larger of the load constraint and the failure constraint, was done in capacity planning, and it is an input to this lesson. The question here is what placement looks like given a fixed number of nodes.

What is measured is not a real cluster but a model built with node. The workload is the services of the regional measurement network (the running fiction): the reading collector, the verifier, billing, the work order service, and the nightly batch job.

The Request Keeps the Ledger, the Limit Does the Cutting

The model consists of three nodes and eighteen pods. The scheduler places on the first node that fits and sees only the declared request; it never sees actual usage. Actual usage is derived from the seed of a generator we wrote ourselves and is spread across a 35 percent band around the baseline.

// scheduling/request.mjs — the effect of the request and the limit on placement. What gets built is a MODEL;
// no real cluster runs. The workload is the regional measurement network's (the running fiction) services.
export const NODE = { cpu: 3000, memory: 4096 };  // SS1: capacity assignable per node
export const NODE_COUNT = 3;

// SS2: service, pod count, base cpu unit, base memory (MiB)
const SERVICE = [
  ["reading-collector", 6, 420, 384], ["verifier", 4, 650, 704],
  ["billing", 3, 300, 512], ["work-order", 3, 180, 256], ["nightly-batch", 2, 900, 1408],
];
const SEED = 20260801;                       // the generator is our own; the seed is visible

export function generator(seed = SEED) {
  let s = seed % 2147483647;
  return () => (s = (s * 48271) % 2147483647) / 2147483647;
}

// SS3: actual usage falls in a +/-35% band around the base; it is measured, not declared
export function workload() {
  const r = generator(), k = [];
  for (const [name, count, c, m] of SERVICE)
    for (let i = 0; i < count; i++) {
      const deviation = 0.65 + 0.7 * r();
      k.push({ name: `${name}-${i + 1}`, baseC: c, baseM: m,
               actualC: Math.round(c * deviation), actualM: Math.round(m * deviation) });
    }
  return k;
}

// The scheduler sees only the DECLARED request; it never sees actual usage.
export function place(pods, n = NODE_COUNT) {
  const nodes = Array.from({ length: n }, () => ({ reqC: 0, reqM: 0, pods: [] }));
  const unplaced = [];
  for (const k of pods) {
    const d = nodes.find((d) => d.reqC + k.reqC <= NODE.cpu &&
                                d.reqM + k.reqM <= NODE.memory);
    if (d === undefined) { unplaced.push(k); continue; }
    d.reqC += k.reqC; d.reqM += k.reqM; d.pods.push(k);
  }
  return { nodes, unplaced };
}

// SS4: limit = request x 1.25. Exceeding the cpu limit throttles, exceeding the memory limit kills.
export const LIMIT_FACTOR = 1.25;

export function measure(pods, n = NODE_COUNT) {
  const { nodes, unplaced } = place(pods, n);
  let waste = 0, contention = 0, throttled = 0, killed = 0, placed = 0;
  for (const d of nodes) {
    const aC = d.pods.reduce((t, k) => t + k.actualC, 0);
    d.actualC = aC;
    contention += Math.max(0, aC - NODE.cpu);
    placed += d.pods.length;
    for (const k of d.pods) {
      waste += Math.max(0, k.reqC - k.actualC);
      if (k.limC > 0 && k.actualC > k.limC) throttled++;
      if (k.limM > 0 && k.actualM > k.limM) killed++;
    }
  }
  return { nodes, placed, unplaced: unplaced.length, waste, contention,
           throttled, killed,
           spareRequest: nodes.reduce((t, d) => t + (NODE.cpu - d.reqC), 0) };
}

export function configure(pods, mode) {
  return pods.map((k) => {
    const c = mode === "none" ? 0 : mode === "realistic" ? k.baseC : k.baseC * 2;
    const m = mode === "none" ? 0 : mode === "realistic" ? k.baseM : k.baseM * 2;
    return { ...k, reqC: c, reqM: m,
             limC: mode === "none" ? 0 : Math.round(c * LIMIT_FACTOR),
             limM: mode === "none" ? 0 : Math.round(m * LIMIT_FACTOR) };
  });
}

if (process.argv[1].endsWith("request.mjs")) {
  const k = workload();
  const totC = k.reduce((t, x) => t + x.actualC, 0), totM = k.reduce((t, x) => t + x.actualM, 0);
  console.log(`model: ${k.length} pods, ${NODE_COUNT} nodes x ${NODE.cpu} cpu ` +
    `units / ${NODE.memory} MiB = ${NODE_COUNT * NODE.cpu} / ` +
    `${NODE_COUNT * NODE.memory}`);
  console.log(`actual total usage ${totC} cpu units, ${totM} MiB (seed ${SEED})\n`);
  const bas = (s) => s.padStart(16);
  console.log(`${"setting".padEnd(18)}${bas("placed")}${bas("unplaced")}${bas("spare request")}` +
    `${bas("waste")}${bas("contention")}${bas("throttled")}${bas("killed")}`);
  for (const mode of ["none", "realistic", "inflated"]) {
    const o = measure(configure(k, mode));
    console.log(`${mode.padEnd(18)}${bas(String(o.placed))}${bas(String(o.unplaced))}` +
      `${bas(String(o.spareRequest))}${bas(String(o.waste))}${bas(String(o.contention))}` +
      `${bas(String(o.throttled))}${bas(String(o.killed))}`);
  }
  console.log(`\nactual occupancy per node (cpu units / ${NODE.cpu}):`);
  for (const mode of ["none", "realistic", "inflated"]) {
    const o = measure(configure(k, mode));
    console.log(`  ${mode.padEnd(12)}` + o.nodes.map((d, i) =>
      `d${i}: ${d.pods.length} pods ${((d.actualC / NODE.cpu) * 100).toFixed(0)}%`)
      .join("   "));
  }
  console.log(`\ninformation needed for the decision: ${k.length} pods x 2 declarations = ${k.length * 2}, ` +
    `${NODE_COUNT} nodes x 2 measurements = ${NODE_COUNT * 2}, total ${k.length * 2 + NODE_COUNT * 2}`);
  console.log(`item the scheduler does not see: actual usage (${k.length} pods x 2 = ${k.length * 2})`);
}

model: 18 pods, 3 nodes x 3000 cpu units / 4096 MiB = 9000 / 12288
actual total usage 8503 cpu units, 10193 MiB (seed 20260801)

setting                     placed        unplaced   spare request           waste      contention       throttled          killed
none                            18               0            9000               0            5503               0               0
realistic                       18               0             640             675             557               1               1
inflated                        11               7             280            3934               0               0               0

actual occupancy per node (cpu units / 3000):
  none        d0: 18 pods 283%   d1: 0 pods 0%   d2: 0 pods 0%
  realistic   d0: 8 pods 108%   d1: 5 pods 110%   d2: 5 pods 65%
  inflated    d0: 4 pods 49%   d1: 4 pods 56%   d2: 3 pods 55%

information needed for the decision: 18 pods x 2 declarations = 36, 3 nodes x 2 measurements = 6, total 42
item the scheduler does not see: actual usage (18 pods x 2 = 36)

Three Settings, Three Separate Failures

Three rows build three separate worlds. In the no request setting, every node in the ledger is empty, so all eighteen pods pile onto the first node: the scheduler thinks it made no mistake while the node’s actual occupancy reaches 283 percent and 5,503 units of contention are born. Resource sharing is left to the operating system here, and who is being slowed down cannot be read from any declaration.

In the realistic request setting, eighteen pods spread across three nodes and spare request capacity drops to 640 units. The setting’s imperfection shows up in two columns: because actual usage falls short of the declaration, 675 units are reserved and unused, while where it exceeds the declaration, 557 units of contention remain. One pod is throttled for exceeding its cpu limit, and one pod is killed for exceeding its memory limit. The limit’s enforcement is independent of placement: both pods are placed, both are within the ledger’s rules.

In the inflated request setting there is neither contention nor a single kill; the price is that seven pods never get placed at all. The nodes actually run half-empty (49, 56, 55 percent) and 3,934 units sit reserved and unused. This setting looks like a safety setting; it is a capacity loss.

Scanning the Drift Separates Two Costs

Once the declaration is written as a multiple of actual usage and scanned, the cost of each direction can be counted separately. The same scan also shows the effect of a declaration that is made once and left in place, and of the choice of limit factor. The factor reads like a measure of information quality: the further it drifts from one, the more badly the declaration misrepresents the truth.

// scheduling/deviation.mjs — the request's drift from actual usage is scanned: the waste of overstatement,
// the contention of understatement. The model and workload come from the previous file.
import { NODE, NODE_COUNT, LIMIT_FACTOR, workload, measure } from "./request.mjs";

const pods = workload();
const CAPACITY = NODE_COUNT * NODE.cpu;

// Request = actual usage x factor. A factor of 1 means the declaration is exact.
const declare = (k, factor, growth = 1) => k.map((x) => {
  const c = Math.round(x.actualC * factor), m = Math.round(x.actualM * factor);
  return { ...x, reqC: c, reqM: m,
           actualC: Math.round(x.actualC * growth), actualM: Math.round(x.actualM * growth),
           limC: Math.round(c * LIMIT_FACTOR), limM: Math.round(m * LIMIT_FACTOR) };
});

const s = (x) => String(x).padStart(12);
console.log(`the request's drift from actual usage (${pods.length} pods, capacity ${CAPACITY}):`);
console.log(`${"factor".padEnd(10)}${s("placed")}${s("unplaced")}${s("waste")}` +
  `${s("contention")}${s("throttled")}${s("killed")}`);
for (const factor of [0.5, 0.75, 0.9, 1.0, 1.25, 1.5, 2.0]) {
  const o = measure(declare(pods, factor));
  console.log(`${("x" + factor.toFixed(2)).padEnd(10)}${s(o.placed)}${s(o.unplaced)}` +
    `${s(o.waste)}${s(o.contention)}${s(o.throttled)}${s(o.killed)}`);
}

// The declaration is made once while usage grows: the cost of a stale declaration.
console.log(`\nwith the declaration held fixed at x1.00 while actual usage grows (SS5):`);
console.log(`${"growth".padEnd(10)}${s("placed")}${s("contention")}${s("throttled")}${s("killed")}`);
for (const g of [1.0, 1.1, 1.2, 1.3, 1.5]) {
  const o = measure(declare(pods, 1.0, g));
  console.log(`${(((g - 1) * 100).toFixed(0) + "%").padEnd(10)}${s(o.placed)}${s(o.contention)}` +
    `${s(o.throttled)}${s(o.killed)}`);
}

// SS6: usage rises 30% during a short-lived spike. The limit factor is scanned.
const SPIKE = 1.3;
console.log(`\nlimit factor (request declared at x1.00, usage 30% higher at the spike):`);
console.log(`${"factor".padEnd(10)}${s("throttled")}${s("killed")}${s("contention")}${s("limit total")}`);
for (const c of [1.0, 1.1, 1.25, 1.5, 2.0]) {
  const k2 = pods.map((x) => ({ ...x, reqC: x.actualC, reqM: x.actualM,
    actualC: Math.round(x.actualC * SPIKE), actualM: Math.round(x.actualM * SPIKE),
    limC: Math.round(x.actualC * c), limM: Math.round(x.actualM * c) }));
  const o = measure(k2);
  console.log(`${("x" + c.toFixed(2)).padEnd(10)}${s(o.throttled)}${s(o.killed)}${s(o.contention)}` +
    `${s(k2.reduce((t, x) => t + x.limC, 0))}`);
}
console.log(`\nat the x2.00 limit factor the limit total reaches ${(pods.reduce((t, x) => t + x.actualC, 0) * 2 /
  CAPACITY).toFixed(2)}x capacity; the limit is not a reservation, it is a ceiling`);

the request's drift from actual usage (18 pods, capacity 9000):
factor          placed    unplaced       waste  contention   throttled      killed
x0.50               18           0           0        2965          18          18
x0.75               18           0           0        1916          18          18
x0.90               18           0           0         557           0           0
x1.00               17           1           0           0           0           0
x1.25               14           4        1720           0           0           0
x1.50               13           5        2876           0           0           0
x2.00                9           9        4322           0           0           0

with the declaration held fixed at x1.00 while actual usage grows (SS5):
growth          placed  contention   throttled      killed
0%                  17           0           0           0
10%                 17         470           0           0
20%                 17        1058           0           0
30%                 17        1646          17          17
50%                 17        2879          17          17

limit factor (request declared at x1.00, usage 30% higher at the spike):
factor       throttled      killed  contention limit total
x1.00               17          17        1646        8503
x1.10               17          17        1646        9355
x1.25               17          17        1646       10630
x1.50                0           0        1646       12760
x2.00                0           0        1646       17006

at the x2.00 limit factor the limit total reaches 1.89x capacity; the limit is not a reservation, it is a ceiling

The Waste of Overstatement, the Contention of Understatement

The scan is not symmetric. As the factor drops below 1, every pod still gets placed and the bill piles up as contention: 2,965 units at half declaration, 1,916 units at three-quarter declaration, and all eighteen of the eighteen pods are both throttled and killed. Once the factor climbs above 1, contention drops to zero and the bill shifts instead to unplaced pods and waste: at the 1.5x factor five pods fail to place and 2,876 units sit idle, and at the 2x factor nine pods fail to place.

The exact-declaration row teaches something too. At a factor of exactly 1, contention is zero and waste is zero, and yet one pod still fails to place. This is not a shortage of information; it is that the first-fit placement rule never looks back — the remaining gaps are, one by one, not quite enough, so fragmentation remains. Even with perfect information, the placement rule produces a loss.

The second table measures how a declaration goes stale. While the declaration stays put, a 10 percent growth in actual usage produces 470 units of contention, a 20 percent growth produces 1,058 units, and at 30 percent seventeen pods are simultaneously throttled and killed. A declaration is not a measurement but a statement, and unless it is refreshed like a measurement, it is the oldest piece of information the system holds.

The third table shows what the limit is not. When the factor is raised from 1 to 2, the number of throttled pods drops from seventeen to zero, while contention never moves from 1,646 units. The limit only cuts off the top of a single pod; it does not protect the node’s total. At the 2x factor the sum of limits can reach 1.89 times the cluster’s capacity, because the limit is not a reservation but a ceiling. The only number that protects the node’s total is the request.

The Cost of Information and the Unseen Item

For the placement decision, the system holds forty-two information items: two declarations each for eighteen pods (thirty-six items) and two measurements each for three nodes (six items). These items do not cost the same. The six measurements arrive regularly from the node agent and are fresh. The thirty-six declarations are written once, nobody verifies them, and they quietly go wrong as load changes.

Measuring this item is not impossible, only costly: it would require collecting, storing, and comparing a continuous usage series against the declaration for every pod. This lesson does not hand that series to the scheduler; what happens when it is handed over is the subject of this topic’s last two lessons.

The forty-third item does not exist at all: actual usage. The scheduler never sees actual usage in any table in this lesson. All three wrong decisions come from the same gap — piling eighteen pods onto a single node in the no-request setting, missing 557 units of contention in the realistic setting, and leaving seven pods waiting while half-empty nodes sit idle in the inflated setting. In all three, the scheduler’s ledger is consistent; what is inconsistent is the ledger and the node.

Summary

• The resource request sets the scheduler’s ledger, the limit sets the node’s runtime: a pod can be placed and still be throttled or killed.
• The three settings’ costs are distinct: no request means eighteen pods on one node at 283 percent occupancy, realistic means 640 units of spare request with 557 units of contention, inflated means seven unplaced pods with 3,934 units of waste.
• Understatement’s cost is contention and cutoffs (2,965 units at half declaration, all eighteen pods throttled); overstatement’s cost is unplaced pods and waste (nine pods fail to place at the 2x factor, 4,322 units sit idle).
• Even with an exact declaration, one pod still fails to place: even with complete information, the first-fit placement rule produces fragmentation.
• Raising the limit factor fourfold drops throttled pods from seventeen to zero and never touches the node’s 1,646 units of contention; the sum of limits can reach 1.89 times capacity.
• The decision rests on forty-two information items, thirty-six of them declarations that are never refreshed; the forty-third item, actual usage, never appears, and it is the source of all three wrong decisions.

Next Step

This lesson’s scheduler asks a single question: does the remaining request fit. Placement has other sides, and none of them concern capacity. Having the verifier run on nodes with fast disks, keeping copies of the same service from piling onto one node, guaranteeing that a surviving copy remains after a zone is lost — each of these adds a new constraint to the scheduler and narrows the set of eligible nodes. The next lesson writes the scheduler as a constraint solver and counts how many pods become unplaceable as constraints are added.