The Serverless Concept

The previous course rented the machine, the network, and the data one at a time. The same thing happened with every rental: a decision passed to the provider, and a constraint came back in return. But even at the end of three courses, what sits underneath had not changed — the rented capacity is a continuously running server, and the meter keeps running in the hours when no work arrives. This course’s question is whether capacity itself can be rented on demand.

The measurement moves along a single axis: the same day, the same workload, two arrangements. In the first, the party that decides the instance count keeps the decision; in the second, the decision sits with the provider. The difference is not only in the bill — idle capacity, requests served at peak, and the hour the night job finishes also change.

FC1. The regional measurement network is a fictional piece of software that collects readings from water meters, validates the readings, generates the bill, and sends notifications to field crews; every duration, count, and price is fictional. FC2. A reading validation takes 400 ms; a continuously running instance carries four validations at once: 600 requests per minute. FC3. Prices are written in a fictional measure — the unit — and are not a real currency: instance-minute 0.6 units, function call 0.0005 units, 400 ms of processing 0.001 units. FC4. The day’s traffic is built with its own generator, and the seed is 20260802. FC5. In the continuous arrangement, the instance count is fixed all day; requests over capacity do not drop, they wait in the queue. FC6. In the function arrangement, every request runs the minute it arrives, and the account-level concurrency limit is 200.

The Same Day, Two Arrangements

The measurement network’s day is not flat: readings are processed in a nightly batch, the bill is generated toward morning, validation traffic flows during the day, and field notifications never stop. The model below builds the day minute by minute.

// measurement-network/two-regimes.mjs — the same day's load in two arrangements: continuous instances and on-demand functions (model)

// All units are fictional; the "unit" is a make-believe measure, not real money.
export const MIN = 1440, PROC_MS = 400, CONCURRENCY = 4;
export const INSTANCE_CAP = 60000 * CONCURRENCY / PROC_MS;      // one instance handles 600 requests/min
export const INSTANCE_MIN = 0.6, CALL = 0.0005, DURATION = 0.001; // units
export const REQUEST_UNIT = CALL + DURATION;

export const rng = (t) => { let s = t >>> 0; return () => {
  s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; };

// The regional measurement network's fictional day: nightly batch read, morning billing, daytime validation.
export const traffic = (base = 0, seed = 20260802) => {
  const r = rng(seed), load = new Array(MIN).fill(0);
  for (let d = 0; d < MIN; d++) {
    let v = base + 60;                                          // field notification: flows every minute
    if (d >= 360 && d < 1320) v += 700 * (0.75 + 0.5 * r());   // reading validation traffic
    if (d >= 120 && d < 145) v += 22000;                       // nightly batch read processing
    if (d >= 300 && d < 310) v += 6000;                        // billing generation
    load[d] = Math.round(v);
  }
  return load;
};

export const clock = (d) => `${String(Math.floor(d / 60)).padStart(2, "0")}:${String(d % 60).padStart(2, "0")}`;

// Continuous arrangement: instance count is fixed all day; requests over capacity wait in a queue.
export const continuous = (load, instances) => {
  const cap = instances * INSTANCE_CAP;
  let queue = 0, longest = 0, finish = -1;
  for (let d = 0; d < MIN; d++) {
    queue += load[d] - Math.min(queue + load[d], cap);
    if (queue > longest) longest = queue;
    if (d >= 145 && queue === 0 && finish < 0) finish = d;
  }
  const total = load.reduce((a, b) => a + b, 0);
  return { label: `continuous ${instances} instances`, cost: instances * MIN * INSTANCE_MIN,
    capacity: instances * MIN * INSTANCE_CAP, utilization: total / (instances * MIN * INSTANCE_CAP),
    atPeak: Math.min(Math.max(...load), cap), longest, backlog: queue, finish };
};

// On-demand arrangement: the capacity decision is handed off; every request runs the minute it arrives.
export const onDemand = (load) => {
  const total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load);
  return { label: "function", cost: total * REQUEST_UNIT, capacity: total, utilization: 1,
    atPeak: peak, longest: 0, backlog: 0, finish: 144,
    concurrency: Math.ceil(peak * (PROC_MS / 1000) / 60) };
};

if (import.meta.url === `file://${process.argv[1]}`) {
  const load = traffic(), total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load);
  console.log(`day: ${total} requests, peak ${peak} req/min, average ${Math.round(total / MIN)} req/min, ` +
    `peak/average ${(peak / (total / MIN)).toFixed(1)}`);
  const arrangements = [continuous(load, Math.ceil(peak / INSTANCE_CAP)), continuous(load, 12), continuous(load, 2), onDemand(load)];
  console.log("\n" + "arrangement".padEnd(25) + "cost".padEnd(11) + "idle slots".padEnd(13) +
    "idle share".padEnd(12) + "served at peak".padEnd(19) + "longest queue".padEnd(16) + "night job ends");
  for (const a of arrangements)
    console.log(a.label.padEnd(25) + a.cost.toFixed(0).padEnd(11) +
      (a.capacity - total).toFixed(0).padEnd(13) +
      `${((1 - a.utilization) * 100).toFixed(1)}%`.padEnd(12) +
      `${a.atPeak}/${peak}`.padEnd(19) + a.longest.toFixed(0).padEnd(16) +
      (a.finish < 0 ? `not finished (${a.backlog} backlog)` : clock(a.finish + 1)));
  const peakSized = arrangements[0], onDemandRow = arrangements[3];
  console.log(`\npeak sizing is ${(peakSized.cost / onDemandRow.cost).toFixed(1)}x more expensive than on-demand; ` +
    `the on-demand arrangement needs ${onDemandRow.concurrency} concurrency at peak`);
}

day: 1370625 requests, peak 22060 req/min, average 952 req/min, peak/average 23.2

arrangement              cost       idle slots   idle share  served at peak     longest queue   night job ends
continuous 37 instances  31968      30597375     95.7%       22060/22060        0               02:26
continuous 12 instances  10368      8997375      86.8%       7200/22060         371500          03:18
continuous 2 instances   1728       357375       20.7%       1200/22060         521500          18:48
function                 2056       0            0.0%        22060/22060        0               02:25

peak sizing is 15.5x more expensive than on-demand; the on-demand arrangement needs 148 concurrency at peak

The continuous arrangement’s three rows are three separate bad choices. Sizing to peak serves every request on time, and in return costs 30.6 million idle request slots in 95.7% of the day; 37 instances stand up all day, though outside twenty-five minutes two would be enough. Sizing to average pulls the bill from 31,968 units down to 1,728 units — but the batch processing that starts at 02:25 at night finishes at 18:48 in the evening, and 521,500 records wait in the queue throughout the day. The 12 instances in between are a compromise: cost 10,368, the night job finishes at 03:18, and idle share is still 86.8%.

The function row resembles none of the three. Idle slots are zero, because nothing sits idle; all 22,060 requests at peak are served, and the night job finishes the minute it arrives — at 02:25. The bill is one-fifteenth of sizing to peak. The difference is not an efficiency trick but a shift in what is measured: there, standing-up time is billed; here, work done is billed.

The last row gives the first number of the constraint: 148 concurrent calls are needed at peak, and the account limit is 200. The instance count is not decided, but 200 is a given ceiling, and if traffic grew 35% the peak would press against it.

Load Profile Scan

The day above is sparse and sudden: the peak is 23.2 times the average. In a profile like this, the function winning is not surprising. The real question is what happens as the profile flattens. The scan below adds a continuous base load on top of the day and reprices the two arrangements at every level.

// measurement-network/load-scan.mjs — where the break-even point sits as continuous base load grows (model)
import { MIN, INSTANCE_CAP, INSTANCE_MIN, REQUEST_UNIT, traffic, continuous, onDemand } from "./two-regimes.mjs";

export const point = (base) => {
  const load = traffic(base), total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load);
  const instances = Math.ceil(peak / INSTANCE_CAP), cont = continuous(load, instances), ond = onDemand(load);
  return { base, peak, average: total / MIN, ratio: peak / (total / MIN), instances,
    utilization: cont.utilization, cont: cont.cost, ond: ond.cost };
};

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log("base load".padEnd(12) + "peak/average".padEnd(15) + "instances".padEnd(11) +
    "utilization".padEnd(13) + "continuous".padEnd(12) + "function".padEnd(10) + "function/continuous".padEnd(22) + "winner");
  for (const t of [0, 2000, 5000, 10000, 20000, 30000, 40000, 50000, 65000]) {
    const n = point(t);
    console.log(`${t}`.padEnd(12) + n.ratio.toFixed(1).padEnd(15) + String(n.instances).padEnd(11) +
      `${(n.utilization * 100).toFixed(1)}%`.padEnd(13) + n.cont.toFixed(0).padEnd(12) +
      n.ond.toFixed(0).padEnd(10) + (n.ond / n.cont).toFixed(2).padEnd(22) + (n.ond < n.cont ? "function" : "continuous"));
  }
  let a = 0, b = 65000;
  for (let k = 0; k < 24; k++) { const o = (a + b) / 2; (point(Math.round(o)).ond < point(Math.round(o)).cont) ? a = o : b = o; }
  const bb = point(Math.round(a));
  console.log(`\nbreak-even base load: ${Math.round(a)} req/min — at this point peak/average ${bb.ratio.toFixed(2)}, ` +
    `utilization ${(bb.utilization * 100).toFixed(1)}%`);
  console.log(`theoretical threshold: an instance's cost per request is ${(INSTANCE_MIN / INSTANCE_CAP).toFixed(6)} units, ` +
    `the function's is ${REQUEST_UNIT.toFixed(6)} units — ratio ${((INSTANCE_MIN / INSTANCE_CAP) / REQUEST_UNIT * 100).toFixed(1)}%`);
}

base load   peak/average   instances  utilization  continuous  function  function/continuous   winner
0           23.2           37         4.3%         31968       2056      0.06                  function
2000        8.2            41         12.0%        35424       6376      0.18                  function
5000        4.5            46         21.6%        39744       12856     0.32                  function
10000       2.9            54         33.8%        46656       23656     0.51                  function
20000       2.0            71         49.2%        61344       45256     0.74                  function
30000       1.7            87         59.3%        75168       66856     0.89                  function
40000       1.5            104        65.6%        89856       88456     0.98                  function
50000       1.4            121        70.2%        104544      110056    1.05                  continuous
65000       1.3            146        75.3%        126144      142456    1.13                  continuous

break-even base load: 42248 req/min — at this point peak/average 1.49, utilization 66.7%
theoretical threshold: an instance's cost per request is 0.001000 units, the function's is 0.001500 units — ratio 66.7%

The scan shows a single rule: the function winning depends not on the size of the traffic but on its shape. When base load is zero, the function is sixteen times cheaper; at 42,248 req/min the two arrangements equalize; beyond that, the continuous arrangement wins. The break-even utilization is 66.7%, and it is not arbitrary — the last row derives the same ratio from price: an instance costs 0.001 per request, the function costs 0.0015 units.

The rule rests on a measurement, not a preference: if two-thirds of capacity is not continuously full, managing capacity yourself is expensive. The peak/average column gives the same boundary from a second angle; once the ratio drops below 1.5, the function’s advantage ends.

The Ledger of the Handed-Off Decision

The bill is only one column of the table. What actually changes is who the decisions sit with.

// measurement-network/decision-ledger.mjs — which decision was handed off, and what number came back in return (model)
import { PROC_MS, INSTANCE_MIN, traffic, onDemand } from "./two-regimes.mjs";

// [decision, in the continuous regime, in the function regime, incoming constraint (fictional number)]
export const DECISIONS = [
  ["instance count", "you", "provider", "concurrency limit 200"],
  ["scaling threshold", "you", "provider", "scaling step 60 instances/min"],
  ["machine size", "you", "provider", "memory tier 128-3008"],
  ["a job's upper duration", "you", "provider", "duration limit 900 s"],
  ["local disk", "you", "provider", "temp space 512 units"],
  ["package contents", "you", "partly you", "package size 250 units"],
  ["runtime version", "you", "partly you", "support window 24 months"],
  ["operating system patching", "you", "provider", "patch schedule invisible"],
  ["placement", "you", "provider", "region chosen, machine not chosen"],
  ["how long an instance lives", "you", "provider", "reclaimed after 300 s idle"],
  ["price per request", "provider", "provider", "—"],
];

// [constraint, workaround path]
export const WORKAROUND = [["duration limit 900 s", "split the job"],
  ["concurrency limit 200", "reserve a share"],
  ["reclaimed after 300 s idle", "pre-warm"],
  ["temp space 512 units", "externalize state"]];

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log("decision".padEnd(30) + "continuous regime".padEnd(20) + "function regime".padEnd(18) + "constraint received in return");
  for (const [a, s, i, k] of DECISIONS) console.log(a.padEnd(30) + s.padEnd(20) + i.padEnd(18) + k);
  const count = (j) => DECISIONS.filter((x) => x[j].includes("you")).length;
  const numbered = DECISIONS.filter((x) => x[2] !== "you" && /\d/.test(x[3])).length;
  console.log(`\n${count(1)} decisions kept in the continuous regime, ${count(2)} kept fully or partly in the function regime; ` +
    `${count(1) - count(2)} decisions were handed off from the start, ${numbered} constraints came back as a number`);

  const load = traffic(), night = load.slice(120, 145).reduce((a, b) => a + b, 0);
  const single = night * (PROC_MS / 1000), warm = 8, idle = warm * 1440, ez = onDemand(load).concurrency;
  const COST = [
    `${night} record night job in one piece takes ${Math.round(single)} s; fitting the limit needs ${Math.ceil(single / 900)} calls`,
    `if the ${ez} needed at peak is reserved, ${200 - ez} concurrency remains for the others`,
    `keeping ${warm} instances continuously warm costs ${idle} idle instance-minutes a day, ${(idle * INSTANCE_MIN).toFixed(0)} units`,
    `every call reads state externally: ${load.reduce((a, b) => a + b, 0)} extra read-write pairs`];
  console.log("\n" + "constraint".padEnd(32) + "workaround path".padEnd(20) + "cost");
  WORKAROUND.forEach(([k, path], j) => console.log(k.padEnd(32) + path.padEnd(20) + COST[j]));
}

decision                      continuous regime   function regime   constraint received in return
instance count                you                 provider          concurrency limit 200
scaling threshold             you                 provider          scaling step 60 instances/min
machine size                  you                 provider          memory tier 128-3008
a job's upper duration        you                 provider          duration limit 900 s
local disk                    you                 provider          temp space 512 units
package contents              you                 partly you        package size 250 units
runtime version               you                 partly you        support window 24 months
operating system patching     you                 provider          patch schedule invisible
placement                     you                 provider          region chosen, machine not chosen
how long an instance lives    you                 provider          reclaimed after 300 s idle
price per request             provider            provider          —

10 decisions kept in the continuous regime, 2 kept fully or partly in the function regime; 8 decisions were handed off from the start, 8 constraints came back as a number

constraint                      workaround path     cost
duration limit 900 s            split the job       551500 record night job in one piece takes 220600 s; fitting the limit needs 246 calls
concurrency limit 200           reserve a share     if the 148 needed at peak is reserved, 52 concurrency remains for the others
reclaimed after 300 s idle      pre-warm            keeping 8 instances continuously warm costs 11520 idle instance-minutes a day, 6912 units
temp space 512 units            externalize state   every call reads state externally: 1370625 extra read-write pairs

Ten of eleven decisions are kept in the continuous regime; in the function regime only two are kept, and both only partly — the package’s contents run up against the package-size limit, and the runtime’s version runs up against the support window. Eight decisions are handed off from the start, and all eight come back as a number. Handing off does not remove the decision; it turns the decision into a ceiling.

The lower table prices the workaround paths around those ceilings. The night batch job takes 220,600 seconds in one piece; to fit the 900-second limit it must be split into 246 separate calls, and 246 calls means a progress record the single job never needed and a question of partial success. If the 148 concurrency needed at peak is reserved, only 52 remains for every other function on the measurement network: protecting one function starves the others. Keeping eight instances continuously warm costs 11,520 idle instance-minutes a day, that is 6,912 units — the idle capacity avoided earlier coming back at a smaller scale.

The four rows share a common shape: the constraint is a number, a workaround path exists, and the cost of the workaround is not in the unit the constraint is measured in. Duration limit turns into call count, concurrency limit turns into the others’ share, instance lifetime turns into idle capacity; each constraint is paid in its own currency.

Summary

• The continuous arrangement sized to peak cost 31,968 units and 95.7% idle capacity; the function arrangement cost 2,056 units and zero idle capacity — a fifteenfold difference.
• Cutting the instance count to 2 to save money pulled the bill down to 1,728 units but pushed the night job that starts at 02:25 out to 18:48 and left 521,500 records in the queue; the savings were paid for in delay.
• The base-load scan found the break-even point at 42,248 req/min: utilization 66.7% and peak/average 1.49. If two-thirds of capacity is not continuously full, managing capacity yourself is expensive.
• Ten of eleven decisions were kept in the continuous regime; only two remained partly kept in the function regime. All eight handed-off decisions came back as a numbered constraint.
• The cost of the workaround is always in a different unit: the duration limit turns into 246 calls, instance lifetime into 11,520 idle instance-minutes a day, and the concurrency limit into the 52 share left for the others.

Next Step

This lesson invoked the function in a single way: a request arrived, the function ran, a result came back. But the night batch job and the field notification do not have to be invoked the same way — one sits at the end of a client waiting for a result, the other at the end of a queue.

The next lesson runs the same validation work through three invocation models and counts four things separately: the latency the caller sees, whether the error is visible to the caller, who owns the retry, and the risk of the same record being processed twice. It then scans batch read size: as size grows, throughput rises while the number of healthy records a single bad record drags down along with it is measured.