--- title: 'The Serverless Concept' source: 'https://academia.sh/en/courses/serverless/serverless-concept' course: 'Serverless and Edge Computing' language: en updated: '2026-09-13T05:07:40+00:00' license: 'CC BY-SA 4.0' objectives: - 'Measures the same workload in two arrangements: idle capacity, requests served at peak, and the hour the job finishes' - 'Scans the base load to separate the region where the function wins from where it loses, and works out the break-even load level' - 'Counts which decisions handing off capacity management takes away, and the number each decision comes back as' - "Measures the workaround path for each constraint and that path's own cost in a separate unit" --- # The Serverless Concept The same day's workload is run in two arrangements: the always-on instance and the function invoked on demand; idle capacity, requests served at peak, and the hour the night job finishes are counted side by side, the base load is scanned from zero to sixty-five thousand to work out the break-even load level, and the ten handed-off decisions with the eight numbered constraints that come back in return are gathered into a single ledger. The previous course rented the machine, the network, and the data one at a time. The same thing happened with every rental: a decision passed to the provider, and a constraint came back in return. But even at the end of three courses, what sits underneath had not changed — the rented capacity is a **continuously running server**, and the meter keeps running in the hours when no work arrives. This course's question is whether capacity itself can be rented on demand. The measurement moves along a single axis: the same day, the same workload, two arrangements. In the first, the party that decides the instance count keeps the decision; in the second, the decision sits with the provider. The difference is not only in the bill — idle capacity, requests served at peak, and the hour the night job finishes also change. **FC1.** The regional measurement network is a **fictional** piece of software that collects readings from water meters, validates the readings, generates the bill, and sends notifications to field crews; every duration, count, and price is fictional. **FC2.** A reading validation takes 400 ms; a continuously running instance carries four validations at once: 600 requests per minute. **FC3.** Prices are written in a fictional measure — the **unit** — and are not a real currency: instance-minute 0.6 units, function call 0.0005 units, 400 ms of processing 0.001 units. **FC4.** The day's traffic is built with its own generator, and the seed is 20260802. **FC5.** In the continuous arrangement, the instance count is fixed all day; requests over capacity do not drop, they wait in the queue. **FC6.** In the function arrangement, every request runs the minute it arrives, and the account-level concurrency limit is 200. ## The Same Day, Two Arrangements The measurement network's day is not flat: readings are processed in a nightly batch, the bill is generated toward morning, validation traffic flows during the day, and field notifications never stop. The model below builds the day minute by minute. ```js // measurement-network/two-regimes.mjs — the same day's load in two arrangements: continuous instances and on-demand functions (model) // All units are fictional; the "unit" is a make-believe measure, not real money. export const MIN = 1440, PROC_MS = 400, CONCURRENCY = 4; export const INSTANCE_CAP = 60000 * CONCURRENCY / PROC_MS; // one instance handles 600 requests/min export const INSTANCE_MIN = 0.6, CALL = 0.0005, DURATION = 0.001; // units export const REQUEST_UNIT = CALL + DURATION; export const rng = (t) => { let s = t >>> 0; return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; }; // The regional measurement network's fictional day: nightly batch read, morning billing, daytime validation. export const traffic = (base = 0, seed = 20260802) => { const r = rng(seed), load = new Array(MIN).fill(0); for (let d = 0; d < MIN; d++) { let v = base + 60; // field notification: flows every minute if (d >= 360 && d < 1320) v += 700 * (0.75 + 0.5 * r()); // reading validation traffic if (d >= 120 && d < 145) v += 22000; // nightly batch read processing if (d >= 300 && d < 310) v += 6000; // billing generation load[d] = Math.round(v); } return load; }; export const clock = (d) => `${String(Math.floor(d / 60)).padStart(2, "0")}:${String(d % 60).padStart(2, "0")}`; // Continuous arrangement: instance count is fixed all day; requests over capacity wait in a queue. export const continuous = (load, instances) => { const cap = instances * INSTANCE_CAP; let queue = 0, longest = 0, finish = -1; for (let d = 0; d < MIN; d++) { queue += load[d] - Math.min(queue + load[d], cap); if (queue > longest) longest = queue; if (d >= 145 && queue === 0 && finish < 0) finish = d; } const total = load.reduce((a, b) => a + b, 0); return { label: `continuous ${instances} instances`, cost: instances * MIN * INSTANCE_MIN, capacity: instances * MIN * INSTANCE_CAP, utilization: total / (instances * MIN * INSTANCE_CAP), atPeak: Math.min(Math.max(...load), cap), longest, backlog: queue, finish }; }; // On-demand arrangement: the capacity decision is handed off; every request runs the minute it arrives. export const onDemand = (load) => { const total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load); return { label: "function", cost: total * REQUEST_UNIT, capacity: total, utilization: 1, atPeak: peak, longest: 0, backlog: 0, finish: 144, concurrency: Math.ceil(peak * (PROC_MS / 1000) / 60) }; }; if (import.meta.url === `file://${process.argv[1]}`) { const load = traffic(), total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load); console.log(`day: ${total} requests, peak ${peak} req/min, average ${Math.round(total / MIN)} req/min, ` + `peak/average ${(peak / (total / MIN)).toFixed(1)}`); const arrangements = [continuous(load, Math.ceil(peak / INSTANCE_CAP)), continuous(load, 12), continuous(load, 2), onDemand(load)]; console.log("\n" + "arrangement".padEnd(25) + "cost".padEnd(11) + "idle slots".padEnd(13) + "idle share".padEnd(12) + "served at peak".padEnd(19) + "longest queue".padEnd(16) + "night job ends"); for (const a of arrangements) console.log(a.label.padEnd(25) + a.cost.toFixed(0).padEnd(11) + (a.capacity - total).toFixed(0).padEnd(13) + `${((1 - a.utilization) * 100).toFixed(1)}%`.padEnd(12) + `${a.atPeak}/${peak}`.padEnd(19) + a.longest.toFixed(0).padEnd(16) + (a.finish < 0 ? `not finished (${a.backlog} backlog)` : clock(a.finish + 1))); const peakSized = arrangements[0], onDemandRow = arrangements[3]; console.log(`\npeak sizing is ${(peakSized.cost / onDemandRow.cost).toFixed(1)}x more expensive than on-demand; ` + `the on-demand arrangement needs ${onDemandRow.concurrency} concurrency at peak`); } ``` ``` day: 1370625 requests, peak 22060 req/min, average 952 req/min, peak/average 23.2 arrangement cost idle slots idle share served at peak longest queue night job ends continuous 37 instances 31968 30597375 95.7% 22060/22060 0 02:26 continuous 12 instances 10368 8997375 86.8% 7200/22060 371500 03:18 continuous 2 instances 1728 357375 20.7% 1200/22060 521500 18:48 function 2056 0 0.0% 22060/22060 0 02:25 peak sizing is 15.5x more expensive than on-demand; the on-demand arrangement needs 148 concurrency at peak ``` The continuous arrangement's three rows are three separate bad choices. Sizing to peak serves every request on time, and in return costs **30.6 million idle request slots** in 95.7% of the day; 37 instances stand up all day, though outside twenty-five minutes two would be enough. Sizing to average pulls the bill from 31,968 units down to 1,728 units — but the batch processing that starts at 02:25 at night finishes at **18:48** in the evening, and 521,500 records wait in the queue throughout the day. The 12 instances in between are a compromise: cost 10,368, the night job finishes at 03:18, and idle share is still 86.8%. The function row resembles none of the three. Idle slots are zero, because nothing sits idle; all 22,060 requests at peak are served, and the night job finishes the minute it arrives — at 02:25. The bill is **one-fifteenth** of sizing to peak. The difference is not an efficiency trick but a shift in what is measured: there, standing-up time is billed; here, work done is billed. The last row gives the first number of the constraint: 148 concurrent calls are needed at peak, and the account limit is 200. The instance count is not decided, but 200 is a given ceiling, and if traffic grew 35% the peak would press against it. ## Load Profile Scan The day above is sparse and sudden: the peak is 23.2 times the average. In a profile like this, the function winning is not surprising. The real question is what happens as the profile flattens. The scan below adds a continuous base load on top of the day and reprices the two arrangements at every level. ```js // measurement-network/load-scan.mjs — where the break-even point sits as continuous base load grows (model) import { MIN, INSTANCE_CAP, INSTANCE_MIN, REQUEST_UNIT, traffic, continuous, onDemand } from "./two-regimes.mjs"; export const point = (base) => { const load = traffic(base), total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load); const instances = Math.ceil(peak / INSTANCE_CAP), cont = continuous(load, instances), ond = onDemand(load); return { base, peak, average: total / MIN, ratio: peak / (total / MIN), instances, utilization: cont.utilization, cont: cont.cost, ond: ond.cost }; }; if (import.meta.url === `file://${process.argv[1]}`) { console.log("base load".padEnd(12) + "peak/average".padEnd(15) + "instances".padEnd(11) + "utilization".padEnd(13) + "continuous".padEnd(12) + "function".padEnd(10) + "function/continuous".padEnd(22) + "winner"); for (const t of [0, 2000, 5000, 10000, 20000, 30000, 40000, 50000, 65000]) { const n = point(t); console.log(`${t}`.padEnd(12) + n.ratio.toFixed(1).padEnd(15) + String(n.instances).padEnd(11) + `${(n.utilization * 100).toFixed(1)}%`.padEnd(13) + n.cont.toFixed(0).padEnd(12) + n.ond.toFixed(0).padEnd(10) + (n.ond / n.cont).toFixed(2).padEnd(22) + (n.ond < n.cont ? "function" : "continuous")); } let a = 0, b = 65000; for (let k = 0; k < 24; k++) { const o = (a + b) / 2; (point(Math.round(o)).ond < point(Math.round(o)).cont) ? a = o : b = o; } const bb = point(Math.round(a)); console.log(`\nbreak-even base load: ${Math.round(a)} req/min — at this point peak/average ${bb.ratio.toFixed(2)}, ` + `utilization ${(bb.utilization * 100).toFixed(1)}%`); console.log(`theoretical threshold: an instance's cost per request is ${(INSTANCE_MIN / INSTANCE_CAP).toFixed(6)} units, ` + `the function's is ${REQUEST_UNIT.toFixed(6)} units — ratio ${((INSTANCE_MIN / INSTANCE_CAP) / REQUEST_UNIT * 100).toFixed(1)}%`); } ``` ``` base load peak/average instances utilization continuous function function/continuous winner 0 23.2 37 4.3% 31968 2056 0.06 function 2000 8.2 41 12.0% 35424 6376 0.18 function 5000 4.5 46 21.6% 39744 12856 0.32 function 10000 2.9 54 33.8% 46656 23656 0.51 function 20000 2.0 71 49.2% 61344 45256 0.74 function 30000 1.7 87 59.3% 75168 66856 0.89 function 40000 1.5 104 65.6% 89856 88456 0.98 function 50000 1.4 121 70.2% 104544 110056 1.05 continuous 65000 1.3 146 75.3% 126144 142456 1.13 continuous break-even base load: 42248 req/min — at this point peak/average 1.49, utilization 66.7% theoretical threshold: an instance's cost per request is 0.001000 units, the function's is 0.001500 units — ratio 66.7% ``` The scan shows a single rule: the function winning depends not on the size of the traffic but on its **shape**. When base load is zero, the function is sixteen times cheaper; at 42,248 req/min the two arrangements equalize; beyond that, the continuous arrangement wins. The break-even utilization is 66.7%, and it is not arbitrary — the last row derives the same ratio from price: an instance costs 0.001 per request, the function costs 0.0015 units. The rule rests on a measurement, not a preference: **if two-thirds of capacity is not continuously full, managing capacity yourself is expensive**. The peak/average column gives the same boundary from a second angle; once the ratio drops below 1.5, the function's advantage ends. ## The Ledger of the Handed-Off Decision The bill is only one column of the table. What actually changes is who the decisions sit with. ```js // measurement-network/decision-ledger.mjs — which decision was handed off, and what number came back in return (model) import { PROC_MS, INSTANCE_MIN, traffic, onDemand } from "./two-regimes.mjs"; // [decision, in the continuous regime, in the function regime, incoming constraint (fictional number)] export const DECISIONS = [ ["instance count", "you", "provider", "concurrency limit 200"], ["scaling threshold", "you", "provider", "scaling step 60 instances/min"], ["machine size", "you", "provider", "memory tier 128-3008"], ["a job's upper duration", "you", "provider", "duration limit 900 s"], ["local disk", "you", "provider", "temp space 512 units"], ["package contents", "you", "partly you", "package size 250 units"], ["runtime version", "you", "partly you", "support window 24 months"], ["operating system patching", "you", "provider", "patch schedule invisible"], ["placement", "you", "provider", "region chosen, machine not chosen"], ["how long an instance lives", "you", "provider", "reclaimed after 300 s idle"], ["price per request", "provider", "provider", "—"], ]; // [constraint, workaround path] export const WORKAROUND = [["duration limit 900 s", "split the job"], ["concurrency limit 200", "reserve a share"], ["reclaimed after 300 s idle", "pre-warm"], ["temp space 512 units", "externalize state"]]; if (import.meta.url === `file://${process.argv[1]}`) { console.log("decision".padEnd(30) + "continuous regime".padEnd(20) + "function regime".padEnd(18) + "constraint received in return"); for (const [a, s, i, k] of DECISIONS) console.log(a.padEnd(30) + s.padEnd(20) + i.padEnd(18) + k); const count = (j) => DECISIONS.filter((x) => x[j].includes("you")).length; const numbered = DECISIONS.filter((x) => x[2] !== "you" && /\d/.test(x[3])).length; console.log(`\n${count(1)} decisions kept in the continuous regime, ${count(2)} kept fully or partly in the function regime; ` + `${count(1) - count(2)} decisions were handed off from the start, ${numbered} constraints came back as a number`); const load = traffic(), night = load.slice(120, 145).reduce((a, b) => a + b, 0); const single = night * (PROC_MS / 1000), warm = 8, idle = warm * 1440, ez = onDemand(load).concurrency; const COST = [ `${night} record night job in one piece takes ${Math.round(single)} s; fitting the limit needs ${Math.ceil(single / 900)} calls`, `if the ${ez} needed at peak is reserved, ${200 - ez} concurrency remains for the others`, `keeping ${warm} instances continuously warm costs ${idle} idle instance-minutes a day, ${(idle * INSTANCE_MIN).toFixed(0)} units`, `every call reads state externally: ${load.reduce((a, b) => a + b, 0)} extra read-write pairs`]; console.log("\n" + "constraint".padEnd(32) + "workaround path".padEnd(20) + "cost"); WORKAROUND.forEach(([k, path], j) => console.log(k.padEnd(32) + path.padEnd(20) + COST[j])); } ``` ``` decision continuous regime function regime constraint received in return instance count you provider concurrency limit 200 scaling threshold you provider scaling step 60 instances/min machine size you provider memory tier 128-3008 a job's upper duration you provider duration limit 900 s local disk you provider temp space 512 units package contents you partly you package size 250 units runtime version you partly you support window 24 months operating system patching you provider patch schedule invisible placement you provider region chosen, machine not chosen how long an instance lives you provider reclaimed after 300 s idle price per request provider provider — 10 decisions kept in the continuous regime, 2 kept fully or partly in the function regime; 8 decisions were handed off from the start, 8 constraints came back as a number constraint workaround path cost duration limit 900 s split the job 551500 record night job in one piece takes 220600 s; fitting the limit needs 246 calls concurrency limit 200 reserve a share if the 148 needed at peak is reserved, 52 concurrency remains for the others reclaimed after 300 s idle pre-warm keeping 8 instances continuously warm costs 11520 idle instance-minutes a day, 6912 units temp space 512 units externalize state every call reads state externally: 1370625 extra read-write pairs ``` Ten of eleven decisions are kept in the continuous regime; in the function regime only two are kept, and both only partly — the package's contents run up against the package-size limit, and the runtime's version runs up against the support window. Eight decisions are handed off from the start, and **all eight come back as a number**. Handing off does not remove the decision; it turns the decision into a ceiling. The lower table prices the workaround paths around those ceilings. The night batch job takes 220,600 seconds in one piece; to fit the 900-second limit it must be split into **246 separate calls**, and 246 calls means a progress record the single job never needed and a question of partial success. If the 148 concurrency needed at peak is reserved, only 52 remains for every other function on the measurement network: protecting one function starves the others. Keeping eight instances continuously warm costs 11,520 idle instance-minutes a day, that is 6,912 units — the idle capacity avoided earlier coming back at a smaller scale. The four rows share a common shape: the constraint is a number, a workaround path exists, and the cost of the workaround is **not in the unit the constraint is measured in**. Duration limit turns into call count, concurrency limit turns into the others' share, instance lifetime turns into idle capacity; each constraint is paid in its own currency. ## Summary - The continuous arrangement sized to peak cost 31,968 units and 95.7% idle capacity; the function arrangement cost 2,056 units and zero idle capacity — a fifteenfold difference. - Cutting the instance count to 2 to save money pulled the bill down to 1,728 units but pushed the night job that starts at 02:25 out to 18:48 and left 521,500 records in the queue; the savings were paid for in delay. - The base-load scan found the break-even point at 42,248 req/min: utilization 66.7% and peak/average 1.49. If two-thirds of capacity is not continuously full, managing capacity yourself is expensive. - Ten of eleven decisions were kept in the continuous regime; only two remained partly kept in the function regime. All eight handed-off decisions came back as a numbered constraint. - The cost of the workaround is always in a different unit: the duration limit turns into 246 calls, instance lifetime into 11,520 idle instance-minutes a day, and the concurrency limit into the 52 share left for the others. ## Next Step This lesson invoked the function in a single way: a request arrived, the function ran, a result came back. But the night batch job and the field notification do not have to be invoked the same way — one sits at the end of a client waiting for a result, the other at the end of a queue. The next lesson runs the same validation work through three invocation models and counts four things separately: the latency the caller sees, whether the error is visible to the caller, who owns the retry, and the risk of the same record being processed twice. It then scans batch read size: as size grows, throughput rises while the number of healthy records a single bad record drags down along with it is measured.