--- title: 'Serverless and Edge Computing' source: 'https://academia.sh/en/courses/serverless' language: en updated: '2026-09-13T05:07:38+00:00' license: 'CC BY-SA 4.0' lessons: 20 outcomes: - 'States the limits of function-based compute with a number' - 'Builds asynchronous integration with event-driven managed services' - 'Turns edge runtime constraints into design decisions' --- # Serverless and Edge Computing Handing over the always-running server as well: giving up capacity management, the synchronous, asynchronous, and stream-based invocation models, where cold start latency comes from and how it is reduced, the resource and duration thresholds that constrain design, gradual release through versions and aliases, package size management with shared layers, the cases where serverless does not fit, the event bus, queue and topic services, step-based workflow orchestration, scheduled triggering, endpoint management, notification services, the edge runtime model, request processing at the edge, edge storage, durable state objects, edge queues, declarative bindings, and edge limits. ## Function-Based Compute Handing off capacity management: the latency and retry difference across synchronous, asynchronous, and stream-based invocation models, where cold start comes from and what mitigating it costs, the design constraint of memory, duration, and concurrency thresholds, gradual release through versioning and aliases, package size management with shared layers, and why serverless does not fit long-running, state-heavy jobs. ### The Serverless Concept Source: https://academia.sh/en/courses/serverless/serverless-concept The same day's workload is run in two arrangements: the always-on instance and the function invoked on demand; idle capacity, requests served at peak, and the hour the night job finishes are counted side by side, the base load is scanned from zero to sixty-five thousand to work out the break-even load level, and the ten handed-off decisions with the eight numbered constraints that come back in return are gathered into a single ledger. The previous course rented the machine, the network, and the data one at a time. The same thing happened with every rental: a decision passed to the provider, and a constraint came back in return. But even at the end of three courses, what sits underneath had not changed — the rented capacity is a **continuously running server**, and the meter keeps running in the hours when no work arrives. This course's question is whether capacity itself can be rented on demand. The measurement moves along a single axis: the same day, the same workload, two arrangements. In the first, the party that decides the instance count keeps the decision; in the second, the decision sits with the provider. The difference is not only in the bill — idle capacity, requests served at peak, and the hour the night job finishes also change. **FC1.** The regional measurement network is a **fictional** piece of software that collects readings from water meters, validates the readings, generates the bill, and sends notifications to field crews; every duration, count, and price is fictional. **FC2.** A reading validation takes 400 ms; a continuously running instance carries four validations at once: 600 requests per minute. **FC3.** Prices are written in a fictional measure — the **unit** — and are not a real currency: instance-minute 0.6 units, function call 0.0005 units, 400 ms of processing 0.001 units. **FC4.** The day's traffic is built with its own generator, and the seed is 20260802. **FC5.** In the continuous arrangement, the instance count is fixed all day; requests over capacity do not drop, they wait in the queue. **FC6.** In the function arrangement, every request runs the minute it arrives, and the account-level concurrency limit is 200. #### The Same Day, Two Arrangements The measurement network's day is not flat: readings are processed in a nightly batch, the bill is generated toward morning, validation traffic flows during the day, and field notifications never stop. The model below builds the day minute by minute. ```js // measurement-network/two-regimes.mjs — the same day's load in two arrangements: continuous instances and on-demand functions (model) // All units are fictional; the "unit" is a make-believe measure, not real money. export const MIN = 1440, PROC_MS = 400, CONCURRENCY = 4; export const INSTANCE_CAP = 60000 * CONCURRENCY / PROC_MS; // one instance handles 600 requests/min export const INSTANCE_MIN = 0.6, CALL = 0.0005, DURATION = 0.001; // units export const REQUEST_UNIT = CALL + DURATION; export const rng = (t) => { let s = t >>> 0; return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; }; // The regional measurement network's fictional day: nightly batch read, morning billing, daytime validation. export const traffic = (base = 0, seed = 20260802) => { const r = rng(seed), load = new Array(MIN).fill(0); for (let d = 0; d < MIN; d++) { let v = base + 60; // field notification: flows every minute if (d >= 360 && d < 1320) v += 700 * (0.75 + 0.5 * r()); // reading validation traffic if (d >= 120 && d < 145) v += 22000; // nightly batch read processing if (d >= 300 && d < 310) v += 6000; // billing generation load[d] = Math.round(v); } return load; }; export const clock = (d) => `${String(Math.floor(d / 60)).padStart(2, "0")}:${String(d % 60).padStart(2, "0")}`; // Continuous arrangement: instance count is fixed all day; requests over capacity wait in a queue. export const continuous = (load, instances) => { const cap = instances * INSTANCE_CAP; let queue = 0, longest = 0, finish = -1; for (let d = 0; d < MIN; d++) { queue += load[d] - Math.min(queue + load[d], cap); if (queue > longest) longest = queue; if (d >= 145 && queue === 0 && finish < 0) finish = d; } const total = load.reduce((a, b) => a + b, 0); return { label: `continuous ${instances} instances`, cost: instances * MIN * INSTANCE_MIN, capacity: instances * MIN * INSTANCE_CAP, utilization: total / (instances * MIN * INSTANCE_CAP), atPeak: Math.min(Math.max(...load), cap), longest, backlog: queue, finish }; }; // On-demand arrangement: the capacity decision is handed off; every request runs the minute it arrives. export const onDemand = (load) => { const total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load); return { label: "function", cost: total * REQUEST_UNIT, capacity: total, utilization: 1, atPeak: peak, longest: 0, backlog: 0, finish: 144, concurrency: Math.ceil(peak * (PROC_MS / 1000) / 60) }; }; if (import.meta.url === `file://${process.argv[1]}`) { const load = traffic(), total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load); console.log(`day: ${total} requests, peak ${peak} req/min, average ${Math.round(total / MIN)} req/min, ` + `peak/average ${(peak / (total / MIN)).toFixed(1)}`); const arrangements = [continuous(load, Math.ceil(peak / INSTANCE_CAP)), continuous(load, 12), continuous(load, 2), onDemand(load)]; console.log("\n" + "arrangement".padEnd(25) + "cost".padEnd(11) + "idle slots".padEnd(13) + "idle share".padEnd(12) + "served at peak".padEnd(19) + "longest queue".padEnd(16) + "night job ends"); for (const a of arrangements) console.log(a.label.padEnd(25) + a.cost.toFixed(0).padEnd(11) + (a.capacity - total).toFixed(0).padEnd(13) + `${((1 - a.utilization) * 100).toFixed(1)}%`.padEnd(12) + `${a.atPeak}/${peak}`.padEnd(19) + a.longest.toFixed(0).padEnd(16) + (a.finish < 0 ? `not finished (${a.backlog} backlog)` : clock(a.finish + 1))); const peakSized = arrangements[0], onDemandRow = arrangements[3]; console.log(`\npeak sizing is ${(peakSized.cost / onDemandRow.cost).toFixed(1)}x more expensive than on-demand; ` + `the on-demand arrangement needs ${onDemandRow.concurrency} concurrency at peak`); } ``` ``` day: 1370625 requests, peak 22060 req/min, average 952 req/min, peak/average 23.2 arrangement cost idle slots idle share served at peak longest queue night job ends continuous 37 instances 31968 30597375 95.7% 22060/22060 0 02:26 continuous 12 instances 10368 8997375 86.8% 7200/22060 371500 03:18 continuous 2 instances 1728 357375 20.7% 1200/22060 521500 18:48 function 2056 0 0.0% 22060/22060 0 02:25 peak sizing is 15.5x more expensive than on-demand; the on-demand arrangement needs 148 concurrency at peak ``` The continuous arrangement's three rows are three separate bad choices. Sizing to peak serves every request on time, and in return costs **30.6 million idle request slots** in 95.7% of the day; 37 instances stand up all day, though outside twenty-five minutes two would be enough. Sizing to average pulls the bill from 31,968 units down to 1,728 units — but the batch processing that starts at 02:25 at night finishes at **18:48** in the evening, and 521,500 records wait in the queue throughout the day. The 12 instances in between are a compromise: cost 10,368, the night job finishes at 03:18, and idle share is still 86.8%. The function row resembles none of the three. Idle slots are zero, because nothing sits idle; all 22,060 requests at peak are served, and the night job finishes the minute it arrives — at 02:25. The bill is **one-fifteenth** of sizing to peak. The difference is not an efficiency trick but a shift in what is measured: there, standing-up time is billed; here, work done is billed. The last row gives the first number of the constraint: 148 concurrent calls are needed at peak, and the account limit is 200. The instance count is not decided, but 200 is a given ceiling, and if traffic grew 35% the peak would press against it. #### Load Profile Scan The day above is sparse and sudden: the peak is 23.2 times the average. In a profile like this, the function winning is not surprising. The real question is what happens as the profile flattens. The scan below adds a continuous base load on top of the day and reprices the two arrangements at every level. ```js // measurement-network/load-scan.mjs — where the break-even point sits as continuous base load grows (model) import { MIN, INSTANCE_CAP, INSTANCE_MIN, REQUEST_UNIT, traffic, continuous, onDemand } from "./two-regimes.mjs"; export const point = (base) => { const load = traffic(base), total = load.reduce((a, b) => a + b, 0), peak = Math.max(...load); const instances = Math.ceil(peak / INSTANCE_CAP), cont = continuous(load, instances), ond = onDemand(load); return { base, peak, average: total / MIN, ratio: peak / (total / MIN), instances, utilization: cont.utilization, cont: cont.cost, ond: ond.cost }; }; if (import.meta.url === `file://${process.argv[1]}`) { console.log("base load".padEnd(12) + "peak/average".padEnd(15) + "instances".padEnd(11) + "utilization".padEnd(13) + "continuous".padEnd(12) + "function".padEnd(10) + "function/continuous".padEnd(22) + "winner"); for (const t of [0, 2000, 5000, 10000, 20000, 30000, 40000, 50000, 65000]) { const n = point(t); console.log(`${t}`.padEnd(12) + n.ratio.toFixed(1).padEnd(15) + String(n.instances).padEnd(11) + `${(n.utilization * 100).toFixed(1)}%`.padEnd(13) + n.cont.toFixed(0).padEnd(12) + n.ond.toFixed(0).padEnd(10) + (n.ond / n.cont).toFixed(2).padEnd(22) + (n.ond < n.cont ? "function" : "continuous")); } let a = 0, b = 65000; for (let k = 0; k < 24; k++) { const o = (a + b) / 2; (point(Math.round(o)).ond < point(Math.round(o)).cont) ? a = o : b = o; } const bb = point(Math.round(a)); console.log(`\nbreak-even base load: ${Math.round(a)} req/min — at this point peak/average ${bb.ratio.toFixed(2)}, ` + `utilization ${(bb.utilization * 100).toFixed(1)}%`); console.log(`theoretical threshold: an instance's cost per request is ${(INSTANCE_MIN / INSTANCE_CAP).toFixed(6)} units, ` + `the function's is ${REQUEST_UNIT.toFixed(6)} units — ratio ${((INSTANCE_MIN / INSTANCE_CAP) / REQUEST_UNIT * 100).toFixed(1)}%`); } ``` ``` base load peak/average instances utilization continuous function function/continuous winner 0 23.2 37 4.3% 31968 2056 0.06 function 2000 8.2 41 12.0% 35424 6376 0.18 function 5000 4.5 46 21.6% 39744 12856 0.32 function 10000 2.9 54 33.8% 46656 23656 0.51 function 20000 2.0 71 49.2% 61344 45256 0.74 function 30000 1.7 87 59.3% 75168 66856 0.89 function 40000 1.5 104 65.6% 89856 88456 0.98 function 50000 1.4 121 70.2% 104544 110056 1.05 continuous 65000 1.3 146 75.3% 126144 142456 1.13 continuous break-even base load: 42248 req/min — at this point peak/average 1.49, utilization 66.7% theoretical threshold: an instance's cost per request is 0.001000 units, the function's is 0.001500 units — ratio 66.7% ``` The scan shows a single rule: the function winning depends not on the size of the traffic but on its **shape**. When base load is zero, the function is sixteen times cheaper; at 42,248 req/min the two arrangements equalize; beyond that, the continuous arrangement wins. The break-even utilization is 66.7%, and it is not arbitrary — the last row derives the same ratio from price: an instance costs 0.001 per request, the function costs 0.0015 units. The rule rests on a measurement, not a preference: **if two-thirds of capacity is not continuously full, managing capacity yourself is expensive**. The peak/average column gives the same boundary from a second angle; once the ratio drops below 1.5, the function's advantage ends. #### The Ledger of the Handed-Off Decision The bill is only one column of the table. What actually changes is who the decisions sit with. ```js // measurement-network/decision-ledger.mjs — which decision was handed off, and what number came back in return (model) import { PROC_MS, INSTANCE_MIN, traffic, onDemand } from "./two-regimes.mjs"; // [decision, in the continuous regime, in the function regime, incoming constraint (fictional number)] export const DECISIONS = [ ["instance count", "you", "provider", "concurrency limit 200"], ["scaling threshold", "you", "provider", "scaling step 60 instances/min"], ["machine size", "you", "provider", "memory tier 128-3008"], ["a job's upper duration", "you", "provider", "duration limit 900 s"], ["local disk", "you", "provider", "temp space 512 units"], ["package contents", "you", "partly you", "package size 250 units"], ["runtime version", "you", "partly you", "support window 24 months"], ["operating system patching", "you", "provider", "patch schedule invisible"], ["placement", "you", "provider", "region chosen, machine not chosen"], ["how long an instance lives", "you", "provider", "reclaimed after 300 s idle"], ["price per request", "provider", "provider", "—"], ]; // [constraint, workaround path] export const WORKAROUND = [["duration limit 900 s", "split the job"], ["concurrency limit 200", "reserve a share"], ["reclaimed after 300 s idle", "pre-warm"], ["temp space 512 units", "externalize state"]]; if (import.meta.url === `file://${process.argv[1]}`) { console.log("decision".padEnd(30) + "continuous regime".padEnd(20) + "function regime".padEnd(18) + "constraint received in return"); for (const [a, s, i, k] of DECISIONS) console.log(a.padEnd(30) + s.padEnd(20) + i.padEnd(18) + k); const count = (j) => DECISIONS.filter((x) => x[j].includes("you")).length; const numbered = DECISIONS.filter((x) => x[2] !== "you" && /\d/.test(x[3])).length; console.log(`\n${count(1)} decisions kept in the continuous regime, ${count(2)} kept fully or partly in the function regime; ` + `${count(1) - count(2)} decisions were handed off from the start, ${numbered} constraints came back as a number`); const load = traffic(), night = load.slice(120, 145).reduce((a, b) => a + b, 0); const single = night * (PROC_MS / 1000), warm = 8, idle = warm * 1440, ez = onDemand(load).concurrency; const COST = [ `${night} record night job in one piece takes ${Math.round(single)} s; fitting the limit needs ${Math.ceil(single / 900)} calls`, `if the ${ez} needed at peak is reserved, ${200 - ez} concurrency remains for the others`, `keeping ${warm} instances continuously warm costs ${idle} idle instance-minutes a day, ${(idle * INSTANCE_MIN).toFixed(0)} units`, `every call reads state externally: ${load.reduce((a, b) => a + b, 0)} extra read-write pairs`]; console.log("\n" + "constraint".padEnd(32) + "workaround path".padEnd(20) + "cost"); WORKAROUND.forEach(([k, path], j) => console.log(k.padEnd(32) + path.padEnd(20) + COST[j])); } ``` ``` decision continuous regime function regime constraint received in return instance count you provider concurrency limit 200 scaling threshold you provider scaling step 60 instances/min machine size you provider memory tier 128-3008 a job's upper duration you provider duration limit 900 s local disk you provider temp space 512 units package contents you partly you package size 250 units runtime version you partly you support window 24 months operating system patching you provider patch schedule invisible placement you provider region chosen, machine not chosen how long an instance lives you provider reclaimed after 300 s idle price per request provider provider — 10 decisions kept in the continuous regime, 2 kept fully or partly in the function regime; 8 decisions were handed off from the start, 8 constraints came back as a number constraint workaround path cost duration limit 900 s split the job 551500 record night job in one piece takes 220600 s; fitting the limit needs 246 calls concurrency limit 200 reserve a share if the 148 needed at peak is reserved, 52 concurrency remains for the others reclaimed after 300 s idle pre-warm keeping 8 instances continuously warm costs 11520 idle instance-minutes a day, 6912 units temp space 512 units externalize state every call reads state externally: 1370625 extra read-write pairs ``` Ten of eleven decisions are kept in the continuous regime; in the function regime only two are kept, and both only partly — the package's contents run up against the package-size limit, and the runtime's version runs up against the support window. Eight decisions are handed off from the start, and **all eight come back as a number**. Handing off does not remove the decision; it turns the decision into a ceiling. The lower table prices the workaround paths around those ceilings. The night batch job takes 220,600 seconds in one piece; to fit the 900-second limit it must be split into **246 separate calls**, and 246 calls means a progress record the single job never needed and a question of partial success. If the 148 concurrency needed at peak is reserved, only 52 remains for every other function on the measurement network: protecting one function starves the others. Keeping eight instances continuously warm costs 11,520 idle instance-minutes a day, that is 6,912 units — the idle capacity avoided earlier coming back at a smaller scale. The four rows share a common shape: the constraint is a number, a workaround path exists, and the cost of the workaround is **not in the unit the constraint is measured in**. Duration limit turns into call count, concurrency limit turns into the others' share, instance lifetime turns into idle capacity; each constraint is paid in its own currency. #### Summary - The continuous arrangement sized to peak cost 31,968 units and 95.7% idle capacity; the function arrangement cost 2,056 units and zero idle capacity — a fifteenfold difference. - Cutting the instance count to 2 to save money pulled the bill down to 1,728 units but pushed the night job that starts at 02:25 out to 18:48 and left 521,500 records in the queue; the savings were paid for in delay. - The base-load scan found the break-even point at 42,248 req/min: utilization 66.7% and peak/average 1.49. If two-thirds of capacity is not continuously full, managing capacity yourself is expensive. - Ten of eleven decisions were kept in the continuous regime; only two remained partly kept in the function regime. All eight handed-off decisions came back as a numbered constraint. - The cost of the workaround is always in a different unit: the duration limit turns into 246 calls, instance lifetime into 11,520 idle instance-minutes a day, and the concurrency limit into the 52 share left for the others. #### Next Step This lesson invoked the function in a single way: a request arrived, the function ran, a result came back. But the night batch job and the field notification do not have to be invoked the same way — one sits at the end of a client waiting for a result, the other at the end of a queue. The next lesson runs the same validation work through three invocation models and counts four things separately: the latency the caller sees, whether the error is visible to the caller, who owns the retry, and the risk of the same record being processed twice. It then scans batch read size: as size grows, throughput rises while the number of healthy records a single bad record drags down along with it is measured. ### Function Invocation Models Source: https://academia.sh/en/courses/serverless/function-invocation-models The same validation job is run through three invocation models and four things are counted separately: the latency the caller sees, who sees the error, who owns the retry, and the rate of repeated processing; batch read size is then scanned from one to two thousand five hundred in the stream-based model to work out call count, extra processing, the healthy records dragged into the dead letter queue, and the batch that no longer fits the duration limit. The previous lesson invoked the function in a single way: a request arrived, the function ran, a result came back. But the network's jobs do not all sit at the same end: no one waits for the reply to the notification sent to the field crew; the nightly batch job is read from a queue; the validation request coming through the gateway sits at a client's end. The invocation model is a choice in which four separate numbers change at once: latency, who sees the error, who owns the retry, and how many times the same record is processed. This lesson counts all four. **FC7.** The 20,000 meter readings entering the regional measurement network's — a fictional piece of software's — nightly batch job are measured; record classes are distributed with a self-written generator, and the seed is 20260803. **FC8.** A validation takes 400 ms, fixed overhead per call is 250 ms, and the asynchronous accept response takes 5 ms. **FC9.** A transient error passes on the second attempt; a poison record never passes. **FC10.** In the synchronous model the caller retries once; in the asynchronous and stream-based models the infrastructure owns the retry, with three attempts and a 60-second gap. **FC11.** In the stream-based model a batch is not split: if a single record errors, the whole batch is re-read, and after the third attempt the whole batch goes to the dead letter queue; records arrive at 250 per second. **FC12.** Prices are written in fictional units (0.0005 per call, 0.0000025 per processing millisecond), and the duration limit is 900 seconds. #### The Same Job, Three Invocation Paths ```js // measurement-network/invocation-models.mjs — the same validation job in three invocation models (model) // Fictional numbers: one reading validation is 400 ms, fixed overhead per call is 250 ms, the accept response is 5 ms. export const N = 20000, PROC_MS = 400, FIXED_MS = 250, ACCEPT_MS = 5, RETRY_MS = 60000; export const CALL_UNIT = 0.0005, MS_UNIT = 0.0000025; // fictional unit: per call and per ms export const rng = (t) => { let s = t >>> 0; return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; }; // The 20000 meter readings entering the nightly batch: most clean, some transient errors, some poison. export const records = (seed = 20260803) => { const r = rng(seed); return Array.from({ length: N }, () => { const v = r(); return v < 0.001 ? "poison" : v < 0.021 ? "transient" : "clean"; }); }; const percentile = (a, p) => [...a].sort((x, y) => x - y)[Math.floor(a.length * p)]; const average = (a) => a.reduce((x, y) => x + y, 0) / a.length; // Synchronous: the caller waits for the reply, sees the error, and owns the retry itself. export const synchronous = (k) => { let calls = 0, retries = 0; const finishTimes = []; for (const d of k) { const attempts = d === "clean" ? 1 : 2; calls += attempts; retries += attempts - 1; finishTimes.push(attempts * (FIXED_MS + PROC_MS) + (attempts - 1) * 200); } return { label: "synchronous", finishTimes, calls, retries, processed: calls, duration: calls * (FIXED_MS + PROC_MS), deadLetter: 0, collateral: 0, errorSeenBy: "caller", retryOwner: "caller" }; }; // Asynchronous: the caller receives the accept response and leaves; the infrastructure owns the retry. export const asynchronous = (k) => { let calls = 0, retries = 0, dead = 0; const finishTimes = []; for (const d of k) { const attempts = d === "clean" ? 1 : d === "transient" ? 2 : 3; calls += attempts; retries += attempts - 1; finishTimes.push(attempts * (FIXED_MS + PROC_MS) + (attempts - 1) * RETRY_MS); if (d === "poison") dead++; } return { label: "asynchronous", ackMs: ACCEPT_MS, finishTimes, calls, retries, processed: calls, duration: calls * (FIXED_MS + PROC_MS), deadLetter: dead, collateral: 0, errorSeenBy: "no one", retryOwner: "infrastructure" }; }; // Stream-based: records are read in order, a batch is a single call, one bad record retries the whole batch. export const streamBased = (k, batchSize, arrivalRate = 250) => { let calls = 0, processed = 0, dead = 0, collateral = 0; const finishTimes = []; for (let i = 0; i < k.length; i += batchSize) { const batch = k.slice(i, i + batchSize); const poisoned = batch.filter((d) => d === "poison").length; const attempts = poisoned ? 3 : batch.some((d) => d === "transient") ? 2 : 1; const duration = attempts * (FIXED_MS + batch.length * PROC_MS) + (attempts - 1) * RETRY_MS; calls += attempts; processed += attempts * batch.length; if (poisoned) { dead += batch.length; collateral += batch.length - poisoned; } for (let j = 0; j < batch.length; j++) finishTimes.push(batch.length / arrivalRate * 1000 + duration); } return { label: `stream-based (batch ${batchSize})`, ackMs: ACCEPT_MS, finishTimes, calls, processed, retries: processed - N, duration: calls * FIXED_MS + processed * PROC_MS, deadLetter: dead, collateral, errorSeenBy: "no one", retryOwner: "infrastructure" }; }; export const measure = (m) => ({ ...m, ackAvg: m.ackMs ?? average(m.finishTimes), finishAvg: average(m.finishTimes), finish99: percentile(m.finishTimes, 0.99), cost: m.calls * CALL_UNIT + m.duration * MS_UNIT }); if (import.meta.url === `file://${process.argv[1]}`) { const k = records(), count = (d) => k.filter((x) => x === d).length; console.log(`${N} records: ${count("clean")} clean, ${count("transient")} transient error, ${count("poison")} poison`); const m = [synchronous(k), asynchronous(k), streamBased(k, 100)].map(measure); console.log("\n" + "model".padEnd(26) + "caller sees".padEnd(19) + "job finishes (avg)".padEnd(20) + "job finishes (p99)".padEnd(20) + "processing calls".padEnd(18) + "extra processing".padEnd(18) + "dead letter".padEnd(13) + "cost"); for (const x of m) console.log(x.label.padEnd(26) + `${x.ackAvg.toFixed(0)} ms`.padEnd(19) + `${(x.finishAvg / 1000).toFixed(1)} s`.padEnd(20) + `${(x.finish99 / 1000).toFixed(1)} s`.padEnd(20) + String(x.calls).padEnd(18) + String(x.retries).padEnd(18) + String(x.deadLetter).padEnd(13) + x.cost.toFixed(2)); console.log("\n" + "model".padEnd(26) + "error seen by".padEnd(16) + "retry owner".padEnd(23) + "extra processing per 100 records"); for (const x of m) console.log(x.label.padEnd(26) + x.errorSeenBy.padEnd(16) + x.retryOwner.padEnd(23) + (x.retries / N * 100).toFixed(1)); const s = m[2]; console.log(`\nof the ${s.deadLetter} records that fell into the dead letter queue, ${s.collateral} were healthy: ` + `${(s.collateral / count("poison")).toFixed(1)} healthy records fell along with each poison record`); } ``` ``` 20000 records: 19551 clean, 436 transient error, 13 poison model caller sees job finishes (avg) job finishes (p99) processing calls extra processing dead letter cost synchronous 669 ms 0.7 s 1.5 s 20449 449 0 43.45 asynchronous 5 ms 2.1 s 61.3 s 20462 462 13 43.48 stream-based (batch 100) 5 ms 130.9 s 241.2 s 380 18000 1300 38.43 model error seen by retry owner extra processing per 100 records synchronous caller caller 2.2 asynchronous no one infrastructure 2.3 stream-based (batch 100) no one infrastructure 90.0 of the 1300 records that fell into the dead letter queue, 1287 were healthy: 99.0 healthy records fell along with each poison record ``` Two columns of the first table move in opposite directions. The latency the caller sees drops from 669 ms to 5 ms: in the asynchronous model the caller waits for the job to be **accepted**, not for it to finish. But the moment the job finishes rises from 0.7 seconds to 2.1 seconds, and the 99th percentile rises to 61.3 seconds. The latency saved does not disappear; it moves somewhere the caller cannot see. The second table shows the actual transfer. In the synchronous model the caller sees the error and decides on the retry itself; all thirteen poison records are reported. In the other two models **no one** sees the error: the infrastructure tries three times, drops the record into the dead letter queue, and the caller is long gone. Handing off the retry does not remove the responsibility; it removes the error's visibility. The last column gives the cost of the transfer. In the first two models, 2.2 extra processing operations fall on every hundred records; in the stream-based model, 90. The reason is that the batch is not split: in a batch of a hundred records, a single transient error reprocesses all hundred records. The same indivisibility holds for the dead letter queue — 1,287 of 1,300 records are healthy, and their only offense is landing in the same batch as a poison record. #### Batch Read Size Batch size is a single number, but it decides three things at once: call count, the amount of reprocessing, and when a record's result is reached. ```js // measurement-network/batch-scan.mjs — batch read size is scanned in the stream-based model (model) import { N, PROC_MS, FIXED_MS, records, streamBased, measure } from "./invocation-models.mjs"; export const LIMIT = 900; // fictional duration limit, seconds export const BATCH_SIZES = [1, 2, 5, 10, 25, 50, 100, 250, 500, 1000, 2500]; if (import.meta.url === `file://${process.argv[1]}`) { const k = records(), poisoned = k.filter((x) => x === "poison").length; console.log("size".padEnd(7) + "calls".padEnd(9) + "processed records".padEnd(19) + "extra".padEnd(11) + "total duration".padEnd(17) + "cost".padEnd(10) + "record latency".padEnd(18) + "dead letter".padEnd(13) + "collateral".padEnd(12) + "batch job / limit"); const rows = BATCH_SIZES.map((b) => measure(streamBased(k, b))); for (let i = 0; i < BATCH_SIZES.length; i++) { const x = rows[i], span = (FIXED_MS + BATCH_SIZES[i] * PROC_MS) / 1000; console.log(String(BATCH_SIZES[i]).padEnd(7) + String(x.calls).padEnd(9) + String(x.processed).padEnd(19) + `${(x.retries / N * 100).toFixed(0)}%`.padEnd(11) + `${(x.duration / 1000).toFixed(0)} s`.padEnd(17) + x.cost.toFixed(2).padEnd(10) + `${(x.finishAvg / 1000).toFixed(1)} s`.padEnd(18) + String(x.deadLetter).padEnd(13) + String(x.collateral).padEnd(12) + `${span.toFixed(0)}/${LIMIT} s` + (span > LIMIT ? " — does not fit" : "")); } const cheapest = rows.reduce((a, b) => (b.cost < a.cost ? b : a)); const fastest = rows.reduce((a, b) => (b.duration < a.duration ? b : a)); console.log(`\ncheapest size: ${cheapest.label} — ${cheapest.cost.toFixed(2)} units; ` + `shortest total duration: ${fastest.label} — ${(fastest.duration / 1000).toFixed(0)} s`); const row1 = rows[0], row500 = rows[8]; console.log(`from size 1 to 500: calls ${row1.calls} -> ${row500.calls} (${(row1.calls / row500.calls).toFixed(0)}x fewer), ` + `processed records ${row1.processed} -> ${row500.processed} (${(row500.processed / row1.processed).toFixed(1)}x more)`); console.log(`${poisoned} poison records dragged ${row1.collateral} healthy records at size 1, and ${row500.collateral} ` + `healthy records at size 500 into the dead letter queue together`); } ``` ``` size calls processed records extra total duration cost record latency dead letter collateral batch job / limit 1 20462 20462 2% 13300 s 43.48 2.1 s 13 0 1/900 s 2 10456 20912 5% 10979 s 32.68 3.8 s 26 13 1/900 s 5 4440 22200 11% 9990 s 27.20 9.1 s 65 52 2/900 s 10 2420 24200 21% 10285 s 26.92 17.8 s 130 117 4/900 s 25 1166 29150 46% 11952 s 30.46 42.5 s 325 312 10/900 s 50 684 34200 71% 13851 s 34.97 77.4 s 650 637 20/900 s 100 380 38000 90% 15295 s 38.43 130.9 s 1300 1287 40/900 s 250 172 43000 115% 17243 s 43.19 285.5 s 3000 2987 100/900 s 500 91 45500 127% 18223 s 45.60 534.1 s 5500 5487 200/900 s 1000 49 49000 145% 19612 s 49.06 1071.6 s 9000 8987 400/900 s 2500 23 57500 188% 23006 s 57.53 2998.2 s 17500 17487 1000/900 s — does not fit cheapest size: stream-based (batch 10) — 26.92 units; shortest total duration: stream-based (batch 5) — 9990 s from size 1 to 500: calls 20462 -> 91 (225x fewer), processed records 20462 -> 45500 (2.2x more) 13 poison records dragged 0 healthy records at size 1, and 5487 healthy records at size 500 into the dead letter queue together ``` The cost column draws a valley: 43.48 units at size 1, 26.92 at size 10, 57.53 at size 2,500. The left slope comes from the fixed overhead per call — every call needs 250 ms of setup, and that is paid twenty thousand times. The right slope comes from reprocessing: as size grows, the odds of a batch containing a bad record rise, and at size 2,500, **188% of records are processed extra**. The cheapest point is not the fastest point: the shortest duration is at size 5, the lowest cost at size 10. The third force is latency, and it rises in a single direction: the time to reach a record's result is 2.1 seconds at size 1 and 534 seconds at size 500; a record waits for 499 records it has nothing to do with to be processed. The fourth is a threshold: at size 2,500 the batch itself takes 1,000 seconds and no longer fits the 900-second duration limit. The row is written but cannot be chosen. The collateral column reduces the cost of indivisibility to a single number: thirteen poison records take down no healthy records at size 1, and drag 5,487 healthy records into the dead letter queue at size 500. Growing the batch size grows the **failure radius** along with the throughput. #### The Cost of Repeated Processing Extra processing was counted as a cost line; the real question is whether processing the same record twice is harmless. It is harmless for validation; it is not for a billing line. ```js // measurement-network/deduplication.mjs — the cost of repeated processing, and the cost of deduplicating it (model) import { N, MS_UNIT, records, synchronous, asynchronous, streamBased, measure } from "./invocation-models.mjs"; export const READ_MS = 15, WRITE_MS = 10, WINDOW = 24 * 3600; // ms, ms, seconds (fictional) export const SWEEP_DELAY = 3 * 24 * 3600; // the dead letter queue is swept by hand after 3 days if (import.meta.url === `file://${process.argv[1]}`) { const k = records(); const m = [synchronous(k), asynchronous(k), streamBased(k, 10)].map(measure); console.log("model".padEnd(25) + "extra processing".padEnd(19) + "extra billing lines".padEnd(21) + "dedup overhead".padEnd(19) + "extra cost".padEnd(12) + "outside-window repeat"); for (const x of m) { const overhead = x.processed * READ_MS + N * WRITE_MS; console.log(x.label.padEnd(25) + String(x.retries).padEnd(19) + String(x.retries).padEnd(21) + `${(overhead / 1000).toFixed(0)} s`.padEnd(19) + (overhead * MS_UNIT).toFixed(2).padEnd(12) + String(x.deadLetter)); } const s = m[2], share = (s.processed * READ_MS + N * WRITE_MS) / s.duration; console.log(`\ndedup window is ${WINDOW / 3600} hours; the dead letter queue is swept ${SWEEP_DELAY / 3600} hours ` + `later — the window is closed, ${s.deadLetter} records are processed a second time`); console.log(`for stream-based (batch 10), deduplication adds overhead equal to ${(share * 100).toFixed(1)}% of job duration; ` + `in return ${s.retries} extra billing lines are prevented`); } ``` ``` model extra processing extra billing lines dedup overhead extra cost outside-window repeat synchronous 449 449 507 s 1.27 0 asynchronous 462 462 507 s 1.27 13 stream-based (batch 10) 4200 4200 563 s 1.41 130 dedup window is 24 hours; the dead letter queue is swept 72 hours later — the window is closed, 130 records are processed a second time for stream-based (batch 10), deduplication adds overhead equal to 5.5% of job duration; in return 4200 extra billing lines are prevented ``` The cost of deduplication is small: 5.5% of job duration and 1.41 units. In return, **4,200 extra billing lines are prevented**; this comparison takes deduplication out of the realm of optional. The last column shows that deduplication comes with its own constraint too. The key is not kept forever; the window is 24 hours, while the dead letter queue is swept by hand after 72 hours. By the time the 130 records are swept, their deduplication record has long been deleted, and all of them are processed a second time. The measure's own window opens a path the measure does not cover. #### Summary - The latency the caller sees drops from 669 ms to 5 ms, but the moment the job finishes rises from 0.7 seconds to 2.1 seconds, and the 99th percentile rises to 61.3 seconds; the latency does not disappear, it becomes invisible. - In the synchronous model the caller sees all thirteen poison records; in the other two no one sees the error — handing off retry responsibility does not remove it, it removes visibility. - In the batch size scan, cost draws a valley: 43.48 units at size 1, 26.92 at size 10, 57.53 at size 2,500. The shortest duration is at size 5, the lowest cost at size 10, and size 2,500 no longer fits the duration limit. - Because the batch is not split, the failure radius grows with size: thirteen poison records drag no healthy records into the dead letter queue at size 1, and 5,487 healthy records at size 500. - Deduplication adds 5.5% to job duration and prevents 4,200 extra billing lines, but it has a 24-hour window; the 130 records swept after 72 hours fall outside the window and are processed a second time. #### Next Step The latency numbers of all three models rest on a single assumption: when the call arrives, an instance ready to run the function is standing by. The 250 ms written as fixed overhead is the cost of that readiness. A ready instance is not always there. Because the capacity decision is handed off, instances are created as requests arrive and reclaimed when idle. The next lesson runs a warm pool model and counts the cold call rate, the duration added to latency, and when a queue forms as pool lifetime and traffic profile change; it then measures three mitigation paths separately. ### Cold Start Source: https://academia.sh/en/courses/serverless/cold-start A warm pool model is run to count whether a request finds an instance: in three fictional traffic profiles, the cold call rate, the duration added to latency, and the requests that land in the queue are measured; pool lifetime is scanned from zero to eighteen hundred seconds to work out the threshold where the cold rate drops and the idle instance-seconds that come back in return, and three mitigation paths are priced separately. The latency numbers of the previous two lessons rested on a single assumption: an instance ready to run the function was standing by when the call arrived. Because the capacity decision is handed off, this is not always true. Instances are created as requests arrive and reclaimed when idle; a request that finds no ready instance experiences a **cold start** — setting up the runtime, unpacking the package, and startup code all land inside the request path. M23/K02 measured the same trade-off — the item embedded in the image versus the work left for boot — at the scale of seconds; that measurement is not repeated here. Here the scale is milliseconds, and the object is not an image but a package. A single ratio is asked: how many calls start cold, what it depends on, and what mitigation costs. **FC13.** The measurement window is 1,800 seconds; the regional measurement network's three fictional traffic profiles are built with a self-written generator, and the seed is 20260804. **FC14.** A warm call takes 400 ms; a cold call adds 900 ms; this extra time enters latency, not the bill. **FC15.** Profiles: field notification is sparse (average 90 s gap), reading validation is regular (average 250 ms gap), and the nightly batch is bursty (once every 600 seconds, a 60-second burst, 25 ms gap). **FC16.** Pool lifetime is how long an instance sits idle before it is reclaimed; the baseline value is 300 seconds. **FC17.** The function's concurrency share is 40; once the pool is full, an incoming request joins the queue rather than being dropped. **FC18.** Prices are written in fictional units: 0.0005 per call, 0.0000025 per processing millisecond, 0.0002 per pre-warmed instance-second. #### Is There an Instance in the Pool The model rests on a single rule: if the pool has an idle instance, the call is warm; if not and the share has not filled, a new instance is created; if the share has also filled, the request joins the queue. ```js // measurement-network/warm-pool.mjs — warm if the pool has an instance when the request arrives, cold otherwise (model) export const WINDOW = 1800000, PROC_MS = 400, COLD_EXTRA_MS = 900; // ms: measurement window, processing, cold extra export const LIFETIME = 300000, SHARE_LIMIT = 40; // ms idle-kept, the function's concurrency share export const CALL_UNIT = 0.0005, MS_UNIT = 0.0000025, WARM_UNIT = 0.0002; // fictional unit export const rng = (t) => { let s = t >>> 0; return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; }; // Three fictional traffic profiles: sparse field notifications, regular validation, bursty nightly batch. export const PROFILE = { sparse: { gap: 90000, windows: [[0, WINDOW]] }, regular: { gap: 250, windows: [[0, WINDOW]] }, bursty: { gap: 25, windows: [[0, 60000], [600000, 660000], [1200000, 1260000]] }, }; export const arrivals = (name, seed = 20260804) => { const r = rng(seed), prof = PROFILE[name], out = []; for (const [start, end] of prof.windows) { let t = start; while (t < end) { t += -Math.log(1 - r()) * prof.gap; if (t < end) out.push(t); } } return out; }; const percentile = (a, p) => [...a].sort((x, y) => x - y)[Math.floor(a.length * p)]; const idle = (inst, end) => Math.max(0, end - inst.born - inst.busy) / 1000; // Pool: every instance gets a "free" timestamp the moment it frees up; once its lifetime elapses from that stamp, it is reclaimed. export const run = (arrivalTimes, { lifetime = LIFETIME, shareLimit = SHARE_LIMIT, coldExtra = COLD_EXTRA_MS, proc = PROC_MS, warm = 0, extraCallMs = 0 } = {}) => { const pool = []; for (let i = 0; i < warm; i++) pool.push({ free: 0, born: 0, busy: 0, pinned: true }); let coldCount = 0, queued = 0, longestWait = 0, idleSec = 0, mostInstances = pool.length, workMs = 0; const latency = []; for (const t of arrivalTimes) { for (let i = pool.length - 1; i >= 0; i--) { const o = pool[i]; if (!o.pinned && o.free + lifetime < t) { idleSec += idle(o, o.free + lifetime); pool.splice(i, 1); } } let inst = pool.find((o) => o.free <= t), start = t, extra = 0, wait = 0; if (!inst && pool.length < shareLimit) { inst = { free: 0, born: t, busy: 0, pinned: false }; pool.push(inst); extra = coldExtra; coldCount++; } else if (!inst) { inst = pool.reduce((a, b) => (b.free < a.free ? b : a)); wait = inst.free - t; start = inst.free; queued++; if (wait > longestWait) longestWait = wait; } const duration = extra + proc + extraCallMs; inst.free = start + duration; inst.busy += duration; workMs += duration; latency.push(wait + duration); if (pool.length > mostInstances) mostInstances = pool.length; } for (const o of pool) idleSec += idle(o, o.pinned ? WINDOW : Math.min(WINDOW, o.free + lifetime)); return { calls: arrivalTimes.length, cold: coldCount, rate: coldCount / arrivalTimes.length, avgLatency: latency.reduce((a, b) => a + b, 0) / latency.length, p99: percentile(latency, 0.99), queued, longestWait, mostInstances, idleSec, cost: arrivalTimes.length * CALL_UNIT + workMs * MS_UNIT + warm * (WINDOW / 1000) * WARM_UNIT }; }; if (import.meta.url === `file://${process.argv[1]}`) { console.log(`window ${WINDOW / 1000} s, processing ${PROC_MS} ms, cold extra ${COLD_EXTRA_MS} ms, ` + `lifetime ${LIFETIME / 1000} s, share ${SHARE_LIMIT}`); console.log("\n" + "profile".padEnd(10) + "calls".padEnd(9) + "cold".padEnd(8) + "cold rate".padEnd(12) + "avg latency".padEnd(14) + "p99".padEnd(12) + "queued".padEnd(15) + "longest wait".padEnd(17) + "most instances".padEnd(16) + "idle instance-s"); for (const name of Object.keys(PROFILE)) { const s = run(arrivals(name)); console.log(name.padEnd(10) + String(s.calls).padEnd(9) + String(s.cold).padEnd(8) + `${(s.rate * 100).toFixed(2)}%`.padEnd(12) + `${s.avgLatency.toFixed(0)} ms`.padEnd(14) + `${s.p99.toFixed(0)} ms`.padEnd(12) + String(s.queued).padEnd(15) + `${s.longestWait.toFixed(0)} ms`.padEnd(17) + String(s.mostInstances).padEnd(16) + s.idleSec.toFixed(0)); } const a = run(arrivals("bursty")); console.log(`\nin the bursty profile, cold calls add ${(a.cold * COLD_EXTRA_MS / 1000).toFixed(0)} s to latency; ` + `a warm call is ${PROC_MS} ms, a cold call is ${PROC_MS + COLD_EXTRA_MS} ms — ${((PROC_MS + COLD_EXTRA_MS) / PROC_MS).toFixed(2)}x`); } ``` ``` window 1800 s, processing 400 ms, cold extra 900 ms, lifetime 300 s, share 40 profile calls cold cold rate avg latency p99 queued longest wait most instances idle instance-s sparse 18 2 11.11% 500 ms 1300 ms 0 0 ms 1 1722 regular 7242 9 0.12% 401 ms 400 ms 0 0 ms 8 10510 bursty 7241 120 1.66% 419 ms 1300 ms 135 539 ms 40 38159 in the bursty profile, cold calls add 108 s to latency; a warm call is 400 ms, a cold call is 1300 ms — 3.25x ``` The three profiles show three separate behaviors. In regular traffic the cold rate is 0.12%: requests arrive 250 ms apart, the pool settles at eight instances, and only nine calls start cold. In sparse traffic the rate is 11.11%, and that rate comes not from load but from how infrequent the calls are. In bursty traffic the rate is 1.66%, but the real number is in a different column: **135 requests join the queue**, the longest wait is 539 ms. The queue is the second face of cold start. When the burst begins the pool is empty; every new request needs a new instance, and every instance spends 900 ms on setup. Once the share reaches 40, requests start waiting. A cold call does not only raise its own latency; by slowing how fast the pool fills, it builds a queue behind it: the average shown is 419 ms, and the 99th percentile is 1,300 ms. #### Pool Lifetime Scan Pool lifetime is a handed-off decision: the provider decides how long an instance is kept idle. The scan raises lifetime from zero to 1,800 seconds. ```js // measurement-network/lifetime-scan.mjs — the cold call rate as pool lifetime and traffic profile change (model) import { PROFILE, arrivals, run } from "./warm-pool.mjs"; export const LIFETIMES = [0, 15, 30, 60, 120, 300, 600, 1800]; // seconds export const NAMES = Object.keys(PROFILE); if (import.meta.url === `file://${process.argv[1]}`) { const arrivalsByProfile = Object.fromEntries(NAMES.map((a) => [a, arrivals(a)])); console.log("pool lifetime".padEnd(15) + NAMES.map((a) => `${a}: cold / idle`.padEnd(24)).join("") + "bursty: queued / avg latency"); const table = {}; for (const lt of LIFETIMES) { const row = Object.fromEntries(NAMES.map((a) => [a, run(arrivalsByProfile[a], { lifetime: lt * 1000 })])); table[lt] = row; console.log(`${lt} s`.padEnd(15) + NAMES.map((a) => `${(row[a].rate * 100).toFixed(2)}% / ${row[a].idleSec.toFixed(0)}`.padEnd(24)).join("") + `${row.bursty.queued} / ${row.bursty.avgLatency.toFixed(0)} ms`); } console.log(""); for (const name of NAMES) { const good = LIFETIMES.find((lt) => table[lt][name].rate < 0.01); const atZero = table[0][name], atMax = table[1800][name]; console.log(`${name.padEnd(9)} at lifetime 0: ${(atZero.rate * 100).toFixed(1)}% cold and ${atZero.idleSec.toFixed(0)} idle instance-s; ` + `at lifetime 1800: ${(atMax.rate * 100).toFixed(2)}% and ${atMax.idleSec.toFixed(0)}; ` + (good === undefined ? "never drops below 1% at any lifetime" : `drops below 1% at a ${good} s lifetime`)); } } ``` ``` pool lifetime sparse: cold / idle regular: cold / idle bursty: cold / idle bursty: queued / avg latency 0 s 100.00% / 0 100.00% / 0 49.99% / 0 3621 / 969 ms 15 s 88.89% / 266 1.35% / 7651 1.78% / 3819 135 / 420 ms 30 s 77.78% / 489 0.65% / 8493 1.73% / 5713 135 / 419 ms 60 s 72.22% / 895 0.33% / 9208 1.66% / 9359 135 / 419 ms 120 s 38.89% / 1364 0.18% / 9883 1.66% / 16559 135 / 419 ms 300 s 11.11% / 1722 0.12% / 10510 1.66% / 38159 135 / 419 ms 600 s 5.56% / 1781 0.11% / 10873 0.59% / 57609 47 / 407 ms 1800 s 5.56% / 1781 0.11% / 10873 0.55% / 69047 47 / 406 ms sparse at lifetime 0: 100.0% cold and 0 idle instance-s; at lifetime 1800: 5.56% and 1781; never drops below 1% at any lifetime regular at lifetime 0: 100.0% cold and 0 idle instance-s; at lifetime 1800: 0.11% and 10873; drops below 1% at a 30 s lifetime bursty at lifetime 0: 50.0% cold and 0 idle instance-s; at lifetime 1800: 0.55% and 69047; drops below 1% at a 600 s lifetime ``` The scan gives three separate curves. In regular traffic, a 15-second lifetime pulls the cold rate from 100% to 1.35%; beyond that it gains nothing but raises idle instance-seconds from 7,651 to 10,873. In bursty traffic the rate **does not change at all** from 15 seconds to 300 seconds — it sticks at 1.66% — and drops suddenly to 0.59% at 600 seconds. The threshold is not a coincidence: the gap between bursts is 540 seconds, and once lifetime crosses that gap, the pool survives between two bursts. On the same row, queued requests drop from 135 to 47. The price of that threshold sits on the same row: idle instance-seconds rise from 38,159 to 57,609. Cutting the cold call rate to a third is paid for by **raising idle capacity by half**. The sparse profile never drops below 1% at any lifetime; even at 1,800 seconds the rate is 5.56%, because one of the eighteen calls is the window's first, and there is no pool before it. In sparse traffic, cold start is not a tuning problem — it is a property of the traffic itself. #### Three Mitigation Paths ```js // measurement-network/mitigation.mjs — three ways to reduce cold start, and the cost of each (model) import { COLD_EXTRA_MS, PROC_MS, WINDOW, arrivals, run } from "./warm-pool.mjs"; // [name, run options, kind of cost] export const PATH = [ ["no mitigation", {}, "—"], ["pre-warming (16)", { warm: 16 }, "16 instances sit idle for 1800 s"], ["package shrink", { coldExtra: 350 }, "3 dependencies are moved out"], ["lazy loading", { coldExtra: 500, extraCallMs: 40 }, "40 ms of lazy loading on every call"], ["shrink + warming", { coldExtra: 350, warm: 16 }, "the cost of both together"], ]; if (import.meta.url === `file://${process.argv[1]}`) { const arr = arrivals("bursty"), baseline = run(arr); console.log(`bursty profile, ${arr.length} calls, pool lifetime 300 s; warm call ${PROC_MS} ms`); console.log("\n" + "path".padEnd(22) + "cold".padEnd(8) + "cold rate".padEnd(12) + "avg latency".padEnd(14) + "p99".padEnd(12) + "queued".padEnd(15) + "idle instance-s".padEnd(17) + "cost".padEnd(10) + "latency gain"); for (const [name, opts] of PATH) { const row = run(arr, opts); console.log(name.padEnd(22) + String(row.cold).padEnd(8) + `${(row.rate * 100).toFixed(2)}%`.padEnd(12) + `${row.avgLatency.toFixed(1)} ms`.padEnd(14) + `${row.p99.toFixed(0)} ms`.padEnd(12) + String(row.queued).padEnd(15) + row.idleSec.toFixed(0).padEnd(17) + row.cost.toFixed(2).padEnd(10) + `${(baseline.avgLatency - row.avgLatency).toFixed(1)} ms`); } console.log("\n" + "path".padEnd(22) + "cost".padEnd(38) + "extra cost per ms gained"); for (const [name, opts, cost] of PATH.slice(1)) { const row = run(arr, opts), gain = baseline.avgLatency - row.avgLatency; console.log(name.padEnd(22) + cost.padEnd(38) + (gain > 0 ? ((row.cost - baseline.cost) / gain).toFixed(3) : "no gain")); } const threshold = 40 / (COLD_EXTRA_MS - 500); console.log(`lazy loading only pays off once the cold rate is above ${(threshold * 100).toFixed(0)}%: ` + `40 ms lands on every call, while the 400 ms gained falls only on cold calls — in this profile the rate is 1.66%`); const warmRow = run(arr, { warm: 16 }); console.log(`\npre-warming raises capacity spent idle from ${baseline.idleSec.toFixed(0)} instance-s to ` + `${warmRow.idleSec.toFixed(0)} instance-s; over the ${WINDOW / 1000} s window that is ` + `${(warmRow.cost - baseline.cost).toFixed(2)} extra units — the cold extra itself is ${COLD_EXTRA_MS} ms`); } ``` ``` bursty profile, 7241 calls, pool lifetime 300 s; warm call 400 ms path cold cold rate avg latency p99 queued idle instance-s cost latency gain no mitigation 120 1.66% 418.9 ms 1300 ms 135 38159 11.13 0.0 ms pre-warming (16) 49 0.68% 406.1 ms 400 ms 0 42736 16.73 12.8 ms package shrink 111 1.53% 405.4 ms 750 ms 0 35500 10.96 13.5 ms lazy loading 120 1.66% 448.5 ms 940 ms 22 38229 11.74 -29.7 ms shrink + warming 43 0.59% 402.1 ms 400 ms 0 40933 16.66 16.8 ms path cost extra cost per ms gained pre-warming (16) 16 instances sit idle for 1800 s 0.438 package shrink 3 dependencies are moved out -0.013 lazy loading 40 ms of lazy loading on every call no gain shrink + warming the cost of both together 0.329 lazy loading only pays off once the cold rate is above 10%: 40 ms lands on every call, while the 400 ms gained falls only on cold calls — in this profile the rate is 1.66% pre-warming raises capacity spent idle from 38159 instance-s to 42736 instance-s; over the 1800 s window that is 5.60 extra units — the cold extra itself is 900 ms ``` The three paths each pay a different kind of cost. Pre-warming pulls the cold rate to 0.68%, the queue to zero, and the 99th percentile from 1,300 ms to 400 ms: because sixteen instances are always waiting in the pool, the burst's first second is not cold. Its cost is the very thing the first lesson avoided — idle instance-seconds rise to 42,736, and the bill climbs from 11.13 units to 16.73 units, a **50% increase**. Package shrinking is the one path that looks free: once the cold extra drops from 900 ms to 350 ms, both latency and cost fall. But the cost is not in the table — three dependencies have been pulled out of the package, and that has to be paid for somewhere. This is the quietest form of a workaround around a constraint. The third path loses money. Moving startup work pulls the cold extra down to 500 ms but adds 40 ms of lazy loading to every call; average latency **rises 29.7 ms**, and twenty-two requests join the queue. The last row gives the threshold: 40 ms lands on every call, while the 400 ms gained falls only on cold calls, so the path pays off only once the cold rate is above 10%. In this profile the rate is 1.66%; in the sparse profile it is 11.11%. #### Summary - A cold call takes 1,300 ms, a warm call 400 ms: 3.25x. Even though the bursty profile's rate is only 1.66%, 135 requests join the queue and the longest wait is 539 ms. - In the pool lifetime scan, the threshold comes from the shape of the traffic: in the bursty profile the rate sticks at 1.66% between 15 and 300 seconds, then drops to 0.59% at 600 seconds, because the gap between bursts is 540 seconds. The threshold's cost is written on the same row: idle instance-seconds rise from 38,159 to 57,609. - Pre-warming drops the queue to zero and raises the bill 50%; package shrinking lowers both, but it demands pulling three dependencies out of the package. - Moving startup work raises latency 29.7 ms in this profile; it pays off only once the cold rate is above 10% — the mitigation path itself depends on a threshold. #### Next Step In this lesson the concurrency share of 40 was a fixed number, and it explained where the queue came from. But it does not stand alone: the memory choice decides the processor share, the processor share decides processing duration, and processing duration decides how much work the same share can do. The next lesson applies three thresholds — memory, duration, and concurrency — to a job set and counts how many jobs do not fit. Memory is scanned, and it shows that duration and unit cost move in opposite directions, with the cheapest point landing in the middle. Once the concurrency limit is exceeded, throttling and the queue behind it are counted; sharing the limit at the account level is measured with an example. ### Resource and Duration Limits Source: https://academia.sh/en/courses/serverless/resource-and-duration-limits Three thresholds — memory, duration, and concurrency — are applied to a set of twelve jobs, and how many jobs do not fit is counted; scanning memory tiers shows duration dropping while unit cost draws a valley, with the cheapest point landing in the middle; when the concurrency limit is shared at the account level, one function starving another is measured with the throttle rate and the lost requests. The same numbers sat in the background across three lessons: the 900-second duration limit, the memory tiers, and the concurrency limit of 200. This lesson moves all three to the center of measurement, because this is exactly what handing off capacity management buys — the capacity decision is gone, and three numbers have arrived in its place. The constraint shows up in two forms: some jobs do not fit the threshold at all and must be split, some jobs fit but cannot run in their cheapest or fastest form. Both are countable. **FC19.** The thresholds are **fictional**: the memory cap is 3008 MB, the duration limit is 900 seconds, the account-level concurrency limit is 200. **FC20.** The measurement network's twelve fictional jobs are listed with memory, duration, and peak concurrency values; durations are at the 1024 MB tier. **FC21.** Processor share is proportional to memory: one unit at 256 MB, doubling at each tier, and capping at eight at 2048 MB. **FC22.** If a job's working set does not fit the chosen memory, garbage collection time is added; the coefficient is 0.15, and below 256 MB the job drops out from insufficient memory. **FC23.** In the concurrency model the night job is split into 900 pieces, each piece is about 245 seconds, and the deviation comes from a self-written generator, seed 20260805; field notification is 25 requests per second. **FC24.** A throttled notification request retries one second later, and is lost after the third attempt; prices are in fictional units. #### Three Thresholds, Twelve Jobs ```js // measurement-network/job-set.mjs — three thresholds are applied to a job set, how many jobs do not fit (model) export const MEMORY_CAP = 3008, DURATION_LIMIT = 900, CONCURRENCY_LIMIT = 200; // MB, s, concurrent calls // The regional measurement network's fictional jobs: [name, working set MB, single run s, peak concurrency] export const JOB = [ ["reading validation", 180, 0.4, 148], ["field notification", 120, 0.3, 25], ["branch endpoint report", 300, 45, 8], ["daily summary", 512, 60, 3], ["billing generation", 900, 240, 40], ["nightly batch job (single piece)", 700, 220600, 1], ["nightly batch job (246 pieces)", 700, 897, 246], ["anomaly scan", 2200, 700, 12], ["archive compression", 1400, 1100, 6], ["monthly billing close", 2600, 1500, 4], ["meter calibration report", 3400, 420, 2], ["meter visual inspection", 5200, 95, 3], ]; export const check = ([name, mb, s, cc]) => { const exceeded = []; if (mb > MEMORY_CAP) exceeded.push("memory"); if (s > DURATION_LIMIT) exceeded.push("duration"); if (cc > CONCURRENCY_LIMIT) exceeded.push("concurrency"); return { name, mb, s, cc, exceeded }; }; // The workaround path for a job that does not fit, and its kind of cost. export const WORKAROUND = { memory: (i) => `externalize state: ${Math.ceil(i.mb / MEMORY_CAP)} pieces and ${i.mb - MEMORY_CAP} MB external read`, duration: (i) => `split: ${Math.ceil(i.s / DURATION_LIMIT)} calls and ${Math.ceil(i.s / DURATION_LIMIT) - 1} progress records`, concurrency: (i) => `throttle: ${i.cc - CONCURRENCY_LIMIT} calls wait, ${Math.ceil(i.cc / CONCURRENCY_LIMIT)} rounds`, }; if (import.meta.url === `file://${process.argv[1]}`) { console.log(`thresholds: memory ${MEMORY_CAP} MB, duration ${DURATION_LIMIT} s, concurrency ${CONCURRENCY_LIMIT}`); console.log("\n" + "job".padEnd(34) + "memory".padEnd(10) + "duration".padEnd(11) + "concurrency".padEnd(14) + "result"); const d = JOB.map(check); for (const i of d) console.log(i.name.padEnd(34) + `${i.mb} MB`.padEnd(10) + `${i.s} s`.padEnd(11) + String(i.cc).padEnd(14) + (i.exceeded.length ? `exceeds ${i.exceeded.join(" + ")}` : "fits")); const unfit = d.filter((i) => i.exceeded.length); const count = (e) => d.filter((i) => i.exceeded.includes(e)).length; console.log(`\n${unfit.length} of ${JOB.length} jobs do not fit: memory ${count("memory")}, ` + `duration ${count("duration")}, concurrency ${count("concurrency")} (a job can exceed more than one threshold)`); console.log("\n" + "unfit job".padEnd(34) + "exceeded threshold".padEnd(20) + "workaround path and cost"); for (const i of unfit) for (const e of i.exceeded) console.log(i.name.padEnd(34) + e.padEnd(20) + WORKAROUND[e](i)); } ``` ``` thresholds: memory 3008 MB, duration 900 s, concurrency 200 job memory duration concurrency result reading validation 180 MB 0.4 s 148 fits field notification 120 MB 0.3 s 25 fits branch endpoint report 300 MB 45 s 8 fits daily summary 512 MB 60 s 3 fits billing generation 900 MB 240 s 40 fits nightly batch job (single piece) 700 MB 220600 s 1 exceeds duration nightly batch job (246 pieces) 700 MB 897 s 246 exceeds concurrency anomaly scan 2200 MB 700 s 12 fits archive compression 1400 MB 1100 s 6 exceeds duration monthly billing close 2600 MB 1500 s 4 exceeds duration meter calibration report 3400 MB 420 s 2 exceeds memory meter visual inspection 5200 MB 95 s 3 exceeds memory 6 of 12 jobs do not fit: memory 2, duration 3, concurrency 1 (a job can exceed more than one threshold) unfit job exceeded threshold workaround path and cost nightly batch job (single piece) duration split: 246 calls and 245 progress records nightly batch job (246 pieces) concurrency throttle: 46 calls wait, 2 rounds archive compression duration split: 2 calls and 1 progress records monthly billing close duration split: 2 calls and 1 progress records meter calibration report memory externalize state: 2 pieces and 392 MB external read meter visual inspection memory externalize state: 2 pieces and 2192 MB external read ``` Six of the twelve jobs do not fit: three exceed duration, two exceed memory, one exceeds the concurrency limit. The lower table writes the cost of the workaround in its own unit — because the nightly batch job takes 220,600 seconds in one piece, it is split into **246 calls and 245 progress records**. The "where was I" information that is free inside a single job turns, once split, into a line item kept outside and read and written on every call. Two rows are the same job's two states. The night job runs into the duration limit in one piece; splitting it into 246 pieces escapes that but runs into the **concurrency limit**. Working around one threshold moves the job into another: 246 pieces fit the 200 limit in two rounds, and 46 calls wait for the first round. The two jobs that exceed the memory cap take a different workaround path: externalizing state. Meter visual inspection asks for 5,200 MB, the cap is 3,008 MB, and the 2,192 MB in between is read externally on every run — trading memory access for network access. #### Memory Tier Scan Memory is not a limit on its own; it also decides the processor share. So increasing memory shortens duration, and unit cost becomes the product of two opposing forces. ```js // measurement-network/memory-scan.mjs — as memory grows duration drops, unit cost first drops then rises (model) import { DURATION_LIMIT } from "./job-set.mjs"; export const TIER = [128, 256, 512, 768, 1024, 1536, 2048, 3008]; // MB, fictional tiers export const BASE_MB = 256, CAP_SHARE = 8; // processor share is proportional to memory, caps at 2048 export const WORKING_SET = 900, GC_FACTOR = 0.15, LOWER_LIMIT = 256; export const SERIAL = 4000, PARALLEL = 944000; // ms: the serial and parallel portion of billing generation export const GB_S = 0.001, CALL_UNIT = 0.0005; // fictional unit export const share = (mb) => Math.min(mb / BASE_MB, CAP_SHARE); export const gc = (mb) => (mb >= WORKING_SET ? 0 : (WORKING_SET / mb - 1) ** 2 * GC_FACTOR * PARALLEL); export const run = (mb) => { if (mb < LOWER_LIMIT) return { mb, fits: false, reason: "not enough memory" }; const duration = (SERIAL + PARALLEL / share(mb) + gc(mb)) / 1000; return { mb, fits: duration <= DURATION_LIMIT, reason: duration > DURATION_LIMIT ? "duration limit" : "", duration, share: share(mb), gc: gc(mb) / 1000, cost: (mb / 1024) * duration * GB_S + CALL_UNIT }; }; if (import.meta.url === `file://${process.argv[1]}`) { console.log(`billing generation: working set ${WORKING_SET} MB, parallel portion ` + `${PARALLEL / 1000} s (at ${BASE_MB} MB), processor share cap ${CAP_SHARE}`); console.log("\n" + "memory".padEnd(10) + "processor share".padEnd(17) + "garbage collection".padEnd(20) + "duration".padEnd(12) + "unit cost".padEnd(16) + "result"); const s = TIER.map(run); for (const x of s) console.log(`${x.mb} MB`.padEnd(10) + (x.fits === false && !x.duration ? "—".padEnd(17) + "—".padEnd(20) + "—".padEnd(12) + "—".padEnd(16) : x.share.toFixed(2).padEnd(17) + `${x.gc.toFixed(1)} s`.padEnd(20) + `${x.duration.toFixed(1)} s`.padEnd(12) + x.cost.toFixed(4).padEnd(16)) + (x.fits ? "fits" : x.reason)); const fitting = s.filter((x) => x.fits); const cheapest = fitting.reduce((a, b) => (b.cost < a.cost ? b : a)); const fastest = fitting.reduce((a, b) => (b.duration < a.duration ? b : a)); const largest = fitting[fitting.length - 1]; console.log(`\ncheapest tier ${cheapest.mb} MB: ${cheapest.duration.toFixed(1)} s, ${cheapest.cost.toFixed(4)} units`); console.log(`fastest tier ${fastest.mb} MB: ${(cheapest.duration / fastest.duration).toFixed(2)}x faster, ` + `${((fastest.cost / cheapest.cost - 1) * 100).toFixed(1)}% more expensive`); console.log(`largest tier ${largest.mb} MB: same duration, ` + `${((largest.cost / cheapest.cost - 1) * 100).toFixed(0)}% more expensive — because the processor share hits its cap`); } ``` ``` billing generation: working set 900 MB, parallel portion 944 s (at 256 MB), processor share cap 8 memory processor share garbage collection duration unit cost result 128 MB — — — — not enough memory 256 MB 1.00 896.1 s 1844.1 s 0.4615 duration limit 512 MB 2.00 81.3 s 557.3 s 0.2792 fits 768 MB 3.00 4.2 s 322.8 s 0.2426 fits 1024 MB 4.00 0.0 s 240.0 s 0.2405 fits 1536 MB 6.00 0.0 s 161.3 s 0.2425 fits 2048 MB 8.00 0.0 s 122.0 s 0.2445 fits 3008 MB 8.00 0.0 s 122.0 s 0.3589 fits cheapest tier 1024 MB: 240.0 s, 0.2405 units fastest tier 2048 MB: 1.97x faster, 1.7% more expensive largest tier 3008 MB: same duration, 49% more expensive — because the processor share hits its cap ``` The duration column falls in a single direction: from 1,844 seconds to 122 seconds. The cost column falls and then rises: it drops from 0.4615 to 0.2405, then climbs to 0.3589; the cheapest tier, **1024 MB**, sits in the middle of the table. The reason for the rise on the right sits in the last row — the processor share caps at 2048 MB, and choosing 3008 MB raises the bill 49% without shortening duration at all. The reason for the rise on the left is different. At 256 MB the job's working set does not fit in memory, and garbage collection adds 896 seconds; the job takes 1,844 seconds and **falls outside the duration limit**. This row shows the two thresholds are linked: dropping the memory tier throws the job into the duration limit. At 512 MB the same effect drops to 81 seconds and the job fits, but it is both slower and more expensive than 1024 MB. The last rows say the decision is not single: 2048 MB is 1.97x faster than 1024 MB and only 1.7% more expensive. For work where latency matters this trade is cheap; for a night job no one is waiting on, it is unnecessary. The same table suggests two different rows for two different jobs. #### The Shared Limit and Throttling Duration and memory are the job's own decisions. Concurrency is not: the limit sits **at the account level**, and every function shares it. ```js // measurement-network/concurrency.mjs — the account-level shared concurrency limit and throttling (model) import { CONCURRENCY_LIMIT } from "./job-set.mjs"; export const STEP_MS = 40, WINDOW_S = 2400, STEPS = WINDOW_S * 1000 / STEP_MS; export const PIECE = 900, PIECE_S = 245, DEVIATION = 20; // night job: 900 pieces, ~245 s export const NOTIFICATION_STEPS = 8, RETRY = 25; // 300 ms processing, retry 1 s later export const rng = (t) => { let s = t >>> 0; return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; }; export const run = (reserve) => { const cap = CONCURRENCY_LIMIT - reserve, r = rng(20260805); const doneNight = new Int32Array(STEPS + 20000), doneNotif = new Int32Array(STEPS + 20000); const wait = [new Int32Array(STEPS + 200), new Int32Array(STEPS + 200), new Int32Array(STEPS + 200)]; let nightActive = 0, notifActive = 0, remaining = PIECE, throttled = 0, lost = 0, success = 0, delay = 0; let nightEnd = -1, peakNight = 0, peakNotif = 0; for (let a = 0; a < STEPS; a++) { nightActive -= doneNight[a]; notifActive -= doneNotif[a]; for (let d = 0; d < 3; d++) { const n = (d === 0 ? 1 : 0) + wait[d][a]; for (let i = 0; i < n; i++) { if (nightActive + notifActive < CONCURRENCY_LIMIT) { notifActive++; doneNotif[a + NOTIFICATION_STEPS]++; success++; delay += d * RETRY * STEP_MS; } else { throttled++; if (d < 2) wait[d + 1][a + RETRY]++; else lost++; } } } while (remaining > 0 && nightActive + notifActive < CONCURRENCY_LIMIT && nightActive < cap) { const s = PIECE_S + (r() * 2 - 1) * DEVIATION; nightActive++; doneNight[a + Math.round(s * 1000 / STEP_MS)]++; remaining--; } if (nightActive > peakNight) peakNight = nightActive; if (notifActive > peakNotif) peakNotif = notifActive; if (remaining === 0 && nightActive === 0 && nightEnd < 0) nightEnd = a; } return { reserve, cap, nightEnd: nightEnd * STEP_MS / 1000, rounds: Math.ceil(PIECE / cap), throttled, lost, success, incoming: STEPS, throttleRate: throttled / STEPS, peakNight, peakNotif }; }; if (import.meta.url === `file://${process.argv[1]}`) { console.log(`account limit ${CONCURRENCY_LIMIT}; night job ${PIECE} x ~${PIECE_S} s, ` + `notification ${1000 / STEP_MS}/s x ${NOTIFICATION_STEPS * STEP_MS} ms`); console.log("\n" + "arrangement".padEnd(24) + "night cap".padEnd(13) + "night ends".padEnd(13) + "rounds".padEnd(9) + "throttled notifications".padEnd(25) + "throttle rate".padEnd(15) + "lost".padEnd(12) + "peak slots (night / notification)"); const p = run(0), y = run(40); for (const s of [p, y]) console.log((s.reserve ? `reserved share ${s.reserve}` : "shared").padEnd(24) + String(s.cap).padEnd(13) + `${s.nightEnd.toFixed(0)} s`.padEnd(13) + String(s.rounds).padEnd(9) + String(s.throttled).padEnd(25) + `${(s.throttleRate * 100).toFixed(1)}%`.padEnd(15) + String(s.lost).padEnd(12) + `${s.peakNight} / ${s.peakNotif}`); console.log(`\nthe reserved share delays the night job by ${(y.nightEnd - p.nightEnd).toFixed(0)} s; ` + `in return it prevents ${p.throttled} throttling events and ${p.lost} lost notifications — ` + `of ${p.incoming} incoming requests the throttle rate is ${(p.throttled / p.incoming * 100).toFixed(1)}%`); } ``` ``` account limit 200; night job 900 x ~245 s, notification 25/s x 320 ms arrangement night cap night ends rounds throttled notifications throttle rate lost peak slots (night / notification) shared 200 1254 s 5 14814 24.7% 4923 199 / 14 reserved share 40 160 1496 s 6 0 0.0% 0 160 / 8 the reserved share delays the night job by 242 s; in return it prevents 14814 throttling events and 4923 lost notifications — of 60000 incoming requests the throttle rate is 24.7% ``` In the shared arrangement, the night job holds 199 slots. **24.7% of notification requests are throttled**, 8.2% are lost after three attempts: 4,923 notifications are never sent. The night job sees no error during this. Starving is a one-sided event; it shows up in the victim's log, not the offender's. The reserved share fixes this and writes its cost openly: once the night cap drops to 160, the job finishes in six rounds instead of five and takes 1,496 seconds instead of 1,254. **242 seconds of delay is the price of 14,814 throttling events and 4,923 lost notifications.** The reserved share does not create new capacity; it divides up what already exists, and it forces that division to be made once, in advance. #### Summary - Six of the twelve jobs do not fit the thresholds: three exceed duration, two exceed memory, one exceeds the concurrency limit. - Working around one threshold moves the job into another: the night job exceeds the duration limit in one piece, runs into the concurrency limit once split into 246 pieces, and pays 245 progress records. - In the memory scan, duration drops from 1,844 seconds to 122 seconds while cost draws a valley; the cheapest tier is 1024 MB, and 3008 MB raises the bill 49% without shortening duration at all. - At 256 MB the working set does not fit, so garbage collection adds 896 seconds and the job falls outside the duration limit: dropping the memory tier throws the job into a second threshold. - Because the concurrency limit is shared at the account level, the night job holds 199 slots; 24.7% of notification requests are throttled, 8.2% are lost, and the night job never sees any of it. - The reserved share drops throttling to zero and writes the cost onto the night job: six rounds instead of five, 242 seconds more. #### Next Step This lesson measured the three walls around the function, and every measurement carried a silent assumption: the code being measured is the code running. But the change that splits the night job into 246 pieces, or the setting that pulls the memory tier from 512 to 1024, gets published on some day, and on that day two versions exist side by side. Which version traffic goes to, and whether settings travel along with the version, calls for a separate measurement. The next lesson measures the function's versioning and how traffic is split across versions. ### Versioning and Aliases Source: https://academia.sh/en/courses/serverless/versioning-and-aliases Weight shifting through the alias is measured: the version stays unchanged, the alias moves. At every step, the affected requests and the round where the difference disappears are counted; cold start's interference with the canary measurement and rollback being a single key turn are worked out separately. Thresholds were counted: the duration limit, the memory limit, the concurrency cap. Which job does not fit inside these thresholds was written down. What happens to the job that does fit was never asked — once that function is written, how does it enter the running system? The pattern side of gradual release — by which metric, which window, which threshold the decision is made — is a topic measured elsewhere and is not repeated here. The question here is narrower: this ladder runs through a single pointer called an **alias**, and the mechanism has two facts of its own. Rollback is a single write; cold start starts from zero on the new version. - **FC25.** The fictional regional measurement network's validation function receives nine hundred requests a minute; variation by time of day is not modeled. Both the network and the numbers are fictional. - **FC26.** On a warm instance processing takes two hundred twenty milliseconds; cold start adds one thousand eight hundred milliseconds. Package size's effect is outside this lesson. - **FC27.** An idle instance is dropped four minutes later. **The pool is separate per version**: even though the alias moves, the two versions' pools do not merge. - **FC28.** The gate reads a single figure: the new version's average latency. The baseline is measured while weight is zero, and the threshold is one point two five times the baseline. - **FC29.** Requests arrive at exponential intervals; the generator is self-written, and the seed is visible. Every weight step contains five two-minute rounds. - **FC30.** The alias write, propagation, and republish durations are fictional. #### The Version Stays Fixed, the Alias Moves What is measured is not a real runtime but a model built with `node`. There are two objects: a **version**, a frozen record whose digest never changes; an **alias**, a pointer that points at one of these versions and can be overwritten. A separate warm pool is kept per version. ```js // alias.mjs — version, alias, and warm pool model; not a real runtime, a model export const SEED = 20260420, REQ_MIN = 900; // fictional: requests per minute to the validation function export const SERVICE_MS = 220, COLD_EXTRA_MS = 1800, IDLE_MS = 240000; // warm, cold extra, idle lifetime export const ROUND_MIN = 2, ROUNDS = 5, GATE = 1.25, REGRESSION_MS = 120; export const WEIGHTS = [1, 2, 5, 25, 100]; export function rng(seed) { // linear congruential generator, seed is visible let x = seed >>> 0; return () => { x = (Math.imul(x, 1664525) + 1013904223) >>> 0; return x / 4294967296; }; } // A version is frozen: its digest never changes again. An alias is a pointer that can be overwritten. export const version = (label, digest) => Object.freeze({ label, digest }); export const alias = (name, target) => ({ name, target, writes: 0 }); export const move = (a, target) => Object.assign(a, { target, writes: a.writes + 1 }); // The pool is per version: an idle instance runs warm if found, otherwise a new instance is opened. export const pool = () => ({ instances: [], cold: 0 }); export const alive = (p, t) => p.instances.filter((o) => o.last + IDLE_MS > t).length; export function call(p, t, extraMs) { p.instances = p.instances.filter((o) => o.last + IDLE_MS > t); const o = p.instances.find((x) => x.busy <= t), duration = SERVICE_MS + extraMs + (o ? 0 : COLD_EXTRA_MS); if (o) { o.busy = t + duration; o.last = t + duration; } else { p.cold += 1; p.instances.push({ busy: t + duration, last: t + duration }); } return duration; } // A step: requests arrive at exponential intervals, and are routed to the old or new version by weight. export function step(rnd, poolOld, poolNew, weight, extraMs, t0, rounds = ROUNDS, roundMin = ROUND_MIN) { const lam = REQ_MIN / 60000, roundData = []; let t = t0 - Math.log(1 - rnd()) / lam; for (let k = 0; k < rounds; k += 1) { const end = t0 + (k + 1) * roundMin * 60000; let nNew = 0, nOld = 0, sumNew = 0, sumOld = 0, coldStart = poolNew.cold; while (t < end) { if (rnd() * 100 < weight) { sumNew += call(poolNew, t, extraMs); nNew += 1; } else { sumOld += call(poolOld, t, 0); nOld += 1; } t -= Math.log(1 - rnd()) / lam; } roundData.push({ nNew, avgNew: nNew ? sumNew / nNew : 0, avgOld: nOld ? sumOld / nOld : 0, cold: poolNew.cold - coldStart }); } return roundData; } // Ladder: baseline is read first while weight is zero, then weight grows. Pools carry over between // steps: the new version's pool is empty only on the first step. export function ladder(seed, extraMs, weights = WEIGHTS, warm = 0) { const rnd = rng(seed), poolOld = pool(), poolNew = pool(); const baseline = step(rnd, poolOld, poolNew, 0, 0, 0, ROUNDS).reduce((x, y) => x + y.avgOld, 0) / ROUNDS; const rows = []; let t = ROUNDS * ROUND_MIN * 60000, affected = 0; for (let i = 0; i < warm; i += 1) call(poolNew, t - 30000, 0); // synchronous synthetic warm-up calls for (const w of weights) { const roundData = step(rnd, poolOld, poolNew, w, extraMs, t, ROUNDS); t += ROUNDS * ROUND_MIN * 60000; affected += roundData.reduce((x, y) => x + y.nNew, 0); rows.push({ weight: w, roundData, affected }); } return { baseline, rows }; } export const sign = (x) => `${x < 0 ? '' : '+'}${Math.round(x)}`; export const row = (g, ...x) => x.map((v, i) => String(v).padEnd(g[i])).join(''); ``` The handed-off decision shows up here: which request goes to which version, how many instances open, is not written down — a single number is written in its place. The constraint sits in the same spot: because the pool is separate per version, the new version does its warming from zero. #### Two Numbers at Every Step ```js // run1.mjs — alias weight is shifted step by step; cold calls, latency, and the gate are read at each step import { ladder, version, alias, move, sign, row, SEED, REQ_MIN, SERVICE_MS, COLD_EXTRA_MS, GATE, REGRESSION_MS, WEIGHTS } from './alias.mjs'; const v7 = version('v7', 'e3bb1237'), v8 = version('v8', 'd8bb00e6'), live = alias('live', v7); try { v7.digest = 'overwritten'; } catch { /* writing to a frozen record is rejected */ } console.log(`seed ${SEED} | ${REQ_MIN} req/min | warm ${SERVICE_MS} ms | cold extra ${COLD_EXTRA_MS} ms`); console.log(`alias "${live.name}" -> ${live.target.label}, moved -> ${move(live, v8).target.label}` + ` | writes ${live.writes} | digest ${v7.digest} (could not be overwritten)`); const G = [9, 11, 13, 12, 12, 13, 13, 13]; for (const [label, extra, show] of [['clean version', 0, WEIGHTS], [`flawed version (+${REGRESSION_MS} ms real regression)`, REGRESSION_MS, [1]]]) { const { baseline, rows } = ladder(SEED, extra); console.log(`\n${label} | ladder ${WEIGHTS.join('-')} | baseline ${Math.round(baseline)} ms | gate threshold ${Math.round(baseline * GATE)} ms`); console.log(row(G, 'weight', 'req/round', 'round1 cold', 'round1 avg', 'round2 avg', 'round1 diff', 'round2 diff', 'first clean') + 'affected'); for (const s of rows.filter((x) => show.includes(x.weight))) { const [a, b] = s.roundData, t = s.roundData.findIndex((x) => x.avgNew <= baseline * GATE); console.log(row(G, `${s.weight}%`, a.nNew, a.cold, Math.round(a.avgNew), Math.round(b.avgNew), sign(a.avgNew - baseline), sign(b.avgNew - baseline), t < 0 ? 'none' : `round ${t + 1}`) + s.affected); } } ``` ``` seed 20260420 | 900 req/min | warm 220 ms | cold extra 1800 ms alias "live" -> v7, moved -> v8 | writes 1 | digest e3bb1237 (could not be overwritten) clean version | ladder 1-2-5-25-100 | baseline 226 ms | gate threshold 283 ms weight req/round round1 cold round1 avg round2 avg round1 diff round2 diff first clean affected 1% 21 2 391 220 +165 -6 round 2 85 2% 38 1 267 220 +41 -6 round 1 266 5% 99 1 238 220 +12 -6 round 1 717 25% 468 2 228 220 +1 -6 round 1 2968 100% 1823 7 227 220 +1 -6 round 1 11969 flawed version (+120 ms real regression) | ladder 1-2-5-25-100 | baseline 226 ms | gate threshold 283 ms weight req/round round1 cold round1 avg round2 avg round1 diff round2 diff first clean affected 1% 21 2 511 340 +285 +114 none 85 ``` The second line confirms the lesson's title: the alias moved from `v7` to `v8` — one write — and at the same moment, the attempt to write to `v7`'s digest was rejected. Release is not changing records; it is updating a single field that can be overwritten. The affected requests column writes the gain from shifting: eighty-five at the end of the 1% step, eleven thousand nine hundred sixty-nine once it reaches 100% — a difference close to one hundred forty times. The round-where-it-clears column, though, breaks down in an unexpected place. In the clean version, the **first round of the 1% step trips the gate**: the average is three hundred ninety-one milliseconds, the threshold two hundred eighty-three. The same version drops to two hundred twenty in the second round; in the flawed version the difference stays around one hundred fourteen milliseconds in every round. What distinguishes the two is not the size of the difference but **whether it dies down**. The reason is that the pool is separate per version: the new version enters the first step with an empty pool, opens two instances, and those instances stay warm for the rest of the ladder. The cold start cost falls only on the first step — by definition the step with the smallest sample. The arithmetic is one line: the gate's margin is fifty-seven milliseconds, the 1% share carries eighteen requests per round, and a single cold call of one thousand eight hundred milliseconds adds one hundred milliseconds to the average; fifty at 2%, twenty at 5%. A small share both lowers the affected requests and makes the measurement sensitive to cold start — the first was wanted, the second arrived as a constraint. #### Rollback Is a Single Key Turn ```js // run2.mjs — two rollback paths, the old pool after a full cutover, the cost of the workaround paths import { ladder, rng, pool, step, alive, sign, row, SEED, REQ_MIN, IDLE_MS, ROUND_MIN, GATE, WEIGHTS } from './alias.mjs'; // Fictional durations: write 120, propagation 900, package load (42 MB / 20 MB-s) 2100, version 3000 ms. const PATHS = { 'alias flip': [120, 900], redeploy: [2100, 3000, 120, 900] }; console.log(row([20, 7, 11], 'rollback path', 'steps', 'duration') + 'requests hitting the flawed version in this time'); for (const [name, a] of Object.entries(PATHS)) { const ms = a.reduce((x, y) => x + y, 0); console.log(row([20, 7, 11], name, a.length, `${(ms / 1000).toFixed(2)} s`) + Math.round(REQ_MIN / 60000 * ms)); } // After a full cutover the old version's pool empties; rolling back has to reopen that pool. function cutoverAndRollback(waitMin) { const rnd = rng(SEED), poolOld = pool(), poolNew = pool(); step(rnd, poolOld, poolNew, 0, 0, 0, 5, 1); // old version warms up const moved = step(rnd, poolOld, poolNew, 100, 0, 300000, waitMin, 1).reduce((x, y) => x + y.nNew, 0); const t = 300000 + waitMin * 60000, remaining = alive(poolOld, t), coldStart = poolOld.cold; const rollback = step(rnd, poolOld, poolNew, 0, 0, t, 1, 1)[0]; // alias rolled back: one minute is read return { moved, remaining, cold: poolOld.cold - coldStart, avg: Math.round(rollback.avgOld) }; } const H = [9, 25, 29, 19]; console.log(` rollback after a full cutover (100%) — an idle instance is dropped after ${IDLE_MS / 60000} min`); console.log(row(H, 'wait', 'moved to new version', 'instances left in old pool', 'cold on rollback') + 'first-min avg'); for (const b of [3, 4, 5]) { const g = cutoverAndRollback(b); console.log(row(H, `${b} min`, g.moved, g.remaining, g.cold) + `${g.avg} ms`); } // Ways to escape cold start on the first step; each path's cost is a separate line item. const margin = ladder(SEED, 0).baseline * (GATE - 1), Y = [32, 16, 16, 13, 11, 12, 13]; console.log(` ways to escape cold start on the first step — gate margin ${Math.round(margin)} ms`); console.log(row(Y, 'path', 'ladder', 'decision round', 'publish min', 'synthetic', 'in round1', 'round1 cold') + 'diff at decision round'); for (const [label, weights, warm, decisionRound] of [['skip round1, decide on round2', WEIGHTS, 0, 2], ['1 synchronous synthetic call', WEIGHTS, 1, 1], ['3 synchronous synthetic calls', WEIGHTS, 3, 1], ['make the first step 25%', [25, 100], 0, 1]]) { const { baseline, rows } = ladder(SEED, 0, weights, warm), round1 = rows[0].roundData[0]; console.log(row(Y, label, weights.join('-'), decisionRound, weights.length * decisionRound * ROUND_MIN, warm, round1.nNew, round1.cold) + `${sign(rows[0].roundData[decisionRound - 1].avgNew - baseline)} ms`); } ``` ``` rollback path steps duration requests hitting the flawed version in this time alias flip 2 1.02 s 15 redeploy 4 6.12 s 92 rollback after a full cutover (100%) — an idle instance is dropped after 4 min wait moved to new version instances left in old pool cold on rollback first-min avg 3 min 2680 12 0 220 ms 4 min 3584 4 14 247 ms 5 min 4490 0 39 296 ms ways to escape cold start on the first step — gate margin 57 ms path ladder decision round publish min synthetic in round1 round1 cold diff at decision round skip round1, decide on round2 1-2-5-25-100 2 20 0 21 2 -6 ms 1 synchronous synthetic call 1-2-5-25-100 1 10 1 21 1 +79 ms 3 synchronous synthetic calls 1-2-5-25-100 1 10 3 21 0 -6 ms make the first step 25% 25-100 1 4 0 477 5 +13 ms ``` The first table shows what the version being unchangeable buys. Because the old version stands as a record, rollback is two steps — one write and propagation — one point zero two seconds in total, and fifteen requests hitting the flawed version. If the version were not kept, the path would climb to four steps, six point one two seconds, and ninety-two requests; a sixfold difference. The second table writes the limit of this comfort. Once weight is raised to 100%, the old version receives no requests at all, and its pool empties in four minutes. Rolled back three minutes later, twelve instances were still standing in the pool, and no cold call occurred; five minutes later zero instances remain, and the first minute pays thirty-nine cold calls, pushing the average up to two hundred ninety-six. Rollback is still a single write, but its cost comes not from the operation but from the time that has passed. The third table writes the bill for the workaround. Discarding the first round clears the decision, but stretches the publish from ten minutes to twenty. Pre-warming keeps the duration, but a single synthetic call is not enough: the pile-up inside the round opens a second instance, and the difference stays at seventy-nine milliseconds. Three synchronous calls zero out the cold count; their cost is fifteen synthetic calls per ladder — requests that mix into the metric and never come from a real user. Starting the ladder from the top is the most expensive: the difference drops to thirteen milliseconds, but the affected requests in the first round climb from twenty-one to four hundred seventy-seven. #### Summary - The alias moved from `v7` to `v8`: a single write. At the same moment, the attempt to write to the version's digest was rejected — release is not changing records, it is updating a pointer. - Affected requests are eighty-five at the end of the 1% step, eleven thousand nine hundred sixty-nine once it reaches 100%. - A clean version tripped the gate in the first round of the 1% step: two cold calls, three hundred ninety-one milliseconds, threshold two hundred eighty-three. It dropped to two hundred twenty in the second round; in the flawed version the difference never died down in any round. - At the 1% share, eighteen requests fall to a round and a single cold call adds one hundred milliseconds to the average; the gate's margin is fifty-seven. - Rollback is two steps and one point zero two seconds; if the version were not kept, it would be four steps, six point one two seconds, and ninety-two requests. - Five minutes after a full cutover, zero instances remain in the old pool; rollback's first minute pays thirty-nine cold calls and the average climbs to two hundred ninety-six. #### Next Step The same number stood still like a constant everywhere in the measurement: the eighteen hundred milliseconds cold start adds. It was what broke the canary round, wrote rollback's delayed cost, and made pre-warming necessary. But that number is not constant — the bigger the package, the more bytes there are to download. The next lesson measures the package: it scans the relationship between size and cold start, counts what moving dependencies into a shared layer shrinks and what it does not, works out how many functions version pinning and a layer update touch — and writes the cost of an unused dependency as bytes moved. ### Shared Layers and Dependencies Source: https://academia.sh/en/courses/serverless/shared-layers-and-dependencies The relationship between package size and cold start is scanned: when dependencies move to a shared layer, the package shrinks but the bytes pulled do not change. Version pinning's storage cost, a layer update's radius, and an unused dependency's cost in bytes moved are counted. In the version-shifting measurement, cold start behaved like a constant of eighteen hundred milliseconds. It is not. That number comes from the function's package: the bigger the package, the more bytes there are to pull, unpack, and load. The question here is this — what are the ways to shrink the package, and what does each way bill? The **layer** here shares the same word as a container image's layer but is a different object: there, the layer was a file system snapshot; here it is a common set of dependencies added to the function's package at publish time. - **FC31.** The fictional regional measurement network has nine functions and a dependency catalog of seven items. Sizes, module counts, and usage relationships are fictional. - **FC32.** Cold start is the sum of four items: a size-independent baseline of one hundred eighty milliseconds, pulling at twenty megabytes per second, unpacking at twelve milliseconds per megabyte, and loading at zero point thirty-five milliseconds per module. Network variability and compression are not modeled. - **FC33.** A dependency moves to the shared layer if it appears in at least four functions. The threshold itself is a design decision and is not scanned. - **FC34.** The layer is pulled together with the package on cold start. Host caching is modeled with a single hit rate; on a hit the layer's cost is zero. - **FC35.** Every function experiences forty cold starts a day. Traffic's distribution across the day is not modeled. - **FC36.** The distribution of live layer versions is fictional: four versions, pinned to three, two, three, and one functions respectively. #### What a Cold Start Is the Sum Of What is measured is not a real deployment but a model built with `node`. The catalog holds each dependency's size and module count; each of the nine functions lists the dependencies it actually calls. One item does not appear in any list. ```js // package.mjs — package size, cold start, and shared layer model; not a measurement, a model export const BANDWIDTH_MB_S = 20; // fictional: bandwidth for pulling the package export const UNPACK_MS_MB = 12; // time to unpack one MB export const MODULE_MS = 0.35; // load time per module export const BASE_MS = 180; // runtime startup baseline, independent of size export const COLD_PER_DAY = 40; // fictional: daily cold starts per function export const LAYER_THRESHOLD = 4; // a dependency used in this many functions moves to the shared layer // The fictional measurement network's dependency catalog: size (MB) and module count. export const CATALOG = { mp: { name: 'measurement parser', mb: 9.4, module: 320 }, sv: { name: 'schema validator', mb: 4.8, module: 210 }, da: { name: 'data access layer', mb: 6.3, module: 240 }, dt: { name: 'date-time library', mb: 2.1, module: 90 }, eh: { name: 'encryption helpers', mb: 1.6, module: 60 }, rf: { name: 'report formatter', mb: 14.2, module: 540 }, ig: { name: 'image generator', mb: 21.5, module: 760 }, }; export const OWN_CODE = { mb: 1.2, module: 45 }; // each function's own code // Nine functions and the dependencies they actually call. 'ig' does not appear in any function. export const FUNCTIONS = { 'reading-validation': ['mp', 'sv', 'da', 'dt', 'eh'], 'reading-writing': ['mp', 'da', 'dt', 'eh'], 'billing-generation': ['mp', 'sv', 'da', 'dt', 'eh', 'rf'], 'billing-delivery': ['da', 'dt', 'eh', 'rf'], 'field-notification': ['da', 'dt', 'eh'], 'nightly-batch-job': ['mp', 'sv', 'da', 'dt', 'eh'], 'meter-registration': ['sv', 'da', 'dt', 'eh'], 'tariff-update': ['sv', 'da', 'eh'], 'health-endpoint': ['da'], }; export const usedBy = (k) => Object.values(FUNCTIONS).filter((d) => d.includes(k)).length; export const size = (ks) => ks.reduce((a, k) => ({ mb: a.mb + CATALOG[k].mb, module: a.module + CATALOG[k].module }), { ...OWN_CODE }); // Cold start: fixed base + pull + unpack + module load. Bytes are both pulled and unpacked. export const cold = (b) => BASE_MS + (1000 * b.mb) / BANDWIDTH_MB_S + UNPACK_MS_MB * b.mb + MODULE_MS * b.module; export const row = (g, ...x) => x.map((v, i) => String(v).padEnd(g[i])).join(''); export const mb = (x) => `${x.toFixed(1)} MB`; export const extra = (b) => cold(b) - BASE_MS; // pull + unpack + load time with the base subtracted ``` ```js // run1.mjs — the relationship between package size and cold start, and four packaging arrangements import { CATALOG, FUNCTIONS, OWN_CODE, LAYER_THRESHOLD, COLD_PER_DAY, BANDWIDTH_MB_S, UNPACK_MS_MB, MODULE_MS, usedBy, size, cold, row, mb } from './package.mjs'; const all = Object.keys(CATALOG), n = Object.keys(FUNCTIONS).length; console.log(`${n} functions | pull ${BANDWIDTH_MB_S} MB/s | unpack ${UNPACK_MS_MB} ms/MB | module ${MODULE_MS} ms`); console.log(`usage count: ${all.map((k) => `${k} ${usedBy(k)}`).join(', ')} | layer threshold ${LAYER_THRESHOLD}`); console.log('\npackage size -> cold start (module count is assumed proportional to size, 36 modules/MB)'); console.log(row([10, 9, 12, 11, 11], 'size', 'modules', 'pull ms', 'unpack ms', 'load ms') + 'cold start'); for (const x of [2, 8, 16, 26, 40, 61]) { const b = { mb: x, module: Math.round(36 * x) }; console.log(row([10, 9, 12, 11, 11], mb(x), b.module, Math.round(1000 * x / BANDWIDTH_MB_S), Math.round(UNPACK_MS_MB * x), Math.round(MODULE_MS * b.module)) + `${Math.round(cold(b))} ms`); } // Four packaging arrangements. C and D carry the same content: the only difference is where the bytes sit. const layerCode = all.filter((k) => usedBy(k) >= LAYER_THRESHOLD), layer = size(layerCode); layer.mb -= OWN_CODE.mb; layer.module -= OWN_CODE.module; // the layer holds no function's own code const arrangement = (name, packer, layered = false) => { const pkgs = Object.values(FUNCTIONS).map((d) => size(packer(d))); const pulled = pkgs.map((x) => (layered ? { mb: x.mb + layer.mb, module: x.module + layer.module } : x)); const c = pulled.map(cold), sizes = pkgs.map((x) => x.mb); return { name, smallest: Math.min(...sizes), largest: Math.max(...sizes), stored: sizes.reduce((a, b) => a + b, 0) + (layered ? layer.mb : 0), coldMin: Math.min(...c), coldMax: Math.max(...c) }; }; const D = [54, 18, 17, 12]; console.log('\npackaging arrangements — nine functions'); console.log(row(D, 'arrangement', 'smallest package', 'largest package', 'stored') + 'cold start range'); for (const r of [ arrangement('A. every function packages the whole catalog', () => all), arrangement('B. entirely unused items are stripped out', () => all.filter((k) => usedBy(k) > 0)), arrangement('C. every function packages only what it uses', (d) => d), arrangement('D. dependencies past the threshold move to the layer', (d) => d.filter((k) => usedBy(k) < LAYER_THRESHOLD), true)]) { console.log(row(D, r.name, mb(r.smallest), mb(r.largest), mb(r.stored)) + `${Math.round(r.coldMin)}-${Math.round(r.coldMax)} ms`); } const ig = CATALOG.ig, singleCold = 1000 * ig.mb / BANDWIDTH_MB_S + UNPACK_MS_MB * ig.mb + MODULE_MS * ig.module; console.log(`\nif the entirely unused "${ig.name}" (${mb(ig.mb)}, ${ig.module} modules) stays in arrangement A:`); console.log(` ${Math.round(singleCold)} ms on every cold start, ${(n * COLD_PER_DAY * singleCold / 1000 / 60).toFixed(1)} min ` + `and ${(n * COLD_PER_DAY * ig.mb / 1024).toFixed(1)} GB of bytes moved across ${n * COLD_PER_DAY} daily cold starts`); console.log(` ${mb(n * ig.mb)} in storage; stripping it shrinks the package by ${mb(ig.mb)} and requires no code change`); ``` ``` 9 functions | pull 20 MB/s | unpack 12 ms/MB | module 0.35 ms usage count: mp 4, sv 5, da 9, dt 7, eh 8, rf 2, ig 0 | layer threshold 4 package size -> cold start (module count is assumed proportional to size, 36 modules/MB) size modules pull ms unpack ms load ms cold start 2.0 MB 72 100 24 25 329 ms 8.0 MB 288 400 96 101 777 ms 16.0 MB 576 800 192 202 1374 ms 26.0 MB 936 1300 312 328 2120 ms 40.0 MB 1440 2000 480 504 3164 ms 61.0 MB 2196 3050 732 769 4731 ms packaging arrangements — nine functions arrangement smallest package largest package stored cold start range A. every function packages the whole catalog 61.1 MB 61.1 MB 549.9 MB 4761-4761 ms B. entirely unused items are stripped out 39.6 MB 39.6 MB 356.4 MB 3162-3162 ms C. every function packages only what it uses 7.5 MB 39.6 MB 185.0 MB 745-3162 ms D. dependencies past the threshold move to the layer 1.2 MB 15.4 MB 63.4 MB 2093-3162 ms if the entirely unused "image generator" (21.5 MB, 760 modules) stays in arrangement A: 1599 ms on every cold start, 9.6 min and 7.6 GB of bytes moved across 360 daily cold starts 193.5 MB in storage; stripping it shrinks the package by 21.5 MB and requires no code change ``` The first scan separates out where cold start comes from. Going from two megabytes to sixty-one, the duration climbs from three hundred twenty-nine milliseconds to four thousand seven hundred thirty-one; the entire difference in between is due to size. The load per megabyte is seventy-four point six milliseconds: fifty pull, twelve unpack, twelve point six module load. The fixed baseline stays at one hundred eighty — in a thirty-megabyte package, more than 90% of cold start is the bytes themselves. #### Four Packaging Arrangements The second table builds the same nine functions with four different packaging arrangements. In arrangement A every function carries the whole catalog; in B the entirely unused item is stripped out; in C every function packages only what it calls; in D the dependencies past the threshold move to the shared layer. Going from A to B is a free gain: the code does not change, the package drops from sixty-one point one to thirty-nine point six, and cold start falls from four thousand seven hundred sixty-one to three thousand one hundred sixty-two. C, by giving each function its own list, brings the smallest package down to seven point five and opens the cold start range to between seven hundred forty-five and three thousand one hundred sixty-two. Row D, though, writes the lesson's real number. Package size falls to as low as one point two megabytes — one-sixth of C's — but **cold start's lower end climbs from seven hundred forty-five to two thousand ninety-three**. The reason is simple: the twenty-four point two megabytes in the layer did not vanish, they were only moved outside the package; they are still pulled on cold start. What is more, functions that do not use everything in the layer now pull bytes they do not use as well. Taking package size as the measure is misleading here; what should be measured is **bytes pulled**. The last rows write the bill for the entirely unused item. The twenty-one point five megabyte image generator adds one thousand five hundred ninety-nine milliseconds to every cold start; across three hundred sixty daily cold starts that comes to nine point six minutes and seven point six gigabytes of bytes moved. #### The Layer Does Not Destroy Bytes, It Relocates Them ```js // run2.mjs — layer cache hit rate, version pinning, and the radius of a layer update import { CATALOG, FUNCTIONS, OWN_CODE, LAYER_THRESHOLD, BASE_MS, usedBy, size, cold, extra, row, mb } from './package.mjs'; const all = Object.keys(CATALOG), n = Object.keys(FUNCTIONS).length; const layerCode = all.filter((k) => usedBy(k) >= LAYER_THRESHOLD), layer = size(layerCode); layer.mb -= OWN_CODE.mb; layer.module -= OWN_CODE.module; const pkgC = Object.values(FUNCTIONS).map((d) => size(d)); const pkgD = Object.values(FUNCTIONS).map((d) => size(d.filter((k) => usedBy(k) < LAYER_THRESHOLD))); console.log(`shared layer: ${layerCode.join(', ')} = ${mb(layer.mb)}, ${layer.module} modules`); // If the layer is cached on the host, it is not pulled again on cold start. Hit rate is scanned. const O = [10, 21, 20, 18]; console.log(row(O, 'hit rate', 'D smallest function', 'D largest function', 'D average') + 'C average'); const avgC = pkgC.map(cold).reduce((a, b) => a + b, 0) / n; for (const h of [0, 0.25, 0.5, 0.75, 1]) { const times = pkgD.map((p) => BASE_MS + extra(p) + (1 - h) * extra(layer)); console.log(row(O, `${h * 100}%`, `${Math.round(Math.min(...times))} ms`, `${Math.round(Math.max(...times))} ms`, `${Math.round(times.reduce((a, b) => a + b, 0) / n)} ms`) + `${Math.round(avgC)} ms`); } // The layer is versioned, and each function is pinned to one version. Fictional split: how many functions on which version. const PINNING = { v4: 3, v3: 2, v2: 3, v1: 1 }; const live = Object.keys(PINNING).length; console.log(`\nversion pinning — ${live} layer versions kept live`); console.log(` ${Object.entries(PINNING).map(([s, k]) => `${s}: ${k} functions`).join(' | ')}`); console.log(` total layer storage ${mb(live * layer.mb)} (single-version assumption ${mb(layer.mb)}); ` + `dropping to one version requires republishing ${n - Math.max(...Object.values(PINNING))} functions`); // A dependency update's two costs: how many publishes it takes and how many functions it hits at once. const Y = [40, 11, 22]; console.log('\na dependency update'); console.log(row(Y, 'where the dependency sits', 'publishes', 'affected at once') + 'stored'); for (const [label, publishes, radius, stored] of [ ['C — separately inside each package', n, 1, pkgC.reduce((a, b) => a + b.mb, 0)], ['D — in the layer, version pinned', n, 1, pkgD.reduce((a, b) => a + b.mb, 0) + live * layer.mb], ['D — in the layer, following an alias', 1, n, pkgD.reduce((a, b) => a + b.mb, 0) + layer.mb]]) { console.log(row(Y, label, publishes, `${radius} functions`) + mb(stored)); } ``` ``` shared layer: mp, sv, da, dt, eh = 24.2 MB, 920 modules hit rate D smallest function D largest function D average C average 0% 2093 ms 3162 ms 2330 ms 1730 ms 25% 1637 ms 2706 ms 1875 ms 1730 ms 50% 1181 ms 2251 ms 1419 ms 1730 ms 75% 726 ms 1795 ms 963 ms 1730 ms 100% 270 ms 1340 ms 508 ms 1730 ms version pinning — 4 layer versions kept live v4: 3 functions | v3: 2 functions | v2: 3 functions | v1: 1 functions total layer storage 96.8 MB (single-version assumption 24.2 MB); dropping to one version requires republishing 6 functions a dependency update where the dependency sits publishes affected at once stored C — separately inside each package 9 1 functions 185.0 MB D — in the layer, version pinned 9 1 functions 136.0 MB D — in the layer, following an alias 1 9 functions 63.4 MB ``` The first table shows when the layer pays off. At zero cache hit rate, arrangement D's average is two thousand three hundred thirty, C's average is one thousand seven hundred thirty milliseconds — the layer **makes the average six hundred milliseconds worse**. At 50% hit rate D drops to one thousand four hundred nineteen and passes C; the break-even point sits around 33%, and at a full hit D falls to five hundred eight. The layer's value is not in the layer itself but in **how often the host holds onto it** — and that is not a decision the function's author makes. #### Version Pinning and the Radius of an Update The version pinning table writes the second cost. Nine functions are pinned to four separate layer versions, and none can be dropped, because each has users. Stored layer bytes climb from twenty-four point two to ninety-six point eight; pulling all of them onto a single version requires republishing six functions. The last table cuts the bill for the handed-off decision. While the dependency sits separately inside each package, an update needs nine publishes, but each publish affects a single function: the radius is one, and it can roll out gradually. Once it moves to the layer and the version is pinned, the number of publishes is **still nine** — pinning takes back the layer's "update from one place" promise — the gain is only in storage, from one hundred eighty-five to one hundred thirty-six megabytes. Once pinning is dropped and the layer follows an alias, publishes fall to one; in return the radius climbs to nine. A single write changes nine functions at once, and one flawed layer version breaks all nine together. #### Summary - Cold start's load per megabyte is seventy-four point six milliseconds: fifty pull, twelve unpack, twelve point six module load. - Stripping the entirely unused item drops the package to thirty-nine point six megabytes and cold start to three thousand one hundred sixty-two milliseconds, with no code change required; bytes moved shrink by seven point six gigabytes a day. - Moving dependencies to the layer pulls the package down to one point two megabytes but pushes cold start's lower end from seven hundred forty-five to two thousand ninety-three milliseconds: the bytes did not vanish, they changed location. - The layer only pays off once the host cache hits above 33%; at zero hit rate it makes the average six hundred milliseconds worse. - With four layer versions kept live, stored layer bytes quadruple; dropping to one version demands republishing six functions. - An update means nine publishes and a one-function radius inside the package, still nine publishes in a pinned layer, and a single publish with a nine-function radius in a layer that follows an alias. #### Next Step The package was measured, the layer was measured, and which item of cold start comes from where was written down. All of this shared a single assumption: the job fits the function. It fits inside the duration limit, its state can be kept outside, it starts when a request arrives and ends with a response. Not every job is like this. The next lesson places five job types side by side on the axes of duration, state, and cost; for each type it writes the item that exceeds the threshold by name, and counts the cost of splitting an unfit job to make it fit — step count, intermediate state storage, failure surface. In the end a single table remains: which job fits serverless, which does not, and why. ### Cases Where Serverless Is Not Suitable Source: https://academia.sh/en/courses/serverless/cases-where-serverless-is-not-suitable Five job types are compared on the axes of duration, state, and cost: for each type the item that exceeds the threshold is written by name, and the cost of splitting an unfit job to make it fit is counted as steps, interim state, and failure points. The topic closes with a table showing which job fits serverless, which does not, and why. Everything measured across this topic shared the same assumption: the job fits the function. A request arrives, finishes inside the limits, keeps its state outside. Thresholds were counted, the package was shrunk, release was tied to a gate — all of it inside that assumption. Now the assumption itself is measured: which job fits inside these limits, which does not, and what is the bill for forcing an unfit job to fit? The question is not "is serverless good or bad"; it is which job type can afford the constraints that come back in exchange for renting, and which cannot. The answer is not an opinion but a number read on three axes: duration, state, cost. - **FC37.** The fictional regional measurement network's five job types are measured. Both the workload numbers and the thresholds are fictional and represent a single runtime. - **FC38.** The thresholds are fixed: a single run is nine hundred seconds, memory is three thousand seventy-two megabytes. A connection cannot outlive the request, so the same nine hundred seconds is the connection lifetime. - **FC39.** Cost is written in **fictional units**; no real currency is used, and only ratios are meaningful. Serverless is priced per call and per gigabyte-second, the rented machine per hour. - **FC40.** A rented machine carries forty concurrent jobs or two thousand persistent connections. Redundancy, load balancing, and operating labor do not enter the cost — meaning the rented machine side is **counted short**. - **FC41.** When splitting, interim state is written and read at every boundary: eighteen plus twelve seconds and two hundred forty megabytes. The full limit cannot be used; the usable share is a scanned setting. - **FC42.** A failure point is the number of places that can fail and be retried: step count plus the number of transitions between steps. The compensation path after partial success is not modeled. #### Five Job Types, Three Axes ```js // jobs.mjs — five job types compared on duration, state, and cost; not a measurement, a model export const DURATION_LIMIT = 900; // fictional upper limit, seconds export const MEMORY_CAP = 3072; // fictional memory cap, MB export const CONNECTION_LIFETIME = 900; // a connection cannot outlive the request: same limit export const MONTH_S = 2592000; // a thirty-day month, seconds // Fictional units — not real currency, only ratios are meaningful. export const CALL = 0.0000002; // unit per call export const GB_S = 0.000017; // unit per GB-second export const MACHINE_HOUR = 0.096; // continuously running rented machine, unit per hour export const MACHINE_CONCURRENCY = 40; // concurrent work a machine carries export const MACHINE_CONNECTIONS = 2000; // Five job types. `month` is the monthly call count; for continuous services it is MONTH_S / s x concurrency. export const JOBS = { 'short and sparse request': { s: 0.12, month: 40000, mb: 256, concurrency: 1, connections: 0, state: 'none' }, 'nightly batch job': { s: 2700, month: 30, mb: 1024, concurrency: 1, connections: 0, state: 'temp file' }, 'long-running conversion': { s: 4200, month: 60, mb: 3072, concurrency: 1, connections: 0, state: 'in-memory interim result' }, 'persistent-connection service': { s: 1500, month: 1382400, mb: 512, concurrency: 20, connections: 800, state: 'session' }, 'high and continuous load': { s: 0.09, month: 1036800000, mb: 512, concurrency: 36, connections: 0, state: 'none' }, }; export const serverlessMonth = (i) => i.month * CALL + (i.month * i.s * i.mb) / 1024 * GB_S; export const machineMonth = (i) => Math.max(1, Math.ceil(i.concurrency / MACHINE_CONCURRENCY), Math.ceil(i.connections / MACHINE_CONNECTIONS)) * 730 * MACHINE_HOUR; // The item that exceeds a threshold: which limit multiplies first. Order matters, the first factor is written. export function exceeded(i) { const items = []; if (i.connections > 0 && i.s > CONNECTION_LIFETIME) items.push('connection lifetime'); else if (i.s > DURATION_LIMIT) items.push('duration limit'); if (i.mb > MEMORY_CAP) items.push('memory cap'); if (i.state === 'in-memory interim result' || i.state === 'session') items.push('state'); if (serverlessMonth(i) > machineMonth(i)) items.push('unit cost'); return items; } export const row = (g, ...x) => x.map((v, i) => String(v).padEnd(g[i])).join(''); export const unit = (x) => (x >= 100 ? x.toFixed(0) : x >= 1 ? x.toFixed(2) : x.toFixed(4)); // The line items of splitting to make a job fit (fictional): interim state is written and read at every boundary. export const INTERIM_WRITE = 18, INTERIM_READ = 12, INTERIM_MB = 240; export function split(s, share) { const usable = DURATION_LIMIT * share - INTERIM_WRITE - INTERIM_READ; if (usable <= 0) return null; const steps = Math.ceil(s / usable); const extra = (steps - 1) * (INTERIM_WRITE + INTERIM_READ); return { steps, extra, total: s + extra, interimMb: (steps - 1) * INTERIM_MB, failurePoints: 2 * steps - 1, pieceS: s / steps + INTERIM_WRITE + INTERIM_READ }; } ``` ```js // run1.mjs — five job types are placed side by side on the axes of duration, state, and cost import { JOBS, DURATION_LIMIT, MEMORY_CAP, CONNECTION_LIFETIME, MACHINE_CONCURRENCY, serverlessMonth, machineMonth, exceeded, row, unit } from './jobs.mjs'; console.log(`thresholds: duration ${DURATION_LIMIT} s | memory ${MEMORY_CAP} MB | connection lifetime ${CONNECTION_LIFETIME} s`); console.log(`fictional units: a machine carries ${MACHINE_CONCURRENCY} concurrent jobs; the numbers below are units, not money\n`); const A = [32, 13, 16, 16, 10, 26]; console.log(row(A, 'job type', 'single run', 'limit multiple', 'monthly calls', 'memory', 'state') + 'concurrent'); for (const [name, i] of Object.entries(JOBS)) { console.log(row(A, name, `${i.s} s`, `${(i.s / DURATION_LIMIT).toFixed(2)}x`, i.month.toLocaleString('en-US'), `${i.mb} MB`, i.state) + i.concurrency); } const B = [32, 14, 16, 9, 26]; console.log('\nmonthly cost and the item that exceeds the threshold'); console.log(row(B, 'job type', 'serverless', 'rented machine', 'ratio') + 'exceeded threshold'); for (const [name, i] of Object.entries(JOBS)) { const s = serverlessMonth(i), m = machineMonth(i), k = exceeded(i); console.log(row(B, name, unit(s), unit(m), `${(s / m).toFixed(2)}x`) + (k.length ? k.join(' + ') : 'none')); } ``` ``` thresholds: duration 900 s | memory 3072 MB | connection lifetime 900 s fictional units: a machine carries 40 concurrent jobs; the numbers below are units, not money job type single run limit multiple monthly calls memory state concurrent short and sparse request 0.12 s 0.00x 40,000 256 MB none 1 nightly batch job 2700 s 3.00x 30 1024 MB temp file 1 long-running conversion 4200 s 4.67x 60 3072 MB in-memory interim result 1 persistent-connection service 1500 s 1.67x 1,382,400 512 MB session 20 high and continuous load 0.09 s 0.00x 1,036,800,000 512 MB none 36 monthly cost and the item that exceeds the threshold job type serverless rented machine ratio exceeded threshold short and sparse request 0.0284 70.08 0.00x none nightly batch job 1.38 70.08 0.02x duration limit long-running conversion 12.85 70.08 0.18x duration limit + state persistent-connection service 17626 70.08 251.51x connection lifetime + state + unit cost high and continuous load 1001 70.08 14.28x unit cost ``` The second table draws the real distinction. Short and sparse request sits at the exact center of what serverless is for: monthly cost is 0.0284 units, one 2,400th of the rented machine's. The job runs forty thousand times a month at one hundred twenty milliseconds; a machine left running continuously sits idle for nearly all of this job. This is where the handed-off capacity decision wins most clearly. Nightly batch job and long-running conversion also win on the cost axis — one at 2% of the rented machine, the other at 18% — but both exceed the **duration limit**. For these two jobs the question is not "is it expensive" but "does it run at all". Long-running conversion also keeps its state in memory: the intermediate result accumulated over the four-thousand-two-hundred-second job sits in a place that disappears when the run ends. The last two rows turn things around. Persistent-connection service exceeds three items at once: connection lifetime, state, and unit cost. Its serverless equivalent is **two hundred fifty-one times** the rented machine's, because each of the eight hundred connections is billed like a call running for its whole duration — including while it waits. High and continuous load, by contrast, falls only on the cost axis: fourteen times more expensive. #### The Cost of Splitting to Fit ```js // run2.mjs — the cost of splitting an unfit job, and the break-even point under continuous load import { JOBS, DURATION_LIMIT, MONTH_S, INTERIM_WRITE, INTERIM_READ, INTERIM_MB, GB_S, CALL, MACHINE_HOUR, MACHINE_CONCURRENCY, split, serverlessMonth, machineMonth, row, unit } from './jobs.mjs'; const conv = JOBS['long-running conversion']; console.log(`"long-running conversion" ${conv.s} s, limit ${DURATION_LIMIT} s, interim state ${INTERIM_MB} MB, ` + `per-boundary write ${INTERIM_WRITE} s + read ${INTERIM_READ} s`); const P = [13, 16, 8, 16, 16, 15, 16]; console.log(row(P, 'limit share', 'piece duration', 'steps', 'extra duration', 'total duration', 'interim state', 'failure points') + 'monthly units'); for (const share of [0.4, 0.6, 0.8, 0.95]) { const r = split(conv.s, share); const cost = conv.month * r.steps * CALL + (conv.month * r.total * conv.mb) / 1024 * GB_S; console.log(row(P, `${share * 100}%`, `${Math.round(r.pieceS)} s`, r.steps, `${r.extra} s`, `${Math.round(r.total)} s`, `${r.interimMb} MB`, r.failurePoints) + unit(cost)); } console.log(` without splitting: 1 step, 0 s extra duration, 0 MB interim state, 1 failure point — but a single ` + `run does not work because it is ${(conv.s / DURATION_LIMIT).toFixed(2)} times the limit`); const night = JOBS['nightly batch job'], nightSplit = split(night.s, 0.8); console.log(` with the same 80% share, "nightly batch job" (${night.s} s): ${nightSplit.steps} steps, ${nightSplit.extra} s extra duration, ` + `${nightSplit.interimMb} MB interim state, ${nightSplit.failurePoints} failure points`); // Persistent connection: the job cannot be split, the connection breaks. Its cost is measured by the reconnect count. const conn = JOBS['persistent-connection service']; const breaks = Math.ceil(MONTH_S / DURATION_LIMIT) * conn.connections; console.log(`\n"persistent-connection service": ${conn.connections} connections, average lifetime ${conn.s} s`); console.log(` because the limit is ${DURATION_LIMIT} s, every connection breaks ${Math.ceil(MONTH_S / DURATION_LIMIT)} times a month: ` + `${breaks.toLocaleString('en-US')} reconnects; session state is read externally every time`); // Break-even under continuous load: at which req/s the rented machine gets cheaper. const load = JOBS['high and continuous load']; const Y = [12, 17, 15, 14, 10]; console.log('\nbreak-even under continuous load — req/s is scanned'); console.log(row(Y, 'req/s', 'concurrent jobs', 'serverless/mo', 'machine/mo', 'machines') + 'cheaper'); for (const r of [1, 10, 28, 100, 400]) { const i = { ...load, month: Math.round(r * MONTH_S), concurrency: Math.ceil(r * load.s) }; const s = serverlessMonth(i), m = machineMonth(i); console.log(row(Y, r, i.concurrency, unit(s), unit(m), Math.max(1, Math.ceil(i.concurrency / MACHINE_CONCURRENCY))) + (s < m ? 'serverless' : 'rented machine')); } ``` ``` "long-running conversion" 4200 s, limit 900 s, interim state 240 MB, per-boundary write 18 s + read 12 s limit share piece duration steps extra duration total duration interim state failure points monthly units 40% 353 s 13 360 s 4560 s 2880 MB 25 13.95 60% 497 s 9 240 s 4440 s 1920 MB 17 13.59 80% 630 s 7 180 s 4380 s 1440 MB 13 13.40 95% 730 s 6 150 s 4350 s 1200 MB 11 13.31 without splitting: 1 step, 0 s extra duration, 0 MB interim state, 1 failure point — but a single run does not work because it is 4.67 times the limit with the same 80% share, "nightly batch job" (2700 s): 4 steps, 90 s extra duration, 720 MB interim state, 7 failure points "persistent-connection service": 800 connections, average lifetime 1500 s because the limit is 900 s, every connection breaks 2880 times a month: 2,304,000 reconnects; session state is read externally every time break-even under continuous load — req/s is scanned req/s concurrent jobs serverless/mo machine/mo machines cheaper 1 1 2.50 70.08 1 serverless 10 1 25.01 70.08 1 serverless 28 3 70.04 70.08 1 serverless 100 9 250 70.08 1 rented machine 400 36 1001 70.08 1 rented machine ``` The splitting scan says this: **the cost of splitting is not slowing the job down**. Even in the worst setting, total duration only climbs from four thousand two hundred to four thousand five hundred sixty seconds, 8.6%. The real bill is in the other three columns. When the limit share is pulled to 40%, step count climbs to thirteen, interim state to two thousand eight hundred eighty megabytes, failure points to twenty-five; at 95% these drop to six steps, one thousand two hundred megabytes, and eleven failure points — but getting closer to the limit means a piece whose duration runs slightly long can exceed it. So splitting turns a job with a single failure point into one with eleven to twenty-five failure points, and makes every point in between dependent on writing and reading an interim state, and therefore on a persistent store. At the same 80% setting, the nightly batch job pays four steps, ninety seconds of extra duration, seven hundred twenty megabytes of interim state, and seven failure points. These are line items that come from serverless's constraint, not from the job itself. The persistent-connection row shows the limit of splitting. This job **cannot be split**: when the limit is reached, what gets cut is not computation but the connection. Eight hundred connections break two million three hundred four thousand times a month, and every time one breaks, the session state is read back from outside. The cost here is not a slowdown but a change in design. The break-even scan ties the last axis to a single number. Up to twenty-eight requests per second serverless is cheaper; past that the rented machine is cheaper, and at four hundred requests the difference climbs to fourteen times. This number does not count redundancy or operating labor, so the real break-even point sits above twenty-eight. What the measurement says is this: serverless's cost advantage is **inversely proportional to occupancy**. #### Which Job Fits, Which Does Not | Job Type | Does It Fit? | Item That Exceeds the Threshold | Cost of the Workaround | |---|---|---|---| | Short and sparse request | **Fits** | none | none — 0.03 units a month, one 2,400th of the machine's cost | | Nightly batch job | **Fits if split** | duration limit: 2,700 s, 3x the limit | 4 steps, 90 s, 720 MB interim state, 7 failure points | | Long-running conversion | **Strained** | duration limit: 4,200 s, 4.67x the limit + in-memory state | 7 steps, 180 s, 1,440 MB interim state, 13 failure points | | Persistent-connection service | **Does not fit** | connection lifetime 900 s < 1,500 s + session state + cost 251x | cannot be split; 2,304,000 reconnects a month | | High and continuous load | **Does not fit** | unit cost: machine gets cheaper above 28 req/s, 14x at 400 | none — the constraint is cost, cannot be worked around by design | #### Summary - Short and sparse request costs 0.0284 units a month on serverless, one 2,400th of the rented machine's; a machine left running continuously sits idle for nearly all of this job. - Nightly batch job and long-running conversion win on cost but exceed the duration limit by three and 4.67 times; conversion also keeps its state in memory. - Persistent-connection service exceeds three items at once, and its serverless equivalent is two hundred fifty-one times the rented machine's; a connection is billed even while it waits. - The cost of splitting is not duration: total duration stretches at most 8.6%. The cost is step count, interim state, and failure points — for the conversion, six to thirteen steps, one thousand two hundred to two thousand eight hundred eighty megabytes, and eleven to twenty-five failure points. - Persistent connection cannot be split: what gets cut at the limit is not computation but the connection, meaning two million three hundred four thousand reconnects a month. - Under continuous load the break-even point is twenty-eight requests per second; above it the rented machine is cheaper, and at four hundred requests the difference is fourteen times. Serverless's cost advantage is inversely proportional to occupancy. #### Next Step The function's limits were counted: its duration, memory, concurrency, package, release, rollback, and where it cannot be used. Everything measured across this topic belonged to **a single function** — a request went in, a response came out, and a limit sat in between. But none of the measurement network's jobs end with a single function: reading arrives, validation fires; validation finishes, billing is generated; billing is generated, a notification goes out. This chain itself was never measured. How will functions connect to one another — will one call the other directly, or will something sit between them? The next topic measures the connection itself. ## Event-Driven Services The layer that connects functions to each other: the event bus's routing and filtering rules, the delivery-guarantee difference between queue and topic services, workflow orchestration with step-based state machines, the missed-run and overlapping-run behavior of scheduled triggering, the API gateway service that handles endpoint management and authorization, and the notification services that manage sending reputation. ### Event Bus Source: https://academia.sh/en/courses/serverless/event-bus Handing the routing decision to the event bus: scanning, as the rule set widens over the same 2,000-event stream, the number of events that match no rule and the number that go to multiple targets; the effect on the number of events carried of whether the filter sits in the rule or in the target; and measuring how a new rule can pick up traffic without disturbing any existing number. The previous lesson counted the limits of the function and wrote down where it cannot be used. One question remains: how do functions connect to each other? A function does nothing on its own; something has to wake it. If the waking party is a calling function, the caller has to know the target — its address, its version, how many there are. As the number of producers grows, this knowledge multiplies separately inside every producer. The **event bus** reverses this link. The producer writes the event and does not know where it will go; the target is chosen by a **rule**. The decision has been taken from the producer and handed to the event bus. This lesson counts what comes back in exchange for the handed-over decision. #### The Handed-Over Decision: Who Wakes Up In a direct call the target list sits inside the code: who calls whom can be read from the call stack. In rule-based routing the target list sits outside the code — a rule is a **data** item. The constraint that comes in exchange follows from this: routing no longer shows up in the source-repository diff, does not sit in the function's call graph, and where an event goes can only be known by reading the rule set. The first payback of the handed-over decision shows up in the link count. The measurement network has six components that produce events and nine targets that act on events. In a direct call, every producer has to know the targets it cares about: 54 links in the worst case, each one an address and a version held on both sides. On the event bus the producer writes to a single place and the targets read from a single place — 6 + 9 = 15 records and a rule set between them. The link count drops from a product to a sum. The constraint comes from the same place: all 54 links were visible in the code; none of the items in the rule set are. The second constraint sits in the matching itself. If an event matches no rule, the event bus's default behavior is to **drop** it; it produces no error, wakes no one, leaves no trace behind. The third is the reverse: if an event matches multiple rules, each rule wakes its own target and the same event is processed more than once. Both depend on how wide the rule set is, and both are a number. The measurement runs on a fictional setup that carries through the whole lesson: the **regional measurement network** — a system where meters send in readings, readings are validated, invoices are produced, and field crews are assigned work. Both the network and the event names are fictional. - **ED1.** The stream is 2,000 events; there are nine event types and the share of each type is fixed. - **ED2.** Every event has three fields: type, region (four regions), priority (1–3). - **ED3.** A rule is a dot-separated type pattern with optional field conditions; a `*` in the pattern leaves everything after that segment unrestricted. - **ED4.** If an event matches the same target through two rules, it counts as a single delivery; delivery is counted per target. - **ED5.** Wherever randomness is needed, the generator is written by hand and the seed is visible. - **ED6.** The model builds only the routing decision; transport latency and ordering are not measured. #### As the Rule Set Widens The code below is not an event bus; it is a countable model of the routing decision. The same stream is run through three rule sets: a four-rule **narrow** set, a seven-rule **medium** set, and a ten-rule **wide** set that includes a catch-all rule. ```js // rule-engine.mjs — event bus model: rules route events carrying a pattern to targets // Model: not a real event bus, but a countable reconstruction of the routing decision. const SEED = 20260415; // generator seed: run is reproducible let state = SEED; const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648; const TYPES = [["meter.reading.received", 46], ["meter.reading.missing", 9], ["meter.validation.failed", 7], ["invoice.draft.created", 12], ["invoice.finalized", 6], ["device.battery.low", 8], ["device.connection.lost", 5], ["field.crew.assigned", 4], ["field.fault.closed", 3]]; const REGIONS = ["north", "east", "west", "central"]; function events(n) { const list = []; for (let i = 0; i < n; i++) { let p = random() * 100, type = TYPES[0][0]; for (const [name, weight] of TYPES) { if ((p -= weight) < 0) { type = name; break; } } list.push({ type, region: REGIONS[Math.floor(random() * 4)], priority: 1 + Math.floor(random() * 3) }); } return list; } // Pattern: dot-separated segments; "*" leaves everything after that segment unrestricted. function matches(pattern, type) { const p = pattern.split("."), t = type.split("."); for (let i = 0; i < p.length; i++) { if (p[i] === "*") return true; if (p[i] !== t[i]) return false; } return p.length === t.length; } const fires = (rule, event) => matches(rule.pattern, event.type) && (rule.region === undefined || rule.region === event.region) && (rule.minPriority === undefined || event.priority >= rule.minPriority); const R = { narrow: [ { name: "R1", pattern: "meter.reading.received", target: "validation" }, { name: "R2", pattern: "invoice.*", target: "billing" }, { name: "R3", pattern: "device.battery.low", target: "maintenance" }, { name: "R4", pattern: "field.*", target: "field" }], }; R.medium = [...R.narrow, { name: "R5", pattern: "meter.*", target: "archive" }, { name: "R6", pattern: "device.*", target: "monitoring" }, { name: "R7", pattern: "meter.validation.failed", region: "north", target: "alert" }]; R.wide = [...R.medium, { name: "R8", pattern: "*", target: "audit" }, { name: "R9", pattern: "invoice.finalized", target: "accounting" }, { name: "R10", pattern: "meter.reading.received", minPriority: 3, target: "priority" }]; const STREAM = events(2000); function scan(rules) { let unmatched = 0, single = 0, multi = 0, delivered = 0; for (const e of STREAM) { const h = new Set(rules.filter((r) => fires(r, e)).map((r) => r.target)); delivered += h.size; if (h.size === 0) unmatched++; else if (h.size === 1) single++; else multi++; } return { unmatched, single, multi, delivered }; } console.log("set rules unmatched single target multi target total delivered"); for (const name of ["narrow", "medium", "wide"]) { const s = scan(R[name]); console.log(name.padEnd(8) + String(R[name].length).padStart(7) + String(s.unmatched).padStart(12) + String(s.single).padStart(15) + String(s.multi).padStart(14) + String(s.delivered).padStart(17)); } console.log("event count: " + STREAM.length + ", seed: " + SEED); ``` ```sh node rule-engine.mjs ``` ``` set rules unmatched single target multi target total delivered narrow 4 433 1567 0 1567 medium 7 0 881 1119 3119 wide 10 0 0 2000 5551 event count: 2000, seed: 20260415 ``` The first row shows silent dropping. The four-rule set delivered 1,567 of the two thousand events to one target; **433 events matched no rule** and were dropped. These events are the missing-reading, validation-failure, and lost-connection notices — two of the three types that most need attention. Dropping is not an error; for the event bus, "no match" is an ordinary outcome. There is no symptom on the producer's side either: the event was written, accepted, and the job is done. The second row is the other end of the constraint. Once three rules were added, unmatched fell to zero, but **1,119 events went to more than one target** and total delivered climbed from 1,567 to 3,119. The same reading now wakes both the validation and the archive target. If the two targets do separate work, this is correct; if both of them also write a billing item, the same item gets written twice. The rule set does not know this difference. The path between these two rows is the usual road taken while a rule set widens: an unmatched event is noticed, the pattern is broadened, the unmatched count falls. That is not the only thing that falls — every broadening also pulls events that previously reached one target into the new rule. With four rules no event was multi-target; after three more rules, 56 percent of the stream is multi-target. Broadening's effect is not confined to the targeted type; the pattern also covers types it was never meant to target. In the third row the catch-all rule keeps unmatched at zero, but now **every event is multi-target**: 2,000 events turned into 5,551 deliveries, 2.78 wake-ups per event. The price of escaping silent drops is triple delivery, and delivery, on the function's side, means a call. #### The Filter's Location Once the routing decision is handed over, a second decision can be handed over too: **filtering**. The validation target actually only cares about high-priority readings from the north region. This condition can be written either into the rule or into the target's first lines. Both do the same job; the number of events carried is not the same. ```js // filter-location.mjs — same stream, filter in the rule and in the target; then a hidden rule is added const SEED = 20260415; // same seed as the first run, same stream let state = SEED; const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648; const TYPES = [["meter.reading.received", 46], ["meter.reading.missing", 9], ["meter.validation.failed", 7], ["invoice.draft.created", 12], ["invoice.finalized", 6], ["device.battery.low", 8], ["device.connection.lost", 5], ["field.crew.assigned", 4], ["field.fault.closed", 3]]; const REGIONS = ["north", "east", "west", "central"]; const STREAM = []; for (let i = 0; i < 2000; i++) { let p = random() * 100, type = TYPES[0][0]; for (const [name, weight] of TYPES) { if ((p -= weight) < 0) { type = name; break; } } STREAM.push({ type, region: REGIONS[Math.floor(random() * 4)], priority: 1 + Math.floor(random() * 3) }); } const relevant = (e) => e.region === "north" && e.priority >= 2; // what the target actually wants const readings = STREAM.filter((e) => e.type === "meter.reading.received"); const UNIT = 1000; // fictional unit: every 1,000 deliveries is 1 transport unit console.log("filter location".padEnd(19) + "carried".padStart(9) + "wasted wake-up".padStart(17) + "transport unit".padStart(16)); for (const [place, carried] of [["filter in rule", readings.filter(relevant)], ["filter in target", readings]]) { const wasted = carried.length - carried.filter(relevant).length; console.log(place.padEnd(19) + String(carried.length).padStart(9) + String(wasted).padStart(17) + (carried.length / UNIT).toFixed(2).padStart(16)); } // Hidden rule: without touching the existing target's numbers, sends the same events to a second target. const before = readings.filter(relevant).length; const shadow = STREAM.filter((e) => e.type.startsWith("meter.reading.")).length; console.log("\nbefore adding a rule validation target: " + before + ", total delivered: " + before); console.log("after adding R11 validation target: " + before + ", total delivered: " + (before + shadow) + ", error: 0"); ``` ```sh node filter-location.mjs ``` ``` filter location carried wasted wake-up transport unit filter in rule 145 0 0.14 filter in target 926 781 0.93 before adding a rule validation target: 145, total delivered: 145 after adding R11 validation target: 145, total delivered: 1257, error: 0 ``` Filtering in the rule carried 145 events, and all of them were processed. Filtering in the target carried 926 events, and **781 of them were discarded on the first line**: the function woke up, read the event, saw it was irrelevant, and returned. The same job's visible cost on the transport side is 0.14 units against 0.93 units — 6.4 times as much. The unit is fictional; the ratio follows from the rule set. The cost of a wasted wake-up is not only transport. The 781 calls take a slot from the concurrency pool, carry a chance of cold start, and enter as noise into every metric on the function's side: call count, error rate, average duration. The cost of moving the filter into the rule, on the other hand, is that the condition moves out of code review and into configuration — the subject of the next section. The source has a limit on filtering, and that limit is the real constraint left outside the measurement: a filter can only look at the event's own fields. If the validation target wanted "only readings from subscriptions whose contract is suspended," this condition could not be written with any of the three fields carried in the event — the subscription's status is held somewhere else. In that case the only way is to add the field the condition cares about to the event; the cost is that the producer now has to know the consumer's need, meaning one end of the handed-over link comes back. If the field cannot be added, the 781 wasted wake-ups are unavoidable, and the choice is not where the filter sits but which cost gets accepted. #### The Rule's Invisibility Is a Surface The last two lines of the output show a single thing. A new rule was added to the rule set that does not touch the validation target: R11, with the pattern `meter.reading.*`, sends events to a second target as well. The validation target's delivery count **stayed at 145**, its error count stayed at zero, no metric moved. Total delivered, however, went from 145 to 1,257 — every reading event in the stream now also goes to a second place. This is the most expensive constraint of handing over routing. Because a rule is a data item: - It does not appear in the source-repository diff; not one line of function code changes. - It does not disturb the metrics of existing targets, so it crosses no alert threshold. - The content of events reaching the new target is independent of the producer's intent: the producer wrote the reading for validation, and the rule also sent it somewhere else. The same invisibility works in the other direction too. When a rule is deleted or its pattern narrowed, the target stops receiving traffic; the function does not error, because it is never called. The call count drops to zero, and zero reads not like a failure symptom but like "there is no work at this hour." Between this and the 433 silent drops from the first measurement there is no observable difference: in both cases the event was written, accepted, and went nowhere. The only way to see the difference is to count the number of events produced and the number of events delivered separately and watch the gap between them. There are three ways to work around the constraint, and all three demand a countable cost. The first is to keep the rule set in the source repository and review changes as code; its cost is that every rule change now enters the release pipeline, and a process that takes minutes gets spread over hours. The second is to keep a per-target delivery counter; the jump in total delivered from 145 to 1,257 when R11 was added is only visible this way, and its cost is ten separate counters in a ten-rule set plus a set of thresholds that reads them. The third is to ban wide patterns — the catch-all rule is removed, `*` is only allowed in the last segment. The cost of this is the first row of the first measurement: as the pattern narrows, the unmatched-event count climbs again and 433 events are silently dropped. Choosing among the three means weighing the cost of a dropped event against the cost of an extra delivery. In the measurement network a dropped validation error shows up as wrong consumption on an invoice; an extra archive delivery only costs storage. In this network the rule set is wide, the filter sits in the rule, and the counter is per target. #### Summary - The event bus takes the routing decision away from the producer: the producer does not know the target, the rule chooses it. In exchange, a rule becomes a data item and steps outside code review. - On the same 2,000-event stream, the four-rule set delivered 433 events to no target and did so without producing an error; silent dropping is the event bus's default behavior. - Once the rule count rose to seven, unmatched fell to zero but 1,119 events became multi-target; with ten rules total delivered reached 5,551, 2.78 wake-ups per event. - Filtering in the rule carried 145 events, filtering in the target carried 926, and 781 of those woke the function for nothing; the transport-side difference is 0.14 units against 0.93 fictional units. - A new rule left the existing target's delivery count at 145 while pushing total delivered to 1,257, and it produced no error; this is why a rule's invisibility is a surface. - Silent dropping and a deleted rule produce the same symptom: in both cases the call count is zero. The only way to tell them apart is to count events produced and events delivered separately. #### Next Step The routing decision has been made: which event wakes whom is now written into the rule set. But this measurement never asked one thing — what happens when the target wakes up and is **not standing**? The model counted every delivery as successful; in reality the function can be throttled, can error, can hit its concurrency limit. Something is needed to hold the event, retry it, and keep it waiting until the consumer is ready. The next lesson measures that layer as a service: which decisions leave your hands when queue and topic services are taken over, at what threshold visibility timeout produces double processing, and at what threshold it produces delay. ### Queue and Topic Services Source: https://academia.sh/en/courses/serverless/queue-and-topic-services The constraints that come from taking the queue as a service: counting the five remaining configuration items, scanning visibility timeout across six values to measure the double processing a short duration produces against the recovery delay a long one produces, the dead-letter drop time turning out to be a derived number, and comparing the fan-out difference between queue and topic services in terms of delivery and configuration items. The previous lesson routed events to targets by rule and counted every delivery as successful. That assumption does not hold in a real stream: the target may be throttled, may error, may not be standing at all. The buffer in between is built by the **queue service** — it holds the event, keeps it waiting until the consumer is ready, and makes failed work visible again. The queue's mechanics are not new. The Delivery Semantics lesson of the Asynchronous Processing course measured the order of acknowledgment against effect and showed the difference between at-least-once and deduplication with numbers; that measurement is not repeated here and its results are taken as given. This lesson's question is different: what changes when the queue is not a tool you run yourself but **a service**? #### Handed-Over Decisions and the Surface Left Behind In a queue taken as a service, the following decisions leave your hands: how many nodes run, how many copies a message is stored with, where the disk writes, when the version is upgraded, how many consumers get added when queue depth grows. The last one matters most; once the queue service is wired to a function-execution environment, the scaling decision is handed over too — instances are added as depth rises, removed as it falls. The cost of the handed-over scaling decision is that the **speed** of the increase leaves your hands too. When 400 messages land in the queue at once, the rule that decides how many instances get added and how fast sits inside the service; the only thing visible from outside is a ceiling number. This produces a two-way constraint. If the ramp-up is slow, the nightly batch job finishes later than expected and queue depth stays high for hours. If the ramp-up is fast, the hundreds of instances that wake at once drain the connection pool of the database behind it — the queue finds its own scale, but the scale it finds exceeds a downstream constraint. A consumer count you set yourself, watching queue depth, would solve both; with the handed-over decision, that lever is gone too. The only control left is lowering the ceiling itself, and the cost of that is making the first case permanent. The remaining surface is small and countable. In the model used in this lesson there are five items per queue: **visibility timeout**, **maximum delivery count**, **dead letter queue**, **retention period**, **batch size**. Five items are the entire control left in exchange for all those handed-over decisions. The constraint is here: what cannot be tuned cannot be fixed either. A message whose retention period expires is deleted, and that deletion, like the unmatched event in the previous lesson, is silent. The measurement again runs on the fictional **regional measurement network**: meter readings are written to a queue, the validation function reads from the queue, and the validated reading enters the invoicing stream. - **ED7.** The workload is 400 messages; processing time falls into three clusters — ordinary validation 2–6 units, batch read 12–31 units, delayed batch 40–69 units. - **ED8.** 12 of the messages are poison (they fail on every attempt), 16 have a consumer that dies before finishing the work; the two clusters never overlap and are fixed by position. - **ED9.** Maximum delivery count is 5; a message that reaches this count moves to the dead letter queue. - **ED10.** When visibility timeout expires the message becomes visible again; a consumer still running is not stopped, meaning a second consumer redoes the work **concurrently**. - **ED11.** The duration unit is fictional; what is measured is ratios, not seconds. - **ED12.** The model builds only a single queue; ordering, partitioning, and transport latency are not measured. #### Visibility Timeout **Visibility timeout** is how long a message stays hidden in the queue after a consumer picks it up. If the consumer finishes the work and deletes the message within that time, the job is done; if it does not finish, the message becomes visible again and another consumer picks it up. It is a single number that decides two separate things at once: when slow work gets started a second time, and when a dead consumer's work gets recovered. These two requirements pull in opposite directions; the scan below measures both against the same workload. ```js // visibility.mjs — queue service model: what happens when visibility timeout changes // Model: not a real queue, but the counted results of the service's one setting. const SEED = 20260518; // generator seed: run is reproducible let state = SEED; const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648; const N = 400, MAX_DELIVERIES = 5; // after MAX_DELIVERIES the message goes to the dead letter queue const MESSAGES = []; for (let i = 0; i < N; i++) { const p = random(); const duration = p < 0.88 ? 2 + Math.floor(random() * 5) // ordinary validation: 2-6 units : p < 0.97 ? 12 + Math.floor(random() * 20) // batch read: 12-31 units : 40 + Math.floor(random() * 30); // delayed batch: 40-69 units MESSAGES.push({ duration, poison: i % 33 === 7, crashed: i % 25 === 3 }); // the two never overlap } const POISON = MESSAGES.filter((m) => m.poison).length; const CRASHED = MESSAGES.filter((m) => m.crashed).length; function run(T) { let delivered = 0, doubled = 0, wait = 0; for (const m of MESSAGES) { if (m.poison) { delivered += MAX_DELIVERIES; continue; } // fails again on every retry if (m.crashed) { delivered += 2; wait += T; continue; } // consumer died: stays hidden for T const tries = Math.ceil(m.duration / T); // work not finished before T reappears delivered += tries; if (tries > 1) doubled++; } return { delivered, doubled, wait: wait / CRASHED, deadLetter: MAX_DELIVERIES * T }; } console.log("visibility".padEnd(12) + "double processing".padStart(20) + "total delivered".padStart(18) + "recovery delay".padStart(20) + "dead-letter drop".padStart(19)); for (const T of [5, 10, 20, 30, 60, 120]) { const s = run(T); console.log(String(T).padEnd(12) + String(s.doubled).padStart(20) + String(s.delivered).padStart(18) + s.wait.toFixed(0).padStart(20) + String(s.deadLetter).padStart(19)); } const LONGEST = Math.max(...MESSAGES.map((m) => m.duration)); console.log(`messages: ${N}, poison: ${POISON}, consumer crashed: ${CRASHED}, ` + `longest job: ${LONGEST} units, seed: ${SEED}`); ``` ```sh node visibility.mjs ``` ``` visibility double processing total delivered recovery delay dead-letter drop 5 102 745 5 25 10 40 562 10 50 20 28 505 20 100 30 11 480 30 150 60 5 469 60 300 120 0 464 120 600 messages: 400, poison: 12, consumer crashed: 16, longest job: 69 units, seed: 20260518 ``` The first row shows the cost of a short timeout. With visibility timeout at 5, **102 messages were double-processed** and a workload of 400 messages turned into 745 deliveries. Double processing here is a different thing from a retry: while the first consumer is still working, the message becomes visible again and a second consumer does the same job **at the same time**. In the measurement network this means the same meter reading gets validated twice and enters the invoicing stream twice. The last row is the cost of a long timeout. At 120 units, double processing falls to zero, but the 16 messages whose consumer died wait an average of **120 units hidden** — invisible for that whole span, unreachable by any consumer, showing up in no metric. A poison message's drop into the dead letter queue, meanwhile, takes 600 units. The practical consequence of this trade-off is that concurrent double processing also stresses the deduplication guard. The guard asks "has this message already been processed"; in a concurrent run, the second consumer asks that question before the first has written its result, and the answer comes back "no." A guard that works fine under sequential retries fails both runs here. What closes double processing is not the guard's existence but whether the guard relies on a uniqueness constraint at write time. The rule sits in the output's last line: **the longest job is 69 units.** For double processing to reach zero, visibility timeout has to exceed the longest job; at 60 units, 5 messages are still double-processed, at 120 it is zero. In other words, the queue of processing durations sets the lower bound of recovery delay. A single long-running job type drags down the recovery behavior of the whole queue. The way around the constraint is to extend visibility timeout while the work is running: the consumer reports at regular intervals that it is still alive, and the service moves the counter forward. This costs two things. First, finishing a 69-unit job with a 10-unit timeout takes six extension calls — a single message's work turns into seven requests, and in a service billed per request this is a direct charge. Second, the consumer is no longer only doing its job; it also has to manage its own lifetime, and a consumer that forgets to extend, or whose extension call fails, falls right back into the first row's double processing. #### The Automatic Nature of the Dead Letter Queue The table's last column is a number no one sets directly. The time it takes a poison message to reach the dead letter queue is the product of maximum delivery count and visibility timeout: 5 × 30 = 150, 5 × 120 = 600. Two separate items chosen for separate reasons, and a third number emerges on its own. This is the sneakiest form of constraint — what is tuned is not what is measured. The gain of being a service here is real: a poison message is not weeded out by hand; after the fifth delivery it is moved automatically and does not clog the main queue. In the model, 12 poison messages, whatever visibility timeout is chosen, spend a total of 60 deliveries and get out of the way. What comes back in exchange is this: **the dead letter queue has no consumer by default.** A message moves there, sits there, and is deleted once retention expires. If no one reads it, the result is the silent drop from the previous lesson wearing a receipt — the loss is the same loss, only written down somewhere. The way to empty the queue is to push the messages back onto the main queue, and this too has a number. If the 12 messages are pushed back without the cause being fixed, they get delivered five more times each; 60 more deliveries are spent and the messages return to the same queue. Pushing back is therefore not a repair but a test: first the poisoning is found, then the queue is emptied. Retention is an upper bound in the same way. If the consumer side has an outage longer than retention, messages in the queue start getting deleted one by one, and once the outage ends the queue looks empty. In the measurement network this means the meter readings that landed during the outage window are never billed at all. The way around the constraint is to push retention to its upper limit; the cost is that the queue starts being used like a backup layer, and if depth is not being watched, the loss is only noticed at billing time. #### The Difference Between a Queue and a Topic The second service type delivers the same message to more than one party. The difference sits in one sentence: a queue **shares out** the work; a topic **duplicates** the message. ```js // fanout.mjs — same 400 events through two services: queue service and topic service const N = 400; const QUEUE_ITEMS = 5; // visibility timeout, max deliveries, dead letter queue, retention, batch size const TOPIC_ITEMS = 2; // the topic's own items: name and access rule console.log("consumer".padEnd(10) + "queue delivered".padStart(17) + "queue items".padStart(14) + "topic delivered".padStart(17) + "topic items".padStart(13)); for (const k of [1, 2, 3, 4]) { const topicItems = TOPIC_ITEMS + k * (QUEUE_ITEMS + 1); // each subscription: its own queue + filter console.log(String(k).padEnd(10) + String(N).padStart(17) + String(QUEUE_ITEMS).padStart(14) + String(N * k).padStart(17) + String(topicItems).padStart(13)); } console.log("work per consumer: queue " + N / 4 + ", topic " + N + " (with four consumers)"); ``` ```sh node fanout.mjs ``` ``` consumer queue delivered queue items topic delivered topic items 1 400 5 400 8 2 400 5 800 14 3 400 5 1200 20 4 400 5 1600 26 work per consumer: queue 100, topic 400 (with four consumers) ``` The queue column stays flat: whatever the consumer count, delivered stays at 400, work per consumer drops to a quarter, and the configuration item count sits fixed at five. Adding a consumer is a scaling decision and does not change the cost. The topic column grows linearly. With a fourth subscriber delivered reaches 1,600 and configuration items reach 26. Adding a subscriber is not a scaling decision, it is opening a new line of work: each subscriber gets its own copy of the message, its own visibility timeout, its own maximum delivery count, its own dead letter queue, and its own filter. 26 items are 26 places that can be misconfigured, and six of them arrive with every new subscriber. Notice that this growth is the same surface measured in the previous lesson. Adding a subscriber does not change one line at the producer, does not disturb the numbers of existing subscribers, and produces no error — the rule's invisibility comes back here as the subscription's invisibility. The difference is this: in the rule, the cost of an extra delivery was a call; in the topic, the cost is a call plus the storage of 400 copies. The two services do not have to be alternatives to each other. A common pattern is to bind each of a topic's subscriptions to a queue: the topic duplicates, each queue shares out on its own side. The structure of the model's 26 items is already this — the topic's two items plus five queue items and a filter per subscriber. In this arrangement the previous lesson's filter measurement applies directly: if the filter sits in the subscription, an irrelevant event is never written to the queue at all; if it sits at the consumer, it is written, stored, delivered, and then discarded. The difference this time shows up in three items at once — delivery count, stored message count, and the consumer's wasted wake-ups. The choice criterion is the nature of the work. A queue is for consumers who will share the same job; a topic is for parties who will each derive a different piece of work from the same event. In the measurement network, a validated reading feeds into a single job — it becomes a billing item — and that job is shared out: a queue. If the same reading is meant to produce a bill, an archive entry, and a consumption curve, there are three separate jobs: a topic. #### Summary - Taking the queue as a service hands over decisions about node count, copy count, version upgrades, and consumer scaling; five tunable items remain. - At a visibility timeout of 5 units, the 400-message workload turned into 745 deliveries and 102 messages were double-processed concurrently; at 120 units double processing hit zero, but the 16 messages whose consumer died waited an average of 120 units hidden. - Because the longest job was 69 units, double processing only reached zero at 120 units: processing time sets the lower bound of recovery delay. - The time it takes to reach the dead letter queue is not a tunable item but the product of maximum deliveries and visibility timeout: 5 × 120 = 600 units. The queue itself has no consumer by default. - Queue delivery stayed at 400 regardless of consumer count and configuration stayed at five items; with four subscribers a topic's delivery reached 1,600 and configuration items reached 26. A queue shares out the work, a topic duplicates the message. - Extending visibility timeout while work runs closes double processing, but finishing a 69-unit job with a 10-unit timeout takes six extension calls: a single message's work turns into seven requests, and the consumer has to manage its own lifetime. #### Next Step The queue safely links two functions: one writes, the other reads, and if something fails in between the message comes back. But invoice generation in the measurement network is not two steps; a reading is validated, consumption is computed, a tariff is applied, an invoice is written, a notification is sent. When the fourth of five steps errors, no queue has written down what happens to the first three — a queue only knows its own message, not the whole flow. The next lesson measures that gap: a step-based state machine is run, retries and compensation steps are counted per step, and the question of where the flow's state is held gets asked. ### Workflow Orchestration Source: https://academia.sh/en/courses/serverless/workflow-orchestration Running a five-step invoicing flow as a state machine: measuring retries and compensation steps per step, the constraints that come from moving a flow's state into the orchestrator, and counting the difference between building the same flow inside a function versus handing it to an orchestrator in terms of cost and locked concurrency. A queue safely links two functions: one writes, the other reads, and if something fails the message comes back. Invoice generation in the measurement network, though, is not two steps but five — a reading is validated, consumption is computed, a tariff is applied, an invoice is written, a notification is sent. When the fourth step fails permanently, nothing in any queue records what should happen to the first three; a queue only knows its own message. The dispatched workflow that runs on compensating transactions — the **saga** pattern — was measured earlier. That pattern is not repeated here, only named. This lesson's question is: when the whole flow is handed to a **workflow orchestrator**, which decision is handed over and what number comes back in exchange? #### Who Holds the Flow Writing all five steps inside a single function is the shortest path: one function, five calls, error handling with `if` blocks in between. The flow's state lives in that function's local variables. This causes no trouble until the flow crashes — when it does, nothing is left behind. Which step a function died at on its third attempt, which step left its side effect, is written down nowhere. An orchestrator takes this state out of the function. The start, end, input, output, and attempt count of every step is written to the orchestrator's record; the function only performs a single step and returns its result. That is the handed-over decision: **the responsibility of knowing where the flow is.** The first constraint that comes back follows from the same place — every piece of data passed between steps goes through that record, so it is subject to a size limit, and large payloads cannot be carried between steps; they can only be written to storage with their key passed along instead. The second handed-over decision is the retry itself. In a flow written inside a function, how many times to retry, how long to wait, and which errors count as retryable are in the code; in an orchestrator these sit as a policy next to the step definition. The gain is that the policy is separated from the code and visible in one place. The cost is that the flow definition falls into the same place as the rule in the previous lesson: because a definition is a configuration item, it does not go through code review, and when a step's retry count is raised from three to five, no function changes — the difference only shows up in billing and in latency. The measurement again runs on the fictional regional measurement network. - **ED13.** The flow has five steps and step durations are in fictional units: 2, 3, 2, 4, 1. - **ED14.** Each step has its own error rate; the retry policy is three attempts with increasing wait between attempts (2 and 4 units). - **ED15.** In one in seventeen of 200 flows, writing the invoice conflicts permanently; these flows exhaust all three attempts and end in failure. - **ED16.** Compensation is defined only for steps that have a side effect: apply-tariff 1 unit, write-invoice 2 units. The first two steps have no compensation. - **ED17.** Transition count is the sum of step attempts and compensation steps. - **ED18.** The model builds only the inside of a single flow; concurrency and ordering across flows are not measured. #### The State Machine ```js // flow.mjs — workflow orchestrator model: steps, retry policy, compensation path // Model: not a real orchestrator, but a countable setup of a step-based state machine. const SEED = 20260603; // generator seed: run is reproducible let state = SEED; const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648; const STEPS = [ // duration and compensation: fictional unit { name: "validate-reading", duration: 2, errorRate: 0.06, compensation: 0 }, { name: "compute-consumption", duration: 3, errorRate: 0.04, compensation: 0 }, { name: "apply-tariff", duration: 2, errorRate: 0.10, compensation: 1 }, { name: "write-invoice", duration: 4, errorRate: 0.08, compensation: 2 }, { name: "send-notification", duration: 1, errorRate: 0.15, compensation: 0 }, ]; const MAX_TRIES = 3, WAIT = [2, 4]; // 3 attempts; increasing wait between attempts const N = 200; const PERMANENT = (n) => n % 17 === 5; // in these flows, writing the invoice conflicts every attempt const s = { attempts: STEPS.map(() => 0), errors: STEPS.map(() => 0), succeeded: 0, compensated: 0, compensationSteps: 0, work: 0, wall: 0, transitions: 0, longest: 0 }; for (let n = 0; n < N; n++) { const done = [], start = s.wall; let cut = false; for (let a = 0; a < STEPS.length && !cut; a++) { let attempt = 0, ok = false; while (attempt < MAX_TRIES) { attempt++; s.attempts[a]++; s.transitions++; s.work += STEPS[a].duration; s.wall += STEPS[a].duration; const permanent = PERMANENT(n) && STEPS[a].name === "write-invoice"; if (!permanent && random() >= STEPS[a].errorRate) { ok = true; break; } s.errors[a]++; if (attempt < MAX_TRIES) s.wall += WAIT[attempt - 1]; // wait: no work, but time passes } if (ok) { done.push(a); continue; } for (const t of done.reverse()) { // compensation: unwind in reverse order if (STEPS[t].compensation === 0) continue; s.transitions++; s.work += STEPS[t].compensation; s.wall += STEPS[t].compensation; s.compensationSteps++; } s.compensated++; cut = true; } if (!cut) s.succeeded++; s.longest = Math.max(s.longest, s.wall - start); // longest wall time of a single flow } console.log("step".padEnd(22) + "attempts".padStart(10) + "errors".padStart(8) + "attempts/flow".padStart(15)); STEPS.forEach((a, i) => console.log(a.name.padEnd(22) + String(s.attempts[i]).padStart(10) + String(s.errors[i]).padStart(8) + (s.attempts[i] / N).toFixed(2).padStart(15))); console.log(`\nflows: ${N}, succeeded: ${s.succeeded}, went to compensation: ${s.compensated}, ` + `compensation steps: ${s.compensationSteps}`); console.log(`work time: ${s.work} units, wall time: ${s.wall} units, transitions: ${s.transitions}`); console.log(`longest single flow: ${s.longest} units, seed: ${SEED}`); ``` ```sh node flow.mjs ``` ``` step attempts errors attempts/flow validate-reading 213 13 1.06 compute-consumption 211 11 1.05 apply-tariff 229 29 1.15 write-invoice 234 46 1.17 send-notification 220 32 1.10 flows: 200, succeeded: 188, went to compensation: 12, compensation steps: 12 work time: 2685 units, wall time: 2967 units, transitions: 1119 longest single flow: 31 units, seed: 20260603 ``` The step table shows where retries pile up. If every one of the five steps had passed on its first attempt, 200 flows would have made 1,000 attempts; the measurement shows 1,107 attempts. The difference is the second and third attempts of the steps that error. The heaviest step is writing the invoice: 234 attempts, 46 errors. 36 of those 46 come from the twelve flows with a permanent conflict; the remaining 10 are transient errors closed by retrying. The number in the last row looks backward at first glance: even though sending the notification has the highest error rate, its attempts-per-flow is 1.10, lower than the tariff step's. The reason is the denominator — only 188 flows reached this step, because 12 flows were cut at the previous one. Divided by the same 188 flows that reached it, the number comes out to 1.17. Step metrics should be divided not by the total flow count but by **the number of flows that reached that step**; otherwise every error rate toward the end of a flow looks smaller than it is. The compensation path produced two numbers: 12 flows went to compensation and 12 compensation steps ran in total, a ratio of one per flow, because when writing the invoice fails, the only side effect that needs undoing is the tariff application; the first two steps have no defined compensation. This shows the real question of compensation design: the number of steps to unwind depends on **which** step the failure happened at. A failure later in the flow means a longer unwind, and a flow that fails on its last step can cost more to compensate than the flow itself. The retry policy itself is inside these numbers too. If the maximum attempt count had been five instead of three, the 12 permanently conflicting flows would have spent 60 attempts on writing the invoice instead of 36: 24 extra attempts, 96 extra units of work, and — since the waits increase — a longer delay per flow before compensation even starts. On a permanent error, retrying rescues nothing; it only delays accepting the failure. That is the cost of loosening the policy; the cost of tightening it runs the other way — two attempts instead of three turns part of the tariff step's 29 transient errors into permanent failures and raises the number of flows that go to compensation. #### Billing the Long-Running Flow Total work time is 2,685 units, wall time is 2,967 units. The 282-unit gap between them is the wait between retries — time in which no work happens, but the flow stays open. In this measurement, wait is about 10 percent of the total, because the wait values in the policy were kept small. In flows waiting on an approval or on a response from an external service, the ratio reverses and wait grows to multiples of work time. The calculation below scans that difference with a multiplier; each row runs the same 200 flows, only the weight of the wait changes. Who ends up paying for it depends on where the flow is built. ```js // flow-cost.mjs — same 200 flows: is the flow built inside the function, or in the orchestrator // Numbers read from the first run. The billing unit is fictional; the ratio is preserved, there is no currency. const N = 200, WORK = 2685, WALL = 2967, TRANSITIONS = 1119; const WAIT = WALL - WORK; // wait between steps: time with no work done const DURATION_RATE = 1; // fictional unit: per unit of running time const TRANSITION_RATE = 0.25; // fictional unit: per state transition in the orchestrator console.log("wait multiplier".padEnd(17) + "in function".padStart(14) + "in orchestrator".padStart(17) + "locked (function)".padStart(20) + "locked (orch.)".padStart(19)); for (const k of [1, 5, 20, 100]) { const wait = WAIT * k; const inFunction = (WORK + wait) * DURATION_RATE; // the function stays up throughout the wait too const inOrchestrator = WORK * DURATION_RATE + TRANSITIONS * TRANSITION_RATE; // wait is free, transitions are billed console.log(String(k).padEnd(17) + inFunction.toFixed(0).padStart(14) + inOrchestrator.toFixed(0).padStart(17) + String(WORK + wait).padStart(20) + String(WORK).padStart(19)); } console.log(`flows: ${N}, work: ${WORK} units, wait (multiplier 1): ${WAIT} units, transitions: ${TRANSITIONS}`); ``` ```sh node flow-cost.mjs ``` ``` wait multiplier in function in orchestrator locked (function) locked (orch.) 1 2967 2965 2967 2685 5 4095 2965 4095 2685 20 8325 2965 8325 2685 100 30885 2965 30885 2685 flows: 200, work: 2685 units, wait (multiplier 1): 282 units, transitions: 1119 ``` The first row is an honest result: while wait is short, the cost of the two designs is nearly the same — 2,967 against 2,965. The 1,119 × 0.25 = 280 units the orchestrator charges per transition almost exactly matches the 282 units the function pays for waiting. Per-step billing is not free: as step count grows, transition cost grows with it, and in flows with many steps that finish fast the orchestrator can stay expensive. A side effect of this billing shape is that it prices the flow's **granularity**. If each of the five steps were split into three — its own validation, its own work, its own write — work time would not change but transition count would triple from 1,119 and transition cost would rise from 280 units to 840. Where step boundaries are drawn directly determines the cost of a flow running in the orchestrator. The reverse holds too: merging two steps into one lowers transition cost, but the merged step's retry now reruns both at once and the failure point gets coarser. The gap widens as wait lengthens. At a multiplier of 100 — a flow waiting on an approval, on a response from an external service, or one that sits until the next day — the in-function design climbs to 30,885 units while the orchestrator stays at 2,965: **10.4 times.** The break-even point can be read straight from the measurement; the moment wait exceeds the total transition cost, the orchestrator gets cheaper. The last two columns show a harder constraint. Locked time is the slot held from the concurrency pool. A flow built inside a function occupies a slot through the wait as well; at a multiplier of 100, 30,885 units of slot-time get locked. In the orchestrator this number stays fixed at 2,685, because no function is up during the wait. In an environment with a concurrency limit, this difference hits **capacity** before it hits cost: waiting flows block the processing of newly arriving readings. #### Visibility, Restartability, Limit Beyond cost and capacity there are three more differences, and all three read straight from the measurement's numbers. **Visibility.** Because the orchestrator records every transition, all 1,119 of them form a queryable record: which flow, at which step, on which attempt, waited for how long. When the same flow is built inside a function, this information lives only in log lines, and stitching it back together per flow becomes a separate job. Here the handed-over decision comes back as a gain. **Restartability.** The 12 flows that went to compensation can be restarted from the failed step in the orchestrator; the result of the first three steps sits in the record and does not rerun. In the in-function design, restarting means starting over — three more steps for 12 flows, 12 × 7 = 84 extra units of work. Once the permanent conflict's cause is fixed, this difference is counted directly. **Limit.** The longest single flow took 31 units. This number is the critical constraint of the in-function design: a flow's wall time cannot exceed the function-execution environment's time limit. 31 units stays well under a reasonable limit, but wall time grows with the wait multiplier; a flow waiting on an approval or held over to the next day fits no function's time limit. The orchestrator has no such limit, because nothing is running during the wait. In exchange, the orchestrator brings its own limits, and these are measured not in time but in **count**: the maximum number of steps a flow can take, the record size it can hold, the lifetime of retained history. Because the measured flow has five steps and averages 5.6 transitions per flow, it approaches none of these limits; but if step count depends on the size of the data — for example a batch flow that opens one step per meter — the limit makes the design directly impossible. In that case the only path is turning the flow into an outer loop with a fixed step count, starting a new flow on every iteration; the cost is that flow count, and transition cost with it, grows together with the data. The way around the constraint is to split the flow in two: short, tight steps inside a function, transitions that involve waiting in the orchestrator. The cost is that the flow is now defined in two places at once — visibility gets split again, and which step runs where becomes a separate piece of information. #### Summary - The responsibility of holding the flow's state is handed to the orchestrator; in exchange, every piece of data passed between steps goes through the orchestrator's record and is subject to a size limit. - 200 flows produced 1,107 step attempts; the heaviest step is writing the invoice with 234 attempts and 46 errors, 36 of which come from twelve permanently conflicting flows. - Step metrics must be divided by the number of flows that reached that step, not by the total flow count: the last step's attempt count comes out to 1.10 divided by 200, and 1.17 divided by the 188 flows that reached it. - 12 flows went to compensation and ran 12 compensation steps; the number of steps to unwind depends on which step of the flow the failure happened at. - The two designs cost the same while wait is short (2,967 against 2,965); once wait grows a hundredfold, the in-function flow costs 30,885 while the orchestrator stays at 2,965, and locked concurrency is 30,885 slot-units against 2,685. - Per-transition billing prices the flow's granularity: splitting the steps into three raises transition cost from 280 units to 840, while work time stays unchanged. #### Next Step This entire flow started with an event: a reading arrived, validation was triggered, the chain ran. Some of the measurement network's jobs have no such event. The nightly batch invoicing, the daily reconciliation, and the weekly meter health scan depend on no reading's arrival; the only thing that starts them is the clock. Something holds the clock too, and it is a service. The next lesson measures that trigger: the kind of guarantee the service gives, what firing twice in the same minute does to invoicing, and why a missed run is not automatically compensated for. ### Scheduled Triggering Source: https://academia.sh/en/courses/serverless/scheduled-triggering The kind of guarantee that comes from handing the clock to a service: measuring, across a thirty-night batch-invoicing run, the number of duplicate invoices two triggers arriving in the same minute produce; the idempotency key closing that gap while leaving a missed run uncompensated; and counting, under clock drift, the difference between deriving the work window from trigger time versus from the data. The whole flow in the previous lesson started with an event: a reading arrived, validation was triggered, the chain ran. Some of the measurement network's jobs have no such event. The nightly batch invoicing, the daily reconciliation, and the weekly meter health scan depend on no reading's arrival; the only thing that starts them is the clock. Scheduled work is not new by itself; missed-run and overlap policies were measured earlier and that comparison is not repeated here. This lesson's question is narrow: what changes when the clock is held by **a service**? #### The Handed-Over Decision and the Kind of Guarantee In a self-run scheduler, the clock is a specific process running on a specific machine. If that machine dies, no trigger fires; if two machines run at once, the trigger fires twice. Both are visible, both are fixable — because whatever holds the clock is in your own hands. In a scheduler taken as a service, this responsibility is handed over. The constraint that comes back in exchange fits one sentence, and it is this lesson's axis: **the guarantee is at least once.** The service undertakes that the trigger will happen, not that it will happen exactly once. Two triggers can fire in the same minute; this is not a malfunction, it is declared behavior. This is why the job being idempotent is not a good practice here — it is the service's precondition. What is handed over is countable: keeping the process that holds the clock alive, keeping that process single-instance, not losing the schedule when the machine changes, keeping a record of triggers. What remains is again a small surface — schedule expression, time zone, target, the input passed to the target, and the retry policy applied if triggering fails: five items. This is the same size of surface as the queue configuration in the earlier lessons, the same kind of constraint. The time zone item is a case on its own. When a schedule is written against local time, two nights a year are not 24 hours long — one is 23, the other 25. A once-a-day job can run twice on one of those nights and not at all on the other. So this measurement's two flaws — double runs and missed runs — can also come from the time zone, with nothing to do with the trigger's own accuracy. The second constraint runs the other way. If the target is not standing at trigger time — the function is throttled, at its account limit, mid-deployment — that run is **missed and not compensated for.** The scheduler only knows "now"; it does not reproduce a past moment. The two constraints together form an odd pair: the same job sometimes runs twice, sometimes not at all. The measurement runs on the fictional regional measurement network's nightly batch invoicing job. - **ED19.** A thirty-night run; 1,200 subscribers are invoiced every night, expected total 36,000 invoices. - **ED20.** In one of eleven nights the trigger arrives twice in the same minute, in one of thirteen it does not arrive at all; the days are fixed, there is no randomness in the model. - **ED21.** The idempotency key is derived from period plus subscriber ID; if the key already exists, the second write is skipped. - **ED22.** The trigger moment drifts between −60 and +60 units from the intended moment; the drift is fixed per day. - **ED23.** A day is 1,440 units and readings are spread evenly across the day. - **ED24.** The model does not build the internal locking of overlapping runs; only the invoices written and the readings processed are counted. #### Two Triggers in the Same Minute ```js // triggering.mjs — scheduled triggering model: at-least-once guarantee and a missed run // Model: not a real scheduler, but the counted effect of the guarantee type on invoice count. const DAYS = 30, SUBSCRIBERS = 1200; const DOUBLE = (g) => g % 11 === 4; // on these nights the trigger arrived twice in the same minute const MISSED = (g) => g % 13 === 9; // on these nights the trigger never arrived function run(hasKey) { const written = new Set(); let invoices = 0, duplicate = 0, missing = 0, triggers = 0; for (let g = 0; g < DAYS; g++) { if (MISSED(g)) { missing += SUBSCRIBERS; continue; } // no compensation: the day never comes back for (let t = 0; t < (DOUBLE(g) ? 2 : 1); t++) { triggers++; for (let a = 0; a < SUBSCRIBERS; a++) { const key = `p${g}:s${a}`; // idempotency key: period + subscriber if (hasKey && written.has(key)) continue; written.add(key); invoices++; if (t > 0) duplicate++; } } } return { triggers, invoices, duplicate, missing }; } console.log("idempotency key".padEnd(21) + "triggers".padStart(11) + "invoices written".padStart(19) + "duplicate".padStart(11) + "missing".padStart(9)); for (const [name, v] of [["no", false], ["yes", true]]) { const s = run(v); console.log(name.padEnd(21) + String(s.triggers).padStart(11) + String(s.invoices).padStart(19) + String(s.duplicate).padStart(11) + String(s.missing).padStart(9)); } console.log(`days: ${DAYS}, subscribers: ${SUBSCRIBERS}, expected invoices: ${DAYS * SUBSCRIBERS}`); ``` ```sh node triggering.mjs ``` ``` idempotency key triggers invoices written duplicate missing no 31 37200 3600 2400 yes 31 33600 0 2400 days: 30, subscribers: 1200, expected invoices: 36000 ``` The table processes the same thirty nights twice; the only difference is whether the idempotency key is checked while writing. 31 triggers arrived across the thirty nights. The number is not thirty, because two nights had no trigger at all and three nights had two each: 28 + 3. On its own this number looks harmless; its effect on invoicing is in the first row. Without an idempotency key, **3,600 duplicate invoices** were written. On each of three nights, 1,200 subscribers were billed twice. In the measurement network this means 3,600 subscribers got a second invoice for the same period, a second debt record was created, and — following the chain from earlier lessons — probably a second notification was sent as well. There is no error anywhere; both runs finished successfully. Alongside the duplicate invoices there is a second number: 3,600 extra function calls. Work per subscriber doubled over those three nights, twice the space was held from the concurrency pool, and the invoicing window stretched. And because the two runs started in the same minute, they overlap: two runs write to the same subscriber's record at once. If the write does not rest on a uniqueness constraint, both rows land in the table and both count as "successful." The cost of a double trigger is therefore not only the duplicate record but also a race at the record level. The second row shows what the key does. Trigger count is **still 31** — the key does not deduplicate the trigger, the service keeps sending the second trigger regardless. What changes is the write: if the key derived from period and subscriber ID has already been written, the second write is skipped and duplicate invoices drop to zero. This is the same distinction built in the delivery-semantics measurement — what gets deduplicated is not the delivery, it is the effect. What the key is derived from is decisive here. In the measurement, the key was built from period plus subscriber ID; the second trigger carried the same period, so it produced the same key and the write was skipped. If the key had been derived from the run's own ID — since every trigger arrives with its own ID — the two runs would have produced two different keys and the table would have fallen back to its first row. The key defines by which measure a repeat counts as "the same job"; in scheduled work this measure is always **the job's scope**, not the run itself. In the measurement network the scope is "which period, which subscriber"; the same period cannot be billed twice, because the second invoice is not a new fact, it is a copy of the first. #### The Column the Key Doesn't Close The table's last column is the same in both rows: **2,400 missing invoices.** The idempotency key does not touch this number, because the key only answers "was this job done before"; there is no such question to ask about a job that was never done at all. The missing column's real difficulty is how long it takes to notice. A duplicate invoice shows itself: a second notification, a doubled charge, a call to the support line. A missing invoice produces no symptom — one never generated bothers no one. 2,400 invoices are 6.7 percent of the thirty-day period, and the gap is only visible at end-of-period reconciliation against revenue expectations. The cost of the two missed nights here is therefore not just 2,400 invoices but also that they are cut a month late. A missed run is the other face of the service's guarantee, and fixing it is not the trigger's responsibility but the **job definition's**. The difference sits in one sentence: if the job is written as "invoice today," a missed day never comes back; if it is written as "invoice unbilled periods," the next night closes both days at once. The second form is also resilient to double triggering, because the second run finds no unbilled period. This form has two costs. First, the job must find its own scope: every run executes a "which periods went unprocessed" query, and it runs uselessly more often the more rarely a run is missed. Second, as missed days pile up, a single run's workload multiplies — two days missed back to back means the third night writes 3,600 invoices, which must finish within the function's time limit. Scope-based work turns a missed run into a time-limit problem. A third option is to mark, in a separate place, that a run happened, and build an auditor that looks for missing days. This makes the miss **visible**, but it is itself a scheduled job and is subject to the same guarantee; there is no third layer to catch the auditor's own missed run. The choice among the three is made by looking at the nature of the job. For work whose scope can be read from the data, like invoicing, the second form applies directly. For work with no scope — say, a job that warms a cache or deletes temporary files — a missed night has no counterpart to compensate for; there, a miss is not a defect but an accepted condition. The test for the distinction is one question: when a run is missed, is there a missing **record** left behind? If yes, scope-based writing is mandatory; if not, the scheduler's guarantee is accepted as is. #### Clock Drift and Where the Window Comes From Triggering also has a moment. The service says "at a given minute every night," but the moment that actually happens drifts around that minute: queue wait, scaling delay, clock synchronization difference. The drift is small; the problem is not the size of the drift but **where the job's window comes from.** The run below processes the same thirty nights with two window definitions: one counts the window backward from the trigger moment, the other starts it from where the previous run left off. The drift is the same in both. ```js // drift.mjs — clock drift: is the work window built on the trigger moment, or on the data const DAYS = 30, DAILY = 1200, DAY_MINUTES = 1440; const drift = (g) => ((g * 37) % 121) - 60; // -60..+60 units, fixed per day const trigger = (g) => (g + 1) * DAY_MINUTES + drift(g); const READINGS = []; // readings are spread evenly across the day for (let g = 0; g < DAYS; g++) for (let i = 0; i < DAILY; i++) READINGS.push(g * DAY_MINUTES + Math.floor((i * DAY_MINUTES) / DAILY)); function run(scoped) { const counter = new Map(READINGS.map((z, i) => [i, 0])); let last = 0; for (let g = 0; g < DAYS; g++) { const T = trigger(g); const start = scoped ? last : T - DAY_MINUTES; // scoped: from the last processed moment READINGS.forEach((z, i) => { if (z >= start && z < T) counter.set(i, counter.get(i) + 1); }); last = T; } let never = 0, double = 0; for (const n of counter.values()) { if (n === 0) never++; else if (n > 1) double += n - 1; } return { never, double }; } console.log("window".padEnd(22) + "never processed".padStart(17) + "double processed".padStart(19)); for (const [name, k] of [["from trigger time", false], ["from last processed", true]]) { const s = run(k); console.log(name.padEnd(22) + String(s.never).padStart(17) + String(s.double).padStart(19)); } console.log(`readings: ${READINGS.length}, drift range: -60..+60 units`); ``` ```sh node drift.mjs ``` ``` window never processed double processed from trigger time 648 560 from last processed 0 0 readings: 36000, drift range: -60..+60 units ``` When the window is counted backward from the trigger moment, **648 of the 36,000 readings were never processed** and 560 were **processed twice**. No error happened and no trigger was missed; every night the job ran and finished successfully. The only thing that drifted was the window's two edges. When one night triggers early and the next triggers late, the slice between them falls inside both windows; the reverse leaves it inside neither. 648 readings is 1.8 percent of 36,000, and the ratio is misleading — the loss is not spread at random. The missed slice always sits at the day boundary, so it always lands on meters that report at the same fixed times. In a fleet reporting on a fixed schedule, this means the same subset of subscribers goes unbilled every month — not a 1.8 percent loss overall, but close to a 100 percent loss for a handful of subscribers. The 560 double-processed readings sit in the same subset, and the two errors hide each other: an audit of total consumption sees the 648 missing and 560 extra partly cancel out and reads the deviation as small. The second row produces zero with the same drift. The difference is one line: the window starts not at `T - 1440` but from the **last processed moment**. The window is now derived not from the clock but from the job's own progress; drift can still move where the window ends, but its start is wherever the previous run left off, so neither a gap nor an overlap remains. From here the lesson's rule follows: scheduled triggering should say **when the job starts**, not **what it processes**. What it processes should be read from the data. Once this distinction is set, three constraints close at once — a double trigger finds no new work, a missed run gets closed by the next run, and clock drift does not break the window. The cost is that every run has to write its own progress marker permanently, and that write has to commit at the same boundary as the work; if the marker is written before the work, a missed slice is silently skipped, and if written after, the same slice is processed twice. #### Summary - Once scheduled triggering is taken as a service, the guarantee is at least once: two triggers can fire in the same minute, and this is declared behavior, not a malfunction. - Across thirty nights, 31 triggers arrived; without an idempotency key, 3,600 duplicate invoices were written and both runs finished without error. - With the key added, trigger count stayed at 31 but duplicate invoices dropped to zero: what gets deduplicated is not the trigger, it is the write. - A missed run is not compensated for; 2,400 missing invoices stayed the same in both rows. What closes this is not the key but writing the job as "invoice unbilled periods" instead of "invoice today." - With the window derived from trigger time, 648 of 36,000 readings were never processed and 560 were processed twice; deriving it from the last processed moment turned both numbers to zero under the same drift. The loss is not random either — the missed slice always sits at the day boundary, landing on meters that report at the same time, and an audit of the total sees the 648 missing and 560 extra partly cancel out. - Scheduled triggering says when a job starts; reading what it processes from the data closes double triggering, missed runs, and clock drift all at once. #### Next Step Up to this point, everything that woke a function came from inside: a rule, a queue, a flow step, a clock. The measurement network also has an outward-facing side — the field crew's app, the branch screens, and the interface subscribers use to query their own consumption. These requests cannot reach the function directly; a layer that authenticates identity, maps the route, and throttles stands in between. The next lesson measures that layer as a service: the number of configuration items, the cost per request, the effect of doing authorization at the gateway on function call count, and what the gateway does and does not do about cold start. ### API Gateway Service Source: https://academia.sh/en/courses/serverless/api-gateway-service The numbers behind taking the gateway as a service: the configuration item count of a twelve-endpoint surface, the effect of doing authorization at the gateway on function call count and the break-even cache hit rate, showing by scanning the timeout threshold that the gateway does not hide cold start, and that a timed-out request keeps completing in the function. Up to this point, everything that woke a function came from inside: a rule, a queue, a flow step, a clock. The measurement network also has an outward-facing side — the field crew's app, the branch screens, and the interface subscribers use to query their own consumption. These requests cannot reach the function directly; a layer that maps the route, authenticates identity, and throttles stands in between. The gateway's jobs were measured earlier; route mapping, authentication, and throttling were the subject of those lessons and this measurement does not repeat them. This lesson's question is again narrow: when the gateway is **a service**, which decision is handed over and what number comes back in exchange? #### The Configuration Surface The handed-over decisions are familiar: the machines that run the gateway itself, their scale, certificate renewal, version upgrades. The constraint that comes back this time is not a limit but a **volume.** In the gateway service, every endpoint carries its own configuration set: route pattern, method, integration target, authorizer, request mapping, throttle limit, cache key, logging. Eight items, per endpoint. The measurement network's outward-facing surface is twelve endpoints. Total configuration items: 12 × 8 + 6 = **102**. This number is not a cost but a surface: each of the 102 places can be misconfigured, and none of them show up in function code. It is the same kind of surface as the rule, the flow definition, and the schedule expression in the earlier lessons; the difference is its size. What makes the surface dangerous is that its items are **independent of each other.** When a thirteenth endpoint is added, eight more items open up, and one of them — the authorizer — can be left empty. An empty authorizer produces no error; the endpoint works, a response comes back, metrics stay green. The only difference is that endpoint asks for no identity. In the previous lesson the rule's invisibility was a surface; here that same invisibility touches an access decision directly. The way around the constraint is to generate endpoint definitions from a single template and make the authorizer field mandatory; the cost is that the gateway configuration can no longer be hand-edited, and every change goes through the release pipeline. - **ED25.** The daily request mix is 50,000 and comes from three fictional sources: field app 12,000, branch screen 18,000, subscriber portal 20,000. - **ED26.** The reason for rejection is authorization — an expired token, a missing scope, a tokenless request; rejection rates range from 0.03 to 0.31 depending on the source. - **ED27.** The cost is a fictional unit: 0.002 gateway share per request, 0.010 per function call. - **ED28.** Configuration count is 12 endpoints × 8 items plus 6 gateway-level items. - **ED29.** Cold start's extra time is 90 units, the gateway's own share is 4 units; cold calls are distributed at fixed intervals. - **ED30.** A client retry counts as a single occurrence; throttling and response caching are not measured. #### Where Authorization Happens A request rejected at the gateway never wakes the function. This sentence is true, but on its own it is incomplete — the rejection itself requires an operation, and where that operation happens changes the number. The model below runs the same daily request mix through four arrangements and counts how many function calls happen in each. ```js // gateway.mjs — API gateway service model: where authorization happens, how often the function wakes // Model: not a real gateway, but the counted effect of authorization's location on call count. const SOURCES = [ // daily request mix from the fictional measurement network { name: "field app", requests: 12000, rejectRate: 0.06 }, { name: "branch screen", requests: 18000, rejectRate: 0.03 }, { name: "subscriber portal", requests: 20000, rejectRate: 0.31 }, ]; const REQUESTS = SOURCES.reduce((t, k) => t + k.requests, 0); const REJECTED = SOURCES.reduce((t, k) => t + Math.round(k.requests * k.rejectRate), 0); const PASSED = REQUESTS - REJECTED; const CACHE_HIT = 0.85; // hit rate of the authorizer's cached result const GATEWAY_RATE = 0.002, CALL_RATE = 0.010; // fictional unit: per request and per call const arrangements = [ ["authorization in function", REQUESTS, 0], ["at gateway, uncached", PASSED, REQUESTS], ["at gateway, cached", PASSED, Math.round(REQUESTS * (1 - CACHE_HIT))], ["at gateway, built-in validation", PASSED, 0], ]; console.log("arrangement".padEnd(34) + "work calls".padStart(12) + "authorizer".padStart(16) + "total calls".padStart(14) + "cost".padStart(9)); for (const [name, work, authorizer] of arrangements) { const cost = REQUESTS * GATEWAY_RATE + (work + authorizer) * CALL_RATE; console.log(name.padEnd(34) + String(work).padStart(12) + String(authorizer).padStart(16) + String(work + authorizer).padStart(14) + cost.toFixed(1).padStart(9)); } console.log(`requests: ${REQUESTS}, rejected: ${REJECTED}, passed: ${PASSED}`); console.log(`configuration: 12 endpoints x 8 items + gateway-level 6 items = ${12 * 8 + 6}`); ``` ```sh node gateway.mjs ``` ``` arrangement work calls authorizer total calls cost authorization in function 50000 0 50000 600.0 at gateway, uncached 42540 50000 92540 1025.4 at gateway, cached 42540 7500 50040 600.4 at gateway, built-in validation 42540 0 42540 525.4 requests: 50000, rejected: 7460, passed: 42540 configuration: 12 endpoints x 8 items + gateway-level 6 items = 102 ``` The first row is authorization done in the function: all 50,000 requests wake it. Of those, **7,460 wake it only to say "no"** — it validates the token, finds it invalid, and returns. No work was done, but a call was made; a slot from the concurrency pool was taken, a chance of cold start was risked. The second row shows the opposite of what one might expect. When authorization moves to the gateway, function calls fall to 42,540, but because the authorizer function the gateway calls runs once per request, total calls climb to **92,540** and cost to 1,025.4 units. Taking rejection out of the function does not lower the call count if the thing doing the rejecting is itself a function — it raises it. The third row is the break-even point: once the authorizer's result is cached at an 85 percent hit rate, total calls come to 50,040 and cost to 600.4 units — almost the same as the first row. The break-even hit rate follows from a simple equation: 50,000 × (1 − h) = 7,460, so h = **85.1 percent.** Below this rate, authorizing at the gateway is expensive; above it, it is cheap, and the gain scales directly with the hit rate. The fourth row shows where the real gain sits: when the gateway can **validate the token itself**, no function sits in between at all. Total calls come to 42,540, cost to 525.4 — 12.4 percent lower than the first row. The measurement's result fits one sentence: "a request rejected at the gateway never wakes the function" is a gain only if the rejection itself is not a function call. The distribution of rejected requests is a finding on its own. Of the 7,460 rejections, **6,200** come from a single source, the subscriber portal, where the rate is 31 percent because browser sessions are short and tokens expire often. The remaining two sources together produce 1,260 rejections. So 83.1 percent of rejections come from a single client's token-refresh behavior. This is not something the gateway configuration solves; the count drops on its own once the client refreshes its token before it expires. The lesson here is that lowering call count is not always a gateway-side setting. A detail of the cost is hidden in these same rows. The gateway's own share is the same in all four arrangements: 50,000 × 0.002 = 100 units. The 7,460 rejected requests, even though they never wake the function, still reach the gateway and are billed 14.9 units. Rejecting at the gateway prevents the call, not the request; a flood of unauthorized requests generates a bill without running a single function, and what stops it is not the gateway itself but the throttle limit in front of it. There is a further gain beyond call count that does not show up in the numbers. In three of the four arrangements, the work function never sees an unauthorized request at all; authorization code sits in one place, the same way for all twelve endpoints. In the first arrangement, the same check is written separately in twelve functions, and being forgotten in one of them is a silent gap. This is what the 102-item configuration surface buys in exchange. In the fourth arrangement, the gateway's own share is 100 out of 525.4, or 19 percent of total cost. This ratio is independent of how much work the request does: the gateway counts requests, not duration. A long-running report query sees the gateway's share shrink to a small slice; a short endpoint returning a handful of fields can see the gateway cost more than the function itself. The finer-grained the endpoint, the larger the gateway's share grows, and this is the invisible cost of designing many small endpoints. Consolidating the same work into a single endpoint lowers the share; the cost is that authorization and throttle limits can no longer be separated per endpoint. #### The Gateway Does Not Hide Cold Start The gateway takes a request, routes it, and waits for the response. While waiting it can do only one thing: give up after a while. It cannot hold the request until the function warms up, nor send it elsewhere until it does. So cold start's delay passes straight through to the client, and the gateway's timeout threshold decides who gets the error. The scan below runs the same 42,540 calls through four different thresholds; each call's latency and whether it is cold is the same in every row — the only thing that changes is how long the gateway waits. ```js // timeout.mjs — gateway service does not hide cold start: the timeout threshold is scanned const SEED = 20260714; // generator seed: run is reproducible let state = SEED; const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648; const N = 42540, GATEWAY = 4, COLD_EXTRA = 90; // fictional unit: gateway's own share and cold extra const CALLS = []; for (let i = 0; i < N; i++) { const work = random() < 0.88 ? 10 + Math.floor(random() * 31) // ordinary query: 10-40 units : 60 + Math.floor(random() * 81); // report query: 60-140 units const cold = i % 47 === 3; // cold calls at fixed intervals CALLS.push({ latency: GATEWAY + work + (cold ? COLD_EXTRA : 0), cold }); } const COLD = CALLS.filter((c) => c.cold).length; console.log("timeout".padEnd(13) + "exceeding requests".padStart(20) + "of these, cold".padStart(17) + "if client retries".padStart(25)); for (const T of [60, 120, 200, 300]) { const exceeding = CALLS.filter((c) => c.latency > T); console.log(String(T).padEnd(13) + String(exceeding.length).padStart(20) + String(exceeding.filter((c) => c.cold).length).padStart(17) + String(N + exceeding.length).padStart(25)); } console.log(`calls: ${N}, cold calls: ${COLD}, gateway share: ${GATEWAY} units, seed: ${SEED}`); ``` ```sh node timeout.mjs ``` ``` timeout exceeding requests of these, cold if client retries 60 5929 906 48469 120 1864 455 44404 200 39 39 42579 300 0 0 42540 calls: 42540, cold calls: 906, gateway share: 4 units, seed: 20260714 ``` The bottom row looks reassuring: at a 300-unit timeout, no request exceeds it. The row above gives the real finding. At a 200-unit threshold, 39 requests timed out, and **all 39 of them were cold calls.** No warm call can ever reach that threshold, because the warm path's worst case is 144 units; the cold path's worst case is 234. When the threshold falls between the two, the exact set of clients that gets an error is the set that hit cold start. The gateway's own share of 4 units looks small, but where it shows up matters: it is invisible in the function's own duration metric. The function reports finishing in 30 units while the client waited 34; on a cold call the function still reports 30 while the client waited 124. This gap between what the two sides see is the gateway service's hidden constraint — when a latency complaint arrives, whoever is looking at function metrics finds no anomaly. The only way to see the gap is to put the gateway's own access log total side by side with the function's reported duration; merging the two sources is a separate job, and it is one of the costs of the gateway being a service. At a 60-unit threshold the table says something else: **all** 906 cold calls exceed it. Because cold start's own duration alone exceeds the threshold, how much work that call does no longer matters. Tightening the gateway's timeout turns cold start from a latency problem into an **error** problem. The last column is the second half of the cost. Timeout is the gateway's decision; the function knows nothing about it and keeps running regardless. When the client gets an error and retries, at a 60-unit threshold call count climbs from 42,540 to 48,469 — 13.9 percent more calls. Worse, the first call already finished its work and wrote it; the second call now lands on a warm instance, finishes fast, and performs the same write a second time. This is the duplicate-effect problem measured in the previous two lessons, showing up on the gateway's surface, and the fix is the same: the write has to rest on an idempotency key. Counting what the gateway cannot do here draws its boundary. The gateway does not queue the request — a queue service is a separate thing, and routing through it would mean converting the request to asynchronous first. The gateway does not retry on its own; if it did, it would stack a second call on top of the one that already timed out, and the 48,469 figure in the last column would come from the gateway instead of the client. It does not hold the response until the function warms up, because it does not know when any function will warm. What it does is three things: map the route, validate the token, cut off the wait. Everything outside these three is built either in the client or in the function; the gateway being a service does not widen this boundary, it only lifts the burden of running these three off your hands. There are three ways around the constraint. Raising the timeout zeroes out the 39 errors but keeps the connection open longer and delays noticing a truly stuck request. Keeping the function warm lowers cold-call count but produces idle calls; the cost of 906 cold calls has to be weighed against the cost of keeping it warm. The third is to convert heavy endpoints to asynchronous: the request is written to a queue, the gateway responds immediately, and the result is queried separately — its cost is a two-step flow on the client side and the question of where the result is held. In the measurement network the choice is made by endpoint type. The branch screen's consumption query is short and takes at most 44 units on the warm path; a 300-unit threshold there is more than enough. The report query can reach 234 units on the cold path, and having the client wait that long does not make sense; that endpoint is converted to asynchronous. Same gateway, same configuration surface, two different timeout values. This is also why the 102 items are 102 — items are kept separate per endpoint, because one threshold cannot be right for every endpoint. #### Summary - The gateway service carries 102 configuration items across a twelve-endpoint surface; none of them show up in function code. - With authorization done in the function, all 50,000 requests woke it and 7,460 calls ran only to reject. - Moving authorization to the gateway, with the rejection done by an authorizer function, pushed total calls to 92,540; an 85 percent cache hit rate brought it back to 50,040 and matched the first arrangement's cost. The break-even hit rate is 85.1 percent. - The gateway validating the token itself is the only arrangement with a net gain: 42,540 calls, 12.4 percent lower cost. "A request rejected at the gateway never wakes the function" is a gain only if the rejection is not itself a function call. - At a 200-unit timeout, all 39 exceeding requests were cold calls; at 60 units, all 906 cold calls exceeded it. The gateway does not hide cold start, it only turns it from a delay into an error. - On a timed-out request the function keeps running; when the client retries, call count climbs to 48,469 and the same write happens a second time. #### Next Step The gateway brings outside requests in. The measurement network also has an outward flow it **sends**: a work notification to the field crew when validation fails, a message to a subscriber when an invoice is cut. These sends are also handed to a service, and that service's constraint resembles nothing seen so far — the send's success is not in the sender's hands, it depends on a decision the receiving side makes. The next lesson measures that constraint: a bounced message and a complaint come back as an event, and what happens if this feedback stream goes unprocessed. ### Notification and Email Services Source: https://academia.sh/en/courses/serverless/notification-and-email-services The constraint that comes from handing sending to a service: bounced messages and complaints coming back as events, scanning this feedback stream's processing rate to measure the reputation score at day thirty, the number of throttled days, and the number of messages delivered, and comparing the cost of processing feedback against the delivery cost of not processing it. The gateway brought incoming requests in. The measurement network also has an outward flow it **sends**: a work notification to the field crew when validation fails, a message to a subscriber when an invoice is cut, an alert to the branch supervisor on a meter fault. These sends, too, are handed to a service. Deliverability itself was measured earlier and that measurement is not repeated here. This lesson's question is: which constraint does handing sending to a service bring back? The answer comes from a place unlike any constraint seen so far — **the send's success is not in the sender's hands.** #### The Handed-Over Decision and What Was Never in Your Hands What is handed over is familiar: running the sending server, keeping address records, managing the retry queue, renewing domain verification records. There is one thing that is not handed over, because it was never in anyone's hands to begin with: the receiving side's decision. A message gets accepted, bounced, silently dropped into a folder, or marked as a complaint; the party making that decision is not the sender. The service measures this decision and reflects it back to the sender as a **reputation**. The constraint that comes back follows from this: sending quota is bound to reputation. As reputation falls, quota falls; as quota falls, the number of messages that can be sent falls. In every lesson so far the constraint was a setting — visibility timeout, maximum deliveries, timeout threshold. Here the constraint is **an earned number**, and it cannot be tuned directly; it can only be raised through behavior. The constraint also has a starting condition. A new sending stream has no history, so it has no reputation either; the service does not open up volume from the start, it opens it as history accumulates. The practical result is that sending capacity **cannot scale instantly.** In every service so far, scale arrived the moment it was demanded — queue depth rose and an instance was added, request count rose and the gateway kept up. Sending does not work this way: going from 8,000 messages a day to 80,000 is not a technical limit but an unearned permit. If the measurement network plans to bring a new region online, sending volume has to be ramped up gradually, weeks ahead of that date. The way the service communicates this decision is also familiar. Bounces and complaints come back to the sender as **events**, written to an event bus or a topic just like the events in this topic's first lesson. A direct consequence follows: if there is no rule to receive those events, they are dropped without producing any error. The 433 silent drops measured in the first lesson turn, here, into a list-hygiene problem. The measurement runs on the fictional regional measurement network's field notifications; the addresses are fictional names in the form `example.invalid`. - **ED31.** The list is 12,000 recipients; 480 are invalid from the start (closed account, bad domain). - **ED32.** The daily base quota is 8,000 messages; the actual quota is proportional to reputation, and once reputation drops below 60 it is halved on top of that. - **ED33.** Reputation starts at 100; if the daily bounce rate exceeds 0.02 it drops 6 points, if complaint rate exceeds 0.001 it drops 3 points, and it rises 2 points if both are below threshold. - **ED34.** A complaining recipient stays on the list if not removed, and complains again on the next send. - **ED35.** Feedback processing rate is how much of a day's bounce and complaint events actually gets reflected onto the list. - **ED36.** The model builds a single sending stream; content, subject line, and time-of-send effects are not measured. There is no randomness in the model. #### The Link Between Processing Rate and Delivery ```js // reputation.mjs — notification service model: what happens to delivery if feedback goes unprocessed // Model: not a real send, but the link between feedback processing rate and delivery. const DAYS = 30, LIST = 12000, BASE = 8000; const INVALID0 = 480; // closed or bad address (fictional list) function run(p) { // p: feedback processing rate let invalid = INVALID0, complainers = 0, reputation = 100; let delivered = 0, bouncedTotal = 0, complaintsTotal = 0, throttledDays = 0; for (let g = 0; g < DAYS; g++) { const throttled = reputation < 60; // sending is throttled once reputation drops below the threshold const quota = Math.round(BASE * (reputation / 100) * (throttled ? 0.5 : 1)); if (throttled) throttledDays++; const sent = Math.min(quota, LIST); const bounced = Math.round(sent * (invalid / LIST)); const complaints = Math.round(sent * (0.0006 + complainers / LIST)); delivered += sent - bounced; bouncedTotal += bounced; complaintsTotal += complaints; const bounceRate = bounced / sent, complaintRate = complaints / sent; if (bounceRate > 0.02) reputation -= 6; if (complaintRate > 0.001) reputation -= 3; if (bounceRate < 0.01 && complaintRate < 0.0005) reputation += 2; reputation = Math.max(0, Math.min(100, reputation)); invalid = Math.max(0, invalid - Math.round(bounced * p)); // processed feedback complainers = Math.max(0, complainers + Math.round(sent * 0.0006) - Math.round(complaints * p)); } return { reputation, delivered, bouncedTotal, complaintsTotal, throttledDays }; } console.log("processing rate".padEnd(16) + "day-30 reputation".padStart(19) + "delivered".padStart(11) + "bounced".padStart(12) + "complaints".padStart(12) + "throttled days".padStart(16)); for (const p of [0, 0.25, 0.5, 0.9, 1]) { const s = run(p); console.log(p.toFixed(2).padEnd(16) + String(s.reputation).padStart(19) + String(s.delivered).padStart(11) + String(s.bouncedTotal).padStart(12) + String(s.complaintsTotal).padStart(12) + String(s.throttledDays).padStart(16)); } console.log(`days: ${DAYS}, list: ${LIST}, daily base quota: ${BASE}, ` + `invalid addresses at start: ${INVALID0}`); ``` ```sh node reputation.mjs ``` ``` processing rate day-30 reputation delivered bounced complaints throttled days 0.00 0 40743 1697 65 25 0.25 7 62990 1450 84 24 0.50 88 211687 953 207 0 0.90 94 225548 532 150 0 1.00 94 225600 480 150 0 days: 30, list: 12000, daily base quota: 8000, invalid addresses at start: 480 ``` The first row is the case where feedback is never processed. By day thirty, reputation is **zero**, sending was throttled on **25 of the thirty days**, and total delivered stayed at 40,743. Nothing broke: the send request was accepted every day, the service returned no error, and bounce and complaint events were published correctly. Nobody just read them. The last row is the same list, the same content, the same thirty days, at the opposite end. Feedback fully processed, reputation stays at 94, no day is throttled, and delivered climbs to **225,600.** The gap between the two is 184,857 messages — 5.5 times as many. The bounced column explains the mechanism. With full processing, total bounces are **480** — exactly the count of invalid addresses in the list: each invalid address bounces once, is processed, is removed, and is never tried again. Without processing, the same 480 addresses get retried for thirty days and produce 1,697 bounces. Bounce rate never falls, stays above threshold, and drags reputation down every single day. The throttled-days column shows how this decline accelerates. The moment reputation drops below 60, quota is halved on top of the drop; in the first row, 25 of the thirty days are in this state. Throttling does not just cut delivery — it also thins the feedback stream, so finding the addresses that need cleaning slows down too. The constraint delays its own recovery. The complaint column looks backward at first glance: as processing rate rises, complaint count climbs from 65 to 150. The reason is the denominator — in the unprocessed run, quota has already collapsed, so very few messages get sent at all. Complaint **rate** falls with processing, its count rises because volume rises. A gauge looking at raw counts reads this exactly backward. #### Where the Threshold Sits The real finding sits between the second and third rows. At a processing rate of 0.25, reputation is 7, delivered is 62,990; at 0.50, reputation is 88, delivered is 211,687. The relationship is not linear — doubling the processing rate more than triples delivered. The reason is threshold logic: the moment bounce rate drops below 0.02, the penalty stops and reputation starts rebuilding on its own. Enough cleanup to cross this threshold is all it takes; everything after that comes for free. This is why "we process half of our feedback" is not a shortfall, it is a question of which side of the threshold you land on. Somewhere between 0.25 and 0.50, the system settles into one of two stable states: either the rate falls below the threshold and delivery climbs to its ceiling, or it stays above it and quota collapses to the floor. There is no middle ground, because a collapsed quota also lowers feedback volume, and the addresses that need cleaning get found even slower. #### Processing Feedback Incorrectly The first measurement showed the benefit of processing, but it assumed a single kind of processing: a bounced address gets removed from the list. Real feedback is not that uniform. Part of it is permanent — no such address, no such domain, account closed. Part of it is transient: mailbox full, receiving server busy, a temporary limit hit. Both often arrive from the same event stream, in similar-looking shapes. ```js // bounce-type.mjs — if the feedback type is not distinguished, who gets removed from the list const DAYS = 30, LIST = 12000, QUOTA = 8000; const PERMANENT0 = 480; // permanent error: no such address, no such domain const TRANSIENT = 0.03; // transient error: mailbox full, receiving server busy function run(typed) { let valid = LIST - PERMANENT0, invalid = PERMANENT0, wronglyRemoved = 0, delivered = 0; for (let g = 0; g < DAYS; g++) { const list = valid + invalid; const sent = Math.min(QUOTA, list); const permanent = Math.round(sent * (invalid / list)); const transient = Math.round((sent - permanent) * TRANSIENT); delivered += sent - permanent - transient; invalid = Math.max(0, invalid - permanent); // permanent errors are removed in both modes if (!typed) { valid -= transient; wronglyRemoved += transient; } // if transient errors are removed too } return { list: valid + invalid, wronglyRemoved, delivered }; } console.log("mode".padEnd(24) + "day-30 list".padStart(15) + "wrongly removed".padStart(17) + "delivered".padStart(11)); for (const [name, v] of [["no type distinction", false], ["only permanent removed", true]]) { const s = run(v); console.log(name.padEnd(24) + String(s.list).padStart(15) + String(s.wronglyRemoved).padStart(17) + String(s.delivered).padStart(11)); } console.log(`days: ${DAYS}, starting list: ${LIST}, daily quota: ${QUOTA}`); ``` ```sh node bounce-type.mjs ``` ``` mode day-30 list wrongly removed delivered no type distinction 5025 6495 210020 only permanent removed 11520 0 232334 days: 30, starting list: 12000, daily quota: 8000 ``` Without distinguishing type, **6,495 valid recipients** were removed from the list over thirty days and the list fell from 12,000 to 5,025. Nothing was wrong with the ones removed; their mailbox just happened to be full that day. In the second row the list stays at 11,520 — the starting list with its 480 permanent-invalid addresses removed. The flaw here runs opposite to the first measurement's: there, feedback was not processed and reputation dropped; here, it is processed, more than enough, and delivered still drops — 210,020 instead of 232,334. Processing the feedback stream by itself is not enough; the event's **type** has to be read. The gap sits in two lines of the function that reads the event, and those two lines cost 6,495 recipients. Getting the line between permanent and transient wrong points the same way as not processing the stream at all. #### The Cost of Processing, the Price of Not Processing the feedback stream is one of the cheapest jobs in this topic. In the fully processed run, thirty days bring in 480 bounce events and 150 complaint events: 630 total. If each event is processed with a single function call, at the call cost from the previous lesson, thirty days' cost is 630 × 0.010 = **6.3 fictional units.** Not processing it costs 184,857 undelivered messages. Given a ratio this stark, it is common for this stream to go unprocessed, and the reason is written in the first lesson's measurement. Feedback events are written to a topic; if that topic has no subscriber, the events drop silently. No error counter rises, no threshold is crossed — a topic with no subscriber looks like an empty queue, healthy. The symptom only shows up thirty days later, in a falling delivery count, and by then the cause read out is not the sending service but something like "the messages are not going through." How this flow should be built also follows from pieces already measured in this topic. Feedback events are written to a topic, the topic's subscription is bound to a queue, the queue wakes a function, and the function writes to the list. The queue's visibility timeout can stay short for this job, because writing to the list takes only a few units. That the same bounce event can be delivered twice is the second lesson's result; removing an address from the list is already an idempotent operation, but if a complaint counter is kept, that counter needs to rest on an idempotency key just like the fourth lesson's. This is where the topic closes: the sending service's feedback stream is the single case in this course that uses all five of its lessons at once. There are ways around the constraint and each carries its own countable cost. Cleaning the list by hand substitutes for processing feedback but lags: a weekly cleanup lands in the 0.25-to-0.50 zone of the table. Splitting sending into two — transactional messages on one stream, bulk announcements on another — protects one's reputation from the other; its cost is two configuration surfaces and two feedback streams. Asking to have the quota raised does not remove the constraint at all: quota is the result of reputation, not its input. This is the strictest constraint counted in this topic — visibility timeout can be changed, a timeout threshold can be raised, a step limit can be redrawn; reputation can only be earned. #### Summary - Once sending is handed to a service, the decision that never comes back is the receiving side's; the service reflects that decision back as reputation, and quota is bound to it. This constraint is not tuned, it is earned. - A bounced message and a complaint come back to the sender as events; if there is no rule or subscription to receive them, they drop without producing any error. - With feedback never processed, sending was throttled on 25 of thirty days, reputation fell to zero, and delivered stayed at 40,743; fully processed, reputation stayed at 94, zero days were throttled, and delivered reached 225,600 — a gap of 184,857 messages. - With full processing, total bounces are 480, exactly the count of invalid addresses on the list; unprocessed, the same 480 addresses produce 1,697 bounces. The relationship is not linear either: raising processing rate from 0.25 to 0.50 raises delivered from 62,990 to 211,687, because the penalty stops the moment bounce rate drops below threshold. - Failing to read the event's type does the same damage in reverse: without distinguishing permanent from transient failures, 6,495 valid recipients were removed over thirty days and delivered fell from 232,334 to 210,020. - Processing 630 feedback events costs 6.3 fictional units; not processing them costs 184,857 undelivered messages. #### Next Step Throughout this topic, functions got linked to each other. Events were routed by rule, queues and topics stood in as a buffer, multi-step flows were handed to a state machine, clock-triggered jobs were made idempotent, incoming requests were passed through a gateway, and outbound messages were tied to a feedback stream. The same account repeated in every case: a decision was handed over, and a countable constraint came back in exchange. There is one property common to this whole arrangement, and it has never been asked so far: all of it runs **in a single region.** The rule engine, the queue, the orchestrator, the scheduler, the gateway, and the sending service all sit in the same place; a query from the branch screen and a request from the field app both go to that one place. The only thing close to the user was the cache — and a cache can only hold an answer already computed. The next topic takes on that limit: what is gained when the computation itself moves closer to the user, and which runtime constraints get accepted in exchange. ## Edge Computing Moving computation to the user: the edge runtime model with isolates, request processing and routing at the edge, edge storage with key-value and object store connections, durable objects that hold coordinated state at the edge, moving asynchronous work to the edge and its duration limit, and how duration, memory, and compatibility constraints shape design. ### Edge Runtime Model Source: https://academia.sh/en/courses/serverless/edge-runtime-model The isolate is compared to the container and the virtual machine on the same metrics: startup latency and memory baseline by order of magnitude, the offered interface surface item by item. Cold start's disappearance is counted, along with the seven missing items it comes with and the cost of working around them. Events were routed, streams were set up, long-running jobs ran step by step. But all of this machinery still sits in a single region. In the fictional **measurement network**, field crews and branch sites are scattered across the region; the only thing close to the user was the cache, and the cache only stores a response already produced. Whenever a new response has to be produced, the request still goes to the region. This course moves the computation itself to the edge. The first question is: what runs at the edge? What runs in the region is a container, and the container's cost has been measured. The same cost cannot be paid at hundreds of edge locations, for dozens of different applications at each one. The thing that runs at the edge is named the **isolate**, and this lesson adds it to the familiar scale. **EC1.** Every number is a model input, not the output of a measured setup; what matters is the order-of-magnitude gap between two tiers. **EC2.** Interface items take a binary value: offered or not offered; an item offered only in restricted form counts as "not offered." **EC3.** The startup and memory values on the container and virtual machine rows are inputs from the earlier measurement; this lesson does not re-measure them, it only adds the third row. #### The Row at the Edge of the Scale Isolation tiers were compared earlier on five metrics and their cost was paid in three items. The isolate is one step further down that scale: it carries no kernel of its own, and no file system of its own either. What it isolates is a single language's **interpreter context** — the variable space, the stack, and the object set are separate, but the process underneath is shared. The new metric is therefore the width of the **offered interface surface**: how many items the runtime offers to the application. ```js // measurement-network/unit.mjs — adding the isolate to the three-tier scale // MODEL: every number is a declared model input. What matters is the // order-of-magnitude gap between two tiers, not a single cell's value. export const LEVELS = { "virtual machine": { startupMs: 25_000, memoryBaselineMB: 512 }, "container": { startupMs: 300, memoryBaselineMB: 12 }, "isolate": { startupMs: 3, memoryBaselineMB: 3 }, }; // The interface items the runtime offers to the application, and at which tier. export const SURFACE = { "request/response handling": ["virtual machine", "container", "isolate"], "timers": ["virtual machine", "container", "isolate"], "outbound network request": ["virtual machine", "container", "isolate"], "crypto primitives": ["virtual machine", "container", "isolate"], "stream processing": ["virtual machine", "container", "isolate"], "in-memory data structure": ["virtual machine", "container", "isolate"], "file system": ["virtual machine", "container"], "local disk persistence": ["virtual machine", "container"], "long-running process": ["virtual machine", "container"], "subprocess spawning": ["virtual machine", "container"], "raw sockets": ["virtual machine", "container"], "native binary addon": ["virtual machine", "container"], "kernel tuning": ["virtual machine"], }; export const NAMES = Object.keys(LEVELS); export const ITEMS = Object.keys(SURFACE); export const offered = (d) => ITEMS.filter((k) => SURFACE[k].includes(d)); const magnitude = (n) => "10^" + Math.round(Math.log10(n)); if (import.meta.main) { console.log("cost and interface surface (model input)"); console.log("tier".padEnd(17) + "startup".padEnd(11) + "magnitude".padEnd(11) + "mem baseline".padEnd(15) + "magnitude".padEnd(11) + "surface"); for (const d of NAMES) { const v = LEVELS[d]; console.log(d.padEnd(17) + `${v.startupMs} ms`.padEnd(11) + magnitude(v.startupMs).padEnd(11) + `${v.memoryBaselineMB} MB`.padEnd(15) + magnitude(v.memoryBaselineMB).padEnd(11) + `${offered(d).length}/${ITEMS.length}`); } console.log("\ninterface item (+ offered, - not offered)"); console.log("item".padEnd(28) + NAMES.map((d) => d.padEnd(17)).join("")); for (const k of ITEMS) console.log(k.padEnd(28) + NAMES.map((d) => (SURFACE[k].includes(d) ? "+" : "-").padEnd(17)).join("")); const missing = ITEMS.filter((k) => !SURFACE[k].includes("isolate")); console.log(`\n${missing.length} items missing in the isolate:`); for (const k of missing) console.log(" - " + k); } ``` ``` cost and interface surface (model input) tier startup magnitude mem baseline magnitude surface virtual machine 25000 ms 10^4 512 MB 10^3 13/13 container 300 ms 10^2 12 MB 10^1 12/13 isolate 3 ms 10^0 3 MB 10^0 6/13 interface item (+ offered, - not offered) item virtual machine container isolate request/response handling + + + timers + + + outbound network request + + + crypto primitives + + + stream processing + + + in-memory data structure + + + file system + + - local disk persistence + + - long-running process + + - subprocess spawning + + - raw sockets + + - native binary addon + + - kernel tuning + - - 7 items missing in the isolate: - file system - local disk persistence - long-running process - subprocess spawning - raw sockets - native binary addon - kernel tuning ``` Startup magnitudes drop in two-digit steps: 10^4, 10^2, 10^0. Moving from the virtual machine to the container gains two orders of magnitude; moving from the container to the isolate gains two more. The memory baseline follows the same 10^3, 10^1, 10^0 sequence. Up to this point the table is familiar — only a cheaper isolation tier has been added. The third column turns the table around. The interface surface is 13/13, 12/13, and 6/13. The virtual machine offers every item; the only item the container loses is kernel tuning, because the kernel is shared. The isolate, by contrast, loses seven of thirteen items at once. This is not a steady decline along the scale — it is a **break**: the container gives up one item for one order of magnitude of cheaper cost, the isolate gives up seven items for two orders of magnitude. #### The Disappearance of Cold Start Startup latency dropping to the 10^0 order of magnitude changes exactly one thing, and that thing is large. The most familiar flaw of a serverless function is that an instance starting from zero delays the first call. That delay is not noise next to the application's own runtime — it is an item that **drowns it out**. The run below runs the same call sequence at three tiers and counts how the delay lands across percentiles. **EC4.** The cold-call rate is 6% and its pattern is fixed; there is no randomness. **EC5.** The application's own processing time is 12 ms and does not change by tier; network latency is not in this model. **EC6.** The cost of working around a limit is measured by the number of network round trips added per request; how many milliseconds a round trip costs is the next lesson's subject. ```js // measurement-network/gain.mjs — how startup latency shows up in percentiles and // how the missing surface converts into workload. MODEL: call distribution is deterministic. import { LEVELS, NAMES, offered } from "./unit.mjs"; const CALLS = 10_000, COLD_PERCENT = 6, PROCESS_MS = 12, THRESHOLD_MS = 100; const isCold = (i) => (i * COLD_PERCENT) % 100 < COLD_PERCENT; // pattern: unseeded, fixed const percentile = (a, p) => a[Math.min(a.length - 1, Math.ceil((p / 100) * a.length) - 1)]; console.log(`${CALLS} calls, ${COLD_PERCENT}% cold, processing ${PROCESS_MS} ms (model)`); console.log("tier".padEnd(17) + "p50".padEnd(11) + "p95".padEnd(11) + "p99".padEnd(11) + "avg extra".padEnd(14) + `over ${THRESHOLD_MS} ms`); for (const d of NAMES) { const g = []; for (let i = 0; i < CALLS; i++) g.push(PROCESS_MS + (isCold(i) ? LEVELS[d].startupMs : 0)); g.sort((x, y) => x - y); const extra = g.reduce((t, v) => t + v - PROCESS_MS, 0) / CALLS; console.log(d.padEnd(17) + `${percentile(g, 50)} ms`.padEnd(11) + `${percentile(g, 95)} ms`.padEnd(11) + `${percentile(g, 99)} ms`.padEnd(11) + `${extra.toFixed(1)} ms`.padEnd(14) + String(g.filter((v) => v > THRESHOLD_MS).length)); } // Zeroing out the cold rate: the memory cost of keeping pre-warmed instances. const WARM = 40; console.log(`\nif ${WARM} instances are kept warm to zero out the cold rate`); for (const d of NAMES) console.log(" " + d.padEnd(17) + `${String(LEVELS[d].memoryBaselineMB * WARM).padStart(6)} MB idle baseline`); const WORK = { "request routing": ["request/response handling"], "header editing": ["request/response handling"], "session validation": ["request/response handling", "crypto primitives"], "read-format conversion": ["request/response handling", "stream processing"], "nightly batch processing": ["long-running process", "file system", "local disk persistence"], "invoice document": ["file system", "native binary addon"], "log archiving": ["local disk persistence", "file system"], "persistent device connection":["raw sockets", "long-running process"], }; const runs = (w, d) => WORK[w].every((k) => offered(d).includes(k)); console.log("\ndoes the workload run at the tier"); console.log("workload".padEnd(30) + NAMES.map((d) => d.padEnd(17)).join("")); for (const w of Object.keys(WORK)) console.log(w.padEnd(30) + NAMES.map((d) => (runs(w, d) ? "runs" : "no run").padEnd(17)).join("")); for (const d of NAMES) console.log(" " + d.padEnd(17) + `${Object.keys(WORK).filter((w) => runs(w, d)).length}/${Object.keys(WORK).length} workloads`); // The cost of working around a missing item: the network round trip added per request. const WORKAROUND = { "file system": [1, "read/write goes out to the object store"], "local disk persistence": [1, "state is kept in an external key-value store"], "long-running process": [3, "the job is split; each part reads state from outside"], "subprocess spawning": [2, "the call is delegated to an instance in the region"], "raw sockets": [2, "the connection terminates in the region; the edge only routes it"], "native binary addon": [0, "a pure-language equivalent is written instead; processing time grows"], "kernel tuning": [0, "there is no setting; the job cannot be held at this tier"], }; console.log("\nworking around a missing item (extra network round trip per request)"); let total = 0; for (const [k, [trip, way]] of Object.entries(WORKAROUND)) { total += trip; console.log(" " + k.padEnd(28) + `+${trip} trip ` + way); } console.log(` ${"total for the seven items".padEnd(28)}+${total} trips`); ``` ``` 10000 calls, 6% cold, processing 12 ms (model) tier p50 p95 p99 avg extra over 100 ms virtual machine 12 ms 25012 ms 25012 ms 1500.0 ms 600 container 12 ms 312 ms 312 ms 18.0 ms 600 isolate 12 ms 15 ms 15 ms 0.2 ms 0 if 40 instances are kept warm to zero out the cold rate virtual machine 20480 MB idle baseline container 480 MB idle baseline isolate 120 MB idle baseline does the workload run at the tier workload virtual machine container isolate request routing runs runs runs header editing runs runs runs session validation runs runs runs read-format conversion runs runs runs nightly batch processing runs runs no run invoice document runs runs no run log archiving runs runs no run persistent device connection runs runs no run virtual machine 8/8 workloads container 8/8 workloads isolate 4/8 workloads working around a missing item (extra network round trip per request) file system +1 trip read/write goes out to the object store local disk persistence +1 trip state is kept in an external key-value store long-running process +3 trip the job is split; each part reads state from outside subprocess spawning +2 trip the call is delegated to an instance in the region raw sockets +2 trip the connection terminates in the region; the edge only routes it native binary addon +0 trip a pure-language equivalent is written instead; processing time grows kernel tuning +0 trip there is no setting; the job cannot be held at this tier total for the seven items +9 trips ``` All three tiers share the same p50: 12 ms. The median call lands on a warm instance and never sees startup latency at all. The difference shows up at p95, because cold calls fill the 6% slice and the slowest 6% starts right there. p95 is **25,012 ms** on the virtual machine, **312 ms** on the container, **15 ms** on the isolate. The average extra delay is 1,500.0 / 18.0 / 0.2 ms respectively. The last column makes the call. At a 100 ms threshold, the virtual machine and the container produce the same number: 600 calls. Even though there is an 80-fold gap between the two tiers, **both cross the threshold**; cold start shrinks on the container but does not disappear. On the isolate, the number of calls that cross the threshold is **0**. Cold start is not improved here — it **stops being a problem**: a 3 ms startup next to a 12 ms operation is not worth measuring. This gain has to be read together with the cost of buying it on the container. The way to bring the cold rate to zero is pre-warming instances, and with forty instances kept warm, the idle memory baseline is 20,480 MB on the virtual machine, 480 MB on the container. On the isolate this row is meaningless: no warming happens, because there is nothing to warm. #### The Missing Interface and the Cost of Working Around It The decision handed over here is clear: the runtime's setup, version, capacity, and placement are gone. What comes back in exchange is a number — **7 of 13 items are missing**. Item by item: no file system, no local disk persistence, no long-running process, no subprocess spawning, no raw sockets, no native binary addon, no kernel tuning. The last item has the widest consequence: a library that depends on a native binary addon does not run at this tier — even if the application never calls it directly, if it sits anywhere in the dependency chain, the package fails to load. The workload table turns this into the fictional network. Four of the eight workloads run on the isolate: request routing, header editing, session validation, read-format conversion. Four do not: nightly batch processing, invoice document generation, log archiving, persistent device connection. The ratio is **4/8**, which means half of this same network's jobs cannot sit at the edge. The container and the virtual machine both run all eight; the split is not about the type of workload, it is about **which item it needs** — the four that run touch only the request-handling, crypto, and stream items. Working around the constraint is possible, and the cost is totaled in the last table. Using an object store instead of a file system adds **+1 network round trip** per access, an external key-value store instead of local disk adds **+1 trip**, splitting a long-running process and reading its state from outside adds **+3 trips**, and handing off subprocess and raw-socket work to the region adds two trips each. The total for the seven items is **+9 trips**. Two items add zero trips, but the cost is not zero: the native binary addon's pure-language equivalent lengthens processing time, and kernel tuning has no equivalent at all — that job cannot be held at this tier. The meaning of those nine trips is still incomplete here, because how many milliseconds a trip costs depends on where it goes from the edge. A trip from the edge to a nearby store and a trip to the region are not the same item; the first table erased 300 ms as a gain, and each workaround trip can claw back a piece of that gain. The real arithmetic of computing at the edge is the difference between these two numbers. #### Summary - The isolate is one step further down the familiar scale: startup magnitudes are 10^4, 10^2, and 10^0; memory baseline magnitudes are 10^3, 10^1, and 10^0. What it isolates is not a kernel or a file system, it is a single language's interpreter context. - The third metric turns the table around: the offered interface surface is 13/13, 12/13, and 6/13. The container gives up one item for one order of magnitude, the isolate gives up seven items for two. - Cold start disappears: p95 is 25,012, 312, and 15 ms; the number of calls crossing the 100 ms threshold is 600, 600, and 0. Keeping a warm pool of forty instances costs 20,480 and 480 MB, while on the isolate there is nothing to warm. - The constraint that comes back in exchange is spelled out item by item: no file system, local disk, long-running process, subprocess, raw sockets, native binary addon, or kernel tuning. Only 4 of the fictional network's eight workloads run at this tier. - The cost of working around the constraint is a network round trip per request: +9 trips total for the seven items. Two items add no trips, but the cost is written into processing time or into moving the job elsewhere. #### Next Step This lesson wrote down what can run at the edge but never measured where the thing that runs actually sits. If four workloads can run at the edge, the same job can also run in the region or at the origin — what decides the choice among the three places? What is the difference, in network round trips, between deciding a request at the edge and moving it to the region, and how many milliseconds do those round trips cost? When the routing decision is made at the edge, what share of requests never reaches the origin at all, and how does that share change when the same decision is authentication or an A/B split instead? The next lesson runs the same request set separately at three places, works out the round-trip count and latency of each placement, and names the work that cannot be done at the edge. ### Request Processing and Routing Source: https://academia.sh/en/courses/serverless/request-processing-and-routing The same request set is processed at three places — the edge, the region, and the origin — and how many network round trips and milliseconds each placement adds is counted. For the four job types that can run at the edge, the share of requests that never reaches the origin is worked out, and the work that cannot be done at the edge is named. The previous lesson wrote down what can run at the edge: six of thirteen interface items, four of the fictional network's eight workloads. But being able to run somewhere is not a placement decision — the same four jobs also run in the region and at the origin. In the fictional **measurement network**, every request from a branch site goes to the region today; knowing that part of it could be decided at the edge does not say which part should be. This lesson measures one thing: how many network round trips and milliseconds a request pays depending on where the decision is made. Once trips are counted, placement stops being a matter of preference and becomes arithmetic. **EC7.** Trips are counted as full round trips (there and back); one-way latency is not modeled. **EC8.** Round-trip and processing times are model inputs; they correspond to the fictional network's edge, region, and origin triad. **EC9.** A request is processed in at most one place; the same job is never done in two places at once. #### Three Places, Three Trips The path has three legs. The client-to-edge leg is short, because the edge sits close to the branch. The edge-to-region leg is long. The region-to-origin leg is short again, because the two sit in the same region. Wherever a request is processed, every leg up to that point is paid for. ```js // measurement-network/placement.mjs — the same request processed at three places: trips and latency // MODEL: leg times are a declared model input; round trips are counted, not one-way hops. export const LEG = { "client-edge": 8, "edge-region": 60, "region-origin": 20 }; export const PLACE = { "edge": { leg: ["client-edge"], processMs: 2 }, "region": { leg: ["client-edge", "edge-region"], processMs: 6 }, "origin": { leg: ["client-edge", "edge-region", "region-origin"], processMs: 15 }, }; export const PLACES = Object.keys(PLACE); export const trips = (y) => PLACE[y].leg.length; export const latency = (y) => PLACE[y].leg.reduce((t, b) => t + LEG[b], 0) + PLACE[y].processMs; // Four job types and where each one can be done. export const WORK = { "routing": { place: ["edge", "region", "origin"], ends: true }, "header editing": { place: ["edge", "region", "origin"], ends: false }, "authentication": { place: ["edge", "region", "origin"], ends: true }, "a/b split": { place: ["edge", "region", "origin"], ends: true }, "relational query": { place: ["origin"], ends: false }, "transactional write":{ place: ["origin"], ends: false }, "dataset scan": { place: ["region", "origin"], ends: false }, }; if (import.meta.main) { console.log("a request processed at three places (model input)"); console.log("place".padEnd(10) + "trips".padEnd(10) + "net time".padEnd(12) + "process".padEnd(10) + "total".padEnd(10) + "vs edge"); for (const y of PLACES) { const net = PLACE[y].leg.reduce((t, b) => t + LEG[b], 0); console.log(y.padEnd(10) + String(trips(y)).padEnd(10) + `${net} ms`.padEnd(12) + `${PLACE[y].processMs} ms`.padEnd(10) + `${latency(y)} ms`.padEnd(10) + `+${latency(y) - latency("edge")} ms`); } console.log("network time per leg: " + Object.entries(LEG).map(([b, v]) => `${b} ${v} ms`).join(", ")); console.log("\nwhere the job type can be done (+ can, - cannot)"); console.log("job type".padEnd(24) + PLACES.map((y) => y.padEnd(10)).join("") + "ends at edge"); for (const [ad, v] of Object.entries(WORK)) console.log(ad.padEnd(24) + PLACES.map((y) => (v.place.includes(y) ? "+" : "-").padEnd(10)).join("") + (v.place.includes("edge") ? (v.ends ? "yes" : "no") : "-")); const atEdge = Object.values(WORK).filter((v) => v.place.includes("edge")).length; console.log(`can be done at the edge: ${atEdge}/${Object.keys(WORK).length}`); } ``` ``` a request processed at three places (model input) place trips net time process total vs edge edge 1 8 ms 2 ms 10 ms +0 ms region 2 68 ms 6 ms 74 ms +64 ms origin 3 88 ms 15 ms 103 ms +93 ms network time per leg: client-edge 8 ms, edge-region 60 ms, region-origin 20 ms where the job type can be done (+ can, - cannot) job type edge region origin ends at edge routing + + + yes header editing + + + no authentication + + + yes a/b split + + + yes relational query - - + - transactional write - - + - dataset scan - + + - can be done at the edge: 4/7 ``` The round-trip count is 1, 2, and 3; latency is 10, 74, and 103 ms. Moving from the edge to the region adds **+64 ms**, moving from the region to the origin adds **+29 ms**. The two steps are not equal, and the reason is the leg times: the long leg sits between the edge and the region. The weight of the placement decision is therefore concentrated in the first step — moving a job from the region to the origin is a minor detail, while moving it from the edge to the region is roughly two-thirds of the latency. Processing times run the opposite way and do not soften the decision. It is 2 ms at the edge, 6 ms in the region, 15 ms at the origin; the farthest place is also the slowest to process, because the work done there is heavier. Across all three items, network time drowns out processing time: at the origin, 88 ms of network against 15 ms of processing. Counting trips as full round trips (EC7) also determines how the number should be read: a placement's cost is not a one-way distance but a duration that includes the response coming back. If a decision made at the edge has to ask the origin a question and wait, that wait adds one more row to the same table. What the model leaves out shows up here too: a job that makes two independent data requests in sequence cannot be represented by a single row — it pays two round trips. This is why the job type best suited to the edge is the one that carries everything the decision needs inside the request itself. The second table places seven job types. Four can be done at all three places, and three of those can close out the request at the edge: routing produces a response, authentication produces a rejection, the A/B split returns a ready copy. Header editing runs at the edge but does not close out the request — the header is written and the request continues on its path. This distinction determines the next count: a job being able to run at the edge, and that job sparing a visit to the origin, are not the same thing. #### The Request That Ends at the Edge Now a request set is run through these four jobs in sequence. The sequence matters: a request that closes out at the routing stage never enters authentication, so the same request is never counted twice. **EC10.** The set is 10,000 requests, and the number of requests ending at the edge per stage is a model input. **EC11.** In the A/B split, the copy ready at the edge is an input; cache hit rate is not re-measured in this lesson. **EC12.** A request processed at the edge and passed along adds the edge's own processing time to the path toward the origin. ```js // measurement-network/filter.mjs — a request set run through a sequential filter at the edge // MODEL: the number of requests ending at the edge per stage is a declared model input. import { PLACE, PLACES, WORK, trips, latency } from "./placement.mjs"; const SET = 10_000; const STAGE = [ ["routing", 1_240, "a stale address closes out at the edge with a permanent redirect"], ["authentication", 860, "an invalid token is rejected at the edge"], ["a/b split", 2_100, "the variant is chosen at the edge and a ready copy is returned"], ["header editing", 0, "the header is written and the request continues on its path"], ]; console.log(`${SET} requests, sequential filter at the edge (model input)`); console.log("stage".padEnd(20) + "touched".padEnd(12) + "ends at edge".padEnd(16) + "remaining".padEnd(11) + "share of set"); let remaining = SET, ended = 0; for (const [ad, done, _] of STAGE) { const touched = remaining; remaining -= done; ended += done; console.log(ad.padEnd(20) + String(touched).padEnd(12) + String(done).padEnd(16) + String(remaining).padEnd(11) + `${((done / SET) * 100).toFixed(1)}%`); } console.log("never reaches origin: " + ended + `/${SET} ` + `${((ended / SET) * 100).toFixed(1)}% goes to origin: ${remaining}`); // The weighted latency and total network trips of two scenarios. const EDGE_EXTRA = PLACE["edge"].processMs; const scenario = { "all at origin": [[SET, latency("origin"), trips("origin")]], "four jobs at edge": [[ended, latency("edge"), trips("edge")], [remaining, latency("origin") + EDGE_EXTRA, trips("origin")]], }; console.log("\nweighted latency and total network trips"); const avg = {}; for (const [ad, row] of Object.entries(scenario)) { const ms = row.reduce((t, [n, g]) => t + n * g, 0) / SET; const tripCount = row.reduce((t, [n, , k]) => t + n * k, 0); avg[ad] = ms; console.log(" " + ad.padEnd(20) + `${ms.toFixed(1)} ms/req ` + `${tripCount} trips ${(tripCount / SET).toFixed(2)} trips/req`); } const gap = avg["all at origin"] - avg["four jobs at edge"]; console.log(` ${"gain".padEnd(20)}${gap.toFixed(1)} ms/req ` + `${((gap / avg["all at origin"]) * 100).toFixed(1)}%`); // The marginal cost of deciding one step further out instead of at the edge. console.log("\ncost of moving the same decision one step further out"); for (let i = 1; i < PLACES.length; i++) { const [o, y] = [PLACES[i - 1], PLACES[i]]; console.log(" " + `${o} -> ${y}`.padEnd(20) + `+${trips(y) - trips(o)} trips +${latency(y) - latency(o)} ms/req ` + `+${(((latency(y) - latency(o)) * SET) / 1000).toFixed(1)} s per set`); } const REASON = { "relational query": "joins many tables; the data is not at the edge", "transactional write": "needs strict ordering and rollback; there is no transaction boundary at the edge", "dataset scan": "input is larger than the memory baseline; there is no disk at the edge", }; console.log("\nwork that cannot be done at the edge"); for (const [ad, v] of Object.entries(WORK)) { if (v.place.includes("edge")) continue; console.log(" " + ad.padEnd(24) + `nearest place: ${v.place[0].padEnd(8)}` + REASON[ad]); } ``` ``` 10000 requests, sequential filter at the edge (model input) stage touched ends at edge remaining share of set routing 10000 1240 8760 12.4% authentication 8760 860 7900 8.6% a/b split 7900 2100 5800 21.0% header editing 5800 0 5800 0.0% never reaches origin: 4200/10000 42.0% goes to origin: 5800 weighted latency and total network trips all at origin 103.0 ms/req 30000 trips 3.00 trips/req four jobs at edge 65.1 ms/req 21600 trips 2.16 trips/req gain 37.9 ms/req 36.8% cost of moving the same decision one step further out edge -> region +1 trips +64 ms/req +640.0 s per set region -> origin +1 trips +29 ms/req +290.0 s per set work that cannot be done at the edge relational query nearest place: origin joins many tables; the data is not at the edge transactional write nearest place: origin needs strict ordering and rollback; there is no transaction boundary at the edge dataset scan nearest place: region input is larger than the memory baseline; there is no disk at the edge ``` The share of requests that never reaches the origin is read separately for each of the four job types: routing 12.4%, authentication 8.6%, A/B split 21.0%, header editing 0.0%. In total, **4,200 requests** never see the origin at all — **42.0%** of the set. Header editing producing zero does not make it worthless; the work it does lightens the origin's code, not its load. But it has no counterpart in the trip count, and the table shows exactly that. Round trips per request drop from 3.00 to **2.16**, weighted latency from 103.0 ms to **65.1 ms**; the gain is 37.9 ms, or 36.8%. This number is the direct counterpart of the 42.0%: every request that does not go to the origin moves from the three-trip path to the one-trip path. The 5,800 requests that do go to the origin, however, pay **105 ms**, not 103; being processed and passed along at the edge carries a 2 ms cost of its own, and it shows up in the average. The filter's order is also a decision, visible in the "touched" column. Authentication sits behind routing, so it looks at 8,760 requests instead of 10,000; the A/B split sits behind both, so it looks at 7,900. Reversing the order would not change the total ending at the edge — 4,200 would still be 4,200 — but the A/B split would then also compute a variant for the 860 requests rejected anyway. What a sequential filter gains is not trips but processing time spent at the edge; the cheap, high-elimination stage goes first. The marginal table reads the same decision from the other direction. Moving an edge-eligible decision to the region costs **+64 ms** per request, **640 seconds** across ten thousand requests; moving it from the region to the origin costs +29 ms and 290 seconds. In the fictional network, deciding authentication in the region instead of at the edge therefore creates more than ten minutes of total waiting across ten thousand requests — and 860 of those requests will be rejected anyway. #### Work That Cannot Be Done at the Edge The decision handed over in this lesson is placement: the application does not choose which node meets the request. What comes back in exchange is again a number — **3 of the seven job types cannot be done at the edge**, and all three are named. The relational query joins multiple tables, and the data to join is not at the edge. The transactional write needs strict ordering and rollback, and no transaction boundary can be established at the edge. The dataset scan's input is larger than the memory baseline, and there is no disk at the edge. The last two follow directly from the previous lesson's missing-item list: no local disk persistence, and no durable log for the transactional write to rely on. The nearest place differs for these three, and the difference is a design lever. The dataset scan can be done in the region; the relational query and the transactional write, only at the origin. So not every job that cannot run at the edge is pushed to the same distance: keeping a job in the region wins back **29 ms per request** compared to moving it to the origin. In the fictional network, nightly reconciliation therefore stays in the region while invoice finalization drops all the way to the origin — both leave the edge, but not to the same distance. How to work around the constraint is clear in this lesson's own terms, and so is its cost. For the relational query to move to the edge, its result would have to be precomputed and sit somewhere at the edge; the question then stops being a query and becomes a **read**. For the transactional write to move to the edge, a single writer that guarantees ordering would have to sit at the edge. Both workarounds want the same thing: state that sits at the edge, readable and writable. This lesson's model never assumed that state existed — all four jobs that ran were stateless. #### Summary - Three placements cost three trip counts: edge 1 trip / 10 ms, region 2 trips / 74 ms, origin 3 trips / 103 ms. The long leg sits between the edge and the region, so the first step is +64 ms, the second +29 ms. - 4 of the seven job types can be done at the edge; 3 of those close out the request at the edge. Being able to run at the edge and sparing a visit to the origin are not the same thing — header editing is the example. - The shares of requests that never reach the origin: routing 12.4%, authentication 8.6%, A/B split 21.0%, header editing 0.0%. The total is 4,200/10,000, or 42.0%. - Round trips per request drop from 3.00 to 2.16, weighted latency from 103.0 ms to 65.1 ms; the gain is 36.8%. A request processed and passed along at the edge pays a 2 ms extra on the path to the origin. - The three job types that cannot be done at the edge are named: the relational query, the transactional write, the dataset scan. Two of them have the origin as their nearest place, one has the region; the difference between them is 29 ms per request. #### Next Step All four jobs in this lesson were stateless: routing looked at a table, authentication at a signature, the A/B split at a split rule. The three jobs that cannot be done at the edge, however, want state. The question becomes: can a value be read at the edge, can it be written? A read is fast close to the edge — but where does a write go, and when does the written value show up at other edge sites? When two edge sites read the same key, for how long do they answer differently, and how many reads go stale in that window? How many round trips does reaching a large object from the edge cost, and which way does it push the egress cost? The next lesson measures edge storage separately along the read and write axes, scans the propagation window, and counts the store's size limit as a constraint. ### Edge Storage Source: https://academia.sh/en/courses/serverless/edge-storage The edge key-value store is fast for reads, slow for writes, and its consistency is delayed. The propagation window is scanned and stale reads are counted for each value; the round-trip count and egress cost of reaching the object store are compared in a fictional unit, and the store's size limit is worked out as a constraint. All four jobs in the previous lesson were stateless: routing looked at a table, authentication at a signature, the A/B split at a split rule. What the three jobs that could not run at the edge had in common was that they wanted state. In the fictional **measurement network**, what branch sites need often is well known: a branch's tariff table, its configuration, a session's validity summary. These are not large, and they are mostly read. A store exists at the edge, and its name is the **edge key-value store**. This lesson does not weigh it with a single metric, because a single number would mislead: the store in this model is faster for reads than anything in the region, and slower for writes than anything in the region — real edge stores do not all make that trade the same way, and some offer strongly consistent writes. The difference between the two is not a latency item — it is a **consistency window**. **EC13.** Writes are applied at the center; the site that wrote sees the new value immediately, other sites return the old value for the length of the propagation window. **EC14.** The window is fixed, and eventually every site sees the new value; partial propagation is not modeled. **EC15.** The event stream is produced by our own generator, and the seed is visible. #### Reads Are Cheap, Writes Are Expensive The length of the read path depends on whether the value sits at the edge. If the value sits at the edge, the read never leaves the network at all. The length of the write path never changes: a write always goes to the center, because that is the single point that determines ordering. ```js // measurement-network/propagation.mjs — read/write asymmetry in the edge key-value store // and the count of stale reads across the propagation window. // MODEL: the event stream is generated by our own generator; the seed is visible. const SEED = 20_260_801; let s = SEED % 2_147_483_647; const rnd = () => (s = (s * 48_271) % 2_147_483_647) / 2_147_483_647; const SITES = 6, KEYS = 40, EVENT_COUNT = 20_000, WRITE_PERCENT = 8; const PATH = { "read (copy at edge)": { trip: 0, ms: 2 }, "read (no copy)": { trip: 1, ms: 62 }, "write (to center)": { trip: 1, ms: 64 }, }; console.log("read and write are not the same thing in this store (model input)"); for (const [ad, v] of Object.entries(PATH)) console.log(" " + ad.padEnd(24) + `${v.trip} extra trip ${String(v.ms).padStart(3)} ms`); console.log(" write/read latency ratio: " + (PATH["write (to center)"].ms / PATH["read (copy at edge)"].ms).toFixed(0) + "x"); const events = []; let t = 0; for (let i = 0; i < EVENT_COUNT; i++) { t += 1 + Math.floor(rnd() * 60); const site = Math.floor(rnd() * SITES), key = Math.floor(rnd() * KEYS); events.push({ t, site, key, write: rnd() * 100 < WRITE_PERCENT }); } const writes = events.filter((o) => o.write).length; console.log(`\n${EVENT_COUNT} events, ${SITES} sites, ${KEYS} keys, seed ${SEED}`); console.log(` writes ${writes} reads ${EVENT_COUNT - writes} stream duration ${(t / 1000).toFixed(1)} s`); console.log("\nstale reads across the scanned propagation window"); console.log("window".padEnd(12) + "reads in window".padEnd(20) + "stale reads".padEnd(14) + "read share".padEnd(14) + "per key"); for (const window of [1_000, 5_000, 10_000, 30_000, 60_000]) { const last = new Array(KEYS).fill(null); let inWindow = 0, stale = 0, reads = 0; for (const o of events) { if (o.write) { last[o.key] = o; continue; } reads++; const y = last[o.key]; if (!y || o.t - y.t >= window) continue; inWindow++; if (y.site !== o.site) stale++; } console.log(`${window} ms`.padEnd(12) + String(inWindow).padEnd(20) + String(stale).padEnd(14) + `${((stale / reads) * 100).toFixed(2)}%`.padEnd(14) + (stale / KEYS).toFixed(1)); } console.log(" a 0 ms window would give 0 stale reads; the cost is every read going to the center"); ``` ``` read and write are not the same thing in this store (model input) read (copy at edge) 0 extra trip 2 ms read (no copy) 1 extra trip 62 ms write (to center) 1 extra trip 64 ms write/read latency ratio: 32x 20000 events, 6 sites, 40 keys, seed 20260801 writes 1582 reads 18418 stream duration 609.8 s stale reads across the scanned propagation window window reads in window stale reads read share per key 1000 ms 1154 957 5.20% 23.9 5000 ms 5019 4147 22.52% 103.7 10000 ms 8690 7230 39.26% 180.8 30000 ms 15466 12857 69.81% 321.4 60000 ms 17572 14621 79.38% 365.5 a 0 ms window would give 0 stale reads; the cost is every read going to the center ``` The asymmetry is read in the first three rows. Reading from the copy at the edge costs 0 extra trips and 2 ms, reading with no copy costs 1 trip and 62 ms, writing costs 1 trip and **64 ms** in every case. The ratio of write to read is **32-fold**. This ratio says what the store is built for: if a value is written once and read many times, the store sits in the right place; if it is written once per request, the store sits in the wrong place. #### The Count of Stale Reads Consistency being delayed is not an adjective; it is measured by how many reads see the old value. The scan below holds the window at five values and counts stale reads at each one. Being counted as stale requires two conditions: the read must arrive after the last write, before the window closes, and the reading site must differ from the writing site. ```js // measurement-network/object.mjs — trips and egress cost of reaching the object store from // the edge, then the edge store's size limit. MODEL: rates are a fictional unit, // not a real currency or price. const REQUESTS = 10_000, OBJECT_KB = 250, GB = 1_048_576; // 1 GB in KB const RATE = { "object->region": 0, "region->edge": 0, "object->edge": 2, "edge->client": 1 }; const PATH = { "via region": { trip: 2, ms: 80, legs: ["object->region", "region->edge", "edge->client"] }, "direct from edge": { trip: 1, ms: 45, legs: ["object->edge", "edge->client"] }, "copy at edge": { trip: 0, ms: 2, legs: ["edge->client"] }, }; const volumeGB = (n) => (n * OBJECT_KB) / GB; const cost = (path, n) => PATH[path].legs.reduce((t, b) => t + RATE[b], 0) * volumeGB(n); console.log(`${REQUESTS} object requests, object ${OBJECT_KB} KB, rates are a fictional unit`); console.log("path".padEnd(18) + "trips".padEnd(10) + "latency".padEnd(10) + "legs".padEnd(14) + "units/GB".padEnd(11) + "set cost"); for (const [ad, v] of Object.entries(PATH)) { const unitsGB = v.legs.reduce((t, b) => t + RATE[b], 0); console.log(ad.padEnd(18) + String(v.trip).padEnd(10) + `${v.ms} ms`.padEnd(10) + String(v.legs.length).padEnd(14) + String(unitsGB).padEnd(11) + `${cost(ad, REQUESTS).toFixed(2)} units`); } console.log(`set volume ${volumeGB(REQUESTS).toFixed(2)} GB ` + `leg rate: ` + Object.entries(RATE).map(([b, u]) => `${b} ${u}`).join(", ")); const HIT_RATE = 70; const hits = Math.round((REQUESTS * HIT_RATE) / 100), misses = REQUESTS - hits; const blendedMs = (hits * PATH["copy at edge"].ms + misses * PATH["via region"].ms) / REQUESTS; const blendedCost = cost("copy at edge", hits) + cost("via region", misses); console.log(`\nblend: ${HIT_RATE}% copy at edge, the rest via region`); console.log(" " + `${hits} + ${misses} requests`.padEnd(22) + `${blendedMs.toFixed(1)} ms/req ${blendedCost.toFixed(2)} units ` + `${(cost("direct from edge", REQUESTS) / blendedCost).toFixed(2)}x cheaper than direct`); // The edge key-value store's limit: 512-byte keys, 25 KB values. const VALUE_KB = 25, KEY_BYTES = 512; const RECORD = { "branch configuration": 2, "tariff table": 18, "session summary": 0.4, "daily reading summary": 140, "monthly invoice document": 900, "raw counter log": 12_000, }; console.log(`\nedge store limit: value ${VALUE_KB} KB, key ${KEY_BYTES} bytes`); console.log("record".padEnd(26) + "size".padEnd(12) + "fits".padEnd(9) + "workaround"); let fits = 0; for (const [ad, kb] of Object.entries(RECORD)) { const ok = kb <= VALUE_KB; if (ok) fits++; const chunks = Math.ceil(kb / VALUE_KB); console.log(ad.padEnd(26) + `${kb} KB`.padEnd(12) + (ok ? "yes" : "no").padEnd(9) + (ok ? "-" : `${chunks} chunks, +${chunks - 1} reads per read or the object store (+2 trips)`)); } console.log(` ${fits}/${Object.keys(RECORD).length} records fit in the store`); ``` ``` 10000 object requests, object 250 KB, rates are a fictional unit path trips latency legs units/GB set cost via region 2 80 ms 3 1 2.38 units direct from edge 1 45 ms 2 3 7.15 units copy at edge 0 2 ms 1 1 2.38 units set volume 2.38 GB leg rate: object->region 0, region->edge 0, object->edge 2, edge->client 1 blend: 70% copy at edge, the rest via region 7000 + 3000 requests 25.4 ms/req 2.38 units 3.00x cheaper than direct edge store limit: value 25 KB, key 512 bytes record size fits workaround branch configuration 2 KB yes - tariff table 18 KB yes - session summary 0.4 KB yes - daily reading summary 140 KB no 6 chunks, +5 reads per read or the object store (+2 trips) monthly invoice document 900 KB no 36 chunks, +35 reads per read or the object store (+2 trips) raw counter log 12000 KB no 480 chunks, +479 reads per read or the object store (+2 trips) 3/6 records fit in the store ``` The stream itself is also an input, and it needs to be read: of 20,000 events, 1,582 are writes and 18,418 are reads, spread across **609.8 seconds**. So the mix matches the direction the store was designed for — one write for every eleven reads. The stale-read count coming out this high is therefore not because writes are frequent, but because forty keys are spread across six sites: every write ties the reads at the other five sites to the old value for the length of the window. The numbers grow in one direction and grow fast. At a 1,000 ms window, stale reads are 957, or 5.20% of reads. At 5,000 ms, 4,147 and 22.52%. At 10,000 ms, 7,230 and 39.26%. At 30,000 ms, 12,857 and 69.81%. At 60,000 ms, 14,621 and 79.38%. As the window grows sixtyfold, stale reads grow fifteenfold; the growth is not linear, because as the window widens, most reads already fall in the shadow of some write, and the count approaches saturation. Stale reads per key climb from 23.9 to 365.5. If each of the forty keys is treated as one branch's tariff table in the fiction, then at a 30,000 ms window, 321.4 reads per key see the old tariff. Whether that number is acceptable depends on what the read is for: tolerable if it is read to display a tariff on a screen, not tolerable if it is read to write an amount onto an invoice. The same store, with the same window, gives two different answers for two different jobs. The decision handed over here is replication: the application does not choose which sites get a copy of the value, or when. What comes back in exchange is the propagation window, and it is a number. How to work around the constraint is written in the last line: if the window is brought to zero, stale reads become zero, but then every read goes to the center — reading climbs from 2 ms to 62 ms, a **31-fold** rise. Since the whole reason for using the store is that 2 ms, this workaround cancels the store itself. #### Reaching Out to the Object Store For things that do not fit in the store, the edge reaches out to the object store. Reaching out has two axes, and they work in opposite directions: round-trip count and egress cost. **EC16.** Rates are a **fictional unit**; they are not a real currency or price. **EC17.** Object size is fixed at 250 KB, and the hit rate of the copy kept at the edge is a 70% input. **EC18.** In this model the value limit is 25 KB and the key limit is 512 bytes — in real services these two limits can differ widely; chunking assumes the chunks are read in sequence. The relationship among the three paths is the reverse of what is expected at first glance. Going directly from the edge to the object store is one of the paths with the fewest trips — 1 trip, 45 ms — but the set cost is **7.15 units**, three times going via the region. The reason is the egress leg: on the direct path, the object leaves the store's network and reaches the edge, and this leg charges 2 units per GB. Going via the region keeps the object inside the same network; the only leg charged is the one from edge to client. Cutting the trip count here **triples** the cost. Keeping a copy at the edge wins on both axes: 0 trips, 2 ms, and 2.38 units. But this row comes with an assumption — that the copy is actually there. The blended row gives the real situation: at 70% hit rate, an average of **25.4 ms/request** and 2.38 units; **3.00 times cheaper** than the direct path. As the hit rate falls, latency drifts toward 80 ms, while the cost does not change. So what keeping a copy at the edge buys is not cost, it is latency; the only thing that determines cost is which legs the traffic passes through. #### What Fits in the Store The last table counts the store's limit as a constraint. The value limit is 25 KB, and 3 of the fiction's six record types fit: branch configuration at 2 KB, the tariff table at 18 KB, the session summary at 0.4 KB. Three do not fit: the daily reading summary at 140 KB, the monthly invoice document at 900 KB, the raw counter log at 12,000 KB. The way to force a value that does not fit into the store is chunking, and the cost is collected on the read side. The 140 KB summary comes to 6 chunks; every full read means **+5 reads**. The invoice document is 36 chunks and +35 reads, the raw log is 480 chunks and +479 reads. Chunking is reasonable for a small overshoot and absurd for a large one: for a value four hundred and eighty times the 25 KB limit, the store is not the right tool. The alternative is written in the last line — put it in the object store and pay +2 trips. The decision point is how many chunks a single read requires; the moment the chunk count passes two or three, the object store's two trips become cheaper. The 512-byte key limit is a separate design constraint, and it does not show up in the table. A short key forces a composite key — the branch, counter, and period triple — to be squeezed into a single string; combined with the store's inability to run range queries, this forces data into the design of the key itself. The store is not a database; it is for small, heavily read values reached by key. #### Summary - The same store is two different things in two directions: reading from the copy at the edge is 0 trips / 2 ms, writing is 1 trip / 64 ms. The ratio of write to read is 32-fold; the store is for values written once and read many times. - Consistency is delayed, and the cost is counted: at 1,000 / 5,000 / 10,000 / 30,000 / 60,000 ms windows, stale reads are 957, 4,147, 7,230, 12,857, and 14,621; the read share climbs from 5.20% to 79.38%, and per key from 23.9 to 365.5. - Zeroing out the window zeroes out stale reads but moves reading from 2 ms to 62 ms — canceling the store's whole reason for being used. - There are three paths to the object store: via region 2 trips / 2.38 units, direct from edge 1 trip / 7.15 units, copy at edge 0 trips / 2.38 units. Cutting trips can triple the cost; the 70%-hit blend costs 25.4 ms/request and 2.38 units. - The value limit is 25 KB, and 3 of six record types fit. The ones that do not come to 6, 36, and 480 chunks; once the chunk count passes a few, the object store's +2 trips become cheaper. #### Next Step This lesson's store never solved one thing: what happens when two edge sites want to change the same key at the same time? The propagation window was making reads return the old value; if two writes race, one overwrites the other, and an update goes missing somewhere in between. Incrementing a counter, counting who has joined a room, locking a resource to a single operation — all three need a "read, modify, write" step, and none of the three is safe in this store. The way to keep coordinated state at the edge is for a single writer to exist for a given key. The next lesson runs a single-writer object model, counts lost updates under concurrent requests, and measures the two constraints that come back in exchange — the single instance's bottleneck, and the object's geographic pinning. ### Durable State Objects Source: https://academia.sh/en/courses/serverless/durable-state-objects A single-writer object model is run for the same key: lost updates drop to zero under concurrent requests. Two constraints that come back in exchange are counted — the single instance queuing per request and the object's geographic pinning — and which of a counter, a room, and a lock scenario fits is written down. The previous lesson's store left one thing unsolved. The propagation window was tying reads to the old value; what happens if two edge sites want to change the same key at the same time? In the fictional **measurement network** this has a concrete counterpart: a branch's validated reading counter is being incremented by records arriving from two different edge sites at once. "Read, modify, write" is three separate steps, and something else can slip in between them. What this lesson measures is how many updates that interleaving drops, and what the model that brings it to zero asks for in exchange. Both models run on the same workload. **EC19.** In the shared-store model, a read takes 20 ms, computation 1 ms, and a write 64 ms; the three are separate steps. **EC20.** Arrivals are deterministic; there is no randomness. **EC21.** In the single-writer object, the same key is routed to a single instance, requests queue in arrival order, state is held in the instance's memory, and processing takes 6 ms per request; there is no concurrent execution. #### Two Models on the Same Key The first model is familiar: the value sits in a remote store, and every client reads it, increments it, and writes it back. In the second model, the key is bound to an instance — there is a single instance in the network for a given key, and every request queues up at it. The step count is now one instead of three. ```js // measurement-network/single-writer.mjs — the same key in two models: shared store // (read-modify-write) and single-writer object (sequential processing). MODEL: arrivals and durations are deterministic. const CLIENTS = 8, STEPS = 50, TOTAL = CLIENTS * STEPS; const READ_MS = 20, COMPUTE_MS = 1, WRITE_MS = 64; // shared store path const PROCESS_MS = 6; // per-request processing at the single instance // --- 1. Shared store: each client does read-modify-write in sequence. const ops = []; for (let c = 0; c < CLIENTS; c++) { let t = c * 3; for (let i = 0; i < STEPS; i++) { ops.push({ readT: t, writeT: t + READ_MS + COMPUTE_MS + WRITE_MS, readValue: 0 }); t = t + READ_MS + COMPUTE_MS + WRITE_MS; } } const events = []; for (const o of ops) { events.push({ t: o.readT, isRead: true, o }); events.push({ t: o.writeT, isRead: false, o }); } events.sort((a, b) => a.t - b.t || (a.isRead ? -1 : 1)); let value = 0, overwritten = 0; for (const e of events) { if (e.isRead) { e.o.readValue = value; continue; } if (value !== e.o.readValue) overwritten++; value = e.o.readValue + 1; } console.log(`${CLIENTS} clients x ${STEPS} increments = ${TOTAL} requests, same key (model)`); console.log("shared store (read-modify-write)"); console.log(` expected ${TOTAL} counter's final value ${value} ` + `lost updates ${TOTAL - value} ${(((TOTAL - value) / TOTAL) * 100).toFixed(1)}%`); console.log(` overwritten writes ${overwritten} operation duration ${READ_MS + COMPUTE_MS + WRITE_MS} ms`); // --- 2. Single-writer object: the same key goes to a single instance, requests processed in order. function queueRun(clients, gapMs) { const arrivals = []; for (let c = 0; c < clients; c++) for (let i = 0; i < STEPS; i++) arrivals.push(c * 3 + i * gapMs); arrivals.sort((a, b) => a - b); let free = 0, processed = 0, waits = []; for (const v of arrivals) { const start = Math.max(v, free); waits.push(start - v); free = start + PROCESS_MS; processed++; // state in memory, step is atomic } waits.sort((a, b) => a - b); const p95 = waits[Math.ceil(0.95 * waits.length) - 1]; const avg = waits.reduce((t, v) => t + v, 0) / waits.length; return { requests: arrivals.length, processed, rate: (clients * 1000) / gapMs, avg, p95, duration: (free - arrivals[0]) / 1000 }; } const CAP = 1000 / PROCESS_MS; console.log(`\nsingle-writer object (${PROCESS_MS} ms per request, cap ${CAP.toFixed(1)} req/s)`); console.log("load".padEnd(22) + "arrival rate".padEnd(14) + "lost".padEnd(9) + "avg wait".padEnd(15) + "p95 wait".padEnd(14) + "queue"); for (const [clients, gap] of [[4, 40], [6, 40], [8, 40]]) { const r = queueRun(clients, gap); console.log(`${clients} clients / ${gap} ms`.padEnd(22) + `${r.rate.toFixed(0)}/s`.padEnd(14) + String(r.requests - r.processed).padEnd(9) + `${r.avg.toFixed(1)} ms`.padEnd(15) + `${r.p95} ms`.padEnd(14) + (r.rate > CAP ? "saturated" : "flowing")); } ``` ``` 8 clients x 50 increments = 400 requests, same key (model) shared store (read-modify-write) expected 400 counter's final value 45 lost updates 355 88.8% overwritten writes 355 operation duration 85 ms single-writer object (6 ms per request, cap 166.7 req/s) load arrival rate lost avg wait p95 wait queue 4 clients / 40 ms 100/s 0 4.5 ms 9 ms flowing 6 clients / 40 ms 150/s 0 7.5 ms 15 ms flowing 8 clients / 40 ms 200/s 0 206.5 ms 386 ms saturated ``` In the shared store, 400 increment requests carry the counter to **45**. Lost updates come to **355**, or 88.8%. The number is this large because the operation takes 85 ms, and for those 85 ms all eight clients read the same stale value; all eight writes then produce the same result, and seven of them go to waste. This is not a flaw in the store — the store wrote exactly what it was told to write. What is lost is that **the interleaving went unseen**. In the single-writer object, lost updates on the same workload are **0** at all three load levels. The reason is the model itself: the read, the computation, and the write happen back to back in a single instance's memory, and that instance never processes another request at the same time. Zero can be written down as a guarantee because the step has become indivisible. #### Two Constraints That Come Back in Exchange Zero is not free. The first cost is queuing, and the table reads it at three levels. The cap per instance is **166.7 req/s**. At a 100/s arrival rate, the average wait is 4.5 ms and p95 is 9 ms. At 150/s, 7.5 ms and 15 ms. At 200/s, the average wait is **206.5 ms**, p95 is **386 ms**, and the queue is saturated — because the arrival rate has crossed the cap, the wait keeps growing for the rest of the run. The gap between the two load levels stands out here: between 150/s and 200/s, arrivals rise 33% while the average wait rises from 7.5 ms to 206.5 ms — roughly 28-fold. A single instance is a unit of scale, and once it crosses its cap it does not stop answering, it **backs up**. The decision handed over is managing concurrency; the constraint that comes back in exchange is a single number: 6 ms per request, 166.7 requests per second. The way to work around the constraint is not replicating the instance — replicating it cancels single-writer status, and with it, the zero. The way is **splitting the key**: keeping one counter per branch instead of one counter, which divides the total load by the instance count. The cost is that a global total no longer sits in one place; whenever the total is requested, every instance has to be read, and the read count grows with the instance count. #### Where the Object Sits The second cost is geography. The object sits in one place, and a client far from that place reaches it **remotely**. All of running at the edge's gain came from proximity, and coordinated state collapses that proximity down to a single point. **EC22.** Distances are round trips and are model input. **EC23.** The object is not moved during the run; a change in placement is not modeled. **EC24.** Scenario rates are total request rates; the rate per instance is computed from how many instances the key is split across. ```js // measurement-network/scenario.mjs — the object's geographic pinning and scenario fit // MODEL: distances are round trips and are model input; the object is not moved during the run. const PROCESS_MS = 6, CAP = 1000 / PROCESS_MS; const EDGE = { "edge-1": { load: 2600, "busiest edge": 4, "in region": 62, "per branch": 4 }, "edge-2": { load: 2100, "busiest edge": 40, "in region": 58, "per branch": 4 }, "edge-3": { load: 1800, "busiest edge": 55, "in region": 66, "per branch": 4 }, "edge-4": { load: 1500, "busiest edge": 70, "in region": 60, "per branch": 4 }, "edge-5": { load: 1200, "busiest edge": 88, "in region": 64, "per branch": 4 }, "edge-6": { load: 800, "busiest edge": 96, "in region": 70, "per branch": 4 }, }; const PLACEMENT = ["busiest edge", "in region", "per branch"]; const TOTAL = Object.values(EDGE).reduce((t, u) => t + u.load, 0); console.log(`object access round trip, ${TOTAL} requests spread across the edges (model input)`); console.log("edge".padEnd(8) + "load".padEnd(9) + PLACEMENT.map((y) => y.padEnd(16)).join("")); for (const [ad, u] of Object.entries(EDGE)) console.log(ad.padEnd(8) + String(u.load).padEnd(9) + PLACEMENT.map((y) => `${u[y]} ms`.padEnd(16)).join("")); console.log("weighted".padEnd(8) + "".padEnd(9) + PLACEMENT.map((y) => { const ms = Object.values(EDGE).reduce((t, u) => t + u.load * u[y], 0) / TOTAL; return `${ms.toFixed(1)} ms`.padEnd(16); }).join("")); console.log("worst edge".padEnd(8) + "".padEnd(9) + PLACEMENT.map((y) => `${Math.max(...Object.values(EDGE).map((u) => u[y]))} ms`.padEnd(16)).join("")); console.log(" coordination-free work finishes at the same edge in 2 ms; the difference is the cost of pinning"); const SCENARIO = { "branch counter": { key: "branch", instances: 6, rate: 480, longJobMs: 0 }, "branch room": { key: "branch", instances: 6, rate: 600, longJobMs: 0 }, "work order lock": { key: "work order",instances: 120, rate: 240, longJobMs: 0 }, "global sequence counter":{ key: "single", instances: 1, rate: 900, longJobMs: 0 }, "nightly reconciliation": { key: "single", instances: 1, rate: 5, longJobMs: 40_000 }, }; console.log(`\nscenario fit (cap per instance ${CAP.toFixed(1)} req/s)`); console.log("scenario".padEnd(26) + "key".padEnd(12) + "instances".padEnd(11) + "total rate".padEnd(13) + "per instance".padEnd(14) + "result"); for (const [ad, s] of Object.entries(SCENARIO)) { const perInstance = s.rate / s.instances; const reason = s.longJobMs > 0 ? `does not fit: ${s.longJobMs / 1000} s job locks the instance, ` + `${Math.round((s.longJobMs / 1000) * s.rate)} requests queue behind it` : perInstance > CAP ? `does not fit: ${perInstance.toFixed(0)}/s per instance, ${(perInstance / CAP).toFixed(1)}x the cap` : `fits: ${((perInstance / CAP) * 100).toFixed(0)}% of the cap`; console.log(ad.padEnd(26) + s.key.padEnd(12) + String(s.instances).padEnd(11) + `${s.rate}/s`.padEnd(13) + `${perInstance.toFixed(0)}/s`.padEnd(14) + reason); } const fits = Object.values(SCENARIO).filter((s) => s.longJobMs === 0 && s.rate / s.instances <= CAP).length; console.log(` ${fits}/${Object.keys(SCENARIO).length} scenarios fit the single-writer object`); ``` ``` object access round trip, 10000 requests spread across the edges (model input) edge load busiest edge in region per branch edge-1 2600 4 ms 62 ms 4 ms edge-2 2100 40 ms 58 ms 4 ms edge-3 1800 55 ms 66 ms 4 ms edge-4 1500 70 ms 60 ms 4 ms edge-5 1200 88 ms 64 ms 4 ms edge-6 800 96 ms 70 ms 4 ms weighted 48.1 ms 62.5 ms 4.0 ms worst edge 96 ms 70 ms 4 ms coordination-free work finishes at the same edge in 2 ms; the difference is the cost of pinning scenario fit (cap per instance 166.7 req/s) scenario key instances total rate per instance result branch counter branch 6 480/s 80/s fits: 48% of the cap branch room branch 6 600/s 100/s fits: 60% of the cap work order lock work order 120 240/s 2/s fits: 1% of the cap global sequence counter single 1 900/s 900/s does not fit: 900/s per instance, 5.4x the cap nightly reconciliation single 1 5/s 5/s does not fit: 40 s job locks the instance, 200 requests queue behind it 3/5 scenarios fit the single-writer object ``` When the object is placed next to the busiest edge, the weighted access round trip is **48.1 ms**; placed in the region, it is **62.5 ms**. The first option beats the second on average but makes the worst edge worse: 96 ms against 70 ms. In other words, placing the object next to the busiest edge gives 4 ms to the edge carrying 26% of the load while charging 96 ms to the edge carrying 8%. Placing it in the region flattens this distribution; it treats no one especially well and no one especially badly. The third column shows the real decision. When the key is split per branch, each object sits at its own edge and the weighted access round trip drops to **4.0 ms** — the worst edge is also 4 ms. Compared with coordination-free work finishing at the same edge in 2 ms, the difference is only 2 ms. Geographic pinning stops being a cost here, because the object already sits where it is used. Pinning becomes a cost only when the key does not line up with geography. #### Which Use Fits The last table weighs five scenarios on the same two measures: rate per instance and job duration. The **branch counter** carries a total load of 480/s, split across six instances, for 80/s per instance — 48% of the cap. The **branch room** runs 600/s, 100/s per instance, and 60% of the cap. The **work order lock** splits across 120 instances for 2/s per instance, 1% of the cap; the lock scenario is the most comfortable fit for the single-writer object, because its keys are naturally numerous and each one is touched rarely. Two scenarios do not fit, and the reasons differ. The **global sequence counter** asks for 900/s on a key that cannot be split; that is 5.4 times the cap, and a single instance cannot carry it. The mismatch here is about rate. **Nightly reconciliation** asks for only 5/s, but the job itself takes 40 seconds and locks the instance for that whole span; **200 requests** queue up behind it. The mismatch here is about duration. The same model rejects at two separate thresholds: the 6 ms-per-request cap, and the request's own indivisible length. **3 of 5** scenarios fit. The rule can be written in one sentence: the single-writer object is for coordination jobs with many keys, each touched rarely, whose operations are short. When any one of the three conditions breaks, the model rejects the job and writes the rejection down as a number. #### Summary - In the shared store, 400 concurrent increments carry the counter to 45: lost updates come to 355, or 88.8%. What is lost is not the store writing incorrectly, but the interleaving going unseen. - In the single-writer object, lost updates on the same workload are 0 at all three load levels; the read, the computation, and the write happen indivisibly in a single instance's memory. - The first constraint that comes back in exchange is the cap: 6 ms per request, 166.7 req/s per instance. At arrival rates of 100, 150, and 200/s, the average wait is 4.5, 7.5, and 206.5 ms; a 33% rise in arrivals grows the wait roughly 28-fold. - The second constraint is geographic pinning: the weighted access round trip is 48.1 ms at the busiest edge, 62.5 ms in the region, and 4.0 ms when split per branch. The cost arises only when the key does not line up with geography. - 3 of 5 scenarios fit: the branch counter (48% of the cap), the branch room (60%), the work order lock (1%). The global sequence counter fails on rate (5.4 times the cap), nightly reconciliation fails on duration (a 40 s job with 200 requests queued behind it). #### Next Step What the two scenarios this lesson rejected have in common is that the request **has to wait for a response**. Nightly reconciliation took 40 seconds and queued two hundred requests behind it, yet no one was waiting on that job's result. Most of the records arriving from the edge in the fictional network are like this: a meter reading is taken, its receipt is acknowledged, and processing it can be put off. If we could defer a job at the edge instead of doing it immediately, would the single instance's cap stop being a limit? Where does deferred work sit, how many times is it retried, and how does the edge runtime's duration limit cut that wait short? The next lesson measures the ways to defer work at the edge: where the deferred work piles up, what the delivery guarantee is, and where a scheduled trigger at the edge diverges from one in the region. ### Edge Queues and Scheduled Jobs Source: https://academia.sh/en/courses/serverless/edge-queues-and-scheduled-jobs The same job set is processed in the center queue and the edge queue; the queue-entry trip, where the job runs, and the result's return path are counted separately. The edge queue's four constraints and the region a scheduled job fires in are measured. The previous lesson managed to hold coordinated state at the edge: the single-writer object queued up writes arriving for its own key. Asynchronous work, though, still runs in the center; validation, invoice lines, and field notifications get written to a queue in the center region. This lesson runs the same job set in two places and counts three things separately: the queue-entry trip, where the job runs, and the result's return path. The measurement runs on the six branch edges of the fictional **measurement network** and is a **model**: the edges, the two queue services, and the scheduler are set up with `node`. No real product is called. **EC25.** For every edge, the one-way latency to the center and to the nearest edge location is given, along with a nightly read count. **EC26.** The queue-entry trip is twice the one-way latency plus the queue's fixed acceptance time. **EC27.** The center queue offers a batch read of 250, 10 retries, 14 days of retention, and ordering per key; the edge queue offers 25 and 3, retains for 4 days, and gives no ordering. **EC28.** The consumer starts once per batch read: 40 ms in the center, 6 ms at the edge. Processing per read is 12 ms on both paths. **EC29.** The result is written to the store where the job runs. The center store is as far away as the center trip; the edge key-value store is read locally, but its write takes 480 ms to become visible at other edges. **EC30.** Scheduled triggering knows only the base clock; the edge scheduler chooses the trigger location itself, and the choice is not recorded. #### The Same Job Set, Two Queues ```js // network.mjs — fictional measurement network model: edges, latencies, the two queues' constraints. export const EDGES = [ // one-way latency ms, band offset from base clock { name: "edge-west", center: 12, edge: 5, band: 0, reads: 610 }, { name: "edge-north", center: 38, edge: 7, band: 0, reads: 520 }, { name: "edge-east", center: 74, edge: 9, band: 2, reads: 430 }, { name: "edge-south", center: 51, edge: 6, band: 1, reads: 350 }, { name: "edge-highland", center: 96, edge: 11, band: 3, reads: 280 }, { name: "edge-port", center: 63, edge: 8, band: 2, reads: 210 }, ]; export const TOTAL = EDGES.reduce((t, u) => t + u.reads, 0); export const QUEUE = { // constraints, item by item center: { accept: 9, batch: 250, retries: 10, retentionDays: 14, startup: 40 }, edge: { accept: 3, batch: 25, retries: 3, retentionDays: 4, startup: 6 }, }; export const PROCESSING = 12, PROPAGATION = 480, STORE_READ = 5; // processing / edge propagation / store response, ms export const roundTrip = (path, u) => 2 * u[path] + QUEUE[path].accept; export const row = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 13 : 15)).join("").trimEnd()); export function rng(seed) { // seed is visible; same sequence run to run let s = seed >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; } export function jobSet(seed) { // reading: counter id, increasing sequence, required retries const r = rng(seed), seq = new Map(), items = []; for (const u of EDGES) for (let i = 0; i < u.reads; i++) { const counterId = `s${String(Math.floor(r() * 900)).padStart(3, "0")}`; seq.set(counterId, (seq.get(counterId) ?? 0) + 1); items.push({ site: u.name, counterId, seq: seq.get(counterId), retries: 1 }); } for (const i of items) while (r() < 0.28 && i.retries < 12) i.retries++; return items; } ``` ```js // trip.mjs — the same job set in two places: queue-entry trip, where the job runs, the result's return path. import { EDGES, TOTAL, QUEUE, PROCESSING, PROPAGATION, STORE_READ, roundTrip, row } from "./network.mjs"; row("edge", "reads", "center trip", "edge trip", "gain"); for (const u of EDGES) row(u.name, u.reads, `${roundTrip("center", u)} ms`, `${roundTrip("edge", u)} ms`, `${(roundTrip("center", u) / roundTrip("edge", u)).toFixed(1)}x`); const weighted = (path) => EDGES.reduce((t, u) => t + u.reads * roundTrip(path, u), 0) / TOTAL; console.log(`weighted queue-entry trip for ${TOTAL} reads: ` + `center ${weighted("center").toFixed(1)} ms, edge ${weighted("edge").toFixed(1)} ms\n`); for (const path of ["center", "edge"]) { // consumer: how many times it starts const k = QUEUE[path], calls = Math.ceil(TOTAL / k.batch); const startup = calls * k.startup, work = TOTAL * PROCESSING; console.log(`${path.padEnd(7)}batch read ${String(k.batch).padStart(3)} -> ${String(calls).padStart(3)} consumer calls; ` + `startup ${String(startup).padStart(4)} ms + processing ${work} ms = ${startup + work} ms`); } console.log(`where the job runs: on the center path, a single known region; ` + `on the edge path, a location the platform chooses, and the choice is not recorded\n`); // return path: the center store is remote, the edge store is local but must propagate first const returnTrip = (path) => EDGES.reduce((t, u) => t + u.reads * (2 * u[path] + STORE_READ), 0) / TOTAL; console.log(`weighted return trip: center ${returnTrip("center").toFixed(1)} ms, edge ${returnTrip("edge").toFixed(1)} ms`); console.log(`until the result is ready: center ${returnTrip("center").toFixed(1)} ms, ` + `edge ${(returnTrip("edge") + PROPAGATION).toFixed(1)} ms (propagation ${PROPAGATION} ms added)`); console.log(`end-to-end total: center ${(weighted("center") + returnTrip("center")).toFixed(1)} ms, ` + `edge ${(weighted("edge") + returnTrip("edge") + PROPAGATION).toFixed(1)} ms`); ``` ``` edge reads center trip edge trip gain edge-west 610 33 ms 13 ms 2.5x edge-north 520 85 ms 17 ms 5.0x edge-east 430 157 ms 21 ms 7.5x edge-south 350 111 ms 15 ms 7.4x edge-highland 280 201 ms 25 ms 8.0x edge-port 210 135 ms 19 ms 7.1x weighted queue-entry trip for 2400 reads: center 106.4 ms, edge 17.5 ms center batch read 250 -> 10 consumer calls; startup 400 ms + processing 28800 ms = 29200 ms edge batch read 25 -> 96 consumer calls; startup 576 ms + processing 28800 ms = 29376 ms where the job runs: on the center path, a single known region; on the edge path, a location the platform chooses, and the choice is not recorded weighted return trip: center 102.4 ms, edge 19.5 ms until the result is ready: center 102.4 ms, edge 499.5 ms (propagation 480 ms added) end-to-end total: center 208.8 ms, edge 517.0 ms ``` On the first measure, the edge wins by a wide margin: the weighted queue-entry trip drops from 106.4 ms to 17.5 ms, and at the farthest edge the gain reaches eightfold. On the second measure, the table turns around. The edge consumer starts in six milliseconds, the center in forty; but because the edge's batch size is twenty-five, 2,400 messages split into ninety-six calls, against ten in the center. **A startup that is cheaper per call ends up costlier in total because it starts nine times as often.** Where the job runs also becomes uncertain: a single known region in the center, an unrecorded location the platform chooses at the edge. The third measure is the return path. The edge's return trip is short at 19.5 ms, but because the result is written to the edge store, becoming visible at the user's edge waits on the 480 ms propagation: the end-to-end total comes to 208.8 ms in the center, 517.0 ms at the edge. **The 89 ms gained on queue entry does not make up for the 397 ms lost in making the result readable.** #### The Edge Queue's Four Constraints ```js // constraint.mjs — the edge queue's four constraints counted on the same job set: retries, retention, ordering, batch. import { TOTAL, QUEUE, jobSet, rng } from "./network.mjs"; const row = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 15 : 17)).join("").trimEnd()); const items = jobSet(20260731); // seed is visible row("constraint", "center", "edge", "measure", "edge's loss"); const deadLettered = (k) => items.filter((i) => i.retries > k.retries).length; // messages that exceed the retry budget row("retries", QUEUE.center.retries, QUEUE.edge.retries, `${TOTAL} messages`, `${deadLettered(QUEUE.edge)} dead letters (${(100 * deadLettered(QUEUE.edge) / TOTAL).toFixed(1)}%)`); const OUTAGE_DAYS = 6, DAILY = TOTAL; // consumer stopped for six days const retained = (k) => Math.min(OUTAGE_DAYS, k.retentionDays) * DAILY; row("retention", `${QUEUE.center.retentionDays} d`, `${QUEUE.edge.retentionDays} d`, `${OUTAGE_DAYS} d outage`, `${OUTAGE_DAYS * DAILY - retained(QUEUE.edge)} messages dropped`); const r = rng(770419), shuffled = items.map((i) => [r(), i]).sort((a, b) => a[0] - b[0]).map((x) => x[1]); const apply = (stream) => { const final = new Map(); for (const i of stream) final.set(i.counterId, i.seq); return final; }; const correct = apply([...items].sort((a, b) => a.seq - b.seq)), actual = apply(shuffled); const wrong = [...correct].filter(([s, n]) => actual.get(s) !== n).length; row("ordering", "per key", "none", `${correct.size} counters`, `${wrong} counters with wrong index (${(100 * wrong / correct.size).toFixed(1)}%)`); const calls = (k) => Math.ceil(TOTAL / k.batch); row("batch read", QUEUE.center.batch, QUEUE.edge.batch, `${TOTAL} messages`, `${calls(QUEUE.edge) - calls(QUEUE.center)} extra calls`); const multiRead = [...correct.values()].filter((n) => n > 1).length, edgeQ = QUEUE.edge; console.log("\nthe cost of working around it, item by item:"); for (const s of [ `retries: ${deadLettered(edgeQ)} dead letters requeued by hand -> ${deadLettered(edgeQ)} extra writes; the retention clock resets, sequence starts over`, `retention: a second durable copy for ${OUTAGE_DAYS * DAILY - retained(edgeQ)} messages; all ${OUTAGE_DAYS * DAILY} messages get written to two places`, `ordering: a buffer for ${multiRead} multi-read counters; ${TOTAL} extra state writes, a 3000 ms wait per message`, `batch read: ${calls(edgeQ)} x ${edgeQ.startup} ms = ${calls(edgeQ) * edgeQ.startup} ms, more than the center's ${calls(QUEUE.center) * QUEUE.center.startup} ms`, ]) console.log(" " + s); ``` ``` constraint center edge measure edge's loss retries 10 3 2400 messages 50 dead letters (2.1%) retention 14 d 4 d 6 d outage 4800 messages dropped ordering per key none 842 counters 438 counters with wrong index (52.0%) batch read 250 25 2400 messages 86 extra calls the cost of working around it, item by item: retries: 50 dead letters requeued by hand -> 50 extra writes; the retention clock resets, sequence starts over retention: a second durable copy for 4800 messages; all 14400 messages get written to two places ordering: a buffer for 667 multi-read counters; 2400 extra state writes, a 3000 ms wait per message batch read: 96 x 6 ms = 576 ms, more than the center's 400 ms ``` The four constraints yield four separate numbers. When the retry allowance drops from ten to three, 50 messages fall into the dead letter queue under the same error distribution; on the center path, all of them get through. When retention drops from fourteen days to four, of the 14,400 messages that pile up during a six-day outage, **4,800 fall out of the queue**; the center queue loses none. The most expensive item is ordering. Without the guarantee, **438 of 842** counters end up with the wrong index: the last read applied may not be the highest sequence number, and an invoice comes out short or over. This is not an item a setting can fix — it is a guarantee the queue never offered in the first place. The cost of working around it sits in the last block. Requeuing dead letters by hand means 50 extra writes, but the requeued message is new: its retention clock resets and its sequence number starts over. Longer retention can only be had with a second durable copy, and ordering only with a buffer kept in the name of 667 counters: all 14,400 messages get written to two places, producing 2,400 extra state writes and a three-second wait per message. **All three are larger than the 89 ms gained at the edge.** #### Which Region a Scheduled Job Triggers In ```js // schedule.mjs — the scheduled job: which region it fires in, and who owns the time-zone conversion. import { EDGES, STORE_READ } from "./network.mjs"; const BASE = 2, DRIFT_DAYS = 217, DRIFTING_BANDS = [2, 3]; // base clock 02:00, seasonal drift window const hour = (h) => `${String(((h % 24) + 24) % 24).padStart(2, "0")}:00`; const dayNote = (h) => (h < 0 ? " (previous day)" : ""); const row = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 16 : 15)).join("").trimEnd()); row("edge", "band", "single trigger", "local time", "drift window"); for (const u of EDGES) row(u.name, `+${u.band}`, hour(BASE), hour(BASE + u.band), DRIFTING_BANDS.includes(u.band) ? `${DRIFT_DAYS} d 60 min` : "none"); const localHours = new Set(EDGES.map((u) => BASE + u.band)); const bands = [...new Set(EDGES.map((u) => u.band))].sort(); console.log(`\nsingle trigger -> ${localHours.size} distinct local times; ` + `${EDGES.filter((u) => u.band === 0).length}/${EDGES.length} edges see the requested ${hour(BASE)}`); console.log(`every edge needs ${bands.length} separate trigger definitions for ${hour(BASE)}: ` + bands.map((s) => `+${s} base ${hour(BASE - s)}${dayNote(BASE - s)}`).join(", ")); const drifting = EDGES.filter((u) => DRIFTING_BANDS.includes(u.band)); console.log(`seasonal drift: ${drifting.length} edges x ${DRIFT_DAYS} days = ${drifting.length * DRIFT_DAYS} edge-days ` + `of 60 min deviation; the scheduler knows only the base clock, the function does the conversion itself\n`); const READS = 2400, PACKET = 100, rounds = Math.ceil(READS / PACKET); const duration = (u) => rounds * (2 * u.center + STORE_READ); const l = EDGES.map(duration), min = Math.min(...l), max = Math.max(...l); console.log(`nightly job: ${rounds} rounds x ${PACKET} reads; because the trigger location cannot be chosen, duration ` + `falls between ${min}-${max} ms, average ${(l.reduce((a, b) => a + b) / l.length).toFixed(0)} ms ` + `(${(max / min).toFixed(1)}x spread)`); console.log(`the cost of pinning: if the job is pinned to the center region, duration is fixed at ${min} ms, ` + `but the ability to run at the edge is given up entirely for this job`); ``` ``` edge band single trigger local time drift window edge-west +0 02:00 02:00 none edge-north +0 02:00 02:00 none edge-east +2 02:00 04:00 217 d 60 min edge-south +1 02:00 03:00 none edge-highland +3 02:00 05:00 217 d 60 min edge-port +2 02:00 04:00 217 d 60 min single trigger -> 4 distinct local times; 2/6 edges see the requested 02:00 every edge needs 4 separate trigger definitions for 02:00: +0 base 02:00, +1 base 01:00, +2 base 00:00, +3 base 23:00 (previous day) seasonal drift: 3 edges x 217 days = 651 edge-days of 60 min deviation; the scheduler knows only the base clock, the function does the conversion itself nightly job: 24 rounds x 100 reads; because the trigger location cannot be chosen, duration falls between 696-4728 ms, average 2792 ms (6.8x spread) the cost of pinning: if the job is pinned to the center region, duration is fixed at 696 ms, but the ability to run at the edge is given up entirely for this job ``` The nightly job's request to "run at 02:00 at every branch" cannot be satisfied by a single trigger definition. A trigger set on the base clock lands on four distinct local times, and only two of the six edges see the requested hour. Every edge needs four definitions for 02:00, and band +3's falls on base 23:00 — that is, **the previous day**. Two bands also drift sixty minutes on 217 days of the year: 651 edge-days of deviation. **The scheduler knows only the base clock; the decision to convert to local time is not handed over, it stays in the function's code.** The last two lines count where the trigger actually runs. The nightly job reads the center store in twenty-four rounds; because where it runs cannot be chosen, the same job finishes anywhere between 696 and 4,728 ms, averaging 2,792 ms — **a 6.8-fold range, and which edge it lands on is not our decision.** The only way to fix the duration is to pin the job to the center; then it does not run at the edge. #### Summary - The queue-entry trip fell at the edge from 106.4 ms to 17.5 ms, and the gain at the farthest edge reached 8-fold; but because the batch read dropped from 250 to 25, 2,400 messages split into 96 calls instead of 10, and total startup came to 576 ms instead of 400. - Because the result was written to the edge store, it waited on a 480 ms propagation; the end-to-end total came to 208.8 ms in the center, 517.0 ms at the edge. Acceptance sped up, delivery slowed down. - The four constraints yielded four numbers: a 3-retry allowance produced 50 dead letters, 4-day retention lost 4,800 messages in a 6-day outage, the absence of an ordering guarantee left 438 of 842 counters with the wrong index, and a batch of 25 cost 86 extra calls. Working around them cost 50 extra writes, writing 14,400 messages to two places, 2,400 extra state writes, and a 3,000 ms wait. - A single trigger fell across 4 distinct local times, and 2 of 6 edges saw the requested hour; seasonal drift came to 651 edge-days, and the nightly job took 696-4,728 ms depending on where it was triggered. #### Next Step The edge runtime reached a queue and a scheduler, but how that access gets set up was never asked. A resource's address and credentials can be handed over by building a client at runtime, or by a **declarative binding** at deploy time. The next lesson reaches the same resource both ways and counts four things: lines of code, startup work, where the credential sits, and the moment a misconfiguration is noticed. ### Bindings and Service Connections Source: https://academia.sh/en/courses/serverless/bindings-and-service-connections The same resource is reached by building a client at runtime and by a declarative binding; lines of code, startup work, where the credential sits, and the moment an error is seen are counted. The binding's scope and portability constraints are also measured. The previous lesson used a queue and a scheduler at the edge, but never asked how access to these resources gets set up. There are two paths. In the first, the function reads the resource's address, credential, and settings on startup, builds a client, and connects. In the second, a **binding** is written into the deployment definition, and the runtime hands the function a ready-made handle. This lesson reaches the same five resources both ways and counts four things: lines of code, startup work, where the credential sits, and when a misconfiguration is noticed. How the resource gets **created** is not this lesson's subject; the provider interface and resource fields were measured in the infrastructure-as-code course. The question here comes after that: how does a resource get connected to running code. The measurement again runs on the fictional **measurement network**; the catalog, the client, and the binding handle are set up with `node` as a **model**. **EC31.** The fictional catalog has nine resources and nine types; for each type, the fields the client has to set up are given. **EC32.** The declarative binding supports only five of these types; the remaining four are reached by building a client at runtime. **EC33.** The client path opens every connection at startup, and each connection has a fixed setup time; the binding path opens no connection at startup, and the handle connects on first use. **EC34.** The edge runtime's cold start baseline is 12 ms; work done at startup is added on top of that baseline. **EC35.** The credential is written to the process environment on the client path and passes from the config object to the client; on the binding path it never enters the code at all. The secret value that appears in the text is a **fictional placeholder**. **EC36.** At deploy time, a validator compares the manifest against the catalog and catches structural errors; there is no such check on the client path — the error is first seen on the first call to the resource. #### The Same Resource, Two Access Paths ```js // resource.mjs — fictional resource catalog, two access paths, and a credential scanner (model). export const TYPES = { // fields the client has to set up data_store: { fields: ["address", "credential", "timeout", "retries", "pool"], bindable: true }, queue: { fields: ["address", "credential", "timeout", "batch"], bindable: true }, object_store: { fields: ["address", "credential", "region", "timeout"], bindable: true }, key_value: { fields: ["address", "credential", "timeout"], bindable: true }, topic: { fields: ["address", "credential", "timeout"], bindable: true }, relational: { fields: ["address", "credential", "timeout", "pool", "schema"], bindable: false }, external_service: { fields: ["address", "credential", "timeout", "retries"], bindable: false }, directory: { fields: ["address", "credential", "timeout"], bindable: false }, stream: { fields: ["address", "credential", "region", "timeout", "batch"], bindable: false }, }; export const RESOURCES = [ // startup: connection setup ms; firstCall: deploy to first call, min { name: "measurement_store", type: "data_store", startup: 34, firstCall: 0.2 }, { name: "counter_queue", type: "queue", startup: 21, firstCall: 0.2 }, { name: "archive_store", type: "object_store", startup: 27, firstCall: 540 }, { name: "rate_counter", type: "key_value", startup: 8, firstCall: 0.2 }, { name: "notification_topic", type: "topic", startup: 15, firstCall: 47 }, { name: "invoice_table", type: "relational", startup: 52, firstCall: 540 }, { name: "field_gateway", type: "external_service", startup: 19, firstCall: 47 }, { name: "identity_directory", type: "directory", startup: 23, firstCall: 0.2 }, { name: "report_stream", type: "stream", startup: 30, firstCall: 540 }, ]; export const BINDABLE = RESOURCES.filter((k) => TYPES[k.type].bindable); export const COLD_BASE = 12; // cold start baseline, ms export const SECRET = "FICTIONAL-SECRET-VALUE"; // fictional placeholder; not a real credential export const clientCode = (k) => [ // runtime client setup code `const ${k.name}_config = {`, ...TYPES[k.type].fields.map((a) => ` ${a}: env("${k.name.toUpperCase()}_${a.toUpperCase()}"),`), `};`, `const ${k.name} = makeClient("${k.type}", ${k.name}_config);`, `await ${k.name}.connect();`, ]; export const bindingLine = (k) => [` { name: "${k.name}", type: "${k.type}", target: "${k.name}-prod" },`]; export const COMMON = 12; // helper functions' lines export const WRAPPER = 2; // manifest's opening and closing lines export function makeClient(type, config) { // credential sits in the config object and the client return { type, config, connect: () => true, call: (n) => `${type}:${n}` }; } export function bindingHandle(k) { // binding handle: no credential field at all return { name: k.name, type: k.type, call: (n) => `${k.type}:${n}` }; } export function scan(obj, path = "root", found = []) { // looks for a credential field in the object graph for (const [a, d] of Object.entries(obj)) { if (a === "credential") found.push(`${path}.${a}`); else if (d && typeof d === "object") scan(d, `${path}.${a}`, found); } return found; } export function rng(seed) { // seed is visible; same sequence run to run let s = seed >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; } ``` ```js // code.mjs — the same five resources, two ways: lines of code and startup work counted. import { TYPES, RESOURCES, BINDABLE, COLD_BASE, clientCode, bindingLine, COMMON, WRAPPER, rng } from "./resource.mjs"; const row = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 16 : 21)).join("").trimEnd()); row("resource", "type", "fields", "client lines", "binding lines"); for (const k of BINDABLE) row(k.name, k.type, TYPES[k.type].fields.length, clientCode(k).length, bindingLine(k).length); const clientLines = BINDABLE.reduce((t, k) => t + clientCode(k).length, 0) + COMMON; const bindingLines = BINDABLE.reduce((t, k) => t + bindingLine(k).length, 0) + WRAPPER; console.log(`total: client ${clientLines} lines (${COMMON} lines of shared helper included), ` + `binding ${bindingLines} lines (${WRAPPER} lines of manifest wrapper included); ` + `ratio ${(clientLines / bindingLines).toFixed(1)}x\n`); const startup = BINDABLE.reduce((t, k) => t + k.startup, 0); console.log(`the client path opens ${BINDABLE.length} connections at startup: ${startup} ms; ` + `cold start ${COLD_BASE} ms -> ${COLD_BASE + startup} ms`); console.log(`the binding path opens no connections at startup: 0 ms; ` + `cold start ${COLD_BASE} ms -> ${COLD_BASE} ms (the handle connects on first use)\n`); const r = rng(19870504), COLD_STARTS = 40; // forty cold starts let clientWork = 0, bindingWork = 0, touched = 0; for (let i = 0; i < COLD_STARTS; i++) { const subset = BINDABLE.filter(() => r() < 0.45); clientWork += startup; // all get set up, whether used or not bindingWork += subset.reduce((t, k) => t + k.startup, 0); touched += subset.length; } console.log(`${COLD_STARTS} cold starts, average resources touched ` + `${(touched / COLD_STARTS).toFixed(2)}/${BINDABLE.length}`); console.log(`work done at startup: client ${clientWork} ms, binding ${bindingWork} ms ` + `(${(100 * (clientWork - bindingWork) / clientWork).toFixed(1)}% less)`); const outside = RESOURCES.filter((k) => !TYPES[k.type].bindable); const outsideLines = outside.reduce((t, k) => t + clientCode(k).length, 0) + COMMON; console.log(`\nthe binding's scope: ${BINDABLE.length}/${RESOURCES.length} resource types; ` + `the remaining ${outside.length} resources (${outside.map((k) => k.type).join(", ")}) are set up with a client`); console.log(`the mixed reality: ${outsideLines} lines of client code and ` + `${outside.reduce((t, k) => t + k.startup, 0)} ms of startup work stay in place`); ``` ``` resource type fields client lines binding lines measurement_store data_store 5 9 1 counter_queue queue 4 8 1 archive_store object_store 4 8 1 rate_counter key_value 3 7 1 notification_topic topic 3 7 1 total: client 51 lines (12 lines of shared helper included), binding 7 lines (2 lines of manifest wrapper included); ratio 7.3x the client path opens 5 connections at startup: 105 ms; cold start 12 ms -> 117 ms the binding path opens no connections at startup: 0 ms; cold start 12 ms -> 12 ms (the handle connects on first use) 40 cold starts, average resources touched 2.20/5 work done at startup: client 4200 ms, binding 1936 ms (53.9% less) the binding's scope: 5/9 resource types; the remaining 4 resources (relational, external_service, directory, stream) are set up with a client the mixed reality: 45 lines of client code and 124 ms of startup work stay in place ``` Lines of code is the first measure. Reaching five resources with a client costs 51 lines, with a binding 7. The difference is not a style choice, it is where the work happens: on the client path every field is a line, in a binding all the fields collapse behind a single line. The second measure is work done at startup, and it is added directly to cold start. The client path opens all five connections on startup: 105 ms, so the 12 ms baseline rises to **117 ms**. The binding path opens no connection; the baseline stays at 12 ms, and connecting happens only when the handle is used. Across a fictional traffic sample of forty cold starts, a call touches 2.20 resources on average, so the binding path does 1,936 ms of connection work instead of 4,200 — **53.9 percent less.** The difference is that the client path also opens connections that will never be used on that call. The third line gives the scope. The binding recognizes **five** of the catalog's nine resource types; for the relational database, external service, directory, and stream, there is no path other than building a client. These four resources keep 45 lines of client code and 124 ms of startup work in place. #### Where the Credential Sits and When the Error Is Seen ```js // trust.mjs — where the credential sits, when the error is seen, what a move retains. import { TYPES, RESOURCES, BINDABLE, SECRET, makeClient, bindingHandle, scan, clientCode, bindingLine, COMMON, WRAPPER } from "./resource.mjs"; const env = {}; // the credential is written to the process environment first for (const k of BINDABLE) for (const a of TYPES[k.type].fields) env[`${k.name.toUpperCase()}_${a.toUpperCase()}`] = a === "credential" ? `${SECRET}-${k.name}` : `value-${a}`; const clients = Object.fromEntries(BINDABLE.map((k) => [k.name, makeClient(k.type, Object.fromEntries(TYPES[k.type].fields.map((a) => [a, env[`${k.name.toUpperCase()}_${a.toUpperCase()}`]])))])); const handles = Object.fromEntries(BINDABLE.map((k) => [k.name, bindingHandle(k)])); const count = (obj) => (JSON.stringify(obj).match(new RegExp(SECRET, "g")) || []).length; const row = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 15 : 28)).join("").trimEnd()); row("where the credential sits", "client", "binding"); row("deploy configuration", BINDABLE.length, BINDABLE.length); row("process environment", Object.values(env).filter((v) => v.startsWith(SECRET)).length, 0); row("object in code", scan(clients).length, scan(handles).length); row("when serialized", count(clients), count(handles)); console.log(`scanned field paths: ${scan(clients).slice(0, 2).join(", ")} ...\n`); const ERRORS = [ // five fictional misconfigurations { resource: "measurement_store", what: "field name misspelled", structural: true }, { resource: "archive_store", what: "required field omitted", structural: true }, { resource: "notification_topic", what: "type not in the catalog", structural: true }, { resource: "counter_queue", what: "wrong resource name", structural: true }, { resource: "archive_store", what: "points to a different existing target", structural: false }, ]; const firstCallOf = (name) => RESOURCES.find((k) => k.name === name).firstCall; const row2 = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 17 : 56)).join("").trimEnd()); row2("misconfiguration", "client", "binding"); for (const e of ERRORS) row2(`${e.resource}: ${e.what}`, `${firstCallOf(e.resource)} min later`, e.structural ? "at deploy, 0 min" : `${firstCallOf(e.resource)} min later`); const late = ERRORS.filter((e) => !e.structural); console.log(`client: 0/${ERRORS.length} caught at deploy, total delay ` + `${ERRORS.reduce((t, e) => t + firstCallOf(e.resource), 0).toFixed(1)} min, worst ${Math.max(...ERRORS.map((e) => firstCallOf(e.resource)))} min`); console.log(`binding: ${ERRORS.length - late.length}/${ERRORS.length} caught at deploy, ` + `the remaining ${late.length} error surfaces ${late.reduce((t, e) => t + firstCallOf(e.resource), 0)} min later\n`); const STUB = 3; // fake handle lines written for each handle const changedClient = BINDABLE.reduce((t, k) => t + TYPES[k.type].fields.length, 0); const changedBinding = BINDABLE.reduce((t, k) => t + bindingLine(k).length, 0) + WRAPPER + BINDABLE.length * STUB; console.log(`moving to a different service: 0 lines of code change on the client path, ` + `${changedClient} configuration values change`); console.log(`on the binding path, ${changedBinding} lines change: ` + `${changedBinding - BINDABLE.length * STUB} manifest lines are service-specific, ` + `${BINDABLE.length * STUB} lines are fake handles for local testing`); console.log(`${BINDABLE.reduce((t, k) => t + clientCode(k).length, 0) + COMMON} lines of client code ` + `move as-is; the binding definition does not move`); ``` ``` where the credential sits client binding deploy configuration 5 5 process environment 5 0 object in code 5 0 when serialized 5 0 scanned field paths: root.measurement_store.config.credential, root.counter_queue.config.credential ... misconfiguration client binding measurement_store: field name misspelled 0.2 min later at deploy, 0 min archive_store: required field omitted 540 min later at deploy, 0 min notification_topic: type not in the catalog 47 min later at deploy, 0 min counter_queue: wrong resource name 0.2 min later at deploy, 0 min archive_store: points to a different existing target 540 min later 540 min later client: 0/5 caught at deploy, total delay 1127.4 min, worst 540 min binding: 4/5 caught at deploy, the remaining 1 error surfaces 540 min later moving to a different service: 0 lines of code change on the client path, 19 configuration values change on the binding path, 22 lines change: 7 manifest lines are service-specific, 15 lines are fake handles for local testing 51 lines of client code move as-is; the binding definition does not move ``` The first table counts how many places the credential sits, and the two paths agree on one row: the five credentials in the deploy configuration exist on both paths, because access authority has to be defined somewhere. The split starts after that. On the client path the credential is written to the process environment, passes from there to the config object, and from there to the client; the scanning function finds five `credential` fields in the object graph, and the value appears five times when the object is serialized. On the binding path all four are zero: **the handle object has no field called credential at all** — the runtime signs the call itself. A log line, an error dump can expose five values on the client path; on the binding path there is no value left to expose. The second table counts when the error is seen. Four of the five fictional misconfigurations are structural: a misspelled field name, a missing required field, a type not in the catalog, a wrong resource name. On the binding path, the deploy validator catches all four at minute zero. On the client path none of them are caught, because there is no declaration to validate; every error is first seen the moment it is called against the resource. **Because the archive store is only called during the nightly job, its error surfaces 540 minutes after deploy** — nine hours. The total delay for the five errors is 1,127.4 minutes on the client path, 540 minutes on the binding path. The fifth error is seen late on both paths, and it draws the boundary of validation: the target name is valid, the type is correct, it has a match in the catalog — it just points to the wrong resource. **The deploy validator checks form, not intent.** #### The Binding's Constraint The last three lines count portability; the binding's second constraint is here. When the client code moves, **zero lines change**, only 19 configuration values change: the code recognizes the resource by its address and settings, not by the service. The binding definition, though, is service-specific: all 7 manifest lines get rewritten for the new service's shape. On top of that, because the handle is handed over by the runtime, it has no counterpart in local testing: five handles need 15 lines of fake handles. The cost of moving is **22 lines** on the binding path, zero lines and 19 values on the client path. Read together, the two constraints show the binding's real scope. Five of nine resources bind, four do not; for those four, 45 lines of client code, 124 ms of startup work, and credentials sitting in the process all stay in place. **The binding does not take the credential out of the code — it takes it out for five of the resources' nine.** The decision handed over is establishing the connection; in exchange, 105 ms is gained at cold start, 44 lines in code, and 587 minutes in when the error is seen — but the choice of resource type is bound to the catalog, and the deployment definition to the service. #### Summary - Reaching five resources with a client took 51 lines, with a binding 7 lines; the ratio is 7.3-fold. - The client path opened 5 connections at startup and raised cold start from 12 ms to 117 ms; the binding stayed at 12 ms. Across forty cold starts, a call touched an average of 2.20 resources, and connection work came to 1,936 ms instead of 4,200. - The credential sat in 4 places on the client path (deploy configuration, process environment, config object, serialized client); on the binding path it sat only in the deploy configuration and appeared 0 times in code. - 4 of 5 misconfigurations were caught at deploy time with the binding; 0 were caught with the client, and the total was seen 1,127.4 minutes later. The worst was 540 minutes, on the resource called by the nightly job. - The binding recognized 5 of 9 resource types; the remaining 4 resources kept 45 lines of client code and 124 ms of startup work in place. - The portability cost came to 22 lines with the binding (7 manifest, 15 fake handles); 0 lines of code and 19 configuration values with the client. #### Next Step Across this topic, the edge won ground one step at a time: request processing, storage, coordinated state, asynchronous work, and declarative access. Every gain came with a constraint. What remains is to look at the sum of those constraints: which limits no workaround gets the edge runtime past, and where the threshold ends past which moving a job to the edge stops making sense. The next lesson adds up these limits and closes the course. ### Edge Limits Source: https://academia.sh/en/courses/serverless/edge-limits The edge's three thresholds are applied to a set of twelve jobs; the cost of splitting is worked out in extra trips and failure points, and the three cases where the edge does not pay off are counted. The course wrap-up sums the delegated decision and the constraint of twenty lessons. The previous lesson made resource access declarative. One thing was left untested: **how much work can actually be done at the edge**. Requests were processed, state was held, a queue was set up, a resource was bound — all of it assumed to fit at the edge. This lesson applies three thresholds to a job set and works out the bill for fitting what does not fit. The measurement runs on the fictional **measurement network** and is a **model**: the thresholds, the job set, and the user groups are set up with `node`, and prices are a fictional unit. **EC37.** The three thresholds are fictional: 50 ms of processing per request, 128 MB of memory, and the missing interface items counted in the first lesson. **EC38.** The job set has twelve jobs; for each one, processing time, peak memory, and requested items are given. **EC39.** A split job uses 80% of the threshold; between parts, intermediate state is written to the store (64 ms) and read (2 ms). A failure point is the step count plus the transition count. **EC40.** A job that hits compatibility cannot be split; it falls back to the center, and the edge only routes it. **EC41.** For six user groups, the request round trip is given both for ending at the edge and for going straight to the center; a request that passes through the edge on its way to the center pays the sum of the two. **EC42.** The standing server does the same job at a 0.55 factor; the function's cold rate is 2%, its cold extra is 900 ms. #### Three Thresholds, Twelve Jobs ```js // limit.mjs — the edge runtime's three thresholds and the fictional job set (model). export const THRESHOLD = { time: 50, memory: 128 }; // ms processing per request, MB memory export const MISSING = ["file system", "local disk", "long-running process", "raw socket", "native binary addon"]; export const JOBS = [ // processing ms, memory MB, requested missing items { name: "request routing", time: 3, memory: 6 }, { name: "header editing", time: 2, memory: 4 }, { name: "session validation", time: 9, memory: 12 }, { name: "tariff lookup", time: 6, memory: 22 }, { name: "read-format conversion", time: 18, memory: 40 }, { name: "counter signature validation", time: 34, memory: 48 }, { name: "anomaly threshold", time: 62, memory: 90 }, { name: "consumption curve", time: 140, memory: 260 }, { name: "log archiving", time: 40, memory: 96, requires: ["file system", "local disk"] }, { name: "invoice document", time: 220, memory: 180, requires: ["file system", "native binary addon"] }, { name: "nightly reconciliation", time: 40000, memory: 700, requires: ["long-running process", "file system"] }, { name: "persistent device connection", time: 5, memory: 30, requires: ["raw socket", "long-running process"] }, ]; export const SHARE = 0.8; // share of the threshold a part may use export const PART_FIXED = 20; // fixed memory per part, MB export const WRITE = 64, READ = 2, FALLBACK = 60; // store write/read, edge-region trip, ms export function check(i) { // the three thresholds are applied separately const exceeded = []; if (i.time > THRESHOLD.time) exceeded.push("time"); if (i.memory > THRESHOLD.memory) exceeded.push("memory"); const missing = (i.requires ?? []).filter((k) => MISSING.includes(k)); if (missing.length) exceeded.push("compatibility"); return { ...i, exceeded, missing }; } export function split(i) { // only a job exceeding time/memory can be split const usable = THRESHOLD.time * SHARE; const parts = Math.ceil(i.time / usable); const peakMemory = Math.ceil(i.memory / parts) + PART_FIXED; return { parts, peakMemory, extraTrips: (parts - 1) * 2, extraTime: (parts - 1) * (WRITE + READ), failurePoints: 2 * parts - 1 }; } export const row = (...s) => console.log(s.map((x, i) => String(x).padEnd(i ? 15 : 33)).join("").trimEnd()); ``` ```js // threshold.mjs — the three thresholds applied to the job set; how many jobs run at the edge and what splitting costs. import { THRESHOLD, MISSING, JOBS, SHARE, check, split, FALLBACK, row } from "./limit.mjs"; console.log(`thresholds: time ${THRESHOLD.time} ms/request, memory ${THRESHOLD.memory} MB, ` + `interface items missing at the edge ${MISSING.length}\n`); row("job", "time", "memory", "missing items", "result"); const d = JOBS.map(check); for (const i of d) row(i.name, `${i.time} ms`, `${i.memory} MB`, i.missing.length, i.exceeded.length ? `${i.exceeded.join(" + ")} exceeded` : "runs"); const countBy = (e) => d.filter((i) => i.exceeded.includes(e)).length; const running = d.filter((i) => !i.exceeded.length); console.log(`\n${running.length} of ${JOBS.length} jobs run at the edge; ` + `time stops ${countBy("time")}, memory stops ${countBy("memory")}, compatibility stops ${countBy("compatibility")} ` + `(a job can hit more than one threshold)\n`); const splittable = d.filter((i) => i.exceeded.length && !i.exceeded.includes("compatibility")); const fallenBack = d.filter((i) => i.exceeded.includes("compatibility")); row("split job", "parts", "peak memory", "extra trips", "extra time", "failure points"); let extraTrips = 0, extraTime = 0, failurePoints = 0; for (const i of splittable) { const p = split(i); extraTrips += p.extraTrips; extraTime += p.extraTime; failurePoints += p.failurePoints; row(i.name, p.parts, `${p.peakMemory} MB`, p.extraTrips, `${p.extraTime} ms`, p.failurePoints); } console.log(`\nsplitting brings ${splittable.length} jobs into the edge: threshold share ${SHARE * 100}%, ` + `total ${extraTrips} extra trips, ${extraTime} ms extra time, ${failurePoints} failure points (1 per job before splitting)`); console.log(`the ${fallenBack.length} jobs that cannot be split fall back to the center (${fallenBack.map((i) => i.name).join(", ")}); ` + `the cost is a ${FALLBACK} ms edge-region trip per request, the edge only routes`); console.log(`final state: ${running.length + splittable.length}/${JOBS.length} at the edge, ` + `${fallenBack.length}/${JOBS.length} at the center`); ``` ``` thresholds: time 50 ms/request, memory 128 MB, interface items missing at the edge 5 job time memory missing items result request routing 3 ms 6 MB 0 runs header editing 2 ms 4 MB 0 runs session validation 9 ms 12 MB 0 runs tariff lookup 6 ms 22 MB 0 runs read-format conversion 18 ms 40 MB 0 runs counter signature validation 34 ms 48 MB 0 runs anomaly threshold 62 ms 90 MB 0 time exceeded consumption curve 140 ms 260 MB 0 time + memory exceeded log archiving 40 ms 96 MB 2 compatibility exceeded invoice document 220 ms 180 MB 2 time + memory + compatibility exceeded nightly reconciliation 40000 ms 700 MB 2 time + memory + compatibility exceeded persistent device connection 5 ms 30 MB 2 compatibility exceeded 6 of 12 jobs run at the edge; time stops 4, memory stops 3, compatibility stops 4 (a job can hit more than one threshold) split job parts peak memory extra trips extra time failure points anomaly threshold 2 65 MB 2 66 ms 3 consumption curve 4 85 MB 6 198 ms 7 splitting brings 2 jobs into the edge: threshold share 80%, total 8 extra trips, 264 ms extra time, 10 failure points (1 per job before splitting) the 4 jobs that cannot be split fall back to the center (log archiving, invoice document, nightly reconciliation, persistent device connection); the cost is a 60 ms edge-region trip per request, the edge only routes final state: 8/12 at the edge, 4/12 at the center ``` The three thresholds pass six of the twelve jobs; among the remaining six, time stops four, memory stops three, compatibility stops four, and two jobs hit all three at once. The second table writes down what splitting buys. Two jobs that hit time and memory can be split, and splitting divides memory too: the consumption curve's peak memory drops from 260 MB to 85 MB. The cost is eight extra trips, 264 ms, and **ten failure points** — before splitting, each job was a single point. The four jobs that hit compatibility cannot be split, because what is missing is not time, it is an interface item. #### Where the Edge Does Not Pay Off ```js // tier.mjs — where the edge does not pay off, and the same job compared across three tiers. import { JOBS, split, row } from "./limit.mjs"; const GROUP = [ // trip durations ms, model input { name: "branch-a", requests: 2600, atEdge: 8, direct: 124 }, { name: "branch-b", requests: 2100, atEdge: 12, direct: 116 }, { name: "branch-c", requests: 1800, atEdge: 18, direct: 80 }, { name: "center-campus", requests: 1500, atEdge: 22, direct: 14 }, { name: "in-datacenter", requests: 1200, atEdge: 28, direct: 6 }, { name: "branch-d", requests: 800, atEdge: 14, direct: 132 }, ]; // a request going to the center passes through the edge, so direct adds the edge's own trip const N = GROUP.reduce((t, g) => t + g.requests, 0); const avg = (f) => GROUP.reduce((t, g) => t + g.requests * f(g), 0) / N; const losing = GROUP.filter((g) => g.direct <= g.atEdge); const kn = losing.reduce((t, g) => t + g.requests, 0); console.log(`${N} requests, ${GROUP.length} groups: weighted gain for a request ending at the edge ` + `${avg((g) => g.direct - g.atEdge).toFixed(1)} ms, loss for a request going to the center ` + `${avg((g) => g.atEdge).toFixed(1)} ms`); console.log(`${losing.length}/${GROUP.length} groups (${kn} requests, ${(100 * kn / N).toFixed(0)}%) ` + `are already close to the resource: the edge charges them ${losing.map((g) => g.direct - g.atEdge).join(" and ")} ms\n`); const WRITE = 64, REGION_WRITE = 10, SERVER_FACTOR = 0.55; const edgeWrite = avg((g) => g.atEdge) + WRITE, regionWrite = avg((g) => g.direct) + REGION_WRITE; console.log(`consistent write: from the edge ${edgeWrite.toFixed(1)} ms, from the region ${regionWrite.toFixed(1)} ms; gain ` + `${(regionWrite - edgeWrite).toFixed(1)} ms, ${(100 * (regionWrite - edgeWrite) / avg((g) => g.direct - g.atEdge)).toFixed(0)}% ` + `of the edge-terminated gain — the ${WRITE} ms never comes off`); const heavy = JOBS.find((i) => i.name === "consumption curve"), heavyParts = split(heavy), serverMs = heavy.time * SERVER_FACTOR; console.log(`heavy computation (${heavy.time} ms): at the edge in ${heavyParts.parts} parts ${heavy.time + heavyParts.extraTime} ms, ` + `on the server ${Math.round(serverMs)} ms — ${((heavy.time + heavyParts.extraTime) / serverMs).toFixed(1)}x\n`); const JOB = JOBS.find((i) => i.name === "read-format conversion"); const MONTHLY = 12_000_000, COLD_RATE = 0.02, COLD_EXTRA = 900; const RATE = { edge: 0.0000003, func: 0.0000012, serverHour: 0.096 }; // fictional unit const concurrent = Math.ceil((MONTHLY / 2_592_000) * (JOB.time / 1000)); const edgeAvg = avg((g) => g.atEdge), regionAvg = avg((g) => g.direct); const TIER = [ ["edge runtime", edgeAvg + JOB.time, "50 ms / 128 MB / 5 missing items", MONTHLY * RATE.edge], ["function execution environment", regionAvg + JOB.time + COLD_RATE * COLD_EXTRA, "900 s / 3008 MB / cold", MONTHLY * RATE.func], ["standing server", regionAvg + JOB.time * SERVER_FACTOR, "capacity upfront / scaling by hand", Math.max(1, Math.ceil(concurrent / 40)) * 730 * RATE.serverHour], ]; console.log(`the same job "${JOB.name}", ${MONTHLY / 1e6} million requests a month; units are fictional`); row("tier", "latency", "monthly units", "constraint"); for (const [name, ms, constraint, units] of TIER) row(name, `${ms.toFixed(1)} ms`, units.toFixed(2), constraint); const [e, f, s] = TIER.map((x) => x[3]); console.log(`the edge is ${(f / e).toFixed(1)}x cheaper than the function, ${(s / e).toFixed(0)}x cheaper than the server; ` + `but this row can only be written for the ${JOBS.length - 4}/${JOBS.length} jobs that fit the three thresholds`); ``` ``` 10000 requests, 6 groups: weighted gain for a request ending at the edge 68.8 ms, loss for a request going to the center 15.6 ms 2/6 groups (2700 requests, 27%) are already close to the resource: the edge charges them -8 and -22 ms consistent write: from the edge 79.6 ms, from the region 94.4 ms; gain 14.8 ms, 21% of the edge-terminated gain — the 64 ms never comes off heavy computation (140 ms): at the edge in 4 parts 338 ms, on the server 77 ms — 4.4x the same job "read-format conversion", 12 million requests a month; units are fictional tier latency monthly units constraint edge runtime 33.6 ms 3.60 50 ms / 128 MB / 5 missing items function execution environment 120.4 ms 14.40 900 s / 3008 MB / cold standing server 94.3 ms 70.08 capacity upfront / scaling by hand the edge is 4.0x cheaper than the function, 19x cheaper than the server; but this row can only be written for the 8/12 jobs that fit the three thresholds ``` Three cases count where the edge does not pay off. Two of the six groups — 2,700 requests, 27% — are closer to the region than to the edge, and the edge **charges** them 8 to 22 ms. In consistent writes, because ordering sits in the center, the 64 ms never comes off: the gain drops from 68.8 to 14.8 ms. In heavy computation, a 140 ms job costs 338 ms at the edge, 77 ms on the server. The last table sets three tiers side by side on the same job. For a job that fits the thresholds, the edge wins on all three measures: 33.6 ms and 3.60 fictional units a month — a quarter of the function's cost, a nineteenth of the server's. The condition is in the last sentence: **it can only be written for eight of the twelve jobs.** #### Summary - The three thresholds passed 6 of the twelve jobs; time stopped 4, memory stopped 3, compatibility stopped 4. - Splitting brought 2 more jobs into the edge and dropped peak memory from 260 MB to 85 MB; the cost is 8 extra trips, 264 ms, and 10 total failure points instead of 1 per job. The 4 jobs that hit compatibility cannot be split and fall back to the center: 8/12 at the edge, 4/12 at the center. - The edge does not pay off in three places: 2,700 requests (27%) are already close to the region, the consistent-write gain drops from 68.8 to 14.8 ms, and a 140 ms computation takes 338 ms at the edge against 77 ms on the server. - For a job that fits, the edge is 4 times cheaper than the function and 19 times cheaper than the server at 33.6 ms and 3.60 units — but only for 8/12 of the set. #### Course Wrap-Up | Lesson | Decision handed over | Constraint that comes back | Cost of working around it | |---|---|---|---| | FC-01 Serverless concept | capacity, 8 decisions | concurrency 200 | 11,520 idle instance-minutes | | FC-02 Invocation models | retries | 5,487 colliding records | deduplication 5.5% | | FC-03 Cold start | pool lifetime | cold call 1,300 ms | warming 50% of the bill | | FC-04 Resource limits | memory tier | 6 of 12 do not fit | 246 calls, 245 records | | FC-05 Aliases | traffic split | +100 ms at a 1% share | 15 synthetic calls | | FC-06 Shared layers | dependency location | cold-start floor 745 → 2,093 | 6 republishes | | FC-07 Jobs that do not fit | standing capacity | 2 of 5 do not fit | 13 failure points | | ED-01 Event bus | routing | 433 silent drops | 781 wasted wake-ups | | ED-02 Queue and topic | consumer scaling | 5 tunable items | 1 message, 7 requests | | ED-03 Orchestration | flow state | 1,119 transitions, 280 units | 840 units if split into three | | ED-04 Scheduled triggering | the clock | 3,600 duplicate invoices | a scope query per run | | ED-05 Gateway service | gateway operation | 102 configuration items | 48,469 re-invocations | | ED-06 Notification services | sending | 25-day throttling | 630 events, 6.3 units | | EC-01 Runtime | setup and version | 7 of 13 items missing | +9 round trips per request | | EC-02 Request processing | placement | 3 of 7 do not run at the edge | +64 ms, 640 s | | EC-03 Edge storage | replication | 12,857 stale reads | read 2 → 62 ms | | EC-04 Durable state objects | concurrency | cap 166.7 req/s | key split, 6 reads | | EC-05 Edge queues | asynchronous work | 438 wrong index | 2,400 extra writes | | EC-06 Bindings | connection setup | 5 of 9 types | 45 lines of code, 124 ms | | EC-07 Edge limits | processing and memory | 6 of 12 fit | 8 extra trips, 10 failure points | The course's rule is the same across all twenty rows: **handing over capacity management does not eliminate it; it comes back as a constraint, and that constraint decides the design.** The third column falls into four families: in eight rows the constraint is **a numbered upper bound** (concurrency, duration, memory, interface items, requests/s), in three rows it is **a startup cost**, in four rows it is **an invisible configuration surface** (a rule, five items, a flow definition, 102 items), in five rows it is **silent loss or duplicated work**. The fourth column, in turn, is never written in the constraint's own unit: a duration limit turns into a call count, a retention period into a second copy, a missing interface into a network round trip. Four courses in a row, every layer got rented: the machine, the network, the data, and computation itself. Every rental handed over a decision and returned a countable constraint in exchange. But who all these resources **belong to**, who can access what, and whose bill it lands on was never asked. **Cloud Governance, Security and Cost** starts with that question.