--- title: 'Edge Runtime Model' source: 'https://academia.sh/en/courses/serverless/edge-runtime-model' course: 'Serverless and Edge Computing' language: en updated: '2026-09-13T05:07:39+00:00' license: 'CC BY-SA 4.0' objectives: - 'Compares three runtime tiers on three metrics: startup latency, memory baseline, offered interface surface' - 'Computes how startup latency lands across percentiles and states at which order of magnitude cold start becomes invisible' - 'Counts the missing interface items, works out which workload each one rules out, and how many network round trips it costs to work around them' --- # Edge Runtime Model The isolate is compared to the container and the virtual machine on the same metrics: startup latency and memory baseline by order of magnitude, the offered interface surface item by item. Cold start's disappearance is counted, along with the seven missing items it comes with and the cost of working around them. Events were routed, streams were set up, long-running jobs ran step by step. But all of this machinery still sits in a single region. In the fictional **measurement network**, field crews and branch sites are scattered across the region; the only thing close to the user was the cache, and the cache only stores a response already produced. Whenever a new response has to be produced, the request still goes to the region. This course moves the computation itself to the edge. The first question is: what runs at the edge? What runs in the region is a container, and the container's cost has been measured. The same cost cannot be paid at hundreds of edge locations, for dozens of different applications at each one. The thing that runs at the edge is named the **isolate**, and this lesson adds it to the familiar scale. **EC1.** Every number is a model input, not the output of a measured setup; what matters is the order-of-magnitude gap between two tiers. **EC2.** Interface items take a binary value: offered or not offered; an item offered only in restricted form counts as "not offered." **EC3.** The startup and memory values on the container and virtual machine rows are inputs from the earlier measurement; this lesson does not re-measure them, it only adds the third row. ## The Row at the Edge of the Scale Isolation tiers were compared earlier on five metrics and their cost was paid in three items. The isolate is one step further down that scale: it carries no kernel of its own, and no file system of its own either. What it isolates is a single language's **interpreter context** — the variable space, the stack, and the object set are separate, but the process underneath is shared. The new metric is therefore the width of the **offered interface surface**: how many items the runtime offers to the application. ```js // measurement-network/unit.mjs — adding the isolate to the three-tier scale // MODEL: every number is a declared model input. What matters is the // order-of-magnitude gap between two tiers, not a single cell's value. export const LEVELS = { "virtual machine": { startupMs: 25_000, memoryBaselineMB: 512 }, "container": { startupMs: 300, memoryBaselineMB: 12 }, "isolate": { startupMs: 3, memoryBaselineMB: 3 }, }; // The interface items the runtime offers to the application, and at which tier. export const SURFACE = { "request/response handling": ["virtual machine", "container", "isolate"], "timers": ["virtual machine", "container", "isolate"], "outbound network request": ["virtual machine", "container", "isolate"], "crypto primitives": ["virtual machine", "container", "isolate"], "stream processing": ["virtual machine", "container", "isolate"], "in-memory data structure": ["virtual machine", "container", "isolate"], "file system": ["virtual machine", "container"], "local disk persistence": ["virtual machine", "container"], "long-running process": ["virtual machine", "container"], "subprocess spawning": ["virtual machine", "container"], "raw sockets": ["virtual machine", "container"], "native binary addon": ["virtual machine", "container"], "kernel tuning": ["virtual machine"], }; export const NAMES = Object.keys(LEVELS); export const ITEMS = Object.keys(SURFACE); export const offered = (d) => ITEMS.filter((k) => SURFACE[k].includes(d)); const magnitude = (n) => "10^" + Math.round(Math.log10(n)); if (import.meta.main) { console.log("cost and interface surface (model input)"); console.log("tier".padEnd(17) + "startup".padEnd(11) + "magnitude".padEnd(11) + "mem baseline".padEnd(15) + "magnitude".padEnd(11) + "surface"); for (const d of NAMES) { const v = LEVELS[d]; console.log(d.padEnd(17) + `${v.startupMs} ms`.padEnd(11) + magnitude(v.startupMs).padEnd(11) + `${v.memoryBaselineMB} MB`.padEnd(15) + magnitude(v.memoryBaselineMB).padEnd(11) + `${offered(d).length}/${ITEMS.length}`); } console.log("\ninterface item (+ offered, - not offered)"); console.log("item".padEnd(28) + NAMES.map((d) => d.padEnd(17)).join("")); for (const k of ITEMS) console.log(k.padEnd(28) + NAMES.map((d) => (SURFACE[k].includes(d) ? "+" : "-").padEnd(17)).join("")); const missing = ITEMS.filter((k) => !SURFACE[k].includes("isolate")); console.log(`\n${missing.length} items missing in the isolate:`); for (const k of missing) console.log(" - " + k); } ``` ``` cost and interface surface (model input) tier startup magnitude mem baseline magnitude surface virtual machine 25000 ms 10^4 512 MB 10^3 13/13 container 300 ms 10^2 12 MB 10^1 12/13 isolate 3 ms 10^0 3 MB 10^0 6/13 interface item (+ offered, - not offered) item virtual machine container isolate request/response handling + + + timers + + + outbound network request + + + crypto primitives + + + stream processing + + + in-memory data structure + + + file system + + - local disk persistence + + - long-running process + + - subprocess spawning + + - raw sockets + + - native binary addon + + - kernel tuning + - - 7 items missing in the isolate: - file system - local disk persistence - long-running process - subprocess spawning - raw sockets - native binary addon - kernel tuning ``` Startup magnitudes drop in two-digit steps: 10^4, 10^2, 10^0. Moving from the virtual machine to the container gains two orders of magnitude; moving from the container to the isolate gains two more. The memory baseline follows the same 10^3, 10^1, 10^0 sequence. Up to this point the table is familiar — only a cheaper isolation tier has been added. The third column turns the table around. The interface surface is 13/13, 12/13, and 6/13. The virtual machine offers every item; the only item the container loses is kernel tuning, because the kernel is shared. The isolate, by contrast, loses seven of thirteen items at once. This is not a steady decline along the scale — it is a **break**: the container gives up one item for one order of magnitude of cheaper cost, the isolate gives up seven items for two orders of magnitude. ## The Disappearance of Cold Start Startup latency dropping to the 10^0 order of magnitude changes exactly one thing, and that thing is large. The most familiar flaw of a serverless function is that an instance starting from zero delays the first call. That delay is not noise next to the application's own runtime — it is an item that **drowns it out**. The run below runs the same call sequence at three tiers and counts how the delay lands across percentiles. **EC4.** The cold-call rate is 6% and its pattern is fixed; there is no randomness. **EC5.** The application's own processing time is 12 ms and does not change by tier; network latency is not in this model. **EC6.** The cost of working around a limit is measured by the number of network round trips added per request; how many milliseconds a round trip costs is the next lesson's subject. ```js // measurement-network/gain.mjs — how startup latency shows up in percentiles and // how the missing surface converts into workload. MODEL: call distribution is deterministic. import { LEVELS, NAMES, offered } from "./unit.mjs"; const CALLS = 10_000, COLD_PERCENT = 6, PROCESS_MS = 12, THRESHOLD_MS = 100; const isCold = (i) => (i * COLD_PERCENT) % 100 < COLD_PERCENT; // pattern: unseeded, fixed const percentile = (a, p) => a[Math.min(a.length - 1, Math.ceil((p / 100) * a.length) - 1)]; console.log(`${CALLS} calls, ${COLD_PERCENT}% cold, processing ${PROCESS_MS} ms (model)`); console.log("tier".padEnd(17) + "p50".padEnd(11) + "p95".padEnd(11) + "p99".padEnd(11) + "avg extra".padEnd(14) + `over ${THRESHOLD_MS} ms`); for (const d of NAMES) { const g = []; for (let i = 0; i < CALLS; i++) g.push(PROCESS_MS + (isCold(i) ? LEVELS[d].startupMs : 0)); g.sort((x, y) => x - y); const extra = g.reduce((t, v) => t + v - PROCESS_MS, 0) / CALLS; console.log(d.padEnd(17) + `${percentile(g, 50)} ms`.padEnd(11) + `${percentile(g, 95)} ms`.padEnd(11) + `${percentile(g, 99)} ms`.padEnd(11) + `${extra.toFixed(1)} ms`.padEnd(14) + String(g.filter((v) => v > THRESHOLD_MS).length)); } // Zeroing out the cold rate: the memory cost of keeping pre-warmed instances. const WARM = 40; console.log(`\nif ${WARM} instances are kept warm to zero out the cold rate`); for (const d of NAMES) console.log(" " + d.padEnd(17) + `${String(LEVELS[d].memoryBaselineMB * WARM).padStart(6)} MB idle baseline`); const WORK = { "request routing": ["request/response handling"], "header editing": ["request/response handling"], "session validation": ["request/response handling", "crypto primitives"], "read-format conversion": ["request/response handling", "stream processing"], "nightly batch processing": ["long-running process", "file system", "local disk persistence"], "invoice document": ["file system", "native binary addon"], "log archiving": ["local disk persistence", "file system"], "persistent device connection":["raw sockets", "long-running process"], }; const runs = (w, d) => WORK[w].every((k) => offered(d).includes(k)); console.log("\ndoes the workload run at the tier"); console.log("workload".padEnd(30) + NAMES.map((d) => d.padEnd(17)).join("")); for (const w of Object.keys(WORK)) console.log(w.padEnd(30) + NAMES.map((d) => (runs(w, d) ? "runs" : "no run").padEnd(17)).join("")); for (const d of NAMES) console.log(" " + d.padEnd(17) + `${Object.keys(WORK).filter((w) => runs(w, d)).length}/${Object.keys(WORK).length} workloads`); // The cost of working around a missing item: the network round trip added per request. const WORKAROUND = { "file system": [1, "read/write goes out to the object store"], "local disk persistence": [1, "state is kept in an external key-value store"], "long-running process": [3, "the job is split; each part reads state from outside"], "subprocess spawning": [2, "the call is delegated to an instance in the region"], "raw sockets": [2, "the connection terminates in the region; the edge only routes it"], "native binary addon": [0, "a pure-language equivalent is written instead; processing time grows"], "kernel tuning": [0, "there is no setting; the job cannot be held at this tier"], }; console.log("\nworking around a missing item (extra network round trip per request)"); let total = 0; for (const [k, [trip, way]] of Object.entries(WORKAROUND)) { total += trip; console.log(" " + k.padEnd(28) + `+${trip} trip ` + way); } console.log(` ${"total for the seven items".padEnd(28)}+${total} trips`); ``` ``` 10000 calls, 6% cold, processing 12 ms (model) tier p50 p95 p99 avg extra over 100 ms virtual machine 12 ms 25012 ms 25012 ms 1500.0 ms 600 container 12 ms 312 ms 312 ms 18.0 ms 600 isolate 12 ms 15 ms 15 ms 0.2 ms 0 if 40 instances are kept warm to zero out the cold rate virtual machine 20480 MB idle baseline container 480 MB idle baseline isolate 120 MB idle baseline does the workload run at the tier workload virtual machine container isolate request routing runs runs runs header editing runs runs runs session validation runs runs runs read-format conversion runs runs runs nightly batch processing runs runs no run invoice document runs runs no run log archiving runs runs no run persistent device connection runs runs no run virtual machine 8/8 workloads container 8/8 workloads isolate 4/8 workloads working around a missing item (extra network round trip per request) file system +1 trip read/write goes out to the object store local disk persistence +1 trip state is kept in an external key-value store long-running process +3 trip the job is split; each part reads state from outside subprocess spawning +2 trip the call is delegated to an instance in the region raw sockets +2 trip the connection terminates in the region; the edge only routes it native binary addon +0 trip a pure-language equivalent is written instead; processing time grows kernel tuning +0 trip there is no setting; the job cannot be held at this tier total for the seven items +9 trips ``` All three tiers share the same p50: 12 ms. The median call lands on a warm instance and never sees startup latency at all. The difference shows up at p95, because cold calls fill the 6% slice and the slowest 6% starts right there. p95 is **25,012 ms** on the virtual machine, **312 ms** on the container, **15 ms** on the isolate. The average extra delay is 1,500.0 / 18.0 / 0.2 ms respectively. The last column makes the call. At a 100 ms threshold, the virtual machine and the container produce the same number: 600 calls. Even though there is an 80-fold gap between the two tiers, **both cross the threshold**; cold start shrinks on the container but does not disappear. On the isolate, the number of calls that cross the threshold is **0**. Cold start is not improved here — it **stops being a problem**: a 3 ms startup next to a 12 ms operation is not worth measuring. This gain has to be read together with the cost of buying it on the container. The way to bring the cold rate to zero is pre-warming instances, and with forty instances kept warm, the idle memory baseline is 20,480 MB on the virtual machine, 480 MB on the container. On the isolate this row is meaningless: no warming happens, because there is nothing to warm. ## The Missing Interface and the Cost of Working Around It The decision handed over here is clear: the runtime's setup, version, capacity, and placement are gone. What comes back in exchange is a number — **7 of 13 items are missing**. Item by item: no file system, no local disk persistence, no long-running process, no subprocess spawning, no raw sockets, no native binary addon, no kernel tuning. The last item has the widest consequence: a library that depends on a native binary addon does not run at this tier — even if the application never calls it directly, if it sits anywhere in the dependency chain, the package fails to load. The workload table turns this into the fictional network. Four of the eight workloads run on the isolate: request routing, header editing, session validation, read-format conversion. Four do not: nightly batch processing, invoice document generation, log archiving, persistent device connection. The ratio is **4/8**, which means half of this same network's jobs cannot sit at the edge. The container and the virtual machine both run all eight; the split is not about the type of workload, it is about **which item it needs** — the four that run touch only the request-handling, crypto, and stream items. Working around the constraint is possible, and the cost is totaled in the last table. Using an object store instead of a file system adds **+1 network round trip** per access, an external key-value store instead of local disk adds **+1 trip**, splitting a long-running process and reading its state from outside adds **+3 trips**, and handing off subprocess and raw-socket work to the region adds two trips each. The total for the seven items is **+9 trips**. Two items add zero trips, but the cost is not zero: the native binary addon's pure-language equivalent lengthens processing time, and kernel tuning has no equivalent at all — that job cannot be held at this tier. The meaning of those nine trips is still incomplete here, because how many milliseconds a trip costs depends on where it goes from the edge. A trip from the edge to a nearby store and a trip to the region are not the same item; the first table erased 300 ms as a gain, and each workaround trip can claw back a piece of that gain. The real arithmetic of computing at the edge is the difference between these two numbers. ## Summary - The isolate is one step further down the familiar scale: startup magnitudes are 10^4, 10^2, and 10^0; memory baseline magnitudes are 10^3, 10^1, and 10^0. What it isolates is not a kernel or a file system, it is a single language's interpreter context. - The third metric turns the table around: the offered interface surface is 13/13, 12/13, and 6/13. The container gives up one item for one order of magnitude, the isolate gives up seven items for two. - Cold start disappears: p95 is 25,012, 312, and 15 ms; the number of calls crossing the 100 ms threshold is 600, 600, and 0. Keeping a warm pool of forty instances costs 20,480 and 480 MB, while on the isolate there is nothing to warm. - The constraint that comes back in exchange is spelled out item by item: no file system, local disk, long-running process, subprocess, raw sockets, native binary addon, or kernel tuning. Only 4 of the fictional network's eight workloads run at this tier. - The cost of working around the constraint is a network round trip per request: +9 trips total for the seven items. Two items add no trips, but the cost is written into processing time or into moving the job elsewhere. ## Next Step This lesson wrote down what can run at the edge but never measured where the thing that runs actually sits. If four workloads can run at the edge, the same job can also run in the region or at the origin — what decides the choice among the three places? What is the difference, in network round trips, between deciding a request at the edge and moving it to the region, and how many milliseconds do those round trips cost? When the routing decision is made at the edge, what share of requests never reaches the origin at all, and how does that share change when the same decision is authentication or an A/B split instead? The next lesson runs the same request set separately at three places, works out the round-trip count and latency of each placement, and names the work that cannot be done at the edge.