--- title: 'Cold Start' source: 'https://academia.sh/en/courses/serverless/cold-start' course: 'Serverless and Edge Computing' language: en updated: '2026-09-13T05:07:39+00:00' license: 'CC BY-SA 4.0' objectives: - 'Runs a warm pool model to count the cold call rate, the added latency, and queue formation across three traffic profiles' - "Scans pool lifetime to work out the threshold where the cold rate drops and that threshold's idle-capacity cost" - 'Measures three mitigation paths separately: pre-warming, package shrinking, and moving startup work' - 'Shows the cold-rate threshold below which moving startup work loses money, as a single number' --- # Cold Start A warm pool model is run to count whether a request finds an instance: in three fictional traffic profiles, the cold call rate, the duration added to latency, and the requests that land in the queue are measured; pool lifetime is scanned from zero to eighteen hundred seconds to work out the threshold where the cold rate drops and the idle instance-seconds that come back in return, and three mitigation paths are priced separately. The latency numbers of the previous two lessons rested on a single assumption: an instance ready to run the function was standing by when the call arrived. Because the capacity decision is handed off, this is not always true. Instances are created as requests arrive and reclaimed when idle; a request that finds no ready instance experiences a **cold start** — setting up the runtime, unpacking the package, and startup code all land inside the request path. M23/K02 measured the same trade-off — the item embedded in the image versus the work left for boot — at the scale of seconds; that measurement is not repeated here. Here the scale is milliseconds, and the object is not an image but a package. A single ratio is asked: how many calls start cold, what it depends on, and what mitigation costs. **FC13.** The measurement window is 1,800 seconds; the regional measurement network's three fictional traffic profiles are built with a self-written generator, and the seed is 20260804. **FC14.** A warm call takes 400 ms; a cold call adds 900 ms; this extra time enters latency, not the bill. **FC15.** Profiles: field notification is sparse (average 90 s gap), reading validation is regular (average 250 ms gap), and the nightly batch is bursty (once every 600 seconds, a 60-second burst, 25 ms gap). **FC16.** Pool lifetime is how long an instance sits idle before it is reclaimed; the baseline value is 300 seconds. **FC17.** The function's concurrency share is 40; once the pool is full, an incoming request joins the queue rather than being dropped. **FC18.** Prices are written in fictional units: 0.0005 per call, 0.0000025 per processing millisecond, 0.0002 per pre-warmed instance-second. ## Is There an Instance in the Pool The model rests on a single rule: if the pool has an idle instance, the call is warm; if not and the share has not filled, a new instance is created; if the share has also filled, the request joins the queue. ```js // measurement-network/warm-pool.mjs — warm if the pool has an instance when the request arrives, cold otherwise (model) export const WINDOW = 1800000, PROC_MS = 400, COLD_EXTRA_MS = 900; // ms: measurement window, processing, cold extra export const LIFETIME = 300000, SHARE_LIMIT = 40; // ms idle-kept, the function's concurrency share export const CALL_UNIT = 0.0005, MS_UNIT = 0.0000025, WARM_UNIT = 0.0002; // fictional unit export const rng = (t) => { let s = t >>> 0; return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; }; // Three fictional traffic profiles: sparse field notifications, regular validation, bursty nightly batch. export const PROFILE = { sparse: { gap: 90000, windows: [[0, WINDOW]] }, regular: { gap: 250, windows: [[0, WINDOW]] }, bursty: { gap: 25, windows: [[0, 60000], [600000, 660000], [1200000, 1260000]] }, }; export const arrivals = (name, seed = 20260804) => { const r = rng(seed), prof = PROFILE[name], out = []; for (const [start, end] of prof.windows) { let t = start; while (t < end) { t += -Math.log(1 - r()) * prof.gap; if (t < end) out.push(t); } } return out; }; const percentile = (a, p) => [...a].sort((x, y) => x - y)[Math.floor(a.length * p)]; const idle = (inst, end) => Math.max(0, end - inst.born - inst.busy) / 1000; // Pool: every instance gets a "free" timestamp the moment it frees up; once its lifetime elapses from that stamp, it is reclaimed. export const run = (arrivalTimes, { lifetime = LIFETIME, shareLimit = SHARE_LIMIT, coldExtra = COLD_EXTRA_MS, proc = PROC_MS, warm = 0, extraCallMs = 0 } = {}) => { const pool = []; for (let i = 0; i < warm; i++) pool.push({ free: 0, born: 0, busy: 0, pinned: true }); let coldCount = 0, queued = 0, longestWait = 0, idleSec = 0, mostInstances = pool.length, workMs = 0; const latency = []; for (const t of arrivalTimes) { for (let i = pool.length - 1; i >= 0; i--) { const o = pool[i]; if (!o.pinned && o.free + lifetime < t) { idleSec += idle(o, o.free + lifetime); pool.splice(i, 1); } } let inst = pool.find((o) => o.free <= t), start = t, extra = 0, wait = 0; if (!inst && pool.length < shareLimit) { inst = { free: 0, born: t, busy: 0, pinned: false }; pool.push(inst); extra = coldExtra; coldCount++; } else if (!inst) { inst = pool.reduce((a, b) => (b.free < a.free ? b : a)); wait = inst.free - t; start = inst.free; queued++; if (wait > longestWait) longestWait = wait; } const duration = extra + proc + extraCallMs; inst.free = start + duration; inst.busy += duration; workMs += duration; latency.push(wait + duration); if (pool.length > mostInstances) mostInstances = pool.length; } for (const o of pool) idleSec += idle(o, o.pinned ? WINDOW : Math.min(WINDOW, o.free + lifetime)); return { calls: arrivalTimes.length, cold: coldCount, rate: coldCount / arrivalTimes.length, avgLatency: latency.reduce((a, b) => a + b, 0) / latency.length, p99: percentile(latency, 0.99), queued, longestWait, mostInstances, idleSec, cost: arrivalTimes.length * CALL_UNIT + workMs * MS_UNIT + warm * (WINDOW / 1000) * WARM_UNIT }; }; if (import.meta.url === `file://${process.argv[1]}`) { console.log(`window ${WINDOW / 1000} s, processing ${PROC_MS} ms, cold extra ${COLD_EXTRA_MS} ms, ` + `lifetime ${LIFETIME / 1000} s, share ${SHARE_LIMIT}`); console.log("\n" + "profile".padEnd(10) + "calls".padEnd(9) + "cold".padEnd(8) + "cold rate".padEnd(12) + "avg latency".padEnd(14) + "p99".padEnd(12) + "queued".padEnd(15) + "longest wait".padEnd(17) + "most instances".padEnd(16) + "idle instance-s"); for (const name of Object.keys(PROFILE)) { const s = run(arrivals(name)); console.log(name.padEnd(10) + String(s.calls).padEnd(9) + String(s.cold).padEnd(8) + `${(s.rate * 100).toFixed(2)}%`.padEnd(12) + `${s.avgLatency.toFixed(0)} ms`.padEnd(14) + `${s.p99.toFixed(0)} ms`.padEnd(12) + String(s.queued).padEnd(15) + `${s.longestWait.toFixed(0)} ms`.padEnd(17) + String(s.mostInstances).padEnd(16) + s.idleSec.toFixed(0)); } const a = run(arrivals("bursty")); console.log(`\nin the bursty profile, cold calls add ${(a.cold * COLD_EXTRA_MS / 1000).toFixed(0)} s to latency; ` + `a warm call is ${PROC_MS} ms, a cold call is ${PROC_MS + COLD_EXTRA_MS} ms — ${((PROC_MS + COLD_EXTRA_MS) / PROC_MS).toFixed(2)}x`); } ``` ``` window 1800 s, processing 400 ms, cold extra 900 ms, lifetime 300 s, share 40 profile calls cold cold rate avg latency p99 queued longest wait most instances idle instance-s sparse 18 2 11.11% 500 ms 1300 ms 0 0 ms 1 1722 regular 7242 9 0.12% 401 ms 400 ms 0 0 ms 8 10510 bursty 7241 120 1.66% 419 ms 1300 ms 135 539 ms 40 38159 in the bursty profile, cold calls add 108 s to latency; a warm call is 400 ms, a cold call is 1300 ms — 3.25x ``` The three profiles show three separate behaviors. In regular traffic the cold rate is 0.12%: requests arrive 250 ms apart, the pool settles at eight instances, and only nine calls start cold. In sparse traffic the rate is 11.11%, and that rate comes not from load but from how infrequent the calls are. In bursty traffic the rate is 1.66%, but the real number is in a different column: **135 requests join the queue**, the longest wait is 539 ms. The queue is the second face of cold start. When the burst begins the pool is empty; every new request needs a new instance, and every instance spends 900 ms on setup. Once the share reaches 40, requests start waiting. A cold call does not only raise its own latency; by slowing how fast the pool fills, it builds a queue behind it: the average shown is 419 ms, and the 99th percentile is 1,300 ms. ## Pool Lifetime Scan Pool lifetime is a handed-off decision: the provider decides how long an instance is kept idle. The scan raises lifetime from zero to 1,800 seconds. ```js // measurement-network/lifetime-scan.mjs — the cold call rate as pool lifetime and traffic profile change (model) import { PROFILE, arrivals, run } from "./warm-pool.mjs"; export const LIFETIMES = [0, 15, 30, 60, 120, 300, 600, 1800]; // seconds export const NAMES = Object.keys(PROFILE); if (import.meta.url === `file://${process.argv[1]}`) { const arrivalsByProfile = Object.fromEntries(NAMES.map((a) => [a, arrivals(a)])); console.log("pool lifetime".padEnd(15) + NAMES.map((a) => `${a}: cold / idle`.padEnd(24)).join("") + "bursty: queued / avg latency"); const table = {}; for (const lt of LIFETIMES) { const row = Object.fromEntries(NAMES.map((a) => [a, run(arrivalsByProfile[a], { lifetime: lt * 1000 })])); table[lt] = row; console.log(`${lt} s`.padEnd(15) + NAMES.map((a) => `${(row[a].rate * 100).toFixed(2)}% / ${row[a].idleSec.toFixed(0)}`.padEnd(24)).join("") + `${row.bursty.queued} / ${row.bursty.avgLatency.toFixed(0)} ms`); } console.log(""); for (const name of NAMES) { const good = LIFETIMES.find((lt) => table[lt][name].rate < 0.01); const atZero = table[0][name], atMax = table[1800][name]; console.log(`${name.padEnd(9)} at lifetime 0: ${(atZero.rate * 100).toFixed(1)}% cold and ${atZero.idleSec.toFixed(0)} idle instance-s; ` + `at lifetime 1800: ${(atMax.rate * 100).toFixed(2)}% and ${atMax.idleSec.toFixed(0)}; ` + (good === undefined ? "never drops below 1% at any lifetime" : `drops below 1% at a ${good} s lifetime`)); } } ``` ``` pool lifetime sparse: cold / idle regular: cold / idle bursty: cold / idle bursty: queued / avg latency 0 s 100.00% / 0 100.00% / 0 49.99% / 0 3621 / 969 ms 15 s 88.89% / 266 1.35% / 7651 1.78% / 3819 135 / 420 ms 30 s 77.78% / 489 0.65% / 8493 1.73% / 5713 135 / 419 ms 60 s 72.22% / 895 0.33% / 9208 1.66% / 9359 135 / 419 ms 120 s 38.89% / 1364 0.18% / 9883 1.66% / 16559 135 / 419 ms 300 s 11.11% / 1722 0.12% / 10510 1.66% / 38159 135 / 419 ms 600 s 5.56% / 1781 0.11% / 10873 0.59% / 57609 47 / 407 ms 1800 s 5.56% / 1781 0.11% / 10873 0.55% / 69047 47 / 406 ms sparse at lifetime 0: 100.0% cold and 0 idle instance-s; at lifetime 1800: 5.56% and 1781; never drops below 1% at any lifetime regular at lifetime 0: 100.0% cold and 0 idle instance-s; at lifetime 1800: 0.11% and 10873; drops below 1% at a 30 s lifetime bursty at lifetime 0: 50.0% cold and 0 idle instance-s; at lifetime 1800: 0.55% and 69047; drops below 1% at a 600 s lifetime ``` The scan gives three separate curves. In regular traffic, a 15-second lifetime pulls the cold rate from 100% to 1.35%; beyond that it gains nothing but raises idle instance-seconds from 7,651 to 10,873. In bursty traffic the rate **does not change at all** from 15 seconds to 300 seconds — it sticks at 1.66% — and drops suddenly to 0.59% at 600 seconds. The threshold is not a coincidence: the gap between bursts is 540 seconds, and once lifetime crosses that gap, the pool survives between two bursts. On the same row, queued requests drop from 135 to 47. The price of that threshold sits on the same row: idle instance-seconds rise from 38,159 to 57,609. Cutting the cold call rate to a third is paid for by **raising idle capacity by half**. The sparse profile never drops below 1% at any lifetime; even at 1,800 seconds the rate is 5.56%, because one of the eighteen calls is the window's first, and there is no pool before it. In sparse traffic, cold start is not a tuning problem — it is a property of the traffic itself. ## Three Mitigation Paths ```js // measurement-network/mitigation.mjs — three ways to reduce cold start, and the cost of each (model) import { COLD_EXTRA_MS, PROC_MS, WINDOW, arrivals, run } from "./warm-pool.mjs"; // [name, run options, kind of cost] export const PATH = [ ["no mitigation", {}, "—"], ["pre-warming (16)", { warm: 16 }, "16 instances sit idle for 1800 s"], ["package shrink", { coldExtra: 350 }, "3 dependencies are moved out"], ["lazy loading", { coldExtra: 500, extraCallMs: 40 }, "40 ms of lazy loading on every call"], ["shrink + warming", { coldExtra: 350, warm: 16 }, "the cost of both together"], ]; if (import.meta.url === `file://${process.argv[1]}`) { const arr = arrivals("bursty"), baseline = run(arr); console.log(`bursty profile, ${arr.length} calls, pool lifetime 300 s; warm call ${PROC_MS} ms`); console.log("\n" + "path".padEnd(22) + "cold".padEnd(8) + "cold rate".padEnd(12) + "avg latency".padEnd(14) + "p99".padEnd(12) + "queued".padEnd(15) + "idle instance-s".padEnd(17) + "cost".padEnd(10) + "latency gain"); for (const [name, opts] of PATH) { const row = run(arr, opts); console.log(name.padEnd(22) + String(row.cold).padEnd(8) + `${(row.rate * 100).toFixed(2)}%`.padEnd(12) + `${row.avgLatency.toFixed(1)} ms`.padEnd(14) + `${row.p99.toFixed(0)} ms`.padEnd(12) + String(row.queued).padEnd(15) + row.idleSec.toFixed(0).padEnd(17) + row.cost.toFixed(2).padEnd(10) + `${(baseline.avgLatency - row.avgLatency).toFixed(1)} ms`); } console.log("\n" + "path".padEnd(22) + "cost".padEnd(38) + "extra cost per ms gained"); for (const [name, opts, cost] of PATH.slice(1)) { const row = run(arr, opts), gain = baseline.avgLatency - row.avgLatency; console.log(name.padEnd(22) + cost.padEnd(38) + (gain > 0 ? ((row.cost - baseline.cost) / gain).toFixed(3) : "no gain")); } const threshold = 40 / (COLD_EXTRA_MS - 500); console.log(`lazy loading only pays off once the cold rate is above ${(threshold * 100).toFixed(0)}%: ` + `40 ms lands on every call, while the 400 ms gained falls only on cold calls — in this profile the rate is 1.66%`); const warmRow = run(arr, { warm: 16 }); console.log(`\npre-warming raises capacity spent idle from ${baseline.idleSec.toFixed(0)} instance-s to ` + `${warmRow.idleSec.toFixed(0)} instance-s; over the ${WINDOW / 1000} s window that is ` + `${(warmRow.cost - baseline.cost).toFixed(2)} extra units — the cold extra itself is ${COLD_EXTRA_MS} ms`); } ``` ``` bursty profile, 7241 calls, pool lifetime 300 s; warm call 400 ms path cold cold rate avg latency p99 queued idle instance-s cost latency gain no mitigation 120 1.66% 418.9 ms 1300 ms 135 38159 11.13 0.0 ms pre-warming (16) 49 0.68% 406.1 ms 400 ms 0 42736 16.73 12.8 ms package shrink 111 1.53% 405.4 ms 750 ms 0 35500 10.96 13.5 ms lazy loading 120 1.66% 448.5 ms 940 ms 22 38229 11.74 -29.7 ms shrink + warming 43 0.59% 402.1 ms 400 ms 0 40933 16.66 16.8 ms path cost extra cost per ms gained pre-warming (16) 16 instances sit idle for 1800 s 0.438 package shrink 3 dependencies are moved out -0.013 lazy loading 40 ms of lazy loading on every call no gain shrink + warming the cost of both together 0.329 lazy loading only pays off once the cold rate is above 10%: 40 ms lands on every call, while the 400 ms gained falls only on cold calls — in this profile the rate is 1.66% pre-warming raises capacity spent idle from 38159 instance-s to 42736 instance-s; over the 1800 s window that is 5.60 extra units — the cold extra itself is 900 ms ``` The three paths each pay a different kind of cost. Pre-warming pulls the cold rate to 0.68%, the queue to zero, and the 99th percentile from 1,300 ms to 400 ms: because sixteen instances are always waiting in the pool, the burst's first second is not cold. Its cost is the very thing the first lesson avoided — idle instance-seconds rise to 42,736, and the bill climbs from 11.13 units to 16.73 units, a **50% increase**. Package shrinking is the one path that looks free: once the cold extra drops from 900 ms to 350 ms, both latency and cost fall. But the cost is not in the table — three dependencies have been pulled out of the package, and that has to be paid for somewhere. This is the quietest form of a workaround around a constraint. The third path loses money. Moving startup work pulls the cold extra down to 500 ms but adds 40 ms of lazy loading to every call; average latency **rises 29.7 ms**, and twenty-two requests join the queue. The last row gives the threshold: 40 ms lands on every call, while the 400 ms gained falls only on cold calls, so the path pays off only once the cold rate is above 10%. In this profile the rate is 1.66%; in the sparse profile it is 11.11%. ## Summary - A cold call takes 1,300 ms, a warm call 400 ms: 3.25x. Even though the bursty profile's rate is only 1.66%, 135 requests join the queue and the longest wait is 539 ms. - In the pool lifetime scan, the threshold comes from the shape of the traffic: in the bursty profile the rate sticks at 1.66% between 15 and 300 seconds, then drops to 0.59% at 600 seconds, because the gap between bursts is 540 seconds. The threshold's cost is written on the same row: idle instance-seconds rise from 38,159 to 57,609. - Pre-warming drops the queue to zero and raises the bill 50%; package shrinking lowers both, but it demands pulling three dependencies out of the package. - Moving startup work raises latency 29.7 ms in this profile; it pays off only once the cold rate is above 10% — the mitigation path itself depends on a threshold. ## Next Step In this lesson the concurrency share of 40 was a fixed number, and it explained where the queue came from. But it does not stand alone: the memory choice decides the processor share, the processor share decides processing duration, and processing duration decides how much work the same share can do. The next lesson applies three thresholds — memory, duration, and concurrency — to a job set and counts how many jobs do not fit. Memory is scanned, and it shows that duration and unit cost move in opposite directions, with the cheapest point landing in the middle. Once the concurrency limit is exceeded, throttling and the queue behind it are counted; sharing the limit at the account level is measured with an example.