Function Invocation Models

The previous lesson invoked the function in a single way: a request arrived, the function ran, a result came back. But the network’s jobs do not all sit at the same end: no one waits for the reply to the notification sent to the field crew; the nightly batch job is read from a queue; the validation request coming through the gateway sits at a client’s end.

The invocation model is a choice in which four separate numbers change at once: latency, who sees the error, who owns the retry, and how many times the same record is processed. This lesson counts all four.

FC7. The 20,000 meter readings entering the regional measurement network’s — a fictional piece of software’s — nightly batch job are measured; record classes are distributed with a self-written generator, and the seed is 20260803. FC8. A validation takes 400 ms, fixed overhead per call is 250 ms, and the asynchronous accept response takes 5 ms. FC9. A transient error passes on the second attempt; a poison record never passes. FC10. In the synchronous model the caller retries once; in the asynchronous and stream-based models the infrastructure owns the retry, with three attempts and a 60-second gap. FC11. In the stream-based model a batch is not split: if a single record errors, the whole batch is re-read, and after the third attempt the whole batch goes to the dead letter queue; records arrive at 250 per second. FC12. Prices are written in fictional units (0.0005 per call, 0.0000025 per processing millisecond), and the duration limit is 900 seconds.

The Same Job, Three Invocation Paths

// measurement-network/invocation-models.mjs — the same validation job in three invocation models (model)

// Fictional numbers: one reading validation is 400 ms, fixed overhead per call is 250 ms, the accept response is 5 ms.
export const N = 20000, PROC_MS = 400, FIXED_MS = 250, ACCEPT_MS = 5, RETRY_MS = 60000;
export const CALL_UNIT = 0.0005, MS_UNIT = 0.0000025;   // fictional unit: per call and per ms

export const rng = (t) => { let s = t >>> 0; return () => {
  s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; };

// The 20000 meter readings entering the nightly batch: most clean, some transient errors, some poison.
export const records = (seed = 20260803) => {
  const r = rng(seed);
  return Array.from({ length: N }, () => { const v = r();
    return v < 0.001 ? "poison" : v < 0.021 ? "transient" : "clean"; });
};

const percentile = (a, p) => [...a].sort((x, y) => x - y)[Math.floor(a.length * p)];
const average = (a) => a.reduce((x, y) => x + y, 0) / a.length;

// Synchronous: the caller waits for the reply, sees the error, and owns the retry itself.
export const synchronous = (k) => {
  let calls = 0, retries = 0; const finishTimes = [];
  for (const d of k) {
    const attempts = d === "clean" ? 1 : 2;
    calls += attempts; retries += attempts - 1;
    finishTimes.push(attempts * (FIXED_MS + PROC_MS) + (attempts - 1) * 200);
  }
  return { label: "synchronous", finishTimes, calls, retries, processed: calls, duration: calls * (FIXED_MS + PROC_MS),
    deadLetter: 0, collateral: 0, errorSeenBy: "caller", retryOwner: "caller" };
};

// Asynchronous: the caller receives the accept response and leaves; the infrastructure owns the retry.
export const asynchronous = (k) => {
  let calls = 0, retries = 0, dead = 0; const finishTimes = [];
  for (const d of k) {
    const attempts = d === "clean" ? 1 : d === "transient" ? 2 : 3;
    calls += attempts; retries += attempts - 1;
    finishTimes.push(attempts * (FIXED_MS + PROC_MS) + (attempts - 1) * RETRY_MS);
    if (d === "poison") dead++;
  }
  return { label: "asynchronous", ackMs: ACCEPT_MS, finishTimes, calls, retries, processed: calls,
    duration: calls * (FIXED_MS + PROC_MS), deadLetter: dead, collateral: 0,
    errorSeenBy: "no one", retryOwner: "infrastructure" };
};

// Stream-based: records are read in order, a batch is a single call, one bad record retries the whole batch.
export const streamBased = (k, batchSize, arrivalRate = 250) => {
  let calls = 0, processed = 0, dead = 0, collateral = 0; const finishTimes = [];
  for (let i = 0; i < k.length; i += batchSize) {
    const batch = k.slice(i, i + batchSize);
    const poisoned = batch.filter((d) => d === "poison").length;
    const attempts = poisoned ? 3 : batch.some((d) => d === "transient") ? 2 : 1;
    const duration = attempts * (FIXED_MS + batch.length * PROC_MS) + (attempts - 1) * RETRY_MS;
    calls += attempts; processed += attempts * batch.length;
    if (poisoned) { dead += batch.length; collateral += batch.length - poisoned; }
    for (let j = 0; j < batch.length; j++) finishTimes.push(batch.length / arrivalRate * 1000 + duration);
  }
  return { label: `stream-based (batch ${batchSize})`, ackMs: ACCEPT_MS, finishTimes, calls, processed,
    retries: processed - N, duration: calls * FIXED_MS + processed * PROC_MS,
    deadLetter: dead, collateral, errorSeenBy: "no one", retryOwner: "infrastructure" };
};

export const measure = (m) => ({ ...m, ackAvg: m.ackMs ?? average(m.finishTimes), finishAvg: average(m.finishTimes),
  finish99: percentile(m.finishTimes, 0.99), cost: m.calls * CALL_UNIT + m.duration * MS_UNIT });

if (import.meta.url === `file://${process.argv[1]}`) {
  const k = records(), count = (d) => k.filter((x) => x === d).length;
  console.log(`${N} records: ${count("clean")} clean, ${count("transient")} transient error, ${count("poison")} poison`);
  const m = [synchronous(k), asynchronous(k), streamBased(k, 100)].map(measure);
  console.log("\n" + "model".padEnd(26) + "caller sees".padEnd(19) +
    "job finishes (avg)".padEnd(20) + "job finishes (p99)".padEnd(20) + "processing calls".padEnd(18) +
    "extra processing".padEnd(18) + "dead letter".padEnd(13) + "cost");
  for (const x of m)
    console.log(x.label.padEnd(26) + `${x.ackAvg.toFixed(0)} ms`.padEnd(19) +
      `${(x.finishAvg / 1000).toFixed(1)} s`.padEnd(20) + `${(x.finish99 / 1000).toFixed(1)} s`.padEnd(20) +
      String(x.calls).padEnd(18) + String(x.retries).padEnd(18) + String(x.deadLetter).padEnd(13) +
      x.cost.toFixed(2));
  console.log("\n" + "model".padEnd(26) + "error seen by".padEnd(16) + "retry owner".padEnd(23) +
    "extra processing per 100 records");
  for (const x of m)
    console.log(x.label.padEnd(26) + x.errorSeenBy.padEnd(16) + x.retryOwner.padEnd(23) +
      (x.retries / N * 100).toFixed(1));
  const s = m[2];
  console.log(`\nof the ${s.deadLetter} records that fell into the dead letter queue, ${s.collateral} were healthy: ` +
    `${(s.collateral / count("poison")).toFixed(1)} healthy records fell along with each poison record`);
}

20000 records: 19551 clean, 436 transient error, 13 poison

model                     caller sees        job finishes (avg)  job finishes (p99)  processing calls  extra processing  dead letter  cost
synchronous               669 ms             0.7 s               1.5 s               20449             449               0            43.45
asynchronous              5 ms               2.1 s               61.3 s              20462             462               13           43.48
stream-based (batch 100)  5 ms               130.9 s             241.2 s             380               18000             1300         38.43

model                     error seen by   retry owner            extra processing per 100 records
synchronous               caller          caller                 2.2
asynchronous              no one          infrastructure         2.3
stream-based (batch 100)  no one          infrastructure         90.0

of the 1300 records that fell into the dead letter queue, 1287 were healthy: 99.0 healthy records fell along with each poison record

Two columns of the first table move in opposite directions. The latency the caller sees drops from 669 ms to 5 ms: in the asynchronous model the caller waits for the job to be accepted, not for it to finish. But the moment the job finishes rises from 0.7 seconds to 2.1 seconds, and the 99th percentile rises to 61.3 seconds. The latency saved does not disappear; it moves somewhere the caller cannot see.

The second table shows the actual transfer. In the synchronous model the caller sees the error and decides on the retry itself; all thirteen poison records are reported. In the other two models no one sees the error: the infrastructure tries three times, drops the record into the dead letter queue, and the caller is long gone. Handing off the retry does not remove the responsibility; it removes the error’s visibility.

The last column gives the cost of the transfer. In the first two models, 2.2 extra processing operations fall on every hundred records; in the stream-based model, 90. The reason is that the batch is not split: in a batch of a hundred records, a single transient error reprocesses all hundred records. The same indivisibility holds for the dead letter queue — 1,287 of 1,300 records are healthy, and their only offense is landing in the same batch as a poison record.

Batch Read Size

Batch size is a single number, but it decides three things at once: call count, the amount of reprocessing, and when a record’s result is reached.

// measurement-network/batch-scan.mjs — batch read size is scanned in the stream-based model (model)
import { N, PROC_MS, FIXED_MS, records, streamBased, measure } from "./invocation-models.mjs";

export const LIMIT = 900;                          // fictional duration limit, seconds
export const BATCH_SIZES = [1, 2, 5, 10, 25, 50, 100, 250, 500, 1000, 2500];

if (import.meta.url === `file://${process.argv[1]}`) {
  const k = records(), poisoned = k.filter((x) => x === "poison").length;
  console.log("size".padEnd(7) + "calls".padEnd(9) + "processed records".padEnd(19) +
    "extra".padEnd(11) + "total duration".padEnd(17) + "cost".padEnd(10) +
    "record latency".padEnd(18) + "dead letter".padEnd(13) + "collateral".padEnd(12) + "batch job / limit");
  const rows = BATCH_SIZES.map((b) => measure(streamBased(k, b)));
  for (let i = 0; i < BATCH_SIZES.length; i++) {
    const x = rows[i], span = (FIXED_MS + BATCH_SIZES[i] * PROC_MS) / 1000;
    console.log(String(BATCH_SIZES[i]).padEnd(7) + String(x.calls).padEnd(9) + String(x.processed).padEnd(19) +
      `${(x.retries / N * 100).toFixed(0)}%`.padEnd(11) + `${(x.duration / 1000).toFixed(0)} s`.padEnd(17) +
      x.cost.toFixed(2).padEnd(10) + `${(x.finishAvg / 1000).toFixed(1)} s`.padEnd(18) +
      String(x.deadLetter).padEnd(13) + String(x.collateral).padEnd(12) +
      `${span.toFixed(0)}/${LIMIT} s` + (span > LIMIT ? " — does not fit" : ""));
  }
  const cheapest = rows.reduce((a, b) => (b.cost < a.cost ? b : a));
  const fastest = rows.reduce((a, b) => (b.duration < a.duration ? b : a));
  console.log(`\ncheapest size: ${cheapest.label} — ${cheapest.cost.toFixed(2)} units; ` +
    `shortest total duration: ${fastest.label} — ${(fastest.duration / 1000).toFixed(0)} s`);
  const row1 = rows[0], row500 = rows[8];
  console.log(`from size 1 to 500: calls ${row1.calls} -> ${row500.calls} (${(row1.calls / row500.calls).toFixed(0)}x fewer), ` +
    `processed records ${row1.processed} -> ${row500.processed} (${(row500.processed / row1.processed).toFixed(1)}x more)`);
  console.log(`${poisoned} poison records dragged ${row1.collateral} healthy records at size 1, and ${row500.collateral} ` +
    `healthy records at size 500 into the dead letter queue together`);
}

size   calls    processed records  extra      total duration   cost      record latency    dead letter  collateral  batch job / limit
1      20462    20462              2%         13300 s          43.48     2.1 s             13           0           1/900 s
2      10456    20912              5%         10979 s          32.68     3.8 s             26           13          1/900 s
5      4440     22200              11%        9990 s           27.20     9.1 s             65           52          2/900 s
10     2420     24200              21%        10285 s          26.92     17.8 s            130          117         4/900 s
25     1166     29150              46%        11952 s          30.46     42.5 s            325          312         10/900 s
50     684      34200              71%        13851 s          34.97     77.4 s            650          637         20/900 s
100    380      38000              90%        15295 s          38.43     130.9 s           1300         1287        40/900 s
250    172      43000              115%       17243 s          43.19     285.5 s           3000         2987        100/900 s
500    91       45500              127%       18223 s          45.60     534.1 s           5500         5487        200/900 s
1000   49       49000              145%       19612 s          49.06     1071.6 s          9000         8987        400/900 s
2500   23       57500              188%       23006 s          57.53     2998.2 s          17500        17487       1000/900 s — does not fit

cheapest size: stream-based (batch 10) — 26.92 units; shortest total duration: stream-based (batch 5) — 9990 s
from size 1 to 500: calls 20462 -> 91 (225x fewer), processed records 20462 -> 45500 (2.2x more)
13 poison records dragged 0 healthy records at size 1, and 5487 healthy records at size 500 into the dead letter queue together

The cost column draws a valley: 43.48 units at size 1, 26.92 at size 10, 57.53 at size 2,500. The left slope comes from the fixed overhead per call — every call needs 250 ms of setup, and that is paid twenty thousand times. The right slope comes from reprocessing: as size grows, the odds of a batch containing a bad record rise, and at size 2,500, 188% of records are processed extra. The cheapest point is not the fastest point: the shortest duration is at size 5, the lowest cost at size 10.

The third force is latency, and it rises in a single direction: the time to reach a record’s result is 2.1 seconds at size 1 and 534 seconds at size 500; a record waits for 499 records it has nothing to do with to be processed. The fourth is a threshold: at size 2,500 the batch itself takes 1,000 seconds and no longer fits the 900-second duration limit. The row is written but cannot be chosen.

The collateral column reduces the cost of indivisibility to a single number: thirteen poison records take down no healthy records at size 1, and drag 5,487 healthy records into the dead letter queue at size 500. Growing the batch size grows the failure radius along with the throughput.

The Cost of Repeated Processing

Extra processing was counted as a cost line; the real question is whether processing the same record twice is harmless. It is harmless for validation; it is not for a billing line.

// measurement-network/deduplication.mjs — the cost of repeated processing, and the cost of deduplicating it (model)
import { N, MS_UNIT, records, synchronous, asynchronous, streamBased, measure } from "./invocation-models.mjs";

export const READ_MS = 15, WRITE_MS = 10, WINDOW = 24 * 3600;   // ms, ms, seconds (fictional)
export const SWEEP_DELAY = 3 * 24 * 3600;           // the dead letter queue is swept by hand after 3 days

if (import.meta.url === `file://${process.argv[1]}`) {
  const k = records();
  const m = [synchronous(k), asynchronous(k), streamBased(k, 10)].map(measure);
  console.log("model".padEnd(25) + "extra processing".padEnd(19) + "extra billing lines".padEnd(21) +
    "dedup overhead".padEnd(19) + "extra cost".padEnd(12) + "outside-window repeat");
  for (const x of m) {
    const overhead = x.processed * READ_MS + N * WRITE_MS;
    console.log(x.label.padEnd(25) + String(x.retries).padEnd(19) + String(x.retries).padEnd(21) +
      `${(overhead / 1000).toFixed(0)} s`.padEnd(19) + (overhead * MS_UNIT).toFixed(2).padEnd(12) +
      String(x.deadLetter));
  }
  const s = m[2], share = (s.processed * READ_MS + N * WRITE_MS) / s.duration;
  console.log(`\ndedup window is ${WINDOW / 3600} hours; the dead letter queue is swept ${SWEEP_DELAY / 3600} hours ` +
    `later — the window is closed, ${s.deadLetter} records are processed a second time`);
  console.log(`for stream-based (batch 10), deduplication adds overhead equal to ${(share * 100).toFixed(1)}% of job duration; ` +
    `in return ${s.retries} extra billing lines are prevented`);
}

model                    extra processing   extra billing lines  dedup overhead     extra cost  outside-window repeat
synchronous              449                449                  507 s              1.27        0
asynchronous             462                462                  507 s              1.27        13
stream-based (batch 10)  4200               4200                 563 s              1.41        130

dedup window is 24 hours; the dead letter queue is swept 72 hours later — the window is closed, 130 records are processed a second time
for stream-based (batch 10), deduplication adds overhead equal to 5.5% of job duration; in return 4200 extra billing lines are prevented

The cost of deduplication is small: 5.5% of job duration and 1.41 units. In return, 4,200 extra billing lines are prevented; this comparison takes deduplication out of the realm of optional.

The last column shows that deduplication comes with its own constraint too. The key is not kept forever; the window is 24 hours, while the dead letter queue is swept by hand after 72 hours. By the time the 130 records are swept, their deduplication record has long been deleted, and all of them are processed a second time. The measure’s own window opens a path the measure does not cover.

Summary

• The latency the caller sees drops from 669 ms to 5 ms, but the moment the job finishes rises from 0.7 seconds to 2.1 seconds, and the 99th percentile rises to 61.3 seconds; the latency does not disappear, it becomes invisible.
• In the synchronous model the caller sees all thirteen poison records; in the other two no one sees the error — handing off retry responsibility does not remove it, it removes visibility.
• In the batch size scan, cost draws a valley: 43.48 units at size 1, 26.92 at size 10, 57.53 at size 2,500. The shortest duration is at size 5, the lowest cost at size 10, and size 2,500 no longer fits the duration limit.
• Because the batch is not split, the failure radius grows with size: thirteen poison records drag no healthy records into the dead letter queue at size 1, and 5,487 healthy records at size 500.
• Deduplication adds 5.5% to job duration and prevents 4,200 extra billing lines, but it has a 24-hour window; the 130 records swept after 72 hours fall outside the window and are processed a second time.

Next Step

The latency numbers of all three models rest on a single assumption: when the call arrives, an instance ready to run the function is standing by. The 250 ms written as fixed overhead is the cost of that readiness.

A ready instance is not always there. Because the capacity decision is handed off, instances are created as requests arrive and reclaimed when idle. The next lesson runs a warm pool model and counts the cold call rate, the duration added to latency, and when a queue forms as pool lifetime and traffic profile change; it then measures three mitigation paths separately.