--- title: 'Rationale for Asynchronous Processing' source: 'https://academia.sh/en/courses/asynchronous-processing/rationale-for-asynchronous-processing' course: 'Caching, Queues and Asynchronous Processing' language: en updated: '2026-08-23T07:00:24+00:00' license: 'CC BY-SA 4.0' --- # Rationale for Asynchronous Processing Shortening the request path by removing work from it: measuring step costs and success probability, separating work that must stay in the request from work that can be deferred, the changed meaning of the response, and the durability difference between handing off work in memory and writing it to the outbox. The previous lesson measured the cache hit ratio and showed with a number how much the read path had shortened. The same lesson also marked a limit: a cache only helps with reads. A loan request is a write, and on top of that it drags along other work — an overdue notification to the member, a recalculation of the branch stock summary, a line added to the monthly report. None of this can be cached, and all of it adds to the request's duration. The second way to shorten the request path is not to speed the work up, but to **remove** it from the request. This lesson answers three questions in order: what must stay inside the request, what can be removed, and what removing it costs. ## The Work Inside the Request The loan request has five steps. Each step's cost is stated in the model below; these values are not measured durations, they are model values that represent the steps' size relative to one another. What actually matters to us is how much each step adds to the total time. ```js // request-path.mjs — the steps of a loan request and each step's modeled cost const STEPS = [ { name: "authorization check", ms: 3, determines_response: true }, { name: "writing the loan record", ms: 9, determines_response: true }, { name: "overdue notification", ms: 140, determines_response: false }, { name: "branch stock summary recalculation", ms: 260, determines_response: false }, { name: "monthly report line", ms: 55, determines_response: false }, ]; let accumulated = 0; for (const s of STEPS) { accumulated += s.ms; const marker = s.determines_response ? "required " : "deferrable"; console.log(`${s.name.padEnd(40)} ${String(s.ms).padStart(4)} ms ${marker} accumulated=${accumulated} ms`); } const total = STEPS.reduce((t, s) => t + s.ms, 0); const required = STEPS.filter((s) => s.determines_response).reduce((t, s) => t + s.ms, 0); console.log(`\nrequest duration (all in request) = ${total} ms`); console.log(`request duration (required work only) = ${required} ms`); console.log(`shortening ratio = ${(1 - required / total).toFixed(3)}`); // How many requests can a single worker finish per second? console.log(`single worker throughput: ${(1000 / total).toFixed(1)} -> ${(1000 / required).toFixed(1)} req/s`); // Each step is an external dependency; the request succeeds only if all of them run together. const p = 0.999; const all = p ** STEPS.length, few = p ** STEPS.filter((s) => s.determines_response).length; console.log(`request success probability: ${all.toFixed(5)} -> ${few.toFixed(5)}`); ``` ```sh node request-path.mjs ``` ``` authorization check 3 ms required accumulated=3 ms writing the loan record 9 ms required accumulated=12 ms overdue notification 140 ms deferrable accumulated=152 ms branch stock summary recalculation 260 ms deferrable accumulated=412 ms monthly report line 55 ms deferrable accumulated=467 ms request duration (all in request) = 467 ms request duration (required work only) = 12 ms shortening ratio = 0.974 single worker throughput: 2.1 -> 83.3 req/s request success probability: 0.99501 -> 0.99800 ``` Three numbers are worth reading. First, duration: the two steps that do the request's own work take 12 milliseconds; the remaining 455 milliseconds is dead time the member spends waiting for a response. Second, throughput: when the same worker no longer carries that dead time, it finishes forty times more requests per unit of time. Third, success probability: each step is a separate external dependency, and the request succeeds only if all of them run together. When five dependencies shrink to two, the failure rate drops from five in a thousand to two in a thousand. The report server being unreachable no longer blocks a loan from being issued. ## Required Work versus Deferrable Work The criterion for the split is not speed. Even if the overdue notification were fast, that would not require it to stay inside the request; the real question is: **would the response be wrong if this step were not done?** Work stays inside the request in three cases: a step determines the response's content — the loan ID returned in it — so writing the record cannot be deferred. A step can cause the request to be rejected — the authorization check or the stock constraint cannot run afterward, since the member would already have been told "issued." A step protects an invariant — the rule that a book is never lent to two members at once has to stay inside the same transaction boundary as the loan record; that is the rule established in the Transaction Boundaries lesson. The remaining work can be deferred. What deferrable work has in common is that doing it **eventually** is enough: the notification is still correct if it goes out two seconds later, the stock summary still shows a true report if it arrives a minute behind. The Business Logic Placement topic sets the same criterion for a domain event's listener: the work handed to the listener is work whose absence still leaves the loan record valid. ## The Cost of the Handoff: The Meaning of the Response Changes Once work is removed, the response no longer says "done" — it says "accepted." The pattern established in the Synchronous and Asynchronous APIs lesson of the Web API Design course finds its match here: the server returns `202` and reports the resource where the work can be tracked. This shift has three consequences. The client cannot learn from the response that the work is finished; it has to ask or be notified. An error no longer returns to the client; when deferred work fails, there is no one waiting on a response, so seeing the failure requires a separate mechanism. And a **message** is born between the two ends: a record one side writes and the other side later reads. Where that message gets written is the subject of this lesson's final measurement. ## The Handoff Must Be Durable The shortest way to remove work from a request is to leave it for later in the same process: put it on an array, return the response, process it afterward. The measurement below shows what this loses. The setup that follows builds the same tables that The Data Access Layer and Business Logic course used. ```sh # setup.sh — builds the library database from scratch rm -f library.db sqlite3 library.db <<'SQL' CREATE TABLE branch (branch_id INTEGER PRIMARY KEY, name TEXT NOT NULL); CREATE TABLE member (member_id INTEGER PRIMARY KEY, name TEXT NOT NULL); CREATE TABLE book (book_id INTEGER PRIMARY KEY, title TEXT NOT NULL, branch_id INTEGER NOT NULL); CREATE TABLE loan (loan_id INTEGER PRIMARY KEY, book_id INTEGER NOT NULL, member_id INTEGER NOT NULL, pickup_date TEXT NOT NULL, return_date TEXT); CREATE TABLE outbox (message_id INTEGER PRIMARY KEY, type TEXT NOT NULL, body TEXT NOT NULL, status TEXT NOT NULL DEFAULT 'pending'); INSERT INTO branch VALUES (1,'Central'),(2,'Bahcelievler'); INSERT INTO member VALUES (1,'Alice Kane'),(2,'Ben Ortiz'),(3,'Clara Diaz'); INSERT INTO book VALUES (1,'Blindness',1),(2,'The Book of Sand',1),(3,'Puslu Kitalar',2); SQL ``` The first version leaves the side jobs in an in-memory array. The process terminating before the array is flushed is represented with `process.exit`; its real-world counterpart is a deployment rolling out a new version or the process crashing. ```js // in-memory.mjs — side jobs are left in an in-memory array; the process exits without flushing it import { DatabaseSync } from "node:sqlite"; const db = new DatabaseSync("library.db"); const sideJobs = []; function issueLoan(memberId, bookId) { db.exec("BEGIN IMMEDIATE"); const loanId = Number(db.prepare( "INSERT INTO loan (book_id, member_id, pickup_date) VALUES (?,?,'2025-07-20')") .run(bookId, memberId).lastInsertRowid); db.exec("COMMIT"); for (const type of ["overdue_notification", "stock_summary", "report_line"]) sideJobs.push({ type, loanId }); return loanId; } for (const [memberId, bookId] of [[1, 1], [2, 2], [3, 3]]) issueLoan(memberId, bookId); console.log(`request done; side jobs pending in memory = ${sideJobs.length}`); process.exit(0); // process restarted: array emptied ``` The second version writes the same side jobs to the outbox **in the same transaction** as the loan record. The outbox pattern was introduced in the Domain Events lesson; its use here is the same, except what gets written is not a domain event but work to be done. ```js // with-outbox.mjs — side jobs are written to the outbox in the same transaction as the loan record import { DatabaseSync } from "node:sqlite"; const db = new DatabaseSync("library.db"); function issueLoan(memberId, bookId) { db.exec("BEGIN IMMEDIATE"); const loanId = Number(db.prepare( "INSERT INTO loan (book_id, member_id, pickup_date) VALUES (?,?,'2025-07-20')") .run(bookId, memberId).lastInsertRowid); for (const type of ["overdue_notification", "stock_summary", "report_line"]) db.prepare("INSERT INTO outbox (type, body) VALUES (?,?)") .run(type, JSON.stringify({ loanId, memberId, bookId })); db.exec("COMMIT"); return loanId; } for (const [memberId, bookId] of [[1, 1], [2, 2], [3, 3]]) issueLoan(memberId, bookId); console.log("request done; side jobs in the outbox"); process.exit(0); // process restarted ``` The restarted process counts what is left behind. ```js // count.mjs — restarted process: what is left import { DatabaseSync } from "node:sqlite"; const db = new DatabaseSync("library.db"); const count = (s) => db.prepare(s).get().n; console.log(`loan rows = ${count("SELECT count(*) AS n FROM loan")}, ` + `recoverable side jobs = ${count("SELECT count(*) AS n FROM outbox WHERE status = 'pending'")}`); ``` ```sh sh setup.sh node in-memory.mjs node count.mjs sh setup.sh node with-outbox.mjs node count.mjs ``` ``` request done; side jobs pending in memory = 9 loan rows = 3, recoverable side jobs = 0 request done; side jobs in the outbox loan rows = 3, recoverable side jobs = 9 ``` In both versions, three loan records were written and all three members got a fast response. The difference is in the number of recoverable side jobs: the nine jobs accumulated in memory vanished along with the process, while the nine jobs written to the outbox were found by the restarted process. Three members never received a notification, and no one noticed — because the response returned to the request had already succeeded. The rule that follows from this is the basic condition of asynchronous processing: **if work is removed from the request, the place it is removed to must be durable.** The handoff of the work must share the same atomicity as the transaction it belongs to; otherwise the loss is silent. ## When Not to Remove It There is a cost in the other direction too. Removed work becomes invisible to whoever made the request: the member sees that they got the book, but does not know whether the notification arrived. If the state of the work is not tracked with a separate resource, the failure accumulates somewhere no one sees. The second cost is delay. Side work gets done "eventually," but if that "eventually" is not measured, it is unbounded. Three criteria justify keeping work inside the request: the response's content depends on that work, the work protects an invariant, or the work's delay is indistinguishable from an error to the user. The third is complexity. A handoff done with two lines in memory, once it becomes durable, requires a table, a reader loop, a retry rule, and failure isolation. Everything after this lesson is about how those four pieces get built. ## Summary - Two of the five steps in the request path determined the response; once the remaining three were removed, the modeled request duration fell from 467 milliseconds to 12 milliseconds, and the number of external dependencies fell from five to two. - The same worker's throughput rose from 2.1 req/s to 83.3 req/s, and the request success probability rose from 0.99501 to 0.99800. - A step stays inside the request if it determines the content of the response, can lead to the request being rejected, or protects an invariant; the remaining work can be deferred. - Once work is removed, the meaning of the response shifts from "done" to "accepted"; because an error no longer returns to the client, its visibility has to be built separately. - When side jobs were accumulated in memory, all nine were lost when the process ended; when written to the outbox in the same transaction, all nine were found by the restarted process. ## Next Step The outbox made the work durable, but on its own it is not a transport mechanism. Who reads the rows? What happens if two readers take the same row at the same time? If a reader takes a row and crashes before processing it, how does that work come back? These are all questions for a single structure: the **queue** that carries messages between two ends. The next lesson builds the queue in its smallest form — producer, consumer, acknowledgement, and visibility timeout — and measures what a delivery guarantee actually means by counting how many times each message gets delivered.