--- title: 'Streaming Responses' source: 'https://academia.sh/en/courses/asynchronous-processing/streaming-responses' course: 'Caching, Queues and Asynchronous Processing' language: en updated: '2026-08-23T07:00:24+00:00' license: 'CC BY-SA 4.0' --- # Streaming Responses Producing the response itself in pieces: the framing of chunked transfer encoding, measuring the time to first byte, the limits of reporting an error after headers have been sent, and backpressure in a streaming response. All four approaches in this topic carried an **event** from server to client: a status changed, a queue advanced, stock was updated. One case remains. Sometimes what needs to be carried is not an event, it is **the response itself**. A monthly report scanning thirty thousand loan records can be sent line by line before the result is complete. This requires no new protocol; it is done by writing the HTTP response body in pieces, and in exchange, the time the client waits to see the first data shrinks significantly. ## Two Forms A response body can be sent in two forms. If its length is known, it is announced in the `Content-Length` header, and the receiver knows from the start how many bytes to expect. If its length is not known, **chunked transfer encoding** is used: the body is split into chunks that each carry their own length, and the last chunk announces the end of the stream with a length of zero. The server below serves the same report in three ways: `/full`, which accumulates the whole thing before sending it; `/streaming`, which writes each line the moment it is produced; and `/cut`, which abandons production partway through. ```js // chunked.mjs — serves the monthly loan report in two forms // /full all lines are accumulated in memory, sent as a single piece // /streaming each line is written the moment it is produced: chunked transfer // /cut production fails after the third line import { createServer } from "node:http"; const LINES = 10; const wait = (ms) => new Promise((c) => setTimeout(c, ms)); const makeLine = (i) => `${String(i).padStart(2, "0")};branch-${1 + (i % 3)};loan=${40 + i * 3}\n`; createServer(async (request, response) => { response.sendDate = false; const path = new URL(request.url, "http://local").pathname; if (path === "/full") { // length known: single piece const lines = []; for (let i = 1; i <= LINES; i++) { await wait(100); lines.push(makeLine(i)); } const body = lines.join(""); response.writeHead(200, { "content-type": "text/csv; charset=utf-8", "content-length": Buffer.byteLength(body) }); return response.end(body); } if (path === "/streaming" || path === "/cut") { // length unknown: chunked transfer response.writeHead(200, { "content-type": "text/csv; charset=utf-8" }); for (let i = 1; i <= LINES; i++) { await wait(100); if (response.writableEnded) return; // client disconnected if (path === "/cut" && i === 4) { // Headers have already been sent: the status code can no longer be changed. response.end("#error;report cut off midway;line=3\n"); return; } if (!response.write(makeLine(i))) await new Promise((c) => response.once("drain", c)); } return response.end(); } response.writeHead(404).end(); }).listen(8401, "127.0.0.1", () => console.log("listening: 127.0.0.1:8401")); ``` Chunked transfer does not need to be requested explicitly. When `Content-Length` is not written and the body is not finished in a single call, the runtime does the framing itself. The only thing written is the line itself. The streaming response checking the `writableEnded` state before every write is the same rule as in server-sent events: continuing production after the client has disconnected means wasted processor time and an error writing to a closed socket. ## Measurement The script below compares the two forms in terms of headers, latency, and chunk arrival times. The measuring script `timer.mjs` writes each chunk's arrival moment in hundred-millisecond slices. ```js // timer.mjs — writes the arrival moment of each chunk of the streaming response as a 100 ms slice const start = performance.now(); let response; try { response = await fetch("http://127.0.0.1:8401/streaming"); } catch { console.log("could not connect to server"); process.exit(0); } for await (const chunk of response.body) { const slice = Math.round((performance.now() - start) / 100); process.stdout.write(`${String(slice).padStart(2)} slice ${Buffer.from(chunk)}`); } ``` Port 8401 is arbitrary and must be free. ```bash #!/usr/bin/env bash # Starts chunked.mjs; measures headers, latencies, and chunk arrivals. node chunked.mjs > /dev/null & server=$! sleep 1 A=http://127.0.0.1:8401 echo "--- response headers ---" for path in full streaming; do printf '%-10s ' "$path" curl -sS -o /dev/null -D - "$A/$path" | grep -i -E 'content-length|transfer-encoding' | tr -d '\r' | paste -sd' ' - done echo "--- first byte and total duration (100 ms slices) ---" for path in full streaming; do curl -sS -o /dev/null -w "$path %{time_starttransfer} %{time_total}\n" "$A/$path" done | awk '{ printf "%-10s first-byte=%2d slice total=%2d slice\n", $1, int($2*10), int($3*10) }' echo "--- arrival times of the streaming response's chunks ---" node timer.mjs echo "--- raw chunked framing (first three chunks) ---" curl -sS --raw -N "$A/streaming" 2>/dev/null | head -c 63 | od -c | head -4 echo "--- if production is cut off midway ---" curl -sS -w 'status=%{http_code}\n' "$A/cut" kill "$server" ``` ``` --- response headers --- full content-length: 200 streaming Transfer-Encoding: chunked --- first byte and total duration (100 ms slices) --- full first-byte=10 slice total=10 slice streaming first-byte= 1 slice total=10 slice --- arrival times of the streaming response's chunks --- 1 slice 01;branch-2;loan=43 2 slice 02;branch-3;loan=46 3 slice 03;branch-1;loan=49 4 slice 04;branch-2;loan=52 5 slice 05;branch-3;loan=55 6 slice 06;branch-1;loan=58 7 slice 07;branch-2;loan=61 8 slice 08;branch-3;loan=64 9 slice 09;branch-1;loan=67 10 slice 10;branch-2;loan=70 --- raw chunked framing (first three chunks) --- 0000000 1 4 \r \n 0 1 ; b r a n c h - 2 ; 0000020 l o a n = 4 3 \n \r \n 1 4 \r \n 0 2 0000040 ; b r a n c h - 3 ; l o a n = 4 0000060 6 \n \r \n 1 4 \r \n 0 3 ; b r a n --- if production is cut off midway --- 01;branch-2;loan=43 02;branch-3;loan=46 03;branch-1;loan=49 #error;report cut off midway;line=3 status=200 ``` Three results stand out. **The headers** give the format away: `/full` announces a length, `/streaming` uses chunked transfer. The two cannot coexist. **The total duration is the same in both forms**: ten slices. Streaming does not speed up the work, because the report is still produced in the same amount of time. The only thing that changes is the **time to first byte**: it drops from ten slices to one. The user sees the report's first lines without waiting for the whole thing to finish. This is the form the shared gain across this entire real-time data topic takes here. **The chunk arrivals** confirm the first result: the lines arrive one slice apart, not all together at the end. ## Framing The raw output shows the format of chunked transfer. Each chunk begins with a line that writes its length in **hexadecimal**: `14` hex, i.e. twenty bytes — the length of one report line. Then comes `\r\n`, then the data itself, then `\r\n` again. This framing solves the question of how the receiver knows where the body ends when the length is not known up front: the last chunk is written with a length of zero, and the stream ends there. It is also technically possible to end a body without announcing a length by closing the connection, but then the receiver cannot tell whether the response finished or was cut off. ## After Headers Have Been Sent The last measurement section shows the most important limit of streaming responses. At the `/cut` endpoint, report production fails at the fourth line, but the status code stays **200**. Because the headers were already sent with the first chunk, and in HTTP, headers cannot be taken back. This has three consequences. The error must now be reported **inside** the body; the output format must have a place for this — here, a notification line starting with `#`. Second, the receiving side must read this line; if it does not, it will mistake the incomplete report for a complete one. Third, writing a **completion marker** that shows the report finished is safer than writing an error marker: if the connection drops, the error line cannot be written either, but the absence of the completion marker is noticed in every case. HTTP also defines trailers for this; they allow a header to be sent after the body. In practice, intermediary support is inconsistent, so an in-body marker is the more common and safer choice. ## Backpressure in a Streaming Response Checking the return value of `response.write(...)` in the code is this lesson's counterpart to the rule measured in the backpressure lesson. If the network client is slow, the lines being written pile up in the server's buffer; if the return value is ignored, the buffer grows without bound. When the report is ten lines long, this is invisible. But a server streaming a thirty-thousand-line report to a slow connection, if it does not listen to the return value, ends up holding the entire report in memory — losing the very memory gain that was the point of streaming. The value of a streaming response lies in production speed being **coupled** to consumption speed. ## When to Stream Streaming is not right for every response. The gain materializes when production takes a long time and the first piece of the result is meaningful on its own: report lines, search results, log entries. For responses that are small and meaningful only as a single piece — the detail of one loan record — streaming only adds framing overhead. There are two practical obstacles. Intermediaries can buffer the response and turn the stream back into a single piece; in that case the measured time to first byte shows no gain. And compression can delay streaming, because it produces no output until its buffer fills; for streaming responses, compression must be configured to flush at chunk boundaries. ## Summary - A response body of unknown length is sent with chunked transfer encoding; each chunk writes its length in hexadecimal, and a zero-length chunk ends the stream. - Streaming does not shorten the total duration; in the measurement, both forms took ten slices, but the time to first byte dropped from ten slices to one. - Because headers are sent with the first chunk, the status code cannot be changed afterward; the error is reported inside the body, and writing a completion marker is safer than writing an error marker. - If the write's return value is not listened to in a streaming response, the entire body piles up in the server's buffer, and streaming's memory gain disappears. - Streaming's gain depends on a long production time and the first piece being meaningful on its own; intermediary buffering and compression can make that gain invisible. ## Course Wrap-Up This course built four separate layers for shortening the path a request is answered on, and for making the work moved outside that path reliable. **Caching** covered reusing a computed result: the client, edge, server, and database layers; the cache-aside, write-through, and write-behind strategies; time-based, event-based, and version-based invalidation; key design with namespace and tenant separation; preventing hot keys and cache stampedes; HTTP validators and cache-control directives; and measuring hit ratio. Its shared lesson: a cache is not an accelerator, it is the controlled acceptance of staleness. **Messaging** built taking work out of the request path: the case for asynchronous processing, the producer–consumer queue, publish–subscribe, the difference between a stream and a queue, at-most-once, at-least-once, and exactly-once delivery semantics, the cost of ordering and partitioning, the dead-letter queue, poison messages, and backoff. Its shared lesson: delivery guarantees are not free; every guarantee trades away something from ordering, volume, or latency. **Background Jobs** treated the side that processes the message as a unit of scaling: worker processes and supervision, recurring task definitions with a lease lock preventing overlap, progress reporting and cooperative cancellation for long-running jobs, backpressure where production speed exceeds consumption speed, and load shedding to enable controlled rejection under overload. Its shared lesson: a rate mismatch cannot be eliminated, only where it gets written can be chosen — to memory, to loss, or to the producer's speed. **Real-Time Data** built the result reaching the client: the request cost of short and long polling, server-sent events and reconnection by event ID, the lifecycle of a WebSocket connection built with its own handshake, channel distribution in a multi-instance deployment, and responses produced piece by piece. Its shared lesson: an open connection is a state; the price of the latency gain it carries is managing that state. There was an assumption held silently throughout the course. The cache, the queue, the worker, and the stream were always treated as parts of a **single application**; all of them inside the same deployment unit, in the same codebase, under the same team's responsibility. This assumption was necessary for learning the concepts, but it does not last long in real systems. The next course, **Service Architectures**, deals with splitting exactly this unit: making a criteria-based choice among monolithic, modular, and distributed architectures, drawing service boundaries according to the domain model, and applying inter-service communication and consistency patterns. None of the concepts measured in this course become invalid there; all of them cross to the other side of a boundary and become more expensive there. A delivery guarantee is no longer a queue's contract but two teams'. An idempotency key must also handle duplicates arriving over the network. Backpressure spreads not within a process but between services. Channel distribution happens not between instances but between services with different ownership. Every number measured in this course becomes a tool for reading the larger-scale counterpart of the same questions there.