Synchronous and Asynchronous APIs

The five styles so far shared a common assumption: the client initiates communication, the server responds, the exchange ends. Some of the library service’s jobs do not fit this pattern.

Three situations break this pattern. First, the event occurs on the server: nobody notifies a client waiting for a book to return to the shelf, and the client does not know when to ask either. Second, the work takes a long time: a branch count job runs for minutes, and keeping the request open for that whole time holds up both the connection and the client. Third, the same notification goes out to many consumers: each consumer asking separately repeats the same work.

This lesson writes and measures the first situation in three forms, then builds the pattern for the second situation.

Publishing the Same Event in Three Forms

The server publishes six events — books returning to the shelf — and serves the same event sequence from three separate endpoints. Event stream keeps the connection open and writes as an event occurs. Long polling holds the request without a response until a new event comes up. Short polling answers every request immediately; it can come back empty, and the client asks again at fixed intervals.

In all three, the client sends the id of the last event it has seen. This is required in the polling forms: events that occur in the gap between two requests would otherwise be lost.

// server.mjs — publishes the same event stream in three forms: event stream, long polling, short polling
import { createServer } from "node:http";

const EVENTS = [];               // published events; the id is the array index
const waiters = [];              // requests waiting for a response in long polling
const streams = new Set();       // open event-stream connections
const jobs = new Map();          // state of long-running jobs
let jobCounter = 0;

const publish = (event) => {
  EVENTS.push(event);
  for (const y of streams) y.write(`id: ${EVENTS.length}\ndata: ${JSON.stringify(event)}\n\n`);
  while (waiters.length) waiters.pop()();
};

const json = (res, body) => {
  const text = JSON.stringify(body);
  res.setHeader("Content-Type", "application/json; charset=utf-8");
  res.setHeader("Content-Length", Buffer.byteLength(text));
  res.writeHead(200).end(text);
};
const after = (n) => EVENTS.slice(n).map((o, i) => ({ id: n + i + 1, ...o }));

createServer((req, res) => {
  res.sendDate = false;
  const url = new URL(req.url, "http://local");
  const since = Number(url.searchParams.get("after") ?? 0);

  // 1) Event stream: the connection stays open, the server writes as events occur.
  if (url.pathname === "/stream") {
    res.writeHead(200, { "Content-Type": "text/event-stream", "Cache-Control": "no-store",
                         Connection: "keep-alive" });
    for (const o of after(since)) res.write(`id: ${o.id}\ndata: ${JSON.stringify(o)}\n\n`);
    streams.add(res);
    req.on("close", () => streams.delete(res));
    return;
  }
  // 2) Long polling: if there is no new event, the response is held; it is written once the event arrives.
  if (url.pathname === "/long") {
    if (EVENTS.length > since) return json(res, { events: after(since) });
    const timeout = setTimeout(() => {
      const k = waiters.indexOf(resume); if (k >= 0) waiters.splice(k, 1);
      json(res, { events: [] });
    }, 2000);
    const resume = () => { clearTimeout(timeout); json(res, { events: after(since) }); };
    waiters.push(resume);
    return;
  }
  // 3) Short polling: every request is answered immediately, the client asks again at fixed intervals.
  if (url.pathname === "/short") return json(res, { events: after(since) });
  // 4) Long-running job: the request is accepted, the result is tracked from a separate resource.
  if (url.pathname === "/counts" && req.method === "POST") {
    const id = `job-${++jobCounter}`;
    jobs.set(id, { status: "running", result: null });
    setTimeout(() => jobs.set(id, { status: "done", result: { missing: 3 } }), 600);
    res.setHeader("Location", `/jobs/${id}`);
    res.setHeader("Content-Type", "application/json; charset=utf-8");
    return res.writeHead(202).end(JSON.stringify({ job: id }));
  }
  if (url.pathname.startsWith("/jobs/")) return json(res, jobs.get(url.pathname.slice(6)));
  // The publication is triggered from outside so the measurements run under the same condition.
  if (url.pathname === "/start") { start(); return json(res, { started: true }); }
  return json(res, { error: "not found" });
}).listen(8491, "127.0.0.1", () => console.log("event endpoint 127.0.0.1:8491"));

// Six events, at 250 ms intervals: a book returning to the shelf is an event that occurs on the server.
const start = () => {
  let n = 0;
  const t = setInterval(() => {
    publish({ type: "book_back_on_shelf", isbn: `978-000000000${n}`, at: Date.now() });
    if (++n === 6) clearInterval(t);
  }, 250);
};

For the measurement to be valid, all three clients have to watch the same event sequence at the same time. So the publication does not start on its own: it is triggered from outside once all three clients are connected.

// measure.mjs — measures three forms on the same event stream, at the same time: connections, bytes, delivery delay
import { Agent, request } from "node:http";
import { connect } from "node:net";

const createClient = () => {
  const state = { connections: 0, requests: 0, bytes: 0, delays: [] };
  const agent = new Agent({ keepAlive: false });     // every request opens a new connection
  agent.createConnection = (s) => {
    state.connections++;
    const b = connect(s);
    b.on("data", (p) => (state.bytes += p.length));
    return b;
  };
  return { state, agent };
};
const record = (state, events) => {
  for (const o of events) state.delays.push(Date.now() - o.at);
  return events.length ? events[events.length - 1].id : null;
};
const get = (client, path) => new Promise((resolve) => {
  client.state.requests++;
  request({ agent: client.agent, host: "127.0.0.1", port: 8491, path }, (y) => {
    let m = ""; y.on("data", (p) => (m += p)); y.on("end", () => resolve(JSON.parse(m)));
  }).end();
});

// 1) EVENT STREAM — a single connection stays open, the client reads as the server writes
const stream = createClient();
const streamTask = new Promise((done) => {
  stream.state.requests++;
  request({ agent: stream.agent, host: "127.0.0.1", port: 8491, path: "/stream?after=0" }, (y) => {
    y.on("data", (p) => {
      for (const block of String(p).split("\n\n")) {
        const e = /data: (.+)/.exec(block);
        if (e) record(stream.state, [JSON.parse(e[1])]);
      }
      if (stream.state.delays.length === 6) { y.destroy(); done(); }
    });
  }).end();
});

// 2) LONG POLLING — the server holds the response until an event arrives; the client reconnects
const long = createClient();
const longTask = (async () => {
  let last = 0;
  while (long.state.delays.length < 6)
    last = record(long.state, (await get(long, `/long?after=${last}`)).events) ?? last;
})();

// 3) SHORT POLLING — asking again at a fixed interval
const short = createClient();
const shortTask = (async () => {
  let last = 0;
  while (short.state.delays.length < 6) {
    last = record(short.state, (await get(short, `/short?after=${last}`)).events) ?? last;
    if (short.state.delays.length < 6) await new Promise((c) => setTimeout(c, 500));
  }
})();

await new Promise((c) => setTimeout(c, 200));       // publication starts once all three clients are connected
await fetch("http://127.0.0.1:8491/start");
await Promise.all([streamTask, longTask, shortTask]);

const summarize = (label, state) => {
  const d = state.delays;
  console.log(`  ${label.padEnd(14)} connections=${String(state.connections).padStart(2)}  ` +
    `bytes=${String(state.bytes).padStart(4)}  events=${d.length}  ` +
    `average delay=${String(Math.round(d.reduce((t, v) => t + v, 0) / d.length)).padStart(3)} ms  ` +
    `max=${String(Math.max(...d)).padStart(3)} ms`);
};
console.log("-- six events, published at 250 ms intervals --");
summarize("event stream", stream.state);
summarize("long polling", long.state);
summarize("short polling", short.state);

// 4) LONG-RUNNING JOB — the request is accepted, the result is tracked from a separate resource
console.log("\n-- long-running job: accepted request --");
const t0 = Date.now();
const accepted = await fetch("http://127.0.0.1:8491/counts", { method: "POST" });
const location = accepted.headers.get("location");
console.log(`  POST /counts -> HTTP ${accepted.status}  Location: ${location}`);
for (let i = 0; i < 5; i++) {
  const d = await (await fetch(`http://127.0.0.1:8491${location}`)).json();
  console.log(`  GET ${location} (+${Date.now() - t0} ms) -> ${JSON.stringify(d)}`);
  if (d.status === "done") break;
  await new Promise((c) => setTimeout(c, 200));
}

node server.mjs > /dev/null & p=$!
curl -s --retry 20 --retry-connrefused --retry-delay 0 -o /dev/null "http://127.0.0.1:8491/short?after=0"
node measure.mjs
kill $p

-- six events, published at 250 ms intervals --
  event stream   connections= 1  bytes= 681  events=6  average delay=  3 ms  max=  4 ms
  long polling   connections= 6  bytes=1182  events=6  average delay=  2 ms  max=  2 ms
  short polling  connections= 5  bytes=1068  events=6  average delay=168 ms  max=294 ms

-- long-running job: accepted request --
  POST /counts -> HTTP 202  Location: /jobs/job-1
  GET /jobs/job-1 (+8 ms) -> {"status":"running","result":null}
  GET /jobs/job-1 (+212 ms) -> {"status":"running","result":null}
  GET /jobs/job-1 (+417 ms) -> {"status":"running","result":null}
  GET /jobs/job-1 (+624 ms) -> {"status":"done","result":{"missing":3}}

Delay values and byte counts depend on the machine and the headers the runtime writes; the count of six events, the connection counts, and the ordering by size do not.

What the Measurement Says

Event stream carries six events over a single connection. The client connects once and reads as the server writes; the delay is on the order of milliseconds. Bytes carried are also the lowest, because a single set of headers was sent for all six events.

Long polling achieves the same delay with six connections. The response closes after every event, and the client reconnects; every reconnection means a new request line and a new set of headers. The byte count is roughly double that of the event stream. The delay is low because the server held the request until the event arrived — the cost of waiting is not in bytes but in the number of open requests: every waiting client holds a response object and a connection on the server.

Short polling ties the delay to the polling interval. The average delay is 168 milliseconds, the largest 294 milliseconds; the polling interval is half a second, and the expected average delay is about half the interval. Connection count here depends not on the number of events but on the time elapsed: even with no events at all, the client keeps asking, and every empty response adds to the bytes carried.

The choice among the three is not made by a single measure. Short polling is the simplest and is enough where delay does not matter; to intermediate layers, it is just an ordinary request. Long polling lowers the delay but raises the number of requests held open on the server. Event stream wins on both counts, and in exchange it requires a long-lived connection: intermediate layers must not close the connection, the server must be able to allocate a connection per client, and the stream must be able to reconnect if it breaks.

This last point corresponds to the after parameter in the measurement. Server-Sent Events sends back the id of the last event received with the Last-Event-ID header when reconnecting a dropped connection, and the server picks up where it left off. Without this mechanism, no asynchronous publication is reliable: events that occur at the moment of disconnection are silently lost.

One-Way and Two-Way

Event stream is one-way: server to client. If the client also needs to send data continuously — like a handheld terminal reporting every tag it reads during a shelf count instantly — a two-way connection is needed, and its counterpart is the WebSocket protocol. Its cost is stepping outside HTTP’s semantics: the notions of method, status code, cache directive, and conditional request no longer apply once the connection is established. None of the gains measured in the third lesson hold over an open socket.

The selection criterion is simple: if data comes only from the server, an event stream is enough; if both sides send data continuously, a two-way connection is needed.

The Accepted Request

The last part of the measurement builds the second situation. When a branch count is requested, the server does not try to finish the work; it accepts the request, returns 202, and reports the resource the job will be tracked from with the Location header. The client learns the status by asking that resource; once the job finishes, the same resource carries the result.

This pattern has three benefits. Request duration is separated from the job’s duration, so the timeout problem disappears. Because the job’s status is a resource, it stays inside the uniform interface; it can be listed, canceled, stored. And what happens when the same job is requested twice becomes a separate decision — this is the subject of this course’s Idempotency Keys lesson.

It is not even required for the client to ask in order to learn that the job is done: when the status resource is combined with an event stream, the notification reaches the client on its own once the job finishes. The two patterns are used together, not as substitutes for each other.

Summary

• The request–response pattern falls short in three situations: an event occurring on the server, work taking a long time, and the same notification going out to many consumers.
• In the measurement, event stream carried six events over one connection and 681 bytes, long polling over six connections and 1182 bytes; both stayed at millisecond-level delay.
• In short polling, the delay depends on the polling interval: at a half-second interval, an average of 168 and a maximum of 294 milliseconds were measured, and the connection count tracked the time elapsed rather than the event count.
• Every polling form has to carry the id of the last event seen; events that occur during a disconnection or in the gap between requests would otherwise be lost.
• A long-running job is accepted with 202 and tracked with a separate status resource; this separates request duration from job duration, and the job turns into a resource inside the uniform interface.

Next Step

This topic compared the styles and measured what each one makes easier and what it makes harder. What comes next is the detail of a choice: actually designing the resource-based style. In the measurements, resources were assumed ready-made — book, member, loan, job — yet their being resources was a decision. Which domain concepts turn into a resource, and which stay as another resource’s field? Which resource does an action like “lend the book” sit on? The next topic’s first lesson starts with this question and derives the resource model from the library domain’s concepts.