Broadcast and Channel Management

The previous lesson built the two-way connection and ended with a warning: a WebSocket connection binds to a server instance and stays there for the connection’s duration. The same holds for server-sent events; every stream kept open sits in the memory of a specific instance.

In a single-instance deployment, this is invisible. In a service spread across two instances, the consequence is this: if two of three clerks watching a branch stock counter are connected to the first instance and one to the second, an update arriving at the first instance does not reach the third clerk. This lesson measures that gap and closes it.

Channel

First, naming. The set of messages a client is interested in is called a channel; in the library service, something like branch-3, loan-1204, or report-2026-01. A channel differs from the queue introduced in the messaging topic in two respects.

A channel has no memory: a message does not reach a subscriber who is not listening at broadcast time, and it does not accumulate anywhere. And a channel is multi-subscriber: a copy of the same message goes to every subscriber, and the first one to receive it does not consume it. In other words, a channel is the connection-level counterpart of the topic and the publish–subscribe model from the messaging topic.

These two properties are generally true for real-time data — nobody cares about a branch stock counter’s value from three seconds ago. If persistence is needed, a channel is not enough; the message must also be written to a durable queue or table.

Application Instance

The instance below offers two endpoints. /subscribe opens an event stream to the client and adds it to the channel set; /publish posts a message to a channel. It runs in two modes: --local mode, which delivers the broadcast only to its own subscribers, and --relay mode, which sends the broadcast to the relay.

// instance.mjs — application instance: gives clients channel subscriptions, produces broadcasts
//   node instance.mjs <port> --local   the broadcast goes only to its own subscribers
//   node instance.mjs <port> --relay   the broadcast goes to the relay, the relay delivers to everyone
import { createServer } from "node:http";

const PORT = Number(process.argv[2] ?? 8392);    // default: first instance
const WITH_RELAY = process.argv.includes("--relay");
const RELAY = "http://127.0.0.1:8391";

const channels = new Map();                        // channel -> Set<subscriber response>
const counter = { subscription: 0, local_delivery: 0, from_relay: 0 };

const readBody = (request) => new Promise((c) => {
  let v = ""; request.on("data", (p) => (v += p)); request.on("end", () => c(v));
});

function localDeliver(channel, body) {
  for (const r of channels.get(channel) ?? []) {
    r.write(`data: ${body}\n\n`);
    counter.local_delivery++;
  }
}

// The side listening to the relay: delivers every incoming broadcast to its own local subscribers.
async function listenToRelay() {
  const response = await fetch(`${RELAY}/listen`);
  let buffer = "";
  for await (const chunk of response.body) {
    buffer += Buffer.from(chunk).toString();
    let k;
    while ((k = buffer.indexOf("\n\n")) >= 0) {
      const block = buffer.slice(0, k); buffer = buffer.slice(k + 2);
      if (!block.startsWith("data: ")) continue;   // comment line: ignore
      const message = JSON.parse(block.slice(6));
      counter.from_relay++;
      localDeliver(message.channel, message.data);
    }
  }
}

createServer(async (request, response) => {
  response.sendDate = false;
  const url = new URL(request.url, "http://local");
  const channel = url.searchParams.get("channel");

  if (url.pathname === "/subscribe") {
    response.writeHead(200, { "content-type": "text/event-stream; charset=utf-8",
                              "cache-control": "no-store" });
    response.write(`: subscribed channel=${channel} instance=${PORT}\n\n`);
    if (!channels.has(channel)) channels.set(channel, new Set());
    channels.get(channel).add(response);
    counter.subscription++;
    return void request.on("close", () => channels.get(channel).delete(response));
  }

  if (url.pathname === "/publish") {
    const data = await readBody(request);
    if (WITH_RELAY) await fetch(`${RELAY}/broadcast`, {
      method: "POST", body: JSON.stringify({ channel, data }),
    });
    else localDeliver(channel, data);
    return response.writeHead(202).end();
  }

  if (url.pathname === "/metrics") {
    response.writeHead(200, { "content-type": "application/json; charset=utf-8" });
    return response.end(JSON.stringify({ instance: PORT, ...counter }) + "\n");
  }
  response.writeHead(404).end();
}).listen(PORT, "127.0.0.1", () => {
  if (WITH_RELAY) listenToRelay();
  console.log(`instance: 127.0.0.1:${PORT}`);
});

The subscription being removed when the connection closes is a critical detail. Without the request.on("close") handler, the channel set keeps holding closed response objects; attempting to write to them produces an error and the set grows without bound. This is the most common leak in any structure that holds long-lived connections.

Inter-Instance Broadcast Channel

The structure the instance talks to in --relay mode is a middle layer whose only job is getting the message to every instance. Each instance connects to it with a stream; a broadcast from one instance is written to all connected instances — including the instance that sent the broadcast.

// relay.mjs — inter-instance broadcast channel; keeps one stream open per application instance
import { createServer } from "node:http";

const instances = new Set();                        // instance responses listening to the relay
const counter = { broadcast: 0, delivery: 0 };

const readBody = (request) => new Promise((c) => {
  let v = ""; request.on("data", (p) => (v += p)); request.on("end", () => c(v));
});

createServer(async (request, response) => {
  response.sendDate = false;
  const url = new URL(request.url, "http://local");

  if (url.pathname === "/listen") {                 // instance connects to the relay
    response.writeHead(200, { "content-type": "text/event-stream; charset=utf-8",
                              "cache-control": "no-store" });
    response.write(": connected to relay\n\n");
    instances.add(response);
    return void request.on("close", () => instances.delete(response));
  }

  if (url.pathname === "/broadcast") {               // a broadcast from one instance: to everyone
    const body = await readBody(request);
    counter.broadcast++;
    for (const i of instances) { i.write(`data: ${body}\n\n`); counter.delivery++; }
    return response.writeHead(202).end();
  }

  if (url.pathname === "/metrics") {
    response.writeHead(200, { "content-type": "application/json; charset=utf-8" });
    return response.end(JSON.stringify({ ...counter, instance: instances.size }) + "\n");
  }
  response.writeHead(404).end();
}).listen(8391, "127.0.0.1", () => console.log("relay: 127.0.0.1:8391"));

Sending the broadcast back to the instance that sent it is not a bug, it is a simplification: local delivery happens in exactly one place, along the path coming back from the relay. Otherwise the message would be delivered twice on its own instance.

Measurement

The script below runs two application instances and one relay. The subscriber connects to the second instance, while the broadcast arrives at the first instance. Ports 8391–8393 are arbitrary and must be free.

#!/usr/bin/env bash
# Two application instances plus one relay; the subscriber is on the second instance, the broadcast arrives at the first instance.
measure() {
  node relay.mjs > /dev/null & relay=$!
  node instance.mjs 8392 "$1" > /dev/null & i1=$!
  node instance.mjs 8393 "$1" > /dev/null & i2=$!
  sleep 1

  curl -sN --max-time 1.2 "http://127.0.0.1:8393/subscribe?channel=branch-3" > subscriber.txt &
  listener=$!
  sleep 0.4
  curl -sS -o /dev/null -X POST -d '{"branch":3,"stock":41}' \
       "http://127.0.0.1:8392/publish?channel=branch-3"
  wait $listener

  echo "--- $2 ---"
  echo "what the subscriber on the second instance saw:"
  sed '/^$/d; s/^/  /' subscriber.txt
  echo "counters:"
  curl -sS http://127.0.0.1:8392/metrics | sed 's/^/  /'
  curl -sS http://127.0.0.1:8393/metrics | sed 's/^/  /'
  curl -sS http://127.0.0.1:8391/metrics | sed 's/^/  relay /'
  kill $relay $i1 $i2 2>/dev/null
  sleep 0.5
}

measure --local "no relay"
measure --relay "with relay"

--- no relay ---
what the subscriber on the second instance saw:
  : subscribed channel=branch-3 instance=8393
counters:
  {"instance":8392,"subscription":0,"local_delivery":0,"from_relay":0}
  {"instance":8393,"subscription":1,"local_delivery":0,"from_relay":0}
  relay {"broadcast":0,"delivery":0,"instance":0}
--- with relay ---
what the subscriber on the second instance saw:
  : subscribed channel=branch-3 instance=8393
  data: {"branch":3,"stock":41}
counters:
  {"instance":8392,"subscription":0,"local_delivery":0,"from_relay":1}
  {"instance":8393,"subscription":1,"local_delivery":1,"from_relay":1}
  relay {"broadcast":1,"delivery":2,"instance":2}

In the first run, the subscriber saw only the subscription confirmation; the broadcast went nowhere. The first instance’s local_delivery counter is zero, because there is no subscriber on that instance. The message was not lost, and it did not produce an error — nothing happened, silently. This is the dangerous side of this flaw in multi-instance deployments: it gives no sign while developing with a single instance.

In the second run, the subscriber received the message. The counters show the path step by step: one broadcast arrived at the relay, it was delivered to two instances, both instances received one message each, but only the instance with a subscriber made a local delivery.

The Cost of Fan-Out

The relay’s delivery counter is a count of copies, not broadcasts. One broadcast turns into as many copies as there are instances. For $N$ instances and $M$ broadcasts per second, the number of messages the relay writes is $N \times M$, and for instances with no subscriber, this is work that is entirely wasted. In the measurement, the first instance received the message and delivered it nowhere.

There are two ways to reduce this. Keeping a subscription registry, the relay knows which channels each instance is listening to and sends the broadcast only to the relevant instances. The gain is significant when the number of channels is larger than the number of instances; the cost is that a registration message goes to the relay every time a subscriber opens or closes. With channel partitioning, channels are assigned to instances in advance, and the client is routed to the instance hosting the channel it is interested in; distribution disappears entirely, but so does flexibility, because a client listening to several channels must open several connections.

As scale grows, a third limit appears: the relay itself is a single point. In real deployments, this job is handed off to the publish–subscribe infrastructure introduced in this course’s messaging topic; the relay becomes a thin adaptation layer in front of that infrastructure.

Is Sticky Routing Enough

A frequently used shortcut is always routing the same client to the same instance; this is called sticky routing. It is useful because it gets the client back to the same instance when a connection drops, but it does not solve the problem here: two different clients listening to the same channel can still land on different instances.

Stickiness becomes a solution only when it is channel-based — that is, when the routing decision looks at the channel rather than the client. This is exactly the channel partitioning approach above, and it requires the load balancer to know about application concepts.

Summary

• A channel is the set of messages a client is interested in; it has no memory and is multi-subscriber, so data that needs persistence must be written separately.
• Because an open connection sits in the memory of a specific instance, local delivery does not get a message to other instances’ subscribers in a multi-instance deployment, and this flaw produces no error.
• An inter-instance broadcast channel closes this gap; the sending instance also getting the message back keeps local delivery on a single path.
• If a subscription is not removed when the connection closes, the channel set keeps holding closed responses and grows without bound.
• The cost of fan-out is multiplied by the number of instances; a subscription registry or channel partitioning reduces it, and sticky routing alone does not solve it.

Next Step

All four approaches in this topic carried an event from server to client: a status changed, a queue advanced, stock was updated. One case remains. Sometimes what needs to be carried is not an event, it is the response itself — the output of a report scanning thirty thousand loan records can be streamed to the client at the rate it is produced. This requires no new protocol; it is done by writing the HTTP response body in pieces. The next lesson builds this final form and closes the course.