Backpressure

The previous lesson dealt with a single long job: breaking it into chunks, recording its progress, and making it cancellable was enough. In a real background, jobs do not arrive one at a time, they flow: every loan transaction spawns a notification job, every cover upload spawns an image-processing job.

The rate jobs arrive at and the rate workers finish them are two independent magnitudes. Nothing keeps them equal; the arrival rate is set by request traffic, the completion rate by worker count and job duration. When the arrival rate exceeds the completion rate, the queue grows. This lesson measures what that growth produces and builds how the system communicates it to the producer.

Rate Mismatch

If a queue receives $\lambda$ jobs per second and drains $\mu$ jobs per second, the queue length changes at a rate of $\lambda - \mu$. As long as $\lambda < \mu$, the queue stays empty and wait time approaches the job’s own duration. The moment $\lambda > \mu$, the queue lengthens by $\lambda - \mu$ every second, and this growth does not stop on its own.

The direct consequence of growth is wait time. Queueing theory’s basic relation says that in a stable queue, average length $L$, arrival rate $\lambda$, and average wait time $W$ satisfy:

$$L = \lambda W$$

Read backward, the relation delivers the real warning: $W = L / \lambda$. If queue length is unbounded, wait time is unbounded too. Long before memory fills up, a job still sitting in the queue loses its meaning even though it will still be processed — an overdue notification delayed four hours is no different from one that was never sent.

This is why queue length is not a resource problem, it is a latency budget problem.

The Bounded Queue and Three Policies

The fix is bounding the queue. Bounding it immediately raises a question: what happens to an incoming job when the queue is full. There are three answers, and each one gives something up.

Blocking: the producer is held until room opens up in the queue. No job is lost, but the slowdown passes to the producer — this, in the narrow sense, is backpressure. Tail drop: an incoming job is rejected while the queue is full. Head drop: the oldest job at the head of the queue is dropped, and the new job is appended.

The simulation below runs all four under the same load: a hundred jobs produced per second, sixty consumed, a ten-second window, the bounded queue’s capacity a hundred.

// simulation.mjs — how the queue behaves under four policies once production rate exceeds consumption rate
const WINDOW = 10_000;             // simulation window (ms)
const PRODUCE = 100;               // jobs produced per second
const CONSUME = 60;                // jobs consumed per second
const CAPACITY = 100;              // the bounded queue's capacity

function run(policy) {
  const queue = [];                // each entry: the instant it was produced (ms)
  let produced = 0, delivered = 0, dropped = 0, blocked = 0;
  let peak = 0, longestWait = 0;
  let produceAcc = 0, consumeAcc = 0;   // accumulators as integers: rise by rate every ms

  for (let t = 1; t <= WINDOW; t++) {
    produceAcc += PRODUCE;
    while (produceAcc >= 1000) {
      produceAcc -= 1000;
      produced++;
      if (policy === "unbounded" || queue.length < CAPACITY) queue.push(t);
      else if (policy === "tail-drop") dropped++;
      else if (policy === "head-drop") { queue.shift(); dropped++; queue.push(t); }
      else { produced--; produceAcc += 1000; blocked++; break; }      // block: the producer waits
    }
    consumeAcc += CONSUME;
    while (consumeAcc >= 1000 && queue.length) {
      consumeAcc -= 1000;
      longestWait = Math.max(longestWait, t - queue.shift());
      delivered++;
    }
    peak = Math.max(peak, queue.length);
  }
  return { policy, produced, delivered, dropped, blocked, peak,
           remaining: queue.length, longestWait };
}

const col = (n) => String(n).padStart(6);
console.log("policy      produced delivered dropped remaining  peak  longest-wait  producer-wait");
for (const p of ["unbounded", "tail-drop", "head-drop", "block"]) {
  const s = run(p);
  console.log(`${s.policy.padEnd(11)}${col(s.produced)}${col(s.delivered)}${col(s.dropped)}` +
    `${col(s.remaining)}${col(s.peak)}${String(s.longestWait).padStart(13)} ms` +
    `${String(s.blocked).padStart(14)} ms`);
}

policy      produced delivered dropped remaining  peak  longest-wait  producer-wait
unbounded    1000   600     0   400   400         4000 ms             0 ms
tail-drop    1000   600   301    99   100         1664 ms             0 ms
head-drop    1000   600   301    99   100          997 ms             0 ms
block         699   600     0    99   100         1666 ms          7490 ms

The simulation does not use real time; it advances a virtual clock millisecond by millisecond and applies the rates through integer accumulators. This is why the numbers come out the same regardless of the machine.

Reading the Numbers

The four rows show four different trade-offs.

Under unbounded, no job was lost, but the queue climbed to four hundred in ten seconds and the longest wait reached four seconds. What matters is not the number itself but its direction: with a twenty-second window, the queue would reach eight hundred and the wait eight seconds. This policy leaves latency unbounded in exchange for not losing work.

Tail drop and head drop delivered the same number of jobs (600) and dropped the same number (301). The only difference between them is in wait time: 1664 ms versus 997 ms. The difference comes from the age of the jobs delivered. Under tail drop the oldest hundred jobs always sit in the queue; under head drop the queue keeps refreshing and younger jobs reach the consumer. At the same loss rate, this makes head drop superior for jobs where freshness matters, like notifications; for jobs where order must be preserved, like a loan request standing in line, dropping the oldest is unacceptable.

Blocking dropped no job, but it could only produce 699 jobs: the producer spent 7490 of the ten seconds’ 10000 milliseconds waiting. No job was lost because none was ever produced. This is the definition of backpressure — the bottleneck reflects its consequence not onto the party downstream from it, but onto the party feeding it.

The shared lesson of the four rows: the rate mismatch cannot be eliminated, only where it gets written down can be chosen. To memory (unbounded queue), to loss (dropping), or to the producer’s own rate (blocking).

How Backpressure Propagates

Blocking sounds abstract, but it is already built into the stream abstraction. node:stream’s writable stream write call returns a boolean: false reports that the buffer has crossed its threshold and production should stop. Once room opens up, the stream emits the drain event.

The measurement below writes the same sixty-four chunks twice. In the first, the return value is listened to; in the second, it is ignored.

// stream.mjs — node:stream backpressure: listening to the write() return value versus ignoring it
import { Writable } from "node:stream";

const CHUNK = Buffer.alloc(1024, 0x61);          // a 1 KiB job body
const COUNT = 64;

const slowSink = () => new Writable({
  highWaterMark: 16 * 1024,                      // our buffer limit: 16 KiB
  write(_chunk, _enc, done) { setTimeout(done, 5); },  // writing each chunk takes 5 ms
});

async function listening() {
  const sink = slowSink();
  let fullSignals = 0, drain = 0, peak = 0;
  for (let i = 0; i < COUNT; i++) {
    const hasRoom = sink.write(CHUNK);
    peak = Math.max(peak, sink.writableLength);
    if (!hasRoom) {                               // buffer full: stop producing
      fullSignals++;
      await new Promise((c) => sink.once("drain", () => { drain++; c(); }));
    }
  }
  await new Promise((c) => sink.end(c));
  return { fullSignals, drain, peak };
}

async function ignoring() {
  const sink = slowSink();
  let fullSignals = 0, peak = 0;
  for (let i = 0; i < COUNT; i++) {
    if (!sink.write(CHUNK)) fullSignals++;         // return value read but not honored
    peak = Math.max(peak, sink.writableLength);
  }
  await new Promise((c) => sink.end(c));
  return { fullSignals, drain: 0, peak };
}

for (const [name, run] of [["listening", listening], ["ignoring", ignoring]]) {
  const s = await run();
  console.log(`${name.padEnd(12)} full-signals=${String(s.fullSignals).padStart(2)}` +
    ` drain=${String(s.drain).padStart(2)} buffer-peak=${s.peak} B`);
}

listening    full-signals= 4 drain= 4 buffer-peak=16384 B
ignoring     full-signals=49 drain= 0 buffer-peak=65536 B

When the return value is listened to, the buffer peak is exactly the threshold itself: 16384 bytes. When it is ignored, all sixty-four chunks pile up in the buffer: 65536 bytes. The ratio across sixty-four chunks is fourfold; if the producer keeps writing without bound, the ratio is unbounded too, because ignoring a false return turns the buffer into a queue, and moreover an unbounded one.

The lesson here is not specific to streams. Every layer that writes a job into a queue faces the same question: does it stop when a write is refused, or does it accumulate its own queue behind it. Any layer that does not stop renders the limit in front of it useless.

How Far Backpressure Propagates

The last link in the chain is the user, and backpressure cannot be carried all the way there. A server handling requests cannot hold an HTTP request indefinitely because the background queue is full; holding it exhausts connections and moves the problem from the queue to the network.

This is why backpressure is applied in the system’s internal links: between a worker and the database, between a queue writer and the queue, between a stream producer and its receiver. At the outer boundary, a different mechanism is needed — explicitly rejecting the incoming request instead of holding it.

Summary

• When production rate exceeds consumption rate, the queue lengthens every second by the gap between them; the growth does not stop on its own.
• Per $L = \lambda W$, unbounded queue length means unbounded wait time; the queue bound is a latency-budget decision, not a memory decision.
• A bounded queue has three policies: blocking loses no job but slows the producer, and tail drop and head drop produce the same loss but deliver jobs of different ages.
• Backpressure is the bottleneck reflecting its consequence onto the party feeding it; node:stream applies this through the write return value and the drain event, and keeps the buffer peak at the threshold.
• A layer that ignores the write return value defeats every limit in front of it; backpressure is not truly applied anywhere in the chain unless it is applied throughout it.

Next Step

Backpressure works in the internal links, but the client sitting at the outer end of the chain cannot be held. Once the queue is full, the workers cannot keep up, and the latency budget is exhausted, the server is left with a single option: refuse the incoming job. Doing this randomly destabilizes the system; which request gets rejected, how the rejection is communicated to the client, and what the latency difference is between the rejected and the accepted are separate decisions. The next lesson builds controlled rejection under overload and measures the effect of rejection on latency.