Zero-Downtime Deployment

The previous lesson’s last measurement raised node count from two to three and counted what that did to sessions. A second event that changes the node set is the release: one version stops, another opens, and for a while the two serve together. Scaling makes this change rarely; releases make it often.

This lesson releases the split loan system’s loan service under three separate layouts, and counts one thing: how many requests dropped during the release. Dropped-request count, divided by the flowing load’s rate, converts to downtime in seconds; downtime is the countable way of saying “the system was down.”

• SD13. A new process coming up and becoming ready to take requests corresponds to a 20-request window in this setup. In a real setup this window depends on process startup time and the request rate at that moment; here it is held fixed so layouts stay comparable.
• SD14. The two versions of the loan service meet the same contract; the client cannot tell which version it landed on. If the contract itself is changing, this lesson does not apply.
• SD15. Load flows sequentially: each request is sent after the previous one’s response. Equivalent downtime is computed assuming 50 requests per second.

Measurement Setup

There are two process types. The first is a version of the loan service; which version it is, and what it does on shutdown, both come from outside.

// version.mjs — a version of the loan service; VERSION, DELAY, and shutdown mode come from outside
import { createServer } from 'node:http';

const VERSION = process.env.VERSION;
const DELAY = Number(process.env.DELAY ?? 0);
const wait = (ms) => new Promise((c) => setTimeout(c, ms));

const server = createServer(async (req, res) => {
  const url = new URL(req.url, 'http://local');
  if (url.pathname === '/shutdown') {
    res.writeHead(200).end('shutting down');
    if (url.searchParams.get('mode') === 'graceful') {
      // No new connections are accepted, in-flight requests finish, then the process exits.
      server.close(() => process.exit(0));
      server.closeIdleConnections();
      setTimeout(() => process.exit(0), 5000).unref();
    } else {
      process.exit(0);                      // in-flight requests drop mid-way
    }
    return;
  }
  if (DELAY) await wait(DELAY);
  res.writeHead(200, { 'content-type': 'application/json' })
    .end(JSON.stringify({ version: VERSION, port: Number(process.env.PORT) }));
});
server.listen(Number(process.env.PORT), () => console.log(`ready ${VERSION}`));

The second is the frontend: the one address the client knows. It changes the pool behind it through a control endpoint; with the pool empty, a request has nowhere to go and gets a 503. The entire difference between release layouts is the order in which this pool gets changed.

// frontend.mjs — dispatches requests round-robin to the processes in the pool; returns 503 when the pool is empty
import { createServer } from 'node:http';

let pool = [];
let turn = 0;

createServer(async (req, res) => {
  const url = new URL(req.url, 'http://local');
  if (url.pathname === '/pool') {
    pool = (url.searchParams.get('target') ?? '').split(',').filter(Boolean);
    turn = 0;
    return res.writeHead(200).end(pool.join(','));
  }
  if (pool.length === 0) return res.writeHead(503).end('pool empty');
  const target = pool[turn++ % pool.length];
  try {
    const upstream = await fetch(`http://127.0.0.1:${target}${req.url}`);
    res.writeHead(upstream.status, { 'content-type': 'application/json' }).end(await upstream.text());
  } catch {
    res.writeHead(502).end('upstream did not respond');
  }
}).listen(Number(process.env.PORT), () => console.log('frontend ready'));

VERSION=v1 PORT=8731 node version.mjs > /dev/null &
OLD=$!
PORT=8730 node frontend.mjs > /dev/null &
FRONT=$!
sleep 0.6
curl -s -o /dev/null -w 'status code with an empty pool: %{http_code}\n' 'http://127.0.0.1:8730/loan'
curl -s -o /dev/null 'http://127.0.0.1:8730/pool?target=8731'
curl -s -w '\n' 'http://127.0.0.1:8730/loan'
kill $OLD $FRONT

status code with an empty pool: 503
{"version":"v1","port":8731}

This 503 is the definition of a dropped request: the client asked, the system could not deliver.

Three Layouts

All three layouts do the same job — v1 leaves, v2 arrives — and run under the same load. Where they split is when the old version gets stopped. In direct replacement, stopping the old is a precondition for opening the new; in blue-green, the new version comes up while the old still serves; in canary, the pool stays mixed and the new version’s share grows step by step.

// release.mjs — load flowing under three release layouts: direct replacement, blue-green, canary
import { spawn } from 'node:child_process';

const FRONTEND = 8730, RATE = 50, LOAD = 20, WINDOW = 20;
const processes = new Map();

async function start(version, port) {
  const child = spawn(process.execPath, ['version.mjs'], { stdio: ['ignore', 'pipe', 'inherit'],
    env: { ...process.env, VERSION: version, PORT: String(port) } });
  await new Promise((c) => child.stdout.once('data', c));
  processes.set(port, child);
}
const stop = (ports) => ports.forEach((p) => { processes.get(p)?.kill('SIGKILL'); processes.delete(p); });
const toPool = (ports) => fetch(`http://127.0.0.1:${FRONTEND}/pool?target=${ports}`).then((y) => y.text());

async function flow(n) {
  const tally = { v1: 0, v2: 0, dropped: 0 };
  for (let i = 0; i < n; i += 1) {
    try {
      const y = await fetch(`http://127.0.0.1:${FRONTEND}/loan`);
      if (y.ok) tally[(await y.json()).version] += 1;
      else { await y.text(); tally.dropped += 1; }
    } catch { tally.dropped += 1; }
  }
  return tally;
}

async function direct() {
  const steps = [], phases = [];
  await start('v1', 8731); await start('v1', 8732); await toPool('8731,8732');
  phases.push(['old version serving', await flow(LOAD)]);
  steps.push('stop old processes and empty the pool'); stop([8731, 8732]); await toPool('');
  phases.push(['old stopped, new not started', await flow(WINDOW)]);
  steps.push('bring up the new version'); await start('v2', 8733); await start('v2', 8734);
  steps.push('put the new version in the pool'); await toPool('8733,8734');
  phases.push(['new version serving', await flow(LOAD)]);
  const rollback = ['stop new processes and empty the pool', 'bring up the old version', 'put the old version in the pool'];
  stop([8733, 8734]); await toPool('');
  const rollbackTally = await flow(WINDOW);
  await start('v1', 8731); await start('v1', 8732); await toPool('8731,8732');
  stop([8731, 8732]);
  return { name: 'direct replacement', steps, phases, rollback, rollbackTally };
}

async function blueGreen() {
  const steps = [], phases = [];
  await start('v1', 8731); await start('v1', 8732); await toPool('8731,8732');
  phases.push(['blue serving', await flow(LOAD)]);
  steps.push('bring up green'); await start('v2', 8733); await start('v2', 8734);
  phases.push(['green up, not in pool', await flow(LOAD)]);
  steps.push('switch the pool to green'); await toPool('8733,8734');
  phases.push(['pool switched to green', await flow(LOAD)]);
  const rollbackTally = await (async () => { await toPool('8731,8732'); return flow(LOAD); })();
  steps.push('empty and stop blue');
  stop([8731, 8732, 8733, 8734]);
  return { name: 'blue-green', steps, phases, rollback: ['switch the pool back to blue'], rollbackTally };
}

async function canary() {
  const steps = [], phases = [];
  await start('v1', 8731); await start('v1', 8732); await toPool('8731,8732');
  phases.push(['new version has no share', await flow(LOAD)]);
  steps.push('bring up the new version\'s first node'); await start('v2', 8733);
  steps.push('one of three pool nodes is new'); await toPool('8731,8732,8733');
  phases.push(['one of three nodes is new', await flow(LOAD)]);
  steps.push('second new node, one old node leaves the pool'); await start('v2', 8734);
  await toPool('8731,8733,8734');
  phases.push(['two of three nodes are new', await flow(LOAD)]);
  steps.push('pool moves entirely to the new version'); await toPool('8733,8734');
  phases.push(['whole pool is new', await flow(LOAD)]);
  const rollbackTally = await (async () => { await toPool('8731,8732'); return flow(LOAD); })();
  steps.push('empty and stop the old nodes');
  stop([8731, 8732, 8733, 8734]);
  return { name: 'canary', steps, phases, rollback: ['switch the pool back to the old version'], rollbackTally };
}

const frontend = spawn(process.execPath, ['frontend.mjs'], { stdio: ['ignore', 'pipe', 'inherit'],
  env: { ...process.env, PORT: String(FRONTEND) } });
await new Promise((c) => frontend.stdout.once('data', c));

const results = [await direct(), await blueGreen(), await canary()];
for (const s of results) {
  console.log(`${s.name} — release steps ${s.steps.length}`);
  for (const [label, tally] of s.phases) {
    console.log(`  ${label.padEnd(32)} v1=${String(tally.v1).padEnd(3)} v2=${String(tally.v2).padEnd(3)} dropped=${tally.dropped}`);
  }
  console.log(`  ${`rollback, ${s.rollback.length} step${s.rollback.length === 1 ? '' : 's'}`.padEnd(32)} ${' '.repeat(14)} dropped=${s.rollbackTally.dropped}`);
}
console.log('');
console.log(`with load flowing at ${RATE} requests/sec`);
console.log('layout                release steps  dropped requests  equivalent downtime (s)  rollback steps  dropped in rollback');
for (const s of results) {
  const dropped = s.phases.reduce((t, [, y]) => t + y.dropped, 0);
  console.log(`${s.name.padEnd(20)} ${String(s.steps.length).padEnd(12)} ${String(dropped).padEnd(12)} ` +
    `${(dropped / RATE).toFixed(2).padEnd(21)} ${String(s.rollback.length).padEnd(16)} ${s.rollbackTally.dropped}`);
}
frontend.kill('SIGKILL');

direct replacement — release steps 3
  old version serving              v1=20  v2=0   dropped=0
  old stopped, new not started     v1=0   v2=0   dropped=20
  new version serving              v1=0   v2=20  dropped=0
  rollback, 3 steps                               dropped=20
blue-green — release steps 3
  blue serving                     v1=20  v2=0   dropped=0
  green up, not in pool            v1=20  v2=0   dropped=0
  pool switched to green           v1=0   v2=20  dropped=0
  rollback, 1 step                                dropped=0
canary — release steps 5
  new version has no share         v1=20  v2=0   dropped=0
  one of three nodes is new        v1=14  v2=6   dropped=0
  two of three nodes are new       v1=7   v2=13  dropped=0
  whole pool is new                v1=0   v2=20  dropped=0
  rollback, 1 step                                dropped=0

with load flowing at 50 requests/sec
layout                release steps  dropped requests  equivalent downtime (s)  rollback steps  dropped in rollback
direct replacement   3            20           0.40                  3                20
blue-green           3            0            0.00                  1                0
canary               5            0            0.00                  1                0

Direct replacement’s downtime does not come from a mistake; it comes from the layout itself. In a setup that cannot place the new version until the old one stops, there is nowhere to go during that window. Twenty requests dropped, and at fifty requests per second that equals 0.40 seconds of downtime. The number looks small; for a team releasing ten times a day it is four seconds of downtime, all of it going to users as errors.

Blue-green finished the same job with zero dropped requests. The difference shows in its second row: blue kept serving while green came up, so the startup window fell outside the load’s reach. The cost is that both versions are up at once through that window — resource demand doubles at release time.

Canary’s table shows a mixed share in its second and third rows: with one of three nodes new, six of twenty requests went to the new version; with two new, thirteen did. Because the frontend resets its turn each time the pool changes, these shares are run-independent. What canary buys is not downtime — that was already zero with blue-green — it is how many requests a broken version reaches: had the new version been broken at the second phase, six of twenty requests would have been touched, versus all twenty under blue-green.

The rollback column carries this lesson’s real decision. In blue-green and canary, rollback is one step — switch the pool back to the old version — and drops zero requests, because the old processes are still up. In direct replacement, rollback is a new release: three steps, twenty dropped requests. The rule to take from this is the rollback window, the name for how long the old processes stay up; the moment that window closes, rollback stops being one step.

Connection Draining

Once the pool changes, no new request reaches the old node — but requests already in flight at that moment exist. What happens to them depends on how the old process gets shut down.

// draining.mjs — requests left in flight while the old node shuts down: hard and graceful shutdown
import { spawn } from 'node:child_process';

const PORT = 8735, INFLIGHT = 8, DELAY = 500;
const wait = (ms) => new Promise((c) => setTimeout(c, ms));

async function measure(mode) {
  const child = spawn(process.execPath, ['version.mjs'], { stdio: ['ignore', 'pipe', 'inherit'],
    env: { ...process.env, VERSION: 'v1', PORT: String(PORT), DELAY: String(DELAY) } });
  await new Promise((c) => child.stdout.once('data', c));

  const inflight = Array.from({ length: INFLIGHT }, () =>
    fetch(`http://127.0.0.1:${PORT}/loan`).then((y) => y.ok).catch(() => false));
  await wait(150);                                    // eight requests in flight, none finished
  await fetch(`http://127.0.0.1:${PORT}/shutdown?mode=${mode}`).catch(() => {});
  const results = await Promise.all(inflight);
  child.kill('SIGKILL');
  return { mode, completed: results.filter(Boolean).length };
}

console.log(`${INFLIGHT} requests are in flight at the moment of shutdown, each taking ${DELAY} ms`);
console.log('shutdown  completed  dropped');
for (const mode of ['hard', 'graceful']) {
  const s = await measure(mode);
  console.log(`${s.mode.padEnd(8)} ${String(s.completed).padEnd(11)} ${INFLIGHT - s.completed}`);
}

8 requests are in flight at the moment of shutdown, each taking 500 ms
shutdown  completed  dropped
hard     0           8
graceful 8           0

This is the measure of a correct setting: under graceful shutdown, all eight in-flight requests completed; under hard shutdown, all eight dropped. The only difference is whether the process receiving the shutdown signal exits immediately or waits for open requests to finish.

What Stays Silent

These two measurements show two sides of the same moment, and appear to contradict each other. In the release table, blue-green’s dropped-request count was zero; in the draining table, the same transition dropped eight requests. Both are correct. The load producing the release table is new requests passing through the frontend, and once the pool switched, all of them went to green — none dropped. The eight dropped requests were routed to blue before the pool switched and are still waiting for a response. They do not show up in the frontend’s counters.

This is exactly the silent result of the wrong setting: leave shutdown mode as hard, and the release keeps getting reported as “zero downtime,” because the frontend never saw an empty pool and the nodes reported ready at startup. The application raises no error either — the process exits with a clean status code. The difference shows up only in the count of in-flight requests, and that count grows with load: an endpoint taking fifty requests per second, averaging half a second, carries roughly twenty-five requests in flight at every shutdown.

Where the settings live also changes across layouts. Pool membership sits in one place, the frontend; it changes several times during a release, always from that same place. Shutdown mode instead sits at every node: which signal gets sent to the process is the process manager’s setting, and how that signal gets handled is in application code. The two disagreeing — the process manager sending a graceful-shutdown signal while the application does not listen for it — produces the hard-shutdown row in the table, without leaving a single error record.

Summary

• A dropped request is one the frontend finds nowhere to send; dividing dropped-request count by load rate gives the equivalent downtime in seconds.
• In direct replacement, downtime comes from the layout itself: because stopping the old is a precondition for opening the new, twenty requests dropped across the startup window — 0.40 seconds.
• Blue-green and canary both dropped zero requests; the cost is that both versions stay up at once during the release window.
• Canary’s gain is not downtime but blast radius: with one of three nodes new, six of twenty requests went to the new version.
• Rollback is one step and zero dropped requests under blue-green and canary; three steps and twenty dropped requests under direct replacement. How long the old processes stay up is the rollback window.
• Under hard shutdown, all eight in-flight requests dropped; under graceful, zero did. This difference does not show up in the frontend’s counters.

Next Step

All three layouts in this lesson leaned on one assumption: two versions can stay up at once and serve the same request the same way. This assumption, true for processes, is not automatically true for the schema underneath them. In a blue-green transition, two versions read the same database; in a canary release, this cohabitation lasts even longer. The next lesson measures that window: which order migration and code release go in, how many requests fail in the wrong order, the span during which two versions can run against the same schema, and where the irreversible step sits.