---
title: 'Process Management and Resilience'
source: 'https://academia.sh/en/courses/nodejs/process-management-and-resilience'
course: 'The Node.js Runtime'
language: en
updated: '2026-08-17T18:09:51+00:00'
license: 'CC BY-SA 4.0'
---

# Process Management and Resilience

Catching signals, the phases of graceful shutdown, the distinction between liveness and readiness probes, the restart contract, and bounding shutdown duration.

So far, the process has always started by hand and stopped by closing the
terminal. In production this job belongs to a supervisor: it starts the process,
reopens it when it crashes, stops it during an update.

There's a contract between supervisor and process, and this lesson covers the
process's side of it: catching the shutdown request, finishing running work
without taking new work, and reporting its own state.

## A Shutdown Request Is a Signal

Signals and traps were covered in the Shell Programming course; the same
mechanism appears here as the process object's events.

The supervisor's shutdown request is the `SIGTERM` signal. Pressing the interrupt
key in a terminal produces `SIGINT`. Both default to ending the process
immediately; once a listener attaches, that default is disabled and the
decision passes to the program.

There's also a signal that cannot be caught. The supervisor uses it if the
process does not shut down within the allowed time after the polite request, and
the process ends without any cleanup. This is why the shutdown needs its own
time limit: one that does not finish before the supervisor's limit counts as
never having happened.

## The Phases of Graceful Shutdown

**Graceful shutdown** is the process stopping taking new work and ending by
finishing the work it already has. It has four phases, and their order matters.

```js
// resilient-server.mjs
import { createServer } from 'node:http';

const EXIT_DELAY_MS = 500;      // time to tell the supervisor "no longer ready"
const SHUTDOWN_LIMIT_MS = 3000; // longest time given to pending requests

let ready = false;
let shuttingDown = false;

const server = createServer((request, response) => {
  if (request.url === '/live') {                 // is the process up
    response.writeHead(200).end('alive\n');
    return;
  }
  if (request.url === '/ready') {                 // ready to accept requests
    const code = ready && !shuttingDown ? 200 : 503;
    response.writeHead(code).end(code === 200 ? 'ready\n' : 'not ready\n');
    return;
  }
  if (request.url === '/slow') {                 // simulates a pending request
    setTimeout(() => response.writeHead(200).end('done\n'), 800);
    return;
  }
  response.writeHead(404).end('not found\n');
});

server.listen(8791, '127.0.0.1', () => {
  ready = true;
  console.log(`listening, process id ${process.pid}`);
});

function shutdown(signal) {
  if (shuttingDown) return;
  shuttingDown = true;
  console.log(`${signal} received; readiness probe now returns 503`);

  setTimeout(() => {
    console.log('no longer accepting new connections');

    const forced = setTimeout(() => {
      console.log('time is up; cutting remaining connections');
      server.closeAllConnections();
    }, SHUTDOWN_LIMIT_MS);
    forced.unref();                           // must not keep the process alive on its own

    server.close(() => {
      clearTimeout(forced);
      console.log('all requests completed; exiting');
    });
    server.closeIdleConnections();
  }, EXIT_DELAY_MS).unref();
}

process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
```

The trial is run with a script:

```sh
#!/bin/sh
# shutdown-test.sh
node resilient-server.mjs &
serverPid=$!
sleep 1

printf 'before shutdown /ready  -> '; curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8791/ready

# Start a pending request, send the signal while waiting for its response
curl -s http://127.0.0.1:8791/slow > slow-response.txt &
requestPid=$!
sleep 0.1
kill -TERM "$serverPid"

sleep 0.2
printf 'during shutdown /ready -> '; curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8791/ready

wait "$requestPid"
echo "response of the pending request: $(cat slow-response.txt)"
wait "$serverPid"
echo "process exit code: $?"
```

```sh
sh shutdown-test.sh
```

```
listening, process id 18312
before shutdown /ready  -> 200
SIGTERM received; readiness probe now returns 503
during shutdown /ready -> 503
no longer accepting new connections
all requests completed; exiting
response of the pending request: done
process exit code: 0
```

The process id changes on every run. Here's the order the output tells.

**Phase one: report not ready.** The moment the signal arrives, the readiness
probe starts returning 503. A load balancer in front of the process needs a
window to take it out of the list; without it, the balancer keeps sending
requests to a process that is shutting down, and those requests get dropped.

**Phase two: stop taking new connections.** When the delay expires, `close`
gets called. This closes the listening socket but does not drop existing
connections. Idle persistent connections are closed separately; otherwise a
client holding a connection open without sending a request would stretch out
the shutdown.

**Phase three: finish running work.** The `close` callback runs once the last
request completes too. In the output above, the pending request gets its `done`
response — the signal had been sent while that request was being processed.

**Phase four: the time limit.** If a client never releases its connection, the
shutdown runs forever. The forcing timer cuts that off. The `unref` call
matters: this timer must not be the sole reason the process stays alive, or it
waits until the limit runs out even after everything else is done.

The exit code is zero. To the supervisor, this means "shut down on request,"
enabling different behavior depending on the restart policy.

There's other work to do at shutdown too: flushing pending log lines, closing
open files, writing the remaining summary to disk — all inside the `close`
callback, before the process ends. As seen in the Process Object lesson,
asynchronous work cannot be done in the `exit` hook.

## Liveness and Readiness

The two probes answer two separate questions, and mixing them up destabilizes
the service.

The **liveness** question: is the process alive, or should it be killed and
restarted? Its answer looks only at the process's own state; it does not check
its dependencies.

The **readiness** question: can this process accept requests right now? Its
answer looks at transient conditions — has startup finished, has shutdown
begun.

The reasoning: when a dependency is temporarily unreachable, the process is not
ready but it is alive. If the liveness probe checked the dependency, a brief
outage there would restart every process — behavior that both stretches out
the outage and produces a failure of its own.

Probes have to be lightweight: if every check runs a database query, the check
itself turns into load.

## The Restart Contract

Restarting when the process dies is not the process's job; it cannot restart
itself. That job belongs to the operating system's service supervisor, or to the
primary process from the Cluster and Worker Threads lesson.

Three things fall to the process.

**Giving a meaningful exit code.** Zero means a requested shutdown, nonzero
means a failure. This is why a separate code was chosen for a configuration
error in the Configuration Management lesson: the supervisor can stop instead
of endlessly retrying a failure that clearly will not fix itself.

**Starting up fast and idempotently.** If restarting is a recovery method,
startup itself has to be reliable. Work done at startup must not fail when it
finds traces of a previous run left half-finished — the
write-to-temporary-name-and-replace pattern from the File System lesson
provides that guarantee.

**Accounting for the restart storm.** Reopening a process that crashes
immediately produces hundreds of startups a second and keeps the machine busy.
Supervisors prevent this with increasing delay; the process must not shorten
that delay, for instance by reporting a fake success while unhealthy.

A process trying to restart itself is a separate mistake: the way out of a
corrupted state is abandoning the memory carrying that state — also why
restarting works at all, the reasoning behind limiting the last-resort hook in
the Error Handling Strategy lesson to "record and exit."

## Summary

- A shutdown request is a signal; once a listener is attached, the default
  termination behavior gives way to the program's decision.
- Graceful shutdown has four phases: report not ready, stop taking new
  connections, finish running work, cut what's left at the time limit.
- The forcing timer is marked with `unref`; otherwise the process stays alive
  until the limit runs out even after everything else is done.
- The liveness probe looks only at the process's own state, the readiness
  probe at transient state; a liveness probe checking a dependency spreads
  outages.
- Restarting is the supervisor's job; what falls to the process is giving a
  meaningful exit code, starting up fast and idempotently, and not feeding the
  storm.

## Next Step

The measurement collector is up and running: configured, classifying its
errors, writing its log in structured form, tested, observable, and shutting
down cleanly. One question remains — how does its reusable part get carried
into other projects? The final lesson takes up the packaging contract:
declaring entry points, hiding internal files, and the rules of publishing a
version.