---
title: 'The Node.js Runtime'
source: 'https://academia.sh/en/courses/nodejs'
language: en
updated: '2026-08-17T04:41:38+00:00'
license: 'CC BY-SA 4.0'
lessons: 20
outcomes:
    - 'Uses the event loop and stream model of a server-side runtime'
    - 'Writes tools and services with the file system, process, and network interfaces'
    - 'Sets up logging, error handling, and monitoring for production conditions'
---

# The Node.js Runtime

The process model of an event-driven server runtime, the phases of the event loop, streams and backpressure, the file system and network interfaces; error handling, logging, and monitoring for production conditions.

## Runtime Model

The capability differences between the browser and the server runtime, the event loop's phases, the process object, and the rules of module resolution.

### Differences from the Browser

Source: https://academia.sh/en/courses/nodejs/differences-from-the-browser

The host environment concept, the global object's contents, the two module systems coexisting, and the server runtime's capability boundaries.

The Asynchronous JavaScript and the Runtime course established the single-threaded
execution model and the event loop. Everything covered there belonged not to the
language but to the environment hosting it: the task queue and the timers are not
defined in the language standard either. This course takes up the same language in a
different host environment.

The question: when the same syntax, value model, and event-loop concept run in a
server process instead of a browser, what changes? The answer falls under two
headings — which names are available, and what the process is authorized to do.

#### The Language Core and the Host Environment

The language standard defines syntax and the value model: `Array`, `Map`, `Promise`,
`JSON`, `Math` belong to this layer. It says how these values behave, not how to
reach the outside world with them.

The **host environment** is the program running the language, and it provides the
contact with the outside world. A browser is a host environment: it offers the
document tree, cookies, screen events. A server runtime is a host environment too: it
offers the file system, process information, network listening.

The distinction can be tested directly. The file below prints whether a set of names
is present on the global object:

```js
// environment.mjs
const names = [
  'window', 'document', 'localStorage', 'alert',
  'process', 'Buffer', 'global',
  'fetch', 'URL', 'TextEncoder', 'AbortController', 'structuredClone',
];
for (const name of names) {
  console.log(`${name.padEnd(16)} ${name in globalThis ? 'yes' : 'no'}`);
}
```

```sh
node environment.mjs
```

```
window           no
document         no
localStorage     no
alert            no
process          yes
Buffer           yes
global           yes
fetch            yes
URL              yes
TextEncoder      yes
AbortController  yes
structuredClone  yes
```

The output shows three groups. The first exists only in the browser: the document
tree, page storage, and UI calls have no counterpart on the server. The second exists
only server-side: `process` represents process information, `Buffer` represents raw
bytes.

The third group is more interesting. `fetch`, `URL`, `TextEncoder`,
`AbortController`, and `structuredClone` are browser interfaces, but they belong to
the web platform, not the language. The server runtime implements a portion of these
under the same contract, so code running in both environments does not need two
separate versions of every interface. This overlap keeps growing: if an interface is
documented as present from a given version onward, it can be relied on; where it
might be missing, a presence check is used.

The name `global` is an older name pointing to the same object as `globalThis`.
Writing `globalThis` in new code makes the same expression work in both environments.

#### Two Module Systems

The browser has a single module form. The server runtime carries two forms together
and picks one per file.

**CommonJS** loads synchronously through a `require` call and exports by writing to a
`module.exports` object. **ES modules** work through `import` and `export`
declarations and resolve dependencies before running them. The difference between the
two was detailed in the Modules, Tooling and the Ecosystem course; what matters here
is how the choice is made.

The choice looks at two rules: file extension and the nearest package manifest. A
`.cjs` file is always CommonJS, `.mjs` is always an ES module. For a `.js` extension,
the decision comes from the `type` field in the first `package.json` found walking up
from the file's directory: `module` means an ES module, `commonjs` or an absent field
means CommonJS.

The runtime difference between the two forms is measurable:

```js
// scope.cjs
console.log('this === module.exports :', this === module.exports);
console.log('__filename last part   :', __filename.split('/').pop());
console.log('typeof require         :', typeof require);
```

```js
// scope.mjs
console.log('this                    :', this);
console.log('import.meta.url last part:', import.meta.url.split('/').pop());
console.log('typeof require          :', typeof require);
```

```sh
node scope.cjs
```

```
this === module.exports : true
__filename last part   : scope.cjs
typeof require         : function
```

```sh
node scope.mjs
```

```
this                    : undefined
import.meta.url last part: scope.mjs
typeof require          : undefined
```

A CommonJS file is wrapped like a function body: `require`, `module`, `__filename`,
and `__dirname` are that wrapper's parameters, not globals. An ES module has none of
these; its own location is given through `import.meta.url`, not as a file path but as
an address. ES modules are used throughout the course; CommonJS appears only where the
difference needs to be seen.

#### Capability and Authority

The real difference is not in the name list, it is in what the process can do.

Code running in a browser is treated as hostile toward the user's machine and kept in
a **sandbox**: it cannot read an arbitrary file, cannot listen on an arbitrary port,
and can only reach another origin to the extent that origin permits. These
restrictions are the guarantees a browser gives the user.

The server runtime has no such sandbox. The process runs with the authority of the
user who started it. The permission model from the Introduction to Linux course
applies here directly: whatever files the process can read, your script can read too.
The same holds for writing and deleting.

This authority shows in the short file below:

```js
// first-read.mjs
import { readFileSync } from 'node:fs';

const text = readFileSync('measurements.ndjson', 'utf8');
const lines = text.trim().split('\n');

console.log('line count:', lines.length);
console.log('first record:', JSON.parse(lines[0]));
```

The `node:` prefix declares that the name points to a **built-in module**. It
resolves without the prefix too; the prefixed spelling settles which one is meant if a
file or package under the same name exists. The course uses the prefixed form
throughout.

#### The Course's Working File

One concrete piece of work will be built throughout this course: a small service that
collects records from remote measurement nodes, summarizes them, and serves them over
HTTP. The input is a text file where every line is an independent JSON object. Save
the command below creates the twelve lines under the name `measurements.ndjson` in
the working directory:

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

The four fields on each line carry, in order, the moment of measurement, the sending
node, the measured quantity, and the value. This one-record-per-line format is
compatible with the line-based filters from the Shell Programming course: it can be
processed line by line without loading the whole file into memory — decisive in the
Streams topic.

After putting the file in place, run the first read:

```sh
node first-read.mjs
```

```
line count: 12
first record: {
  time: '2024-02-07T09:12:44Z',
  node: 'edge-01',
  metric: 'temperature',
  value: 21.4
}
```

This shows work with no browser counterpart: a local file was read by name, without
user consent. The rest of the course builds on this capability — and takes up the
responsibility it brings.

The error that comes up when a file is not found is a form you will see often
throughout the course:

```sh
node -e "import('node:fs').then(m => m.readFileSync('missing.ndjson','utf8')).catch(e => console.log(e.code, '|', e.syscall, '|', e.path))"
```

```
ENOENT | open | missing.ndjson
```

`ENOENT` is the error name the operating system returns from the system call;
`syscall` says which call failed, `path` says for which path. These three fields let
you decide without reading the message text — the text can change, the code does not.

#### Summary

- The language standard defines the value model; access to the outside world is the
  host environment's responsibility. The browser and the server runtime are two
  separate host environments.
- The global object has three groups: names found only in the browser, names found
  only on the server, and shared interfaces inherited from the web platform.
- Module form is chosen by file extension (`.cjs`, `.mjs`) or the `type` field in
  the nearest package manifest; a CommonJS file is wrapped and receives the names
  `require`, `module`, `__filename` as parameters.
- A server process has no sandbox; the process runs with the file and network
  authority of the user who started it.
- Writing built-in modules with the `node:` prefix removes the chance of a name
  collision; file system errors are distinguished by their `code`, `syscall`, and
  `path` fields.

#### Next Step

In this lesson, the file was read with `readFileSync` — by making the process wait.
In a single-threaded server, that waiting is costly: no other request can be
processed during it. The next lesson opens up, phase by phase, the mechanism used to
avoid that cost — with measurable examples of the order the event loop drains its
queues, when timers run, and where input/output callbacks enter that order.

### Event Loop Phases

Source: https://academia.sh/en/courses/nodejs/event-loop-phases

The loop's phase order, the difference between the timers and check phases, when the two microtask queues drain, and the measurable cost of blocking work.

The previous lesson read the file with `readFileSync`: when the call returned, the
data was ready, but until it returned, no other work in the process advanced. In a
single-threaded server, this behavior is unacceptable; every millisecond spent waiting
delays every request that arrives at the same time.

This lesson's question: what exactly does the mechanism that replaces waiting do? The
Asynchronous JavaScript and the Runtime course introduced the event loop as a cycle
that takes a task from a queue when the call stack empties. In the server runtime,
that cycle's inside becomes visible: not one queue, but several phases visited in
order.

#### The Loop's Phases

The process enters the event loop after running the main module's body start to
finish. Every turn of the loop passes through several phases in a fixed order. Each
phase has its own callback queue; entering a phase drains that queue, then the loop
moves to the next phase.

| Phase | Callbacks it runs |
|---|---|
| timers | expired `setTimeout` and `setInterval` callbacks |
| pending callbacks | system-level callbacks deferred from the previous turn |
| poll | callbacks for completed input/output operations |
| check | callbacks set up with `setImmediate` |
| close callbacks | callbacks for `close` events |

The **poll** phase is where the loop actually waits. If there is no other work to run,
the process stops here and asks the operating system "is there input/output ready?"
This wait is not a CPU-busy wait; the kernel wakes the process when a socket becomes
readable or a file read finishes.

This is how "waiting" separates from "blocking." The process can be asleep in the
poll phase while waiting for a request's response; if data arrives from another
connection in the meantime, it wakes up and runs that connection's callback.

#### The Order Between the Timers and Check Phases

If something is added to a phase's queue while inside a phase that comes after it,
the callback runs in the same turn; adding to an earlier phase's queue makes it wait
for the next turn. This rule produces a definite order when viewed from inside an
input/output callback:

```js
// phases.mjs
import { readFile } from 'node:fs';

console.log('sync body');

readFile('measurements.ndjson', () => {
  console.log('poll: file read');

  process.nextTick(() => console.log('  nextTick queue'));
  Promise.resolve().then(() => console.log('  promise microtask'));
  setTimeout(() => console.log('  timers phase'), 0);
  setImmediate(() => console.log('  check phase'));

  console.log('poll: callback body done');
});
```

```sh
node phases.mjs
```

```
sync body
poll: file read
poll: callback body done
  nextTick queue
  promise microtask
  check phase
  timers phase
```

The order is not chance. The callback ran in the poll phase; because the check phase
comes **after** poll, `setImmediate` ran in the same turn. The timers phase comes
**before** poll, so `setTimeout` waited for the next turn. This relationship always
holds inside an input/output callback.

When the same two calls are made from the main module's body, the order is not
guaranteed:

```js
// race.mjs
setTimeout(() => console.log('timers'), 0);
setImmediate(() => console.log('check'));
```

Because the order changes from run to run, the two blocks below are not measured; the outputs are example runs taken from consecutive runs of the same file:

{/* run: skip */}
```sh
node race.mjs
```

```
check
timers
```

{/* run: skip */}
```sh
node race.mjs
```

```
timers
check
```

The reason is the time elapsed before the loop's first turn is entered. The zero
given to `setTimeout` gets rounded up to at least one millisecond by the runtime. If
the main module runs in under a millisecond, the timer has not expired yet when the
loop reaches the timers phase, and the callback waits for the next turn; if it takes
longer, the timer has expired and the callback runs on the first turn. Since the time
depends on machine load, the result varies from run to run. The rule that follows:
**program logic is never anchored to the relative order of callbacks belonging to two
different phases.**

#### Two Microtask Queues

Between phases and after every callback, two extra queues get drained. These do not
belong to any phase; they are handled at every **checkpoint** between phases.

- The **`process.nextTick` queue** is specific to the runtime.
- The **promise microtask queue** is defined in the language standard and exists in
  the browser too.

At a checkpoint, the `nextTick` queue drains first, then the promise queue. This
order shows in the output above: after the callback body finished, `nextTick queue`
printed first, then `promise microtask`.

There is a case where this order looks different at the module's top level:

```js
// micro.cjs
console.log('1 sync');
process.nextTick(() => console.log('2 nextTick'));
Promise.resolve().then(() => console.log('3 microtask'));
console.log('4 sync end');
```

```sh
node micro.cjs
```

```
1 sync
4 sync end
2 nextTick
3 microtask
```

When the same lines are saved with a `.mjs` extension, the order reverses:

```js
// micro.mjs
console.log('1 sync');
process.nextTick(() => console.log('2 nextTick'));
Promise.resolve().then(() => console.log('3 microtask'));
console.log('4 sync end');
```

```sh
node micro.mjs
```

```
1 sync
4 sync end
3 microtask
2 nextTick
```

The reason is how ES modules get evaluated: the module body itself runs as a promise
job. When the body finishes, the process is still draining the promise queue, so a
newly added promise job runs in the same drain pass; the `nextTick` queue, though,
waits for the checkpoint at the turn's end. The lesson: the two queues' relative
order is not a contract, it depends on where the code runs. Program correctness must
not rest on this order.

The `process.nextTick` queue has a hazard of its own. This queue is drained
**entirely** before moving to the next phase; if a callback adds a new one to the
queue every time it runs, the loop can never advance and the process stops
responding.

#### The Measured Cost of Blocking

The phase model has exactly one weak point: while a phase's callback is running,
nothing else can run. Long-running synchronous work stops the entire loop.

```js
// blocking.mjs
const start = process.hrtime.bigint();

setTimeout(() => {
  const elapsed = Number(process.hrtime.bigint() - start) / 1e6;
  console.log(`timer set for 10 ms fired after ${elapsed.toFixed(0)} ms`);
}, 10);

// Synchronous work that deliberately keeps the event loop busy
const deadline = Date.now() + 200;
while (Date.now() < deadline) { /* empty loop */ }
console.log('sync work done');
```

```sh
node blocking.mjs
```

{/* mask: (?<=fired after )2[0-9]{2}(?= ms) */}
```
sync work done
timer set for 10 ms fired after 202 ms
```

The printed duration varies by a few milliseconds depending on machine and load;
what matters is the order of magnitude. The 10 ms given to the timer is not a
promise but a **lower bound**: the callback runs once its time has expired and the
loop reaches the timers phase. Since the loop was kept busy for 200 ms, the delay
came out close to that.

The same measurement means this on a server: a handler doing 200 ms of synchronous
work per request answers at most five requests a second, and the sixth request waits
in queue. This limit comes not from the hardware but from how the work is carried
out.

Two rules follow. First, synchronous input/output interfaces (calls ending in
`...Sync`) are not used while handling a server request; work done before the loop is
running, like reading configuration at startup, is the exception. Second, long
CPU-bound computations have to be moved off the main thread; how to do that is taken
up in the Cluster and Worker Threads lesson.

#### Summary

- Every turn of the event loop passes through phases in a fixed order: timers,
  pending callbacks, poll, check, close callbacks. Each phase has its own queue.
- The poll phase is the wait point that does not keep the CPU busy; the kernel
  wakes the process when input/output is ready.
- Inside an input/output callback, `setImmediate` runs in the same turn,
  `setTimeout` in the next. In the main module's body, this order is not
  guaranteed.
- The `nextTick` and promise queues drain at checkpoints between phases; their
  relative order depends on where the code runs and is never used as a basis for
  program logic.
- Timer delay is a lower bound; synchronous work that keeps the loop busy directly
  extends the delay by its own duration.

#### Next Step

In this lesson, the process was treated as the container that runs the loop. The
process itself is also an object exposed to the program: you read and write, through
that object, what arguments it started with, which environment variables it
inherited, where its standard streams connect, and what exit code it will end with.
The next lesson takes up the process object and builds the measurement collector's
first command-line behavior.

### The Process Object

Source: https://academia.sh/en/courses/nodejs/process-object

The argument array, environment variables, the object counterpart of the standard streams, the two ways to set an exit code, and the exit hook.

The previous lesson examined the loop's turns. The program running inside those turns
talks to the outside world not only through files and the network but also with the
environment that started it: what arguments was it called with, which environment
variables did it inherit, where is its output connected, and what will it report to
its caller when it ends?

All of this is gathered in a single global object. What a shell script did with `$1`,
`$?`, and `$HOME` in the Shell Programming course, this object's fields do here.

#### Arguments and Environment

`process.argv` carries the command line the process was started with, as an array of
strings. The array's first two elements are fixed: the executable's path and the
running main module's path. The arguments the user typed start at the third element.

```js
// process-info.mjs
console.log('argv[0] last part:', process.argv[0].split('/').pop());
console.log('argv[1] last part:', process.argv[1].split('/').pop());
console.log('remaining args   :', process.argv.slice(2));
console.log('platform         :', process.platform);
console.log('working dir      :', process.cwd().split('/').pop());
console.log('MEASUREMENTS_DIR :', process.env.MEASUREMENTS_DIR ?? '(undefined)');
```

```sh
node process-info.mjs --node edge-01 --format summary
```

{/* mask: (?<=working dir      : )[A-Za-z0-9_-]+ */}
```
argv[0] last part: node
argv[1] last part: process-info.mjs
remaining args   : [ '--node', 'edge-01', '--format', 'summary' ]
platform         : linux
working dir      : lab
MEASUREMENTS_DIR : (undefined)
```

The full paths on the first two lines vary by machine, so the example prints only
their last parts. `platform` gives a different value per operating system (`linux` in the example);
`working dir` depends on which directory you called the command from. The next topic
covers the right way to pin paths down.

Environment variables are read through `process.env`. Every value is a string; if you
expect a number, converting and validating it is on you.

```sh
MEASUREMENTS_DIR=/data/measurements node process-info.mjs
```

{/* mask: (?<=working dir      : )[A-Za-z0-9_-]+ */}
```
argv[0] last part: node
argv[1] last part: process-info.mjs
remaining args   : []
platform         : linux
working dir      : lab
MEASUREMENTS_DIR : /data/measurements
```

Writing the variable before the command defines it only for that call; the shell
session is unaffected — the same environment-inheritance rule from the Shell
Programming course.

#### The Object Counterpart of the Standard Streams

In the Shell Programming course, the three standard streams were referred to by
number: 0 input, 1 output, 2 error. The runtime offers these three as objects:
`process.stdin`, `process.stdout`, `process.stderr`. Not ordinary objects but stream
objects, detailed in later lessons; for now, their `write` methods are enough.

The stream-separation rule holds here too: data goes to standard output, every
description about it goes to standard error. The tool below reads the measurement
file and prints two tab-separated columns; the counter and error messages go to the
second stream.

```js
// stream-separation.mjs
import { readFileSync } from 'node:fs';

const file = process.argv[2] ?? 'measurements.ndjson';

let text;
try {
  text = readFileSync(file, 'utf8');
} catch (error) {
  process.stderr.write(`error: could not read ${file} (${error.code})\n`);
  process.exitCode = 66;                    // no data input
  text = '';
}

let count = 0;
for (const line of text.split('\n')) {
  if (line.trim() === '') continue;
  const { node, value } = JSON.parse(line);
  process.stdout.write(`${node}\t${value}\n`);
  count += 1;
}

process.stderr.write(`${count} records processed\n`);

process.on('exit', (code) => {
  process.stderr.write(`process exiting with code ${code}\n`);
});
```

The tool reads the `measurements.ndjson` file created in the first lesson. If it is
not in your working directory, recreate it with this command:

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

To see only the data, the second stream is discarded:

```sh
node stream-separation.mjs 2>/dev/null | head -3
```

```
edge-01	21.4
edge-01	48
edge-02	19.8
```

To see only the diagnostic messages, the first stream:

```sh
node stream-separation.mjs 1>/dev/null
```

```
12 records processed
process exiting with code 0
```

This separation is the precondition for the tool to be pluggable into a pipeline. If
the counter line went to standard output, the next command reading that output would
mistake it for a data line.

`console.log` writes to standard output, `console.error` to standard error. Both are
convenience functions built on `write`; formatting objects readably is useful, but
where the output feeds another program, `write` is preferred for controlling the
format directly.

#### Exit Code

The only numeric result a process reports to its caller is the exit code. The rule
from the Shell Programming course holds here too: zero means success, any nonzero
value is a kind of failure.

There are two ways to set the code, and the difference between them matters.

```js
// exit-hard.mjs
setTimeout(() => console.log('pending work'), 0);
console.log('before exit');
process.exit(3);
```

```sh
node exit-hard.mjs; echo "exit code: $?"
```

```
before exit
exit code: 3
```

```js
// exit-soft.mjs
setTimeout(() => console.log('pending work'), 0);
console.log('before exit');
process.exitCode = 3;
```

```sh
node exit-soft.mjs; echo "exit code: $?"
```

```
before exit
pending work
exit code: 3
```

`process.exit` stops the event loop right then; queued callbacks never run — the
absence of `pending work` in the first output above is the result. The same
interruption can apply to output not yet flushed: in a process whose output is
connected to a pipe, the write can run asynchronously, and a hard exit can cut it
off midway. A write to a file is synchronous and is not affected by this.

Writing to `process.exitCode`, though, only records the result. The process ends with
this code once the loop finishes its work and empties naturally. A pending write, a
closing connection, and an ongoing file operation get to complete.

The rule: **a hard exit is used only when the process's state is judged to be
corrupted.** On ordinary error paths, the code is written to the field and the
process ends on its own.

The tool above follows this rule. When the file is not found, the process is given
code 66, but it is not cut short; the counter line and the exit hook still run:

```sh
node stream-separation.mjs missing.ndjson; echo "exit code: $?"
```

```
error: could not read missing.ndjson (ENOENT)
0 records processed
process exiting with code 66
exit code: 66
```

#### The Exit Hook

The callback set up with `process.on('exit', ...)` runs right before the process
ends and takes the exit code as its argument — what wrote the last line in both runs
above.

This hook has one strict rule: **asynchronous work cannot be started inside it.** The
process is about to close; the loop is never returned to after the hook, so a timer
or file write set up inside it never runs. The hook is only for short, synchronous
work — writing a counter, releasing a lock, that sort of thing.

Real cleanup work is done when a shutdown **signal** is received; signals and
graceful shutdown are the subject of the Process Management and Robustness lesson.

Two more fields describe the process itself. `process.pid` is the identifier the
operating system gives it, different on every run, used to tell processes apart in
log lines. `process.uptime()` gives the seconds since the process started, written as
age information at health probes.

#### Summary

- The first two elements of `process.argv` are the executable and main-module
  paths; user arguments start at the third element.
- `process.env` values are always strings; numeric or boolean interpretation is the
  reading code's responsibility.
- The three standard streams are offered as objects; a tool becomes pluggable into
  a pipeline once data goes to standard output and diagnostics go to standard
  error.
- `process.exit` cuts the loop right then and drops pending work; `process.exitCode`
  only records the result and lets the process end naturally.
- The `exit` hook is suited only to synchronous work; since the event loop is never
  returned to after it, asynchronous cleanup cannot be done there.

#### Next Step

In this lesson, the file name was given as `measurements.ndjson`, relative to the
working directory; the program cannot find it when called from another directory. The
next lesson takes up the mechanism that removes this dependency: the rules the
runtime uses to decide which file to load when a name is written — the distinction
between a built-in module, a file path, and a package name.

### Module Resolution

Source: https://academia.sh/en/courses/nodejs/module-resolution

The three specifier types, the file-extension requirement, the package directory search chain, the exports map, and querying resolution from the program.

The previous lesson established how the process talks to the outside world. This
lesson's question is narrower but its reach is wide: the name written on an `import`
line — which file does it correspond to?

The answer splits across three rule sets. Which set the specifier falls into is clear
from its first character, and that choice determines everything else.

#### Three Specifier Types

A **built-in specifier** starts with the `node:` prefix: `node:fs`, `node:path`,
`node:http`. It points to modules embedded in the runtime and is never searched for
on the file system.

A **file specifier** starts with `./`, `../`, or a root path, and resolves relative to
the file it appears in.

A **package specifier** is a name matching neither form: `measurement-tools`,
`measurement-tools/units`. It gets searched for in package directories.

All three can appear side by side in a single file. The layout below repeats
throughout the course: a package manifest, a local module, and a package in the
dependency directory.

```
resolution/
  package.json
  main.js
  summary.js
  node_modules/
    measurement-tools/
      package.json
      index.js
      units.js
      internal.js
```

The source files are given with their path in the first-line comment. The two
`package.json` files are created by command:

```sh
mkdir -p resolution/node_modules/measurement-tools
cat > resolution/package.json <<'SON'
{
  "name": "measurement-collector",
  "version": "1.0.0",
  "type": "module"
}
SON
cat > resolution/node_modules/measurement-tools/package.json <<'SON'
{
  "name": "measurement-tools",
  "version": "1.0.0",
  "type": "module",
  "exports": {
    ".": "./index.js",
    "./units": "./units.js"
  }
}
SON
```

```js
// resolution/node_modules/measurement-tools/index.js
export const name = 'measurement-tools';
```

```js
// resolution/node_modules/measurement-tools/units.js
export const units = { temperature: 'C', humidity: '%' };
```

```js
// resolution/node_modules/measurement-tools/internal.js
export const hidden = true;
```

```js
// resolution/summary.js
export function average(numbers) {
  return numbers.reduce((t, s) => t + s, 0) / numbers.length;
}
```

```js
// resolution/main.js
import { readFileSync } from 'node:fs';
import { average } from './summary.js';
import { name } from 'measurement-tools';
import { units } from 'measurement-tools/units';

console.log('built-in module :', typeof readFileSync);
console.log('file module     :', average([21.4, 19.8, 24.1]).toFixed(2));
console.log('package module  :', name);
console.log('package subpath :', units.temperature);
```

```sh
node resolution/main.js
```

```
built-in module : function
file module     : 21.77
package module  : measurement-tools
package subpath : C
```

Four files have the `.js` extension; the `"type": "module"` field in the nearest
manifest is what gets them interpreted as ES modules. The dependency directory's own
manifest carries the same field — a package's module form comes from its own
manifest, not the manifest of whatever uses it.

#### The Extension Is Required for File Specifiers

In CommonJS, `require('./summary')` was enough; the runtime tried `./summary.js`,
`./summary.json`, `./summary.node` in turn, and failing those, looked at the
directory file in `./summary/`. This guessing costs several file-system calls per
import and means different environments could pick a different file for the same
name.

There is no guessing in ES modules. The parentheses make `cd` affect only that line:

```sh
(cd resolution && node -e "import('./summary').catch(e => console.log(e.code))")
```

```
ERR_MODULE_NOT_FOUND
```

Writing the extension is not a burden, it is a contract that reduces resolution to a
single step: a specifier maps to exactly one address, computable without touching the
file system.

#### The Package Search Chain

When a package specifier is seen, the search starts from the importing file's
directory and checks the `node_modules` subdirectory at every step. If not found, it
climbs one directory up; this continues up to the root directory.

```js
// resolution/deep/deeper/deep.js
import { name } from 'measurement-tools';
console.log('found node_modules in an ancestor dir:', name);
```

```sh
node resolution/deep/deeper/deep.js
```

```
found node_modules in an ancestor dir: measurement-tools
```

The file is two directories deep with no dependency directory beside it; the name was
found by climbing the chain upward. This rule underlies the dependency-tree layout
explained in the Modules, Tooling and the Ecosystem course: different versions of the
same package can coexist by sitting at different depths.

#### The Exports Map

A package's `exports` field determines which paths can be imported from outside. The
package above defines two entries: the root (`.`) and `./units`. The third file has
no place in the map:

```sh
(cd resolution && node -e "import('measurement-tools/internal.js').catch(e => console.log(e.code))")
```

```
ERR_PACKAGE_PATH_NOT_EXPORTED
```

This differs from the file not existing: the file is right where it is, but the
package does not count it as part of its public interface. The map gives the package
author a way to apply the interface–implementation distinction from the Programming
Fundamentals course. Without a map, every file in the package is open, and any
internal reorganization can break its users.

A package can also define aliases for its own internal use. Names starting with `#`
in the `imports` field resolve only from that package's own files and stay invisible
from outside. This mechanism returns in the Package Publishing lesson.

#### Querying Resolution

Which specifier corresponds to which file can be asked from the program itself:

```js
// resolution/resolve.js
const paths = ['node:fs', './summary.js', 'measurement-tools', 'measurement-tools/units'];
for (const path of paths) {
  const resolved = import.meta.resolve(path);
  console.log(path.padEnd(22), resolved.replace(/^.*\/resolution\//, './'));
}
```

```sh
node resolution/resolve.js
```

```
node:fs                node:fs
./summary.js           ./summary.js
measurement-tools      ./node_modules/measurement-tools/index.js
measurement-tools/units ./node_modules/measurement-tools/units.js
```

The output paths are file addresses; the example trims the common prefix for
readability, and the real values start with `file://`. The built-in module, though,
comes back as its own specifier rather than an address, since it has no counterpart
on the file system.

This query is the shortest way to verify a dependency comes from the expected file.
`import.meta.resolve` exists only in ES modules; its CommonJS counterpart is
`require.resolve`.

When a bridge between the two forms is needed, `createRequire` from `node:module`
produces a `require` that works inside an ES module — the way to use a dependency not
published as an ES module, used again in the Package Publishing lesson.

#### Modules Are Evaluated Once

A module is evaluated once per resolved address; even imported from many places, its
body never runs again, and the same export object is shared. Module-level state is
therefore singular for the whole process — a counter, a connection pool, a
configuration object created in a module body is the same instance for everyone who
imports it.

The detail: singularity depends on the resolved address, not the specifier. Two
copies of the same package in two different directories give two addresses and
produce two instances — the root of the problem duplicated versions cause in a
dependency tree.

An `import` declaration can only be written at a module's top level and
unconditionally, since dependencies resolve before the body runs. When conditional
loading is needed, the promise-returning `import()` form is used — the measurement
collector can use it for output formatters that load only when requested.

#### Summary

- Specifiers come in three types: `node:`-prefixed built-ins, file paths starting
  with `./` or `../`, and package names outside those forms. The type determines
  the resolution rule.
- File extensions are required in ES modules; extension guessing and directory-file
  lookup are specific to CommonJS.
- A package name is searched for in the `node_modules` directory at every level,
  starting from the importing file's directory and going up.
- The `exports` map fixes a package's public interface; access to a file not in the
  map is rejected even if the file exists.
- A module is evaluated once per resolved address; module-level state is singular
  for everyone sharing that address.

#### Next Step

Four parts of the runtime model are complete: the host environment, the event loop,
the process object, and module resolution. The next topic takes up, in order, the
built-in modules built on this model. The first stop is the file system: three
interfaces offer the same read operation, and the choice among them directly
determines the blocking cost measured in the event-loop lesson.

## Built-in Modules

The file system, path operations, streams, buffers, the event emitter, the HTTP server, child processes, and clustering with worker threads.

### File System

Source: https://academia.sh/en/courses/nodejs/file-system

The difference between three interfaces offering the same work, the role of the thread pool, branching on error codes, directory operations, and updating a file without ever leaving it half-written.

The runtime model topic measured how blocking work stops the event loop. The file
system interface is the direct application ground of that measurement: the same read
operation is offered in three separate call forms, and the three behave differently
toward the event loop.

This lesson compares the three forms, determines which one is right in which
situation, and writes the measurement collector's first real step: reading a file and
producing its summary.

#### Three Interfaces

The built-in file system module offers the same operations in three separate forms.
The sync form lives in the `node:fs` module with a `Sync` suffix, the callback form in
the same module without a suffix, and the promise-based form in the
`node:fs/promises` module.

The examples in this lesson read the course's working file, `measurements.ndjson`.
If it is not in your working directory, the command below creates it; the content is
the same twelve lines given in the runtime model topic:

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

```js
// interfaces.mjs
import { readFileSync, readFile } from 'node:fs';
import { readFile as readFilePromise } from 'node:fs/promises';

const file = 'measurements.ndjson';
const countLines = (text) => text.trim().split('\n').length;

// 1. Sync
console.log('sync    :', countLines(readFileSync(file, 'utf8')));

// 2. Callback
readFile(file, 'utf8', (error, text) => {
  if (error) throw error;
  console.log('callback:', countLines(text));
});

// 3. Promise
readFilePromise(file, 'utf8').then((text) => {
  console.log('promise :', countLines(text));
});

console.log('synchronous flow continued here');
```

```sh
node interfaces.mjs
```

```
sync    : 12
synchronous flow continued here
callback: 12
promise : 12
```

The order of the output shows the difference between the three forms. The sync call
stopped the process's progress until it returned a result; it wrote its result before
the second line. The other two started the work and returned right away, so the line
`synchronous flow continued here` appeared before their results.

The callback form's signature carries a contract: the first parameter is the error,
the result is the second parameter. If there is no error, the first parameter comes
as `null`. This arrangement was the only way to do asynchronous work before promises
entered the language, and it still exists in most built-in modules.

The promise-based form makes the same work readable in sequence with `async`/`await`.
It is the default choice throughout the course.

The selection rule is this:

- The **sync form** is used while the event loop is not doing any work yet: an
  initial configuration read, the body of a one-shot script, a final sync write at
  shutdown. It is not used while handling a request.
- The **promise-based form** is the default for service code.
- The **callback form** is used on hot paths where the cost of a promise wrapper has
  to be avoided, and in calls with no promise-based counterpart.

#### Where the Asynchrony Comes From

For network sockets, the kernel's notification mechanism takes over the waiting; the
process sleeps in the poll phase and wakes when data arrives. The file system is
different: on common operating systems there is no comparable notification mechanism
for ordinary files.

The runtime fills this gap with a **thread pool**. File read, write, and directory
listing calls are handed to a thread in the pool; that thread makes the blocking
system call, and once the work is done the result is placed on the main thread's
queue. The main thread keeps handling other requests in the meantime.

This has two consequences. First, file operations genuinely run in parallel; second,
the pool has a fixed width. If more file operations than the pool's width are started
at once, the excess queues up. The pool width can be adjusted with an environment
variable; changing it without measuring first does no good.

#### Branching on Error Codes

File system errors are distinguished not by their message but by their `code` field.
The message text is for readability and can change; the code corresponds to the name
the operating system's system call returned.

```js
// errors.mjs
import { readFile } from 'node:fs/promises';

async function safeRead(path) {
  try {
    return await readFile(path, 'utf8');
  } catch (error) {
    if (error.code === 'ENOENT') return '';       // file missing: empty data
    if (error.code === 'EACCES') {                 // no permission: leave to caller
      throw new Error(`no read permission: ${path}`, { cause: error });
    }
    throw error;
  }
}

console.log('existing:', (await safeRead('measurements.ndjson')).length, 'characters');
console.log('missing :', (await safeRead('missing.ndjson')).length, 'characters');

try {
  await readFile('missing.ndjson', 'utf8');
} catch (error) {
  console.log('code:', error.code, '| syscall:', error.syscall, '| path:', error.path);
}
```

```sh
node errors.mjs
```

```
existing: 1008 characters
missing : 0 characters
code: ENOENT | syscall: open | path: missing.ndjson
```

`ENOENT` (does not exist), `EACCES` (no permission), `EISDIR` (a directory was
given), `ENOSPC` (disk full), and `EMFILE` (open file limit exceeded) are the most
frequently encountered. The permission model from the Introduction to Linux course
determines when `EACCES` arrives: the process's user id and group membership are
compared against the file's permission bits.

It matters that every caught error is not swallowed. The function above handles only
the two cases it knows about, and rethrows the rest as is. Silently turning an unknown
code into empty data hides the failure.

#### Directory Operations

Creating, listing, and removing a directory live in the same module. The block below
sets up an example tree itself, then lists it:

```js
// directories.mjs
import { mkdir, writeFile, readdir, stat, rm, readFile } from 'node:fs/promises';

// This block sets up the example tree itself
await rm('data', { recursive: true, force: true });
await mkdir('data/2024-02-07/raw', { recursive: true });   // also opens intermediate directories
await writeFile('data/2024-02-07/measurements.ndjson', await readFile('measurements.ndjson'));
await writeFile('data/2024-02-07/note.txt', 'edge-03 node has been hot since morning\n');

const entries = await readdir('data/2024-02-07', { withFileTypes: true });
for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) {
  if (entry.isDirectory()) {
    console.log(`directory - ${entry.name}`);
  } else {
    const { size } = await stat(`data/2024-02-07/${entry.name}`);
    console.log(`file  ${String(size).padStart(5)} ${entry.name}`);
  }
}
```

```sh
node directories.mjs
```

```
file   1008 measurements.ndjson
file     40 note.txt
directory - raw
```

The `recursive` option appears in two places and does two different jobs. In the
`mkdir` call it opens intermediate directories and does not error if the directory
already exists; this makes the operation idempotent. In the `rm` call it removes the
entire subtree — together with `force`, it also does not error if the target does not
exist.

An `rm` call cannot be undone. The rule stated for the shell in the Introduction to
Linux course holds here too: if the path to be removed comes from a variable, verify
before removing that the path stays under the root you expect. How to write that
verification is the subject of the next lesson.

The `withFileTypes` option on the `readdir` call also brings type information along
with the names; without it, a separate `stat` call would be needed for each name. The
order of the returned entries depends on the file system and cannot be assumed to be
alphabetical — that is why the example sorts them.

#### Writing Without Leaving a Half-Written File

The measurement collector's first output is a file producing a summary per node and
metric. Writing directly over the target file carries a risk: if the process ends
mid-write, the file is left half-written and the program reading it sees corrupt
data.

The fix is writing to a temporary name and putting it in place once complete. Within
the same file system, a rename operation is atomic: readers see either the entirety
of the old file or the entirety of the new one.

```js
// summarize.mjs
import { readFile, writeFile, rename, stat } from 'node:fs/promises';

async function summarize(source, destination) {
  const text = await readFile(source, 'utf8');
  const buckets = new Map();

  for (const line of text.split('\n')) {
    if (line.trim() === '') continue;
    const { node, metric, value } = JSON.parse(line);
    const key = `${node}/${metric}`;
    const record = buckets.get(key) ?? { count: 0, total: 0 };
    record.count += 1;
    record.total += value;
    buckets.set(key, record);
  }

  const summary = [...buckets].map(([key, { count, total }]) => ({
    key,
    count,
    average: Number((total / count).toFixed(2)),
  }));

  // To avoid leaving a half-written file, write to a temp name first, then rename into place
  const temp = `${destination}.temp`;
  await writeFile(temp, JSON.stringify(summary, null, 2) + '\n');
  await rename(temp, destination);

  const info = await stat(destination);
  return { recordCount: summary.length, bytes: info.size };
}

const result = await summarize('measurements.ndjson', 'summary.json');
console.log(`${result.recordCount} summary records, ${result.bytes} bytes written`);
```

```sh
node summarize.mjs
```

```
6 summary records, 471 bytes written
```

The temporary file has to be created in the **same directory** as the target. A
rename cannot span file systems: the kernel does not carry it out as a copy-then-delete,
it fails with `EXDEV`. The guarantee therefore never weakens silently; the operation
does not happen at all.

This approach has a limit too: the data becomes visible only after the rename
operation, not right after the process finishes, but exactly when it is really
written to disk depends on the operating system's cache. If resilience against a power
outage is needed, a sync call is made through the file descriptor; ordinary services
usually do not take on this extra cost.

The produced `summary.json` file is the content the service will serve throughout the
course; in later lessons the same summary is produced first through streams, then over
HTTP.

#### Summary

- File system operations are offered through three interfaces: sync, callback, and
  promise-based. The sync form is used only while the event loop is doing no work.
- File operations' asynchrony comes from the thread pool; the pool has a fixed width,
  and excess concurrent requests queue up.
- Errors are distinguished by their `code` field; branching based on message text is
  fragile.
- The recursive option in an `mkdir` call makes the operation idempotent; the order
  `readdir` returns depends on the file system.
- When an output file is updated by writing to a temporary name and renaming within
  the same directory, readers never see half-written content.

#### Next Step

Every path in this lesson was written relative to the working directory; when the
program is invoked from a different directory, it cannot find the file. Writing the
separator character by hand also ties the tool to a single platform. The next lesson
covers building paths: the rules for joining parts, making a relative path absolute,
and verifying that a piece of user input does not escape the root directory.

### Path Operations

Source: https://academia.sh/en/courses/nodejs/path-operations

Joining and resolving path parts, platform rules offered separately, anchoring from the script's own location, and preventing an escape outside the root directory.

Every filename in the previous lesson was written bare: `measurements.ndjson`,
`data/2024-02-07`. These names are resolved against the working directory, so the same
program cannot find the file when invoked from a different directory. Writing the
separator character by hand also ties the tool to a single platform's path rules.

This lesson introduces the built-in module that treats paths as data, and answers two
questions: where should the program look for its own files, and how far should a path
piece coming from outside be allowed to go?

#### Joining and Resolving

The `node:path` module has two core functions, and the difference between them is
frequently confused.

`join` appends the given parts to each other with the platform's separator and
normalizes the result; if given a relative result, it stays relative. `resolve`
instead walks right to left and produces an absolute path; if no part is absolute, it
prepends the working directory.

```js
// paths.mjs
import path from 'node:path';

console.log('join      :', path.join('data', '2024-02-07', 'measurements.ndjson'));
console.log('join + .. :', path.join('data', '2024-02-07', '..', 'summary.json'));
console.log('sep       :', JSON.stringify(path.sep));
console.log();

const filePath = 'data/2024-02-07/measurements.ndjson';
console.log('dirname   :', path.dirname(filePath));
console.log('basename  :', path.basename(filePath));
console.log('no ext    :', path.basename(filePath, path.extname(filePath)));
console.log('extname   :', path.extname(filePath));
console.log('parse     :', path.parse(filePath));
```

```sh
node paths.mjs
```

```
join      : data/2024-02-07/measurements.ndjson
join + .. : data/summary.json
sep       : "/"

dirname   : data/2024-02-07
basename  : measurements.ndjson
no ext    : measurements
extname   : .ndjson
parse     : {
  root: '',
  dir: 'data/2024-02-07',
  base: 'measurements.ndjson',
  ext: '.ndjson',
  name: 'measurements'
}
```

The second line shows normalization: a `..` part means going up one directory and
disappears in the result. The `sep` value varies by the platform you run on; the
forward slash in the example was taken on a Unix derivative.

None of these functions look at the file system. `extname` does not say a file
genuinely is JSON, it says how its name ends; `dirname` does not say that directory
exists. Path operations are pure string operations — they fit the pure function
definition from the Programming Fundamentals course, and this is why they are easy to
test.

#### Platform Rules Are Offered Separately

The module applies the rules of the platform it runs on by default, but keeps both
rule sets separately accessible too.

```js
// platforms.mjs
import path from 'node:path';

console.log('this platform :', path.join('data', '2024-02-07', 'summary.json'));
console.log('posix         :', path.posix.join('data', '2024-02-07', 'summary.json'));
console.log('win32         :', path.win32.join('data', '2024-02-07', 'summary.json'));
console.log();
console.log('is "C:\\data" absolute by win32 rules:', path.win32.isAbsolute('C:\\data'));
console.log('is "C:\\data" absolute by posix rules:', path.posix.isAbsolute('C:\\data'));
console.log('is "/data"   absolute by posix rules:', path.posix.isAbsolute('/data'));
```

```sh
node platforms.mjs
```

```
this platform : data/2024-02-07/summary.json
posix         : data/2024-02-07/summary.json
win32         : data\2024-02-07\summary.json

is "C:\data" absolute by win32 rules: true
is "C:\data" absolute by posix rules: false
is "/data"   absolute by posix rules: true
```

The first line reflects the rules of whatever platform the example runs on; on a
Windows machine it would print the backslash form. The existence of separate rule
sets matters for work unrelated to local file paths: parsing an archive entry or an
HTTP path should give the same result on every platform, and `path.posix` is chosen
directly for that.

For address paths there is also the `URL` object, and the two domains should not be
mixed up. Character escaping in a request path follows rules the `path` functions
know nothing about; on the server side, path parsing is done with the address object.

#### Deriving One Path From Another

A frequent job for the measurement collector is producing an output filename from an
input filename: `measurements.ndjson` is read, `measurements-summary.json` is written
next to it. Done through string concatenation, this work rests on an assumption about
how many characters the extension is, and breaks between `.ndjson` and `.json`.

`parse` splits a path into its fields, `format` builds a path from fields. Used
together, the derivation is done without counting characters — but the `format` call
has a priority:

```js
// derive.mjs
import path from 'node:path';

const source = 'data/2024-02-07/measurements.ndjson';
const parts = path.parse(source);

// if base is present it overrides ext and name; base is dropped to derive correctly
const destination = path.format({ dir: parts.dir, name: `${parts.name}-summary`, ext: '.json' });
const wrong = path.format({ ...parts, name: `${parts.name}-summary`, ext: '.json' });

console.log('source     :', source);
console.log('destination:', destination);
console.log('base kept  :', wrong);
console.log();

console.log('relative (neighbor):', path.relative('data/2024-02-07', 'data/2024-02-08/measurements.ndjson'));
console.log('relative (same)    :', JSON.stringify(path.relative('data', 'data')));
console.log('relative (outside) :', path.relative('data', 'secret.txt'));
```

```sh
node derive.mjs
```

```
source     : data/2024-02-07/measurements.ndjson
destination: data/2024-02-07/measurements-summary.json
base kept  : data/2024-02-07/measurements.ndjson

relative (neighbor): ../2024-02-08/measurements.ndjson
relative (same)    : ""
relative (outside) : ../secret.txt
```

The third line shows the trap. `format` ignores the `name` and `ext` fields if the
given object has a `base` field; because `parse`'s output carries a `base` field,
directly spreading it returns the unchanged path unmodified. While deriving, `base` is
either omitted or recomputed.

The output's second block introduces the `relative` function: given two paths, it
gives the relative step to walk from the first to the second. Three behaviors each
mean something separate. The path to a neighboring directory goes up a level and back
down. If the two paths are the same, the result is an empty string — not a child
item, the path itself is meant. If the target falls outside the base, the result
starts with `..`.

The last behavior is the second notation for the root check in the next section: if
the result of `path.relative(root, full)` does not start with `..`, the target is
under the root. This form does not require adding a separator, but on a
case-insensitive file system both forms hit the same limit: string comparison does not
know the file system's name-matching rule.

#### The Script's Own Location

A program finding files next to itself should not depend on the working directory. In
ES modules the anchor is the `import.meta.url` value; this is a file address and has
to be converted to a file system path.

```js
// safety.mjs
import path from 'node:path';
import { fileURLToPath } from 'node:url';

// The script's own location: a fixed anchor independent of the working directory
const thisFile = fileURLToPath(import.meta.url);
const thisDir = path.dirname(thisFile);
console.log('script directory (last part):', path.basename(thisDir));

const root = path.resolve(thisDir, 'data');

function safeResolve(userPath) {
  const full = path.resolve(root, userPath);
  if (full !== root && !full.startsWith(root + path.sep)) {
    throw new Error(`outside root directory: ${userPath}`);
  }
  return path.relative(root, full);
}

for (const request of ['2024-02-07/measurements.ndjson', './a/../b.json', '../secret.txt']) {
  try {
    console.log(`${request.padEnd(28)} -> ${safeResolve(request)}`);
  } catch (error) {
    console.log(`${request.padEnd(28)} -> REJECTED: ${error.message}`);
  }
}
```

The same script gives the same result when invoked from two different directories:

```sh
node safety.mjs
```

{/* mask: (?<=script directory \(last part\): )[^ ]+ */}
```
script directory (last part): lab
2024-02-07/measurements.ndjson -> 2024-02-07/measurements.ndjson
./a/../b.json                -> b.json
../secret.txt                -> REJECTED: outside root directory: ../secret.txt
```

The directory name on the first line depends on which directory you saved the script
in; in the example it was taken from a directory named `lab`. What matters is that
this value does not change whether you invoke the command from the root directory or
somewhere else.

The address-to-path conversion cannot be done by hand. A space in a file address is
encoded as `%20`, and on Windows a drive letter follows an extra rule;
`fileURLToPath` takes care of these details. In the reverse direction, converting a
path to an address uses `pathToFileURL` from the same module.

#### Preventing an Escape Outside the Root Directory

The `safeResolve` function above already solves a problem the measurement collector
will run into later. If a service is going to open a file using a name coming from a
request path, it has to be verified that name stays under the designated root
directory. Without verification, a request containing a `../` sequence could reach
files outside the root.

Verification has three steps, and their order matters:

1. Convert the root directory to an **absolute** path.
2. Resolve the user path against the root. `resolve` applies the `..` parts at this
   step.
3. Verify the result equals the root directory, or starts with the root plus a
   separator.

The separator detail in the third step looks unnecessary but is not. If only
`full.startsWith(root)` is checked, a neighboring directory named `data-backup` also
passes the check for root `data`; adding the separator closes this leak.

This check operates at the string level and does not account for symbolic links. If
there is a symbolic link inside the root directory pointing outside, the resolved
path can appear to be under the root while the file that opens is actually outside.
The symbolic link behavior introduced in the Introduction to Linux course comes into
play here; a `realpath` call that asks the file system for the real path is used, and
the check is repeated on its result.

#### Summary

- `join` combines and normalizes parts while preserving relativity; `resolve` walks
  right to left to produce an absolute path.
- Path functions do not look at the file system; they speak about the shape of the
  name, not the file's existence.
- The module offers both platforms' rules separately; the rule set is chosen
  explicitly for paths outside the local file system.
- With `parse` and `format`, one path is derived from another without counting
  characters; `format` ignores `name` and `ext` if the object has a `base` field.
- In an ES module, the script's own location is obtained from the `import.meta.url`
  address with `fileURLToPath`; this anchor is independent of the working directory.
- A path coming from outside is verified by resolving it against the absolute root
  and testing the root-plus-separator prefix; for symbolic links, the real path is
  also asked for.

#### Next Step

Up to here, files were loaded into memory all at once. This is fine for a
twelve-line file; it is not for a measurement file accumulating over days. The next
lesson builds the stream abstraction that moves data piece by piece: readable,
writable, and transform streams, the correct way to connect them, and the
backpressure that kicks in when the producer is faster than the consumer.

### Streams

Source: https://academia.sh/en/courses/nodejs/streams

The four stream types, why chunk boundaries do not line up with record boundaries, object mode, building a pipeline, and measuring backpressure.

The previous two lessons loaded a file into memory all at once. For a twelve-line file
this costs nothing. When measurement nodes write for days, the file reaches hundreds
of megabytes, and the same code demands memory equal to the whole file — then copies
that memory once more to split it into lines.

This lesson's question is: how do we process data without ever seeing all of it? The
answer is the same as the pipeline model from the Shell Programming course — data
enters at one end, passes through a few transformations, exits the other end; no step
holds all of it. The runtime offers this model as first-class objects.

#### Four Types

A **stream** is an abstraction that carries data piece by piece. It has four types:

- A **readable stream** produces data. File reading, an incoming HTTP request, a
  child process's standard output are of this kind.
- A **writable stream** consumes data. File writing, an outgoing HTTP response,
  standard output are of this kind.
- A **duplex stream** carries two independent directions that are both of the above
  at once; a network socket is like this.
- A **transform stream** is a special case of duplex: it processes written data and
  emits it on its readable side. Compression, encryption, and parsing are of this
  kind.

All streams are event emitters: they emit events like `data`, `end`, `error`,
`drain`. The event emitter itself is the subject of a later lesson; here, higher-level
tools will be used instead of listening to events directly.

#### Chunk Boundaries Are Not Record Boundaries

A readable stream hands out data in chunks of a size it decides on its own. This size
is set with the `highWaterMark` option and has nothing to do with the line boundaries
in the file.

The examples below again read the course's working file; if it is not in your working
directory, create it first with this command:

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

```js
// chunks.mjs
import { createReadStream } from 'node:fs';

const stream = createReadStream('measurements.ndjson', { highWaterMark: 256 });

let index = 0;
for await (const chunk of stream) {
  index += 1;
  console.log(`chunk ${index}: ${chunk.length} bytes, type ${chunk.constructor.name}`);
}
console.log(`total ${index} chunks`);
```

```sh
node chunks.mjs
```

```
chunk 1: 256 bytes, type Buffer
chunk 2: 256 bytes, type Buffer
chunk 3: 256 bytes, type Buffer
chunk 4: 240 bytes, type Buffer
total 4 chunks
```

The file is 1008 bytes; it split into four chunks and the last one carried the
remainder. None of the chunks end at a line boundary. This is the fundamental
difficulty of working with streams: **establishing the record boundary is your job,
not the stream's.**

The `for await` loop in the example takes advantage of a readable stream being an
asynchronous iterable. While the loop body runs, the stream pauses; this is the
plainest form of backpressure.

#### Object Mode and a Transform

By default, streams carry bytes. When **object mode** is turned on, an arbitrary
value is carried instead of a chunk; from some point in the pipeline onward, records
flow instead of bytes.

The transform below splits byte chunks into lines and keeps a leftover partial line
in itself to join with the next chunk. This is the classic pattern of stream writing.

```js
// summary-stream.mjs
import { createReadStream } from 'node:fs';
import { Transform, Writable } from 'node:stream';
import { pipeline } from 'node:stream/promises';

// Splits byte chunks into lines; keeps a leftover partial line for the next chunk
class LineSplitter extends Transform {
  #remainder = '';

  constructor() {
    super({ readableObjectMode: true });
  }

  _transform(chunk, encoding, done) {
    const text = this.#remainder + chunk.toString('utf8');
    const lines = text.split('\n');
    this.#remainder = lines.pop();          // the last piece may be partial
    for (const line of lines) {
      if (line !== '') this.push(line);
    }
    done();
  }

  _flush(done) {
    if (this.#remainder !== '') this.push(this.#remainder);
    done();
  }
}

// Turns lines into records and collects them bucket by bucket
class SummaryCollector extends Writable {
  buckets = new Map();

  constructor() {
    super({ objectMode: true });
  }

  _write(line, encoding, done) {
    let record;
    try {
      record = JSON.parse(line);
    } catch (error) {
      return done(new Error(`broken line: ${line.slice(0, 30)}`, { cause: error }));
    }
    const key = `${record.node}/${record.metric}`;
    const bucket = this.buckets.get(key) ?? { count: 0, total: 0 };
    bucket.count += 1;
    bucket.total += record.value;
    this.buckets.set(key, bucket);
    done();
  }
}

const collector = new SummaryCollector();
await pipeline(
  createReadStream('measurements.ndjson', { highWaterMark: 256 }),
  new LineSplitter(),
  collector,
);

for (const [key, { count, total }] of collector.buckets) {
  console.log(`${key.padEnd(18)} ${String(count).padStart(2)} measurements, average ${(total / count).toFixed(2)}`);
}
```

```sh
node summary-stream.mjs
```

```
edge-01/temperature  3 measurements, average 21.87
edge-01/humidity    2 measurements, average 47.60
edge-02/temperature  3 measurements, average 20.23
edge-02/humidity    1 measurements, average 52.50
edge-03/temperature  2 measurements, average 24.35
edge-03/humidity    1 measurements, average 41.30
```

Three details deserve attention.

`readableObjectMode` puts only the readable side into object mode; the writable side
keeps accepting bytes. A transform's two sides can be configured separately, and this
is exactly what this example requires.

`_flush` is called once when the input ends and gives the chance to emit the
leftover partial line. Without this hook, the last record of a file whose last line
does not end with a newline would be lost.

When an error is given to the `done` callback, the error propagates through the
stream and tears down the pipeline. Using `throw` inside a writable stream does not
give the same result.

#### Building a Pipeline

There are two ways to connect streams to each other. The `pipe` method is old and has
a flaw: when a stream in the middle of the chain errors, the others do not close on
their own, and open file descriptors and buffered memory are left behind.

`pipeline` closes this gap. When any stream in the chain errors, it destroys all of
them and reports the error from a single place.

The behavior is tested below with a file containing a broken line. First create the file:

```sh
printf '{"node":"edge-01","metric":"humidity","value":48.0}\nbroken line\n{"node":"edge-02","metric":"humidity","value":50.0}\n' > broken.ndjson
```

```js
// pipeline-error.mjs
import { createReadStream } from 'node:fs';
import { Transform, Writable } from 'node:stream';
import { pipeline } from 'node:stream/promises';

const parser = new Transform({
  readableObjectMode: true,
  transform(chunk, encoding, done) {
    for (const line of chunk.toString('utf8').split('\n')) {
      if (line === '') continue;
      try {
        this.push(JSON.parse(line));
      } catch (error) {
        return done(new Error(`unparseable line: ${line}`, { cause: error }));
      }
    }
    done();
  },
});

let counter = 0;
const count = new Writable({
  objectMode: true,
  write(record, encoding, done) { counter += 1; done(); },
});

const source = createReadStream('broken.ndjson');

try {
  await pipeline(source, parser, count);
  console.log('completed, record count:', counter);
} catch (error) {
  console.log('pipeline error  :', error.message);
  console.log('root cause      :', error.cause.name);
  console.log('source destroyed:', source.destroyed);
  console.log('target destroyed:', count.destroyed);
}
```

```sh
node pipeline-error.mjs
```

```
pipeline error  : unparseable line: broken line
root cause      : SyntaxError
source destroyed: true
target destroyed: true
```

The last two lines matter: by the time the error was reported, the source and the
target had already been closed. No file descriptor leaked. The promise-returning form
in the `node:stream/promises` module combines this cleanup with `try`/`catch`.

#### Backpressure

**Backpressure** was defined while covering queues in the Data Structures course: if
the producer is faster than the consumer, the only way to stop the buffer in between
from growing is to slow the producer down. The same problem also came up describing
the pipeline in the Shell Programming course — there, the kernel did the slowing
down.

In streams, this job is entrusted to a write call's return value. A `write` call
returns `false` if the data in the buffer has crossed a threshold. This is not an
error, it means "slow down"; the producer waits for the `drain` event and continues.

Two words will be kept separate throughout the course. **Buffer**, lowercase, is the
area inside a stream holding data waiting to be processed; its size is read from the
`writableLength` field. **Buffer object** — capital-B `Buffer` — is the object
representing raw bytes and is the subject of the next lesson. English uses the word
*buffer* for both; the distinction is kept through context here.

```js
// backpressure.mjs
import { Writable } from 'node:stream';
import { once } from 'node:events';

// Target that processes each record in 5 ms, buffering at most 4 records
class SlowWriter extends Writable {
  constructor() {
    super({ objectMode: true, highWaterMark: 4 });
  }
  _write(record, encoding, done) {
    setTimeout(done, 5);
  }
}

function records(count) {
  return Array.from({ length: count }, (_, i) => ({ index: i }));
}

// 1. Producer that ignores the return value
const a = new SlowWriter();
for (const record of records(50)) a.write(record);
console.log('ignoring   -> records waiting in buffer:', a.writableLength);
a.end();

// 2. Producer that listens to the return value
const b = new SlowWriter();
let highest = 0;
for (const record of records(50)) {
  if (!b.write(record)) {
    highest = Math.max(highest, b.writableLength);
    await once(b, 'drain');
  }
}
console.log('listening  -> highest buffer value seen:', highest);
b.end();
```

```sh
node backpressure.mjs
```

```
ignoring   -> records waiting in buffer: 50
listening  -> highest buffer value seen: 4
```

The difference between the two numbers is backpressure in its entirety. The producer
ignoring the return value piled all fifty records into memory; even though the
threshold was 4, the buffer rose to 50. The producer listening to the return value
never let the buffer rise above 4.

The production counterpart of this difference is memory consumption. A server writing
fast-produced data to a slow client, if it ignores the return value, accumulates
unbounded memory per client; a handful of slow clients push the process against its
memory limit.

The good news: `pipeline` and `pipe` do this control on their own, and so does the
`for await` loop. Managing the return value by hand is only needed where you drive
streams directly.

There is also a built-in convenience for line-based reading: the `node:readline`
module's interface turns a readable stream into a line-by-line iterable and takes on
the splitting logic above. Writing your own transform is needed in formats where a
record is bounded by something other than a line.

#### Summary

- Streams carry data piece by piece; their four types are readable, writable,
  duplex, and transform.
- Chunk boundaries do not line up with record boundaries; establishing the record is
  the job of a transform that stores leftover partial data, and the `_flush` hook
  rescues the last record.
- Object mode lets records flow instead of bytes from some point in the pipeline
  onward; a transform's two sides can be configured separately.
- `pipeline` destroys every stream in the chain on an error and reports it from one
  place; `pipe` does not do this cleanup.
- A `write` call returning `false` is a backpressure signal; ignored, the buffer
  grows without bound. `pipeline`, `pipe`, and `for await` take on this control.

#### Next Step

In this lesson, chunks appeared as type `Buffer`, and the encoding name was given by
hand when converting to text. What happens if a chunk splits in the middle of a
character was not asked. The next lesson covers the representation of raw bytes: what
the buffer object is, how multi-byte characters get corrupted at a chunk boundary,
and how a binary record format is encoded and decoded.

### Buffers

Source: https://academia.sh/en/courses/nodejs/buffers

How a raw byte array differs from a string, zeroed versus unzeroed allocation, the view-versus-copy distinction, the problem of splitting at a character boundary, and binary record encoding.

In the previous lesson, chunks coming from a stream showed up with type `Buffer`, and
the encoding name was given by hand when converting to text. What happens if a chunk
splits in the middle of a multi-byte character was left unasked.

This lesson answers that question. The subject is a direct continuation of the
character encoding and byte order lessons from the How Computers Work course: the
concepts established there turn into working code here.

#### A Buffer as a Byte Array

A **buffer** is an object representing a fixed-length raw byte array. It derives
from the language's `Uint8Array` type, so its indexed access, length, and iteration
behavior are the same as a typed array. What is added on top is encoding conversions
and methods for reading and writing numeric fields.

The difference between a string and a buffer shows up in the unit of measure:

```js
// buffers.mjs
const text = 'sıcaklık';

const buf = Buffer.from(text, 'utf8');
console.log('character count :', text.length);
console.log('byte count      :', buf.length, '=', Buffer.byteLength(text, 'utf8'));
console.log('hex             :', buf.toString('hex'));
console.log('is Uint8Array   :', buf instanceof Uint8Array);
console.log('first three bytes:', [...buf.subarray(0, 3)]);
```

```sh
node buffers.mjs
```

```
character count : 8
byte count      : 10 = 10
hex             : 73c4b163616b6cc4b16b
is Uint8Array   : true
first three bytes: [ 115, 196, 177 ]
```

An eight-character string held ten bytes; the letter `ı` is encoded in UTF-8 with two
bytes (`c4 b1`) and appears twice in the word. This distinction matters when writing
a `Content-Length` header on an HTTP response: the number that has to go in the
header is the byte count, not the character count. `Buffer.byteLength` gives this
number without producing a buffer.

#### Allocation, View, and Copy

There are three ways to produce a buffer, and the difference between them is in
memory behavior.

```js
// allocation.mjs
const zeroed = Buffer.alloc(8);
console.log('alloc(8)          :', zeroed.toString('hex'));
console.log('allocUnsafe(8) len:', Buffer.allocUnsafe(8).length);

// subarray is not a copy, it is a view opened onto the same memory
const full = Buffer.from('edge-01', 'utf8');
const view = full.subarray(0, 5);
view[0] = 'E'.charCodeAt(0);
console.log('view changed ->', full.toString('utf8'));

// A copy is requested explicitly when one is needed
const copy = Buffer.from(full.subarray(0, 5));
copy[0] = 'x'.charCodeAt(0);
console.log('copy changed ->', full.toString('utf8'), '|', copy.toString('utf8'));

// Joining pieces into a single buffer
const joined = Buffer.concat([Buffer.from('edge'), Buffer.from('-'), Buffer.from('01')]);
console.log('concat        ->', joined.toString('utf8'), `(${joined.length} bytes)`);
```

```sh
node allocation.mjs
```

```
alloc(8)          : 0000000000000000
allocUnsafe(8) len: 8
view changed -> Edge-01
copy changed -> Edge-01 | xdge-
concat        -> edge-01 (7 bytes)
```

`Buffer.alloc` gives zeroed memory of the requested size. `Buffer.allocUnsafe` skips
the zeroing step and is faster; in exchange, the buffer contains whatever was
previously in that memory. Using this buffer without writing over its content start
to finish can leak data found elsewhere in the process. Rule: **use zeroed
allocation unless you are going to fill the buffer immediately and completely.**

The third line of the output shows view behavior: `subarray` does not allocate new
memory, it returns a window opened onto the same memory. Writing to the window
changes the original buffer. This is valuable in stream code because it allows
slicing without copying; unnoticed, it produces hard-to-explain bugs. When an
independent copy is needed, it is requested explicitly with `Buffer.from`.

`Buffer.concat` joins a list of pieces into a single buffer. This is the standard way
to gather chunks coming from a stream and convert them to a single piece of text at
the end; converting pieces to strings and joining them leads to the problem below.

#### Splitting at a Character Boundary

A chunk can end in the middle of a multi-byte character. If that chunk is converted
to text on its own, a replacement character is put in place of the incomplete
character, and information is lost irreversibly.

```js
// boundary.mjs
import { StringDecoder } from 'node:string_decoder';

const buf = Buffer.from('sıcaklık', 'utf8');   // 10 bytes
const chunkA = buf.subarray(0, 2);              // cuts through the middle of 'ı'
const chunkB = buf.subarray(2);

console.log('direct decode :', chunkA.toString('utf8') + chunkB.toString('utf8'));

const decoder = new StringDecoder('utf8');
console.log('with decoder  :', decoder.write(chunkA) + decoder.write(chunkB) + decoder.end());
```

```sh
node boundary.mjs
```

```
direct decode : s��caklık
with decoder  : sıcaklık
```

The two replacement characters on the first line came from the two bytes of the
split `ı` being decoded separately. The decoder in the `node:string_decoder` module
keeps an incomplete byte sequence in itself and joins it with the next write; the
`end` call reports any leftover incomplete sequence.

This exposes a hidden assumption in the previous lesson's `LineSplitter` class.
There, every chunk was converted directly with `toString('utf8')`; because the
measurement file only contains ASCII characters, no problem was visible. The same
code breaks on a dataset with non-ASCII characters in node names. The correct fix is
keeping a decoder in the transform and writing `decoder.write(chunk)` instead of
`chunk.toString('utf8')`.

#### Binary Record Format

A buffer offers methods that read and write numeric fields at a given width and byte
order. This is how you parse a binary protocol coming over the network, and how you
define your own format to save space.

A measurement record is fit into six bytes below: one byte for the node number, one
byte for the metric number, four bytes for a single-precision floating-point value.

```js
// binary.mjs
// Binary format fitting a measurement into 6 bytes:
// [0] node no (uint8)  [1] metric no (uint8)  [2..5] value (float32, big-endian)
function encode(nodeNo, metricNo, value) {
  const buf = Buffer.alloc(6);          // zeroed memory
  buf.writeUInt8(nodeNo, 0);
  buf.writeUInt8(metricNo, 1);
  buf.writeFloatBE(value, 2);
  return buf;
}

function decode(buf) {
  return {
    nodeNo: buf.readUInt8(0),
    metricNo: buf.readUInt8(1),
    value: Number(buf.readFloatBE(2).toFixed(2)),
  };
}

const record = encode(1, 0, 21.4);
console.log('binary form   :', record.toString('hex'));
console.log('decoded record:', decode(record));

const littleEndian = Buffer.alloc(4);
littleEndian.writeFloatLE(21.4, 0);
console.log('little-endian :', littleEndian.toString('hex'));
console.log('big-endian    :', record.subarray(2).toString('hex'));

const textSize = Buffer.byteLength(JSON.stringify({ node: 'edge-01', metric: 'temperature', value: 21.4 }));
console.log(`text form ${textSize} bytes, binary form ${record.length} bytes`);
```

```sh
node binary.mjs
```

```
binary form   : 010041ab3333
decoded record: { nodeNo: 1, metricNo: 0, value: 21.4 }
little-endian : 3333ab41
big-endian    : 41ab3333
text form 54 bytes, binary form 6 bytes
```

The `BE` and `LE` suffixes in the method names select the big-endian and
little-endian layouts introduced in the How Computers Work course. The third and
fourth lines of the output show the same number's two layouts side by side: the byte
sequence is reversed. The conventional choice in network protocols is big-endian
layout; when defining a format, which endianness is used has to be written down, or
the two ends read each other's data wrong.

The last line shows why binary formats exist: the same measurement takes 54 bytes as
text, 6 bytes as binary. In exchange, the binary form is unreadable by a human, does
not describe itself, and breaks old decoders when a field is added. This trade-off
was discussed in the How the Internet Works course while justifying why HTTP is
text-based; the choice is between size and inspectability.

Squeezing the value into four bytes has one more cost. Single-precision
floating-point representation carries about seven significant decimal digits; the
value `21.4` cannot be represented exactly in this form. This is why the example
rounds it with `toFixed(2)`. If measurement precision needs to exceed this limit,
double-precision read and write methods are used.

Buffer methods do bounds checking: a call that tries to write outside the array's
range throws `RangeError`. This blocks, at the runtime level, the class of
unbounded-memory-access errors covered in the How Computers Work course.

#### Summary

- A buffer is a raw byte array built on `Uint8Array`; character count and byte count
  diverge in multi-byte encodings, and it is the byte count that goes into protocol
  headers.
- `Buffer.alloc` gives zeroed memory; allocation that skips zeroing leaks old memory
  content when not filled completely.
- `subarray` returns a view, not a copy; an independent copy is requested explicitly
  with `Buffer.from`.
- When a multi-byte character splits at a chunk boundary, direct decoding corrupts
  the data; `node:string_decoder` prevents this by holding onto partial bytes.
- Numeric fields are read and written by choosing a width and an endianness; a
  binary format saves space at the cost of inspectability and extensibility.

#### Next Step

Up to here, streams were said to emit events, but the event mechanism itself was
never opened up. The measurement collector's next step needs a structure that
notifies interested parties when data arrives: a mechanism that produces a warning
when a threshold is crossed, reports a broken line, announces that the source has
closed. The next lesson covers this mechanism's base class — the event emitter.

### Event Emitter

Source: https://academia.sh/en/courses/nodejs/event-emitter

The built-in implementation of the observer pattern, the synchrony of emission, the error event's special behavior, the max-listeners warning, and awaiting events in promise form.

The names `data`, `end`, and `drain` from the streams lesson all come from a single
mechanism. The same mechanism is used in the HTTP server, in child processes, and in
the process object's signal notifications: most of the runtime's asynchronous
interfaces are built on top of this one class.

This lesson covers that class and turns the measurement collector's data source into
an event-based one: a source that announces when a record arrives, warns when a
threshold is crossed, and reports a broken line.

#### The Built-in Implementation of the Observer Pattern

The `EventEmitter` class in the `node:events` module keeps a listener list per name.
Listeners are added to the list with `on`, and the list is called in order with
`emit`. This is a direct application of the higher-order function concept from the
Programming Fundamentals course: behavior is stored as a value and called when its
time comes.

The pattern's gain is that it loosens the coupling between producer and consumer. The
source does not know who is listening; the listener does not know the source's inner
workings. The only contract between them is the event name and the shape of the
payload.

```js
// emitter.mjs
import { EventEmitter } from 'node:events';

class MeasurementSource extends EventEmitter {
  feed(line) {
    try {
      const record = JSON.parse(line);
      this.emit('measurement', record);
      if (record.metric === 'temperature' && record.value > 24) {
        this.emit('threshold-exceeded', record);
      }
    } catch (error) {
      this.emit('bad-line', { line, error });
    }
  }
}

const source = new MeasurementSource();

source.on('measurement', (r) => console.log('measurement :', r.node, r.metric, r.value));
source.once('threshold-exceeded', (r) => console.log('threshold (once):', r.node, r.value));
source.on('bad-line', ({ line }) => console.log('bad line    :', line));

source.feed('{"node":"edge-01","metric":"temperature","value":21.4}');
source.feed('{"node":"edge-03","metric":"temperature","value":24.1}');
source.feed('{"node":"edge-03","metric":"temperature","value":24.6}');
source.feed('broken');

console.log('measurement listener count:', source.listenerCount('measurement'));
console.log('threshold listener count :', source.listenerCount('threshold-exceeded'));
```

```sh
node emitter.mjs
```

```
measurement : edge-01 temperature 21.4
measurement : edge-03 temperature 24.1
threshold (once): edge-03 24.1
measurement : edge-03 temperature 24.6
bad line    : broken
measurement listener count: 1
threshold listener count : 0
```

A listener added with `once` is removed from the list after the first emission; the
zero on the output's last line confirms this. This is also why no warning was printed
on the second threshold crossing. `on` is used for a persistent listener, `once` for
a one-time one; when a listener needs to be removed by hand, the `off` call has to be
given the listener's **exact same reference** — a listener added as an anonymous
function expression can never be removed.

#### Emission Is Synchronous

Because the name is "event," it might be assumed to enter a queue. It does not:
`emit` calls listeners **right then**, in the order they were added, and returns once
all of them are done.

```js
// sync.mjs
import { EventEmitter } from 'node:events';
const e = new EventEmitter();
e.on('event', () => console.log('2 listener'));
console.log('1 before emit');
e.emit('event');
console.log('3 after emit');
```

```sh
node sync.mjs
```

```
1 before emit
2 listener
3 after emit
```

This has two consequences. First, long synchronous work done inside a listener stops
the event loop like everything else; emitting an event does not defer the work.
Second, an error thrown inside a listener propagates to wherever the `emit` call was
made — into the middle of source code that never expected it.

If a listener needs to defer its work to free the caller, the deferral is written
explicitly: the work is handed to a timer or a microtask. The source itself cannot
make that decision.

#### The Privilege of the Error Event

The name `error` is handled specially inside the class. If an emission is made under
this name while there is no listener, the payload is thrown, and if not caught, the
process terminates.

```js
// errors.mjs
import { EventEmitter } from 'node:events';

const withListener = new EventEmitter();
withListener.on('error', (e) => console.log('caught:', e.message));
withListener.emit('error', new Error('source closed'));

const withoutListener = new EventEmitter();
withoutListener.emit('error', new Error('this will be thrown'));
console.log('this line is unreachable');
```

Only a single line reaches standard output:

```sh
node errors.mjs 2>/dev/null
```

```
caught: source closed
```

The process terminates on the second emission; the line `this line is unreachable`
is never written, and a stack trace lands on standard error. The trace's content
carries file paths and so varies by machine.

This behavior is a deliberate design. If an asynchronous source's error is swallowed
silently, the program keeps working with wrong data, and the failure shows up much
later, somewhere unrelated. The rule is: **every object that can emit an error gets
an `error` listener attached.** Since stream objects are event emitters too, this
rule applies to them as well; this was the reason for using `pipeline` in the previous
lesson: it reports the error from one place and destroys the remaining streams itself.

#### The Listener Limit

When the number of listeners added under one name crosses the default limit, the
runtime writes a warning:

```js
// limit.mjs
import { EventEmitter } from 'node:events';
const e = new EventEmitter();
for (let i = 0; i < 11; i += 1) e.on('measurement', () => {});
console.log('listener count:', e.listenerCount('measurement'));
```

```sh
node limit.mjs
```

{/* mask: (?<=\(node:)[0-9]+(?=\)) */}
```
listener count: 11
(node:80078) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 measurement listeners added to [EventEmitter]. MaxListeners is 10. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
```

The number at the start of the warning line is the process id and changes with every
run. The warning is not an error; the program keeps running.

The reason: if the listener list keeps growing, this usually means a new listener is
added on every request and never removed. As the list grows, the values held by the
closed-over functions cannot be released either, and a memory leak forms — exactly
the situation described while covering reachability-based garbage collection in the
Asynchronous JavaScript and the Runtime course.

If genuinely many listeners are needed, the limit is raised with
`setMaxListeners`. Before raising the limit, why the list is growing has to be
explained.

#### Awaiting Events as Promises

To combine the event-based interface with `async`/`await`, the module offers two
helpers. The `once` function returns a promise that waits for a single event; the
`on` function turns events into an asynchronous iterable.

```js
// promise.mjs
import { EventEmitter, once, on } from 'node:events';

const source = new EventEmitter();

// Waiting for a single event: as a promise
setTimeout(() => source.emit('ready', { node: 'edge-01' }), 20);
const [payload] = await once(source, 'ready');
console.log('waited with once:', payload);

// Iterating a continuously flowing stream of events
setTimeout(() => {
  source.emit('measurement', 21.4);
  source.emit('measurement', 21.9);
  source.emit('done');
}, 20);

const controller = new AbortController();
source.once('done', () => controller.abort());

try {
  for await (const [value] of on(source, 'measurement', { signal: controller.signal })) {
    console.log('flowing event    :', value);
  }
} catch (error) {
  if (error.name !== 'AbortError') throw error;
  console.log('iteration ended  :', error.name);
}
```

```sh
node promise.mjs
```

```
waited with once: { node: 'edge-01' }
flowing event    : 21.4
flowing event    : 21.9
iteration ended  : AbortError
```

Both helpers give listener arguments as an array; for single-argument events, the
first element is taken by destructuring.

The controller in the second part is the way to end an endless iteration. A loop set
up with `on` does not end on its own — even if the source stops emitting events, the
loop keeps waiting. Ending it is done with an abort signal, and the abort surfaces
from the iterator as an `AbortError`. This is the same mechanism introduced for
request cancellation in the Asynchronous JavaScript and the Runtime course.

One warning: iteration set up with `on` accumulates events arriving while the loop
body is running. If the body is slow, this accumulation is unbounded — unlike
streams, an event emitter carries no backpressure. This is the reason a stream is
used instead of an event emitter for high-volume sources.

#### Summary

- An event emitter is the observer-pattern implementation that keeps a listener list
  per name; most of the runtime's asynchronous interfaces are built on top of this
  class.
- `emit` is synchronous: it calls listeners right then, in the order they were
  added, and returns once all of them are done; emission does not defer work.
- The `error` name is privileged; an emission with no listener throws the payload and
  terminates the process. Every object that can emit an error gets a listener
  attached.
- The warning written when the listener count exceeds the limit usually reports an
  accumulation of un-removed listeners and the memory leak that follows from it.
- The `once` and `on` helpers turn events into a promise and an asynchronous
  iteration form; iteration is ended with an abort signal and carries no
  backpressure.

#### Next Step

All of the measurement collector's pieces are ready: reading a file with a stream,
building the record, summarizing, and event-based notification. The only thing
missing is serving the summary outward. The next lesson builds the HTTP server: that
the request is a readable stream and the response a writable one, and how the
request and response structure examined line by line in the How the Internet Works
course is met in code.

### HTTP Server

Source: https://academia.sh/en/courses/nodejs/http-server

The stream counterpart of request and response objects, dispatch by path and method, a size limit on reading the body, and the choice between declaring a length and chunked transfer.

The How the Internet Works course parsed an HTTP request line by line: the request
line, headers, an empty line, the body. There, the party reading that text was an
example program, and the protocol's parsing was never shown.

This lesson hands that job to the built-in module and turns the measurement collector
into a service. The key observation is this: the request object is a readable stream,
the response object a writable one. Everything established in the streams lesson
applies directly here.

#### Moving Summary Accumulation Into a Separate Module

Before writing the service code, record validation and summary accumulation have to
move into a separate module. Data coming from the network and data coming from a file
have to pass through the same path, and that path has to be testable independently of
the server.

```js
// summarizer.mjs
export function recordFromLine(line) {
  const record = JSON.parse(line);
  if (typeof record.node !== 'string' || typeof record.metric !== 'string') {
    throw new TypeError('node and metric must be strings');
  }
  if (!Number.isFinite(record.value)) {
    throw new TypeError('value must be a finite number');
  }
  return record;
}

export class Summarizer {
  #buckets = new Map();

  add({ node, metric, value }) {
    const key = `${node}/${metric}`;
    const bucket = this.#buckets.get(key) ?? { count: 0, total: 0, max: -Infinity };
    bucket.count += 1;
    bucket.total += value;
    bucket.max = Math.max(bucket.max, value);
    this.#buckets.set(key, bucket);
  }

  summary() {
    return [...this.#buckets].map(([key, { count, total, max }]) => ({
      key,
      count,
      average: Number((total / count).toFixed(2)),
      max,
    }));
  }
}
```

The validation function does not just parse, it also checks field types. Data coming
from the network is not trusted: when the `value` field arrives as a string, the
addition operation silently turns into concatenation and the summary becomes
meaningless.

#### The Server and the Handler

`createServer` in the `node:http` module takes a handler called for every request.
The handler has two arguments, and both are streams.

The server reads the course's working file, `measurements.ndjson`, at startup. If it is
not in your working directory, the command below creates it; the content is the same
twelve lines given in the runtime model topic:

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

```js
// service.mjs
import { createServer } from 'node:http';
import { createReadStream } from 'node:fs';
import { createInterface } from 'node:readline';
import { pipeline } from 'node:stream/promises';
import { Summarizer, recordFromLine } from './summarizer.mjs';

const summarizer = new Summarizer();

// On startup, stream-read the file and fill the buckets
async function loadStartupData(file) {
  const lines = createInterface({
    input: createReadStream(file),
    crlfDelay: Infinity,
  });
  let counter = 0;
  for await (const line of lines) {
    if (line.trim() === '') continue;
    summarizer.add(recordFromLine(line));
    counter += 1;
  }
  return counter;
}

async function readBody(request, maxBytes = 64 * 1024) {
  const chunks = [];
  let size = 0;
  for await (const chunk of request) {          // request is a readable stream
    size += chunk.length;
    if (size > maxBytes) {
      request.destroy();
      throw Object.assign(new Error('body too large'), { status: 413 });
    }
    chunks.push(chunk);
  }
  return Buffer.concat(chunks).toString('utf8');
}

function writeJson(response, status, value) {
  const body = JSON.stringify(value) + '\n';
  response.writeHead(status, {
    'content-type': 'application/json; charset=utf-8',
    'content-length': Buffer.byteLength(body),
  });
  response.end(body);
}

const server = createServer(async (request, response) => {
  const address = new URL(request.url, `http://${request.headers.host}`);
  try {
    if (request.method === 'GET' && address.pathname === '/summary') {
      writeJson(response, 200, summarizer.summary());
      return;
    }
    if (request.method === 'GET' && address.pathname === '/raw') {
      response.writeHead(200, { 'content-type': 'application/x-ndjson; charset=utf-8' });
      await pipeline(createReadStream('measurements.ndjson'), response);   // body written as a stream
      return;
    }
    if (request.method === 'POST' && address.pathname === '/measurement') {
      const body = await readBody(request);
      let counter = 0;
      for (const line of body.split('\n')) {
        if (line.trim() === '') continue;
        summarizer.add(recordFromLine(line));
        counter += 1;
      }
      writeJson(response, 201, { received: counter });
      return;
    }
    writeJson(response, 404, { error: 'not found' });
  } catch (error) {
    writeJson(response, error.status ?? 400, { error: error.message });
  }
});

const loaded = await loadStartupData('measurements.ndjson');
server.listen(8791, '127.0.0.1', () => {
  const { address, port } = server.address();
  console.log(`${loaded} records loaded; listening: http://${address}:${port}`);
});
```

```sh
node service.mjs
```

```
12 records loaded; listening: http://127.0.0.1:8791
```

The number 8791 in the example is a choice; if there is a port already in use on
your machine, write a different number. The next lesson will read this value from
configuration.

The `127.0.0.1` given as the second argument to `listen` makes the server listen
only on the loopback interface; this address was introduced in the How the Internet
Works course. When no argument is given, the server becomes reachable from every
interface — an unwanted openness during development.

#### Parsing the Request

The request object offers the protocol's parsed form as fields. `request.method` is
the request line's first field, `request.url` the second, and `request.headers` is
an object made of lowercased header names.

`request.url` carries only the path and query part; the hostname is in the `host`
header. This split was explained in the How the Internet Works course as the basis
of name-based virtual hosting. Combining the two parts to build a `URL` object is
safer than splitting the path and query by hand: escape decoding and normalization
rules live inside the object.

The body is not found among the fields — it is a stream. The reason is that the body
can be of arbitrary size. The `readBody` function above collects chunks with
`for await` while also applying a limit. The limit is mandatory: an endpoint that
reads an unbounded body can exhaust the process's memory with a single request.

#### Writing the Response

The response object is a writable stream. `writeHead` sets the status code and
headers; `end` writes the body and closes the response.

The `curl` outputs below are example runs taken while the server was running in a
separate terminal.

{/* run: skip */}
```sh
curl -s -i http://127.0.0.1:8791/summary
```

```
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
content-length: 390
Date: Fri, 14 Aug 2026 23:21:55 GMT
Connection: keep-alive
Keep-Alive: timeout=5

[{"key":"edge-01/temperature","count":3,"average":21.87,"max":22.3},{"key":"edge-01/humidity","count":2,"average":47.6,"max":48},{"key":"edge-02/temperature","count":3,"average":20.23,"max":20.7},{"key":"edge-02/humidity","count":1,"average":52.5,"max":52.5},{"key":"edge-03/temperature","count":2,"average":24.35,"max":24.6},{"key":"edge-03/humidity","count":1,"average":41.3,"max":41.3}]
```

The value of the `Date` header changes with the moment of the request; the
`Connection` and `Keep-Alive` headers also depend on client negotiation. What is
fixed is the two headers the handler writes, and the body.

The `content-length` header was computed by hand here. It has to be the byte count,
not the character count; this is exactly why the distinction in the buffers lesson
mattered.

For responses whose length you do not know in advance, this header is not written.
When the body is written through a stream, the runtime switches to chunked transfer:

{/* run: skip */}
```sh
curl -s -i http://127.0.0.1:8791/raw | head -8
```

```
HTTP/1.1 200 OK
content-type: application/x-ndjson; charset=utf-8
Date: Fri, 14 Aug 2026 23:22:02 GMT
Connection: keep-alive
Keep-Alive: timeout=5
Transfer-Encoding: chunked

{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
```

The `Transfer-Encoding: chunked` header announces that each piece of the body is
preceded by its own length; the receiver knows the end this way. This is the
mechanism described in the How the Internet Works course as "if the length is not
known in advance, the body is sent in pieces."

The choice is made this way: if the response is small and ready in memory,
declaring a length gives the client progress information; if the response is large
or flows while being produced, chunked transfer keeps memory from being occupied for
the whole response. Using `pipeline` also guarantees that backpressure is applied: if
the client is slow, the file read slows down too.

#### Sending Data and Errors

Sending a measurement is done with a request carrying a body:

{/* run: skip */}
```sh
curl -s -X POST --data-binary '{"node":"edge-01","metric":"temperature","value":25.9}' -w 'status: %{http_code}\n' http://127.0.0.1:8791/measurement
```

```
{"received":1}
status: 201
```

The 201 code means "a new resource was created" per the classification from the How
the Internet Works course; choosing it separately from 200 announces the effect of
the request.

A broken body falls into a different class:

{/* run: skip */}
```sh
curl -s -X POST --data-binary 'broken' -w 'status: %{http_code}\n' http://127.0.0.1:8791/measurement
```

```
{"error":"Unexpected token 'b', \"broken\" is not valid JSON"}
status: 400
```

{/* run: skip */}
```sh
curl -s -w 'status: %{http_code}\n' http://127.0.0.1:8791/missing
```

```
{"error":"not found"}
status: 404
```

The 400 class reports the sender's error, the 500 class the server's error. Getting
this distinction right is not just courtesy, it is an operational necessity: the 500
rate is a signal of failure, the 400 rate a signal of client behavior. If the two get
mixed, the signal loses its meaning.

The error catcher in the handler above has a shortcoming: it reports every caught
error as 400, but a programmer error occurring during summarization should be 500.
This distinction will be fixed in the Error Handling Strategy lesson.

One warning: because the handler is asynchronous, an error occurring inside it that
is not caught is not seen by `createServer`. This is why the `try`/`catch` block has
to wrap the entire body — without it, no response is ever written and the client
times out.

#### Summary

- The request object is a readable stream, the response object a writable one; every
  rule from the streams lesson applies here directly.
- The request line and headers are offered as parsed fields; `request.url` carries
  only the path and query, the hostname is in the `host` header.
- Because the body is a stream, its size has to be limited; unbounded reading can
  exhaust memory with a single request.
- A length-declaring response writes the byte count; when the length is unknown, the
  runtime switches to chunked transfer, and writing through a stream preserves
  backpressure.
- The status code class reports the source of the error; mixing client and server
  errors makes operational signals meaningless.

#### Next Step

The service runs in a single process and does all its work on a single thread. When
an expensive computation is needed on the measurement data — resummarizing a long
period, say — this single thread makes every request wait. The next lesson covers
the first way to move work outside the process: starting a child process, talking to
it through streams, and safely calling external commands.

### Child Processes

Source: https://academia.sh/en/courses/nodejs/child-processes

Two ways to run an external command, the risk brought by the shell layer, the distinction between an exit code and a terminating signal, and passing messages between runtime processes.

The previous lesson set up the service in a single process. What a single process can
do is limited, and some jobs it should not do at all: rewriting the work of a tool
that already exists on the system means trying to reproduce that tool's years of
tested behavior.

This lesson covers moving work outside the process. There are two separate needs:
running an external command, and starting a second process from the same runtime and
talking to it.

#### Running an External Command

`spawn` in the `node:child_process` module starts a program as a separate process and
gives its three standard streams as an object. The stream model from the Shell
Programming course finds its exact counterpart here.

The examples in this lesson use the course's working file, `measurements.ndjson`.
If it is not in your working directory, the command below creates it; the content is
the same twelve lines given in the runtime model topic:

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

```js
// child-process.mjs
import { spawn } from 'node:child_process';
import { once } from 'node:events';

function run(command, args) {
  const child = spawn(command, args);      // no shell: args pass as an array
  const output = [];
  const errorOutput = [];
  child.stdout.on('data', (c) => output.push(c));
  child.stderr.on('data', (c) => errorOutput.push(c));
  return once(child, 'close').then(([code, signal]) => ({
    code,
    signal,
    output: Buffer.concat(output).toString('utf8').trim(),
    error: Buffer.concat(errorOutput).toString('utf8').trim(),
  }));
}

console.log('wc -l    :', await run('wc', ['-l', 'measurements.ndjson']));
console.log('missing  :', await run('cat', ['missing.txt']));
```

```sh
node child-process.mjs
```

```
wc -l    : { code: 0, signal: null, output: '12 measurements.ndjson', error: '' }
missing  : {
  code: 1,
  signal: null,
  output: '',
  error: 'cat: missing.txt: No such file or directory'
}
```

The difference between the two calls is in the exit code. Zero reports success, one
failure; the rule from the Shell Programming course has not changed. The diagnostic
message came from the second stream and did not get mixed with the data — this is the
runtime-side counterpart of that same distinction.

Chunks are collected as buffers and joined at the end. Converting each chunk to a
string separately and concatenating would have produced the boundary corruption shown
in the buffers lesson.

#### The Shell Layer and Argument Passing

The same module has another function called `exec`. The difference is that it hands
the command to a shell process as text. The shell interprets that text with its own
rules: variable expansion, path expansion, pipelines, semicolon-separated commands.

This interpretation becomes a problem when a piece coming from outside is inserted
into the command text:

```js
// shell-injection.mjs
import { execFile, exec } from 'node:child_process';
import { promisify } from 'node:util';

const execAsync = promisify(exec);
const execFileAsync = promisify(execFile);

// An untrusted file name coming from outside
const fileName = 'measurements.ndjson; echo INJECTED';

// 1. Call that inserts a shell: the text is interpreted by the shell
const { stdout: shellOutput } = await execAsync(`wc -l ${fileName}`);
console.log('exec     :', JSON.stringify(shellOutput.trim()));

// 2. Call without a shell: the name passes as a single argument
try {
  await execFileAsync('wc', ['-l', fileName]);
} catch (error) {
  console.log('execFile :', error.stderr.trim());
}
```

```sh
node shell-injection.mjs
```

```
exec     : "12 measurements.ndjson\nINJECTED"
execFile : wc: 'measurements.ndjson; echo INJECTED': No such file or directory
```

In the first call, the command after the semicolon also ran. Had the text substituted
for the file name been an account-deleting or data-exfiltrating command, the result
would have been run in the exact same way. In the second call, the entire text passed
as a single argument; `wc` looked for a file by that name and did not find one.

Rule: **do not build a command's text by string concatenation.** When the program
name and argument array are given separately, no shell sits in between and no text is
ever formed for interpretation. If a shell feature is genuinely needed — a pipeline
or redirection, say — a fixed command text is written and the variable parts are
passed as environment variables.

`promisify` is the built-in helper that converts a callback-style signature into a
promise-returning one. Most of the runtime's callback-based interfaces support this
conversion.

`exec` and `execFile` also accumulate output in memory and terminate the process
once it crosses a limit. `spawn` is used for commands producing large output, and the
output is processed through the stream.

#### Exit Code and Terminating Signal

A child process can end in two different ways: exiting by its own request, or being
terminated by a signal. The `close` event reports these two cases in separate
fields.

```js
// signal.mjs
import { spawn } from 'node:child_process';
import { once } from 'node:events';

// A long-running job; we will not wait for it to finish on its own
const child = spawn('sleep', ['30']);
console.log('child started, separate process:', child.pid !== process.pid);

setTimeout(() => child.kill('SIGTERM'), 100);

const [code, signal] = await once(child, 'close');
console.log('exit code:', code, '| terminating signal:', signal);
```

```sh
node signal.mjs
```

```
child started, separate process: true
exit code: null | terminating signal: SIGTERM
```

A process terminated by a signal has no exit code; the field comes as `null`. A
check that looks only at the code cannot pass a `code === 0` test, but it could also
misread a `code !== 0` test. The correct check reads both fields together: success is
a code of zero **and** no signal.

The signal name itself was introduced in the Shell Programming course; `SIGTERM` is a
polite termination request, and the process can catch it and clean up. A forceful
terminating signal cannot be caught and gives no chance to clean up; for this reason
it is used only when the polite request goes unanswered. A process's own shutdown
behavior is the subject of the Process Management and Robustness lesson.

#### A Second Process From the Same Runtime

The measurement collector's expensive work is its own code, not an external tool.
`fork` is used for this: it starts a new instance of the same runtime and sets up a
message channel between the two processes.

```js
// summary-child.mjs
import { readFile } from 'node:fs/promises';

process.on('message', async ({ file }) => {
  const text = await readFile(file, 'utf8');
  const buckets = new Map();
  for (const line of text.split('\n')) {
    if (line.trim() === '') continue;
    const { node, metric, value } = JSON.parse(line);
    const key = `${node}/${metric}`;
    const bucket = buckets.get(key) ?? { count: 0, total: 0 };
    bucket.count += 1;
    bucket.total += value;
    buckets.set(key, bucket);
  }
  process.send({ pidIsNumber: typeof process.pid === 'number', bucketCount: buckets.size });
  process.disconnect();
});
```

```js
// summary-main.mjs
import { fork } from 'node:child_process';
import { once } from 'node:events';

const child = fork('./summary-child.mjs');
console.log('separate process:', child.pid !== process.pid);

child.send({ file: 'measurements.ndjson' });
const [message] = await once(child, 'message');
console.log('from child:', message);

const [code] = await once(child, 'exit');
console.log('child exit code:', code);
```

```sh
node summary-main.mjs
```

```
separate process: true
from child: { pidIsNumber: true, bucketCount: 6 }
child exit code: 0
```

No memory is shared between the two processes. The value given to `send` is
serialized, passed through the channel, and reconstructed on the other side; only
serializable values can be carried — a function, a class instance, and a file
descriptor do not pass directly. This copying is a measurable cost for large data:
rather than sending millions of records to the child, sending the file name and
leaving the reading to it is preferred. The example above does exactly that.

The `disconnect` call closes the channel. As long as the channel stays open, the two
processes keep each other alive; if a child has finished its work but does not exit,
this is usually why.

Isolation has one more benefit: a crash in the child process does not bring down the
main process. The main process sees the `exit` event and makes the decision itself —
it can restart, answer the request with an error, or do the work itself.

#### Summary

- `spawn` starts a program as a separate process and gives its three standard
  streams as an object; large output is processed through the stream.
- Running through a shell interprets the command text; when a piece coming from
  outside is inserted into that text, an extra command can run. The program name and
  argument array are given separately.
- The exit code and terminating signal are separate fields; a process terminated by
  a signal has no code, and a success check reads both fields together.
- `fork` starts a second instance of the same runtime and sets up a message channel;
  messages are copied through serialization, no memory is shared.
- Process isolation turns a child's crash into a decision the main process makes.

#### Next Step

A child process is a way to move work out of the main thread, but a heavy one: each
process carries its own memory and its own runtime instance, and every message
between them is copied. The next lesson covers two alternatives — a set of processes
sharing the same port, and threads that can share memory within the same process —
and determines which fits which kind of work.

### Cluster and Worker Threads

Source: https://academia.sh/en/courses/nodejs/cluster-and-worker-threads

A process cluster sharing the same port, a worker thread's memory model, which kind of work each approach fits, and the measured difference in blocking.

The previous lesson covered starting a child process and stated its cost: each
process carries its own memory and its own runtime instance, and every message
between them is copied.

This lesson examines two special forms of it. The first is a cluster of processes
sharing the same port, and its purpose is genuinely handling multiple requests at
once on a multi-core machine. The second is threads running inside the same process
that can share memory; their purpose is moving an expensive computation off the main
thread.

#### Why One Process Is Not Enough

The limit measured in the event loop lesson becomes binding here. A single process
handles requests on a single thread running on a single core. For an
I/O-heavy service this is enough: the process spends most of its time waiting in the
poll phase, and one core is enough for dozens of concurrent connections.

The picture changes once part of the work is CPU-bound. How many independent
execution units the machine has can be asked:

```js
// cores.mjs
import os from 'node:os';
console.log('available parallelism:', os.availableParallelism());
```

```sh
node cores.mjs
```

```
available parallelism: 12
```

This number varies by machine and by the limits granted to the process; it was
taken on a sample run. A single process uses only one of these units.

#### A Process Cluster

The `node:cluster` module sets up a primary process and several worker processes.
Workers are separate processes forked by the primary; incoming connections are
distributed among them through a **single** port.

```js
// cluster.mjs
import cluster from 'node:cluster';
import { createServer } from 'node:http';

const WORKER_COUNT = 2;

if (cluster.isPrimary) {
  console.log(`primary process ${process.pid}, starting ${WORKER_COUNT} workers`);
  for (let i = 0; i < WORKER_COUNT; i += 1) cluster.fork();

  cluster.on('exit', (worker, code, signal) => {
    console.log(`worker ${worker.process.pid} exited (code ${code}, signal ${signal}); starting a replacement`);
    cluster.fork();
  });
} else {
  createServer((request, response) => {
    response.writeHead(200, { 'content-type': 'text/plain' });
    response.end(`request handled by worker: ${process.pid}\n`);
  }).listen(8791, '127.0.0.1', () => {
    console.log(`worker ${process.pid} listening`);
  });
}
```

Running the server in one terminal and sending four requests from another:

```sh
node cluster.mjs
```

{/* run: skip */}
```sh
for i in 1 2 3 4; do curl -s http://127.0.0.1:8791/; done
```

```
primary process 81263, starting 2 workers
worker 81265 listening
worker 81266 listening
request handled by worker: 81265
request handled by worker: 81266
request handled by worker: 81265
request handled by worker: 81266
```

Process ids change on every run; the order in which the two workers start
listening also changes. What is fixed is that requests get handled by two different
processes — an even, alternating distribution like the example's is not guaranteed.

Two processes being able to listen on the same port rests on the listening socket concept
introduced in the How the Internet Works course. Distribution happens in one of two ways.
In the round-robin approach, the default on every platform except Windows, the primary
accepts the connection and hands it to the workers in turn; in the second approach the
primary passes the listening socket's descriptor to the workers and each worker accepts
connections directly. Either way there is one port number; there are multiple processes
using it.

The callback attached to the `exit` event puts a replacement in place of a worker
that goes down. This is the cluster's real gain in terms of robustness: a fatal error
in one worker does not stop the service entirely. The restart rate has to be limited
so it does not enter an infinite loop; if a worker crashes immediately every time it
starts, opening a new one just hides the failure.

The cluster's fundamental constraint is that **memory is not shared** between
workers. The measurement collector's bucket map accumulates separately in each
worker, and the `/summary` response varies depending on which worker handled the
request. Clustering a stateful service requires moving its state out of the process
— to a shared store.

#### Worker Threads

The `node:worker_threads` module starts a separate thread within the same process.
Each thread has its own event loop and its own heap; memory is not shared by default
but can be explicitly shared through a shareable buffer.

An expensive computation is handed to a thread below:

{/* run: fail */}
```js
// heavy-work.mjs
import { parentPort, workerData } from 'node:worker_threads';

// A deliberately expensive computation on top of measurement values
function heavySum(iterations) {
  let total = 0;
  for (let i = 0; i < iterations; i += 1) total += Math.sqrt(i) % 1;
  return total;
}

parentPort.postMessage(Number(heavySum(workerData.iterations).toFixed(3)));
```

This file is not run on its own with `node heavy-work.mjs`: `parentPort` is only filled in code that was started as a thread. The main script below starts the file.

```js
// worker-main.mjs
import { Worker } from 'node:worker_threads';
import { once } from 'node:events';

const ITERATIONS = 3e8;

setTimeout(() => console.log('2 main thread free: timer ran'), 0);
console.log('1 worker started');

const worker = new Worker('./heavy-work.mjs', { workerData: { iterations: ITERATIONS } });
const [result] = await once(worker, 'message');
console.log('3 result from worker:', result);
await once(worker, 'exit');
```

```sh
node worker-main.mjs
```

```
1 worker started
2 main thread free: timer ran
3 result from worker: 149989897.293
```

When the same computation is done on the main thread, the order changes:

```js
// no-worker.mjs
const ITERATIONS = 3e8;

function heavySum(iterations) {
  let total = 0;
  for (let i = 0; i < iterations; i += 1) total += Math.sqrt(i) % 1;
  return total;
}

setTimeout(() => console.log('3 timer only ran now'), 0);
console.log('1 computation starting');
console.log('2 result:', Number(heavySum(ITERATIONS).toFixed(3)));
```

```sh
node no-worker.mjs
```

```
1 computation starting
2 result: 149989897.293
3 timer only ran now
```

The difference between the two outputs is the measurement itself. In the first, the
timer ran while the computation was in progress; the main thread was free and could
have handled an incoming request at that moment. In the second, the timer waited for
the computation to finish — on a server, this means every concurrent request waits.

The result value comes out the same in both runs; the computation is deterministic
and moving it to a thread does not change the result.

#### Which One, When

The two approaches do not substitute for each other.

| Criterion | Process cluster | Worker thread |
|---|---|---|
| Isolation | complete; if one crashes, the other continues | weak; a fatal error can bring down the process |
| Memory | separate; each process carries its own instance | shared process, separate heap; a shareable buffer is possible |
| Startup cost | high | low |
| Message cost | serialization + inter-process copy | structured copy; large data can be transferred |
| Suited work | multiplying request-handling capacity | moving a single expensive computation off the main path |

The rule for the measurement collector is this: a cluster is used when request
handling capacity needs to grow, a worker thread when an expensive computation
inside a single request needs to come off the main thread. The two can also be used
together — each cluster worker can keep its own thread pool.

Starting a thread has a cost too: a new execution environment is set up and modules
are re-evaluated. Opening a thread per request can end up costlier than the
computation itself. The common solution is keeping a few pre-opened threads in a
pool and distributing jobs to them — the same logic the file system calls' pool
uses.

When shared memory is needed, a buffer can be given to both threads through a
`SharedArrayBuffer`. In that case race conditions appear, and access is coordinated
with `Atomics` operations; every concurrency problem the single-threaded model
removed comes back in through this door. Use sharing only when measurement shows it
is needed.

#### Summary

- A single process uses a single core; this is enough for I/O-heavy work, not for
  CPU-bound work.
- A cluster sets up processes sharing the connections arriving on a single port; the
  distribution is round-robin by default, and on Windows the socket descriptor is passed.
- Cluster workers do not share memory; a stateful service has to move its state out
  of the process before being clustered.
- A worker thread runs a separate event loop in the same process; once the
  expensive computation moves there, the main thread keeps accepting requests.
- The selection axis is isolation versus sharing: a cluster to multiply capacity, a
  thread to take a single computation off the main path.

#### Next Step

The built-in modules topic is complete: file, path, stream, buffer, event, HTTP,
child process, and multi-core usage. What comes after this is about turning these
pieces into a working application. The first step is giving the measurement
collector a command-line face: parsing arguments, reading from a pipeline, and
writing usage text to the correct stream.

## Writing Applications

Command-line tools, separating configuration from secrets, error handling strategy, structured logging, testing, debugging, graceful shutdown, and package publishing.

### Command-Line Applications

Source: https://academia.sh/en/courses/nodejs/command-line-applications

Option parsing, taking input from a file or a pipeline, checking terminal attachment, writing the usage text to the right stream, and meaningful exit codes.

The Built-in Modules topic built every piece of the measurement collector. The
service form is not the only way to use it: the same summarizing logic also works as
a command-line tool that plugs into a pipeline — quickly inspecting a log file
should not require standing up a server.

This lesson writes that tool, to the standard set by the Shell Programming course's
filter model: arguments taken in a defined form, input read from a file or a
pipeline, data written to the first stream and descriptions to the second, result
reported with an exit code.

#### Parsing Options

Scanning `process.argv` by hand gets error-prone past the third option: short and
long forms, the value separator, combined flags, arguments after `--`. `parseArgs` in
`node:util` applies these rules in a defined way. It's considered stable from a
certain version onward; targeting an older runtime means checking for its presence.

Every option's type and default are declared to the parser. In `strict` mode, an
undeclared option throws; this keeps a typo from being silently ignored.

The tool needs two files: the `summarizer.mjs` module written in the HTTP Server
lesson and the twelve-line `measurements.ndjson` file from the Built-in Modules topic.
Both must be in this lesson's working directory; if they are missing, the files and
commands below create them.

```js
// summarizer.mjs
export function recordFromLine(line) {
  const record = JSON.parse(line);
  if (typeof record.node !== 'string' || typeof record.metric !== 'string') {
    throw new TypeError('node and metric must be strings');
  }
  if (!Number.isFinite(record.value)) {
    throw new TypeError('value must be a finite number');
  }
  return record;
}

export class Summarizer {
  #buckets = new Map();

  add({ node, metric, value }) {
    const key = `${node}/${metric}`;
    const bucket = this.#buckets.get(key) ?? { count: 0, total: 0, max: -Infinity };
    bucket.count += 1;
    bucket.total += value;
    bucket.max = Math.max(bucket.max, value);
    this.#buckets.set(key, bucket);
  }

  summary() {
    return [...this.#buckets].map(([key, { count, total, max }]) => ({
      key,
      count,
      average: Number((total / count).toFixed(2)),
      max,
    }));
  }
}
```

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

```js
// collect.mjs
import { parseArgs } from 'node:util';
import { createReadStream } from 'node:fs';
import { createInterface } from 'node:readline';
import { Summarizer, recordFromLine } from './summarizer.mjs';

const USAGE = `usage: collect.mjs [--file PATH] [--metric NAME] [--format text|json]

  --file    measurement file; standard input is read if omitted
  --metric  summarize only this metric
  --format  output format: text or json (default: text)
  --help    print this text and exit
`;

function readOptions(args) {
  const { values } = parseArgs({
    args,
    options: {
      file: { type: 'string' },
      metric: { type: 'string' },
      format: { type: 'string', default: 'text' },
      help: { type: 'boolean', default: false },
    },
    strict: true,
  });
  if (!['text', 'json'].includes(values.format)) {
    throw new TypeError(`--format can only be text or json, got: ${values.format}`);
  }
  return values;
}

let options;
try {
  options = readOptions(process.argv.slice(2));
} catch (error) {
  process.stderr.write(`${error.message}\n\n${USAGE}`);
  process.exit(2);
}

if (options.help) {
  process.stdout.write(USAGE);
  process.exit(0);
}

const input = options.file ? createReadStream(options.file) : process.stdin;
if (!options.file && process.stdin.isTTY) {
  process.stderr.write(`no input\n\n${USAGE}`);
  process.exit(2);
}

const summarizer = new Summarizer();
let read = 0;
let skipped = 0;

try {
  for await (const line of createInterface({ input, crlfDelay: Infinity })) {
    if (line.trim() === '') continue;
    read += 1;
    let record;
    try {
      record = recordFromLine(line);
    } catch {
      skipped += 1;
      continue;
    }
    if (options.metric && record.metric !== options.metric) continue;
    summarizer.add(record);
  }
} catch (error) {
  process.stderr.write(`could not read input: ${error.code ?? error.message}\n`);
  process.exitCode = 66;
}

const summary = summarizer.summary();
if (options.format === 'json') {
  process.stdout.write(JSON.stringify(summary) + '\n');
} else {
  for (const row of summary) {
    process.stdout.write(`${row.key}\t${row.count}\t${row.average}\t${row.max}\n`);
  }
}
process.stderr.write(`${read} lines read, ${skipped} lines skipped\n`);
```

The tool uses the `summarizer.mjs` module written in the HTTP Server lesson as is.
The same validation and the same accumulation apply in both interfaces; this is the
payoff of keeping the logic in a module separate from the server.

```sh
node collect.mjs --file measurements.ndjson
```

```
edge-01/temperature	3	21.87	22.3
edge-01/humidity	2	47.6	48
edge-02/temperature	3	20.23	20.7
edge-02/humidity	1	52.5	52.5
edge-03/temperature	2	24.35	24.6
edge-03/humidity	1	41.3	41.3
12 lines read, 0 lines skipped
```

The last line was written to standard error; redirecting one of the streams is
enough to tell them apart.

#### Two Sources of Input

In the Shell Programming course, filters split into three groups; the third accepts
both a file name and standard input. The tool above belongs to that group:
`process.stdin` is read if `--file` is not given.

```sh
cat measurements.ndjson | node collect.mjs --metric temperature --format json
```

```
[{"key":"edge-01/temperature","count":3,"average":21.87,"max":22.3},{"key":"edge-02/temperature","count":3,"average":20.23,"max":20.7},{"key":"edge-03/temperature","count":2,"average":24.35,"max":24.6}]
12 lines read, 0 lines skipped
```

Line-by-line reading goes through `node:readline`, which takes over the
line-splitting work written by hand in the Streams lesson and preserves backpressure:
the source pauses while the loop body runs. The whole file never loads into memory.

The `crlfDelay: Infinity` option counts the `\r\n` pair as a single line ending. This
setting is necessary when reading files produced on different platforms.

The tool's output can also be fed into a pipeline:

```sh
node collect.mjs --file measurements.ndjson 2>/dev/null | sort -t$'\t' -k3 -n -r | head -3
```

```
edge-02/humidity	1	52.5	52.5
edge-01/humidity	2	47.6	48
edge-03/humidity	1	41.3	41.3
```

The tab-separated column format was chosen for this composition. The JSON format is
for output that will be processed by a program; the column format works directly
with the field-based tools from the Shell Programming course.

#### Checking Terminal Attachment

If standard input is attached to a terminal and the user gave no file name, the
program silently starts waiting. To the user, this looks like a freeze.

The `process.stdin.isTTY` field reports whether the stream is attached to a
terminal. In a run connected to a pipeline, the value is undefined:

```js
// tty.mjs
console.log('stdout connected to terminal:', Boolean(process.stdout.isTTY));
console.log('stdin  connected to terminal:', Boolean(process.stdin.isTTY));
```

```sh
node tty.mjs | cat
```

```
stdout connected to terminal: false
stdin  connected to terminal: false
```

When the same script is run directly in a terminal, both lines print `true`.

This check is used not only to avoid the freeze but also to choose the output
format: color and alignment can be added for a terminal, but not for a file or a
pipe — color escape sequences mix into the data, and the next command mistakes them
for field content.

#### Separating Errors

The tool has three error conditions, and each is reported with a separate code.

An unrecognized option is a usage error:

```sh
node collect.mjs --unknown; echo "exit code: $?"
```

```
Unknown option '--unknown'

usage: collect.mjs [--file PATH] [--metric NAME] [--format text|json]

  --file    measurement file; standard input is read if omitted
  --metric  summarize only this metric
  --format  output format: text or json (default: text)
  --help    print this text and exit
exit code: 2
```

Failing to read the input is a separate condition and gets a separate code:

```sh
node collect.mjs --file missing.ndjson; echo "exit code: $?"
```

```
could not read input: ENOENT
0 lines read, 0 lines skipped
exit code: 66
```

Individual broken lines, though, are not an error; they are counted and skipped.
First, we prepare a small file with one broken line in it:

```sh
cat > broken.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":
{"time":"2024-02-07T09:14:19Z","node":"edge-02","metric":"humidity","value":50}
SON
```

Then we run the tool on that file:

```sh
node collect.mjs --file broken.ndjson
```

```
edge-01/humidity	1	48	48
edge-02/humidity	1	50	50
3 lines read, 1 lines skipped
```

This distinction is deliberate: dropping the entire report over one broken line in
bulk data makes all of it unusable, but the skipped count must not be silently
hidden — it goes to the diagnostic stream.

The usage text is written to **standard error**, because it is part of an error.
When requested with the `--help` option, though, it goes to standard output and the
exit code is zero: there, the usage text is the requested result itself.

With the interpreter declaration on the file's first line and execute permission
granted, the tool can be called directly. We add the declaration to the top of the
file and grant the permission:

```sh
{ printf '#!/usr/bin/env node\n'; cat collect.mjs; } > collect.tmp && mv collect.tmp collect.mjs
chmod +x collect.mjs
```

Now the tool can be called by name:

```sh
./collect.mjs --file measurements.ndjson --metric humidity
```

```
edge-01/humidity	2	47.6	48
edge-02/humidity	1	52.5	52.5
edge-03/humidity	1	41.3	41.3
12 lines read, 0 lines skipped
```

#### Summary

- The built-in option parser takes option names and types declaratively; strict
  mode treats an undeclared option as an error and catches typos.
- The same summarizing module is used in both the service and the command-line
  tool; this is the payoff of separating logic from interface.
- Input is taken from a file or standard input; line-by-line reading preserves
  backpressure and never loads the file into memory.
- Checking terminal attachment both prevents the wait and helps choose the output
  format; color escapes mix into data in a pipeline.
- Usage errors, input errors, and individual broken records are handled
  separately; the first two with separate exit codes, the third with a counter.

#### Next Step

The tool and the service still have hardcoded values: the file name, the port, the
body-size limit. These should change with the environment, but the source code
should not. The next lesson takes up configuration management: where values are read
from, how they're validated, and why secrets stay separate from ordinary settings.

### Configuration Management

Source: https://academia.sh/en/courses/nodejs/configuration-management

Separating settings from source code, validating environment variables, precedence order, separating secrets from ordinary settings, and keeping configuration immutable.

The tool from the previous lesson and the service from the HTTP Server lesson still
have hardcoded values: port 8791, the data file's name, the body-size limit. Correct
on a development machine, wrong somewhere else.

This lesson's question: where should a setting's value come from, and what happens
when it is wrong? The answer has two halves — the order of sources and the timing of
validation.

#### Settings Are Separated from Source Code

Every value that varies by environment is pulled out of the code. The test is
simple: the same build has to run unchanged in another environment. Changing the
environment without rebuilding guarantees the code running is the code that was
tested.

Environment variables carry settings. As seen in the Process Object lesson, these
are inherited when the process starts and read through `process.env`; their values
are always strings.

Reading and validation are gathered into a single module. The rest of the program
never touches `process.env`:

```js
// config.mjs
function requireEnv(name) {
  const value = process.env[name];
  if (value === undefined || value === '') {
    throw new Error(`required environment variable missing: ${name}`);
  }
  return value;
}

function integer(name, fallback) {
  const raw = process.env[name];
  if (raw === undefined) return fallback;
  const number = Number(raw);
  if (!Number.isInteger(number) || number <= 0) {
    throw new Error(`${name} must be a positive integer, got: ${JSON.stringify(raw)}`);
  }
  return number;
}

export function readConfig() {
  return Object.freeze({
    listenAddress: process.env.MEASUREMENTS_ADDRESS ?? '127.0.0.1',
    listenPort: integer('MEASUREMENTS_PORT', 8791),
    dataDir: process.env.MEASUREMENTS_DATA ?? './data',
    logLevel: process.env.MEASUREMENTS_LOG_LEVEL ?? 'info',
    adminKey: requireEnv('MEASUREMENTS_ADMIN_KEY'),
  });
}

// Secrets are never printed directly
export function printableView(config) {
  return { ...config, adminKey: `***${config.adminKey.slice(-2)}` };
}
```

```js
// try-config.mjs
import { readConfig, printableView } from './config.mjs';
try {
  console.log(printableView(readConfig()));
} catch (error) {
  console.error('configuration error:', error.message);
  process.exitCode = 78;
}
```

The variable names sharing a common prefix is deliberate: the process inherits
dozens of unrelated variables from the shell that started it, and the prefix removes
any ambiguity about which belong to this program.

#### Validation Happens at Startup

The worst moment for a wrong setting to surface is the first time its code path
runs, hours into the service running. Configuration is read and validated **at
startup**, before any request is accepted; if invalid, the process never comes up.

```sh
node try-config.mjs; echo "exit code: $?"
```

```
configuration error: required environment variable missing: MEASUREMENTS_ADMIN_KEY
exit code: 78
```

```sh
MEASUREMENTS_ADMIN_KEY=abc123XYZ MEASUREMENTS_PORT=eight node try-config.mjs; echo "exit code: $?"
```

```
configuration error: MEASUREMENTS_PORT must be a positive integer, got: "eight"
exit code: 78
```

Both messages state what's missing and the expected form. Configuration errors are
meant to be read by a person; a message like "invalid configuration" gives someone
looking for the fix nothing to go on.

Startup with correct values:

```sh
MEASUREMENTS_ADMIN_KEY=abc123XYZ MEASUREMENTS_PORT=9100 node try-config.mjs
```

```
{
  listenAddress: '127.0.0.1',
  listenPort: 9100,
  dataDir: './data',
  logLevel: 'info',
  adminKey: '***YZ'
}
```

The value `9100`, arriving as a string, was converted to a number and verified as an
integer. Converting in one place means the rest of the program never asks the type
question.

#### Precedence Order

A setting can come from more than one source. The order is fixed and documented
from the start; the most common arrangement, from narrowest to widest, is:

1. Command-line argument — affects a single run.
2. Environment variable — affects that process.
3. Env file — affects every run in a directory.
4. Default in code — applies when none of the above is given.

Each level overrides the one below it. The runtime directly supports the third
level: from a certain version onward, `NAME=value` lines in a file given with a flag
load into `process.env`.

```sh
printf 'MEASUREMENTS_ADMIN_KEY=fromfile99\nMEASUREMENTS_PORT=9200\n' > .env
node --env-file=.env try-config.mjs
```

```
{
  listenAddress: '127.0.0.1',
  listenPort: 9200,
  dataDir: './data',
  logLevel: 'info',
  adminKey: '***99'
}
```

A variable defined in the environment overrides the value from the file:

```sh
MEASUREMENTS_PORT=9999 node --env-file=.env try-config.mjs
```

```
{
  listenAddress: '127.0.0.1',
  listenPort: 9999,
  dataDir: './data',
  logLevel: 'info',
  adminKey: '***99'
}
```

The env file is a development convenience. It is not checked into version control;
the repository root keeps only an example file with field names and descriptions,
and the real file goes on the ignore list — exactly what the ignore mechanism from
the Introduction to Version Control course exists for.

#### Implementing the Top Level

The list's first item is not in the code yet: the reader only looks at the
environment, not command-line arguments. This level redirects a single run — a
diagnostic session, a test startup — without changing the environment.

The reader that merges the layers in one place also records which source won for
each field:

```js
// layered-config.mjs
import { parseArgs } from 'node:util';

const DEFINITIONS = {
  listenAddress: { option: 'address',    variable: 'MEASUREMENTS_ADDRESS',   type: 'string',  fallback: '127.0.0.1' },
  listenPort:    { option: 'port',       variable: 'MEASUREMENTS_PORT',      type: 'integer', fallback: 8791 },
  logLevel:      { option: 'log-level',  variable: 'MEASUREMENTS_LOG_LEVEL', type: 'string',  fallback: 'info' },
};

function convert(sourceName, raw, type) {
  if (type === 'string') return raw;
  const number = Number(raw);
  if (!Number.isInteger(number) || number <= 0) {
    throw new Error(`${sourceName} must be a positive integer, got: ${JSON.stringify(raw)}`);
  }
  return number;
}

// Arguments and environment are passed in: testing never touches the real process
export function readConfig(args = process.argv.slice(2), env = process.env) {
  const { values } = parseArgs({
    args,
    options: Object.fromEntries(Object.values(DEFINITIONS).map((d) => [d.option, { type: 'string' }])),
    strict: true,
  });

  const value = {};
  const source = {};
  for (const [field, d] of Object.entries(DEFINITIONS)) {
    if (values[d.option] !== undefined) {
      value[field] = convert(`--${d.option}`, values[d.option], d.type);
      source[field] = 'argument';
    } else if (env[d.variable] !== undefined && env[d.variable] !== '') {
      value[field] = convert(d.variable, env[d.variable], d.type);
      source[field] = 'environment';
    } else {
      value[field] = d.fallback;
      source[field] = 'default';
    }
  }
  return Object.freeze({ value: Object.freeze(value), source: Object.freeze(source) });
}
```

```js
// try-layered-config.mjs
import { readConfig } from './layered-config.mjs';

try {
  const { value, source } = readConfig();
  for (const field of Object.keys(value)) {
    console.log(`${field.padEnd(15)} ${String(value[field]).padEnd(12)} <- ${source[field]}`);
  }
} catch (error) {
  console.error('configuration error:', error.message);
  process.exitCode = 78;
}
```

When no source is given, every field falls back to the default in code:

```sh
node try-layered-config.mjs
```

```
listenAddress   127.0.0.1    <- default
listenPort      8791         <- default
logLevel        info         <- default
```

An environment variable overrides the default:

```sh
MEASUREMENTS_PORT=9100 MEASUREMENTS_LOG_LEVEL=verbose node try-layered-config.mjs
```

```
listenAddress   127.0.0.1    <- default
listenPort      9100         <- environment
logLevel        verbose      <- environment
```

An argument overrides the environment too:

```sh
MEASUREMENTS_PORT=9100 node try-layered-config.mjs --port 9500
```

```
listenAddress   127.0.0.1    <- default
listenPort      9500         <- argument
logLevel        info         <- default
```

Validation stays the same at every layer; whichever source's name the error message
carries is where the fix belongs:

```sh
node try-layered-config.mjs --port eight; echo "exit code: $?"
```

```
configuration error: --port must be a positive integer, got: "eight"
exit code: 78
```

Keeping the source record looks like an extra field, but it is decisive for
diagnosis: an unexpected value comes either from a wrong value being given or from
the expected layer never being read. The source field separates the two, and this
information can go into the startup log — for fields that are not secret.

The reader taking the argument array and the environment as parameters is also a
testability decision: a test can try every combination of layers without touching
real process state. This design returns in the Test lesson.

#### Secrets Are Handled Separately

An admin key and a port number are not in the same class. The second can be written
to diagnostic output, error messages, and logs; the first cannot.

Applying this rests on two rules. First, the configuration object is never printed
directly; `printableView` above masks the secret field and leaves only the last two
characters — enough to recognize the wrong key was loaded, not enough to give it
away.

Second, the secret is never passed as an argument. Command-line arguments show up
in the machine's process list; environment variables do not. This is why secrets
are only ever read from the environment or a file.

Error objects need attention too: a connection error object can carry the
connection string in one of its fields. Writing that object to the log as is makes
the masking pointless — why the Logging lesson chooses log fields deliberately.

#### Configuration Is Immutable

The configuration object is read once and frozen. A setting that can change while
running lets the same request show two different behaviors and makes diagnosis
impossible.

```js
// freeze-config.mjs
import { readConfig } from './config.mjs';

const config = readConfig();
console.log('listen port      :', config.listenPort);
console.log('is frozen        :', Object.isFrozen(config));

try {
  config.listenPort = 1234;      // ES modules run in strict mode
} catch (error) {
  console.log('write attempt    :', error.constructor.name);
}
console.log('value unchanged  :', config.listenPort);
```

```sh
MEASUREMENTS_ADMIN_KEY=abc123XYZ node freeze-config.mjs
```

```
listen port      : 8791
is frozen        : true
write attempt    : TypeError
value unchanged  : 8791
```

Freezing is one of the immutability techniques from the Objects and Functions in
JavaScript course. Because ES modules run in strict mode, a write on a frozen
object is not silently ignored; it throws, and the code that got it wrong shows up
immediately.

Freezing is shallow: nested objects have to be frozen separately. The configuration
above carries only primitive values, so a single call is enough.

#### Summary

- Every value that varies by environment is pulled out of the code; the same build
  has to run in another environment unchanged.
- Configuration is read and validated in a single module; the rest of the program
  never touches the environment object and never asks the type question.
- Validation happens at startup, and the process never comes up if it fails; the
  error message states the missing field and the expected form.
- The source precedence order is fixed from the start: argument, environment
  variable, env file, default in code; recording the winning source for each field
  shortens diagnosis.
- Secrets are masked before printing and never passed as arguments; command-line
  arguments show up in the process list.

#### Next Step

A configuration error was one that stopped the process from coming up. Errors that
occur while running do not call for the same behavior: a request's malformed body
should not shut down the service, but a corrupted in-memory state should. The next
lesson separates these two classes and decides which error gets caught and which
one ends the process.

### Error Handling Strategy

Source: https://academia.sh/en/courses/nodejs/error-handling-strategy

The distinction between operational errors and programmer errors, chaining error causes, the forms of asynchronous errors that escape catching, and the correct use of last-resort hooks.

The previous lesson said a configuration error should stop the process from coming
up. Not every error occurring while running deserves the same treatment: a
request's malformed body should not shut down the service, but a corrupted
in-memory state should.

This lesson's question: which error gets caught, and which one ends the process?
The distinction rests on a single criterion, and the whole strategy follows from it.

#### Two Classes of Error

An **operational error** is a condition a correctly written program expects to meet
in its running environment: a missing file, an unreachable network, a client that
sent invalid data, a full disk. Not the program's fault — anticipated and handled.

A **programmer error** is the code violating its own contract: accessing a field on
an undefined value, forgetting a required argument, giving an object where a number
is expected. Unanticipated, and it shows the assumptions about the program's state
at that moment were wrong.

The rule: **an operational error is handled, a programmer error is not.** Catching
the second and continuing means running on with a corrupted state — a failure of
unknown duration and much harder to diagnose.

For the distinction to be visible in code, operational errors are marked with their
own type:

```js
// error-types.mjs
export class OperationalError extends Error {
  constructor(message, { code, status = 500, cause } = {}) {
    super(message, { cause });
    this.name = 'OperationalError';
    this.code = code;
    this.status = status;
    this.operational = true;
  }
}

export function isOperational(error) {
  return error instanceof OperationalError && error.operational === true;
}
```

#### Chaining the Cause

An error from a lower layer gets translated moving up to a higher one: the higher
layer throws an error that speaks its own language, without losing the real cause.
The `cause` option builds that link.

The experiment needs the twelve-line measurement file from the Built-in Modules
topic; if it is not in the working directory, this command creates it:

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

```js
// try-errors.mjs
import { readFile } from 'node:fs/promises';
import { OperationalError, isOperational } from './error-types.mjs';

async function readMeasurements(path) {
  try {
    return await readFile(path, 'utf8');
  } catch (error) {
    if (error.code === 'ENOENT') {
      throw new OperationalError(`measurement file missing: ${path}`, {
        code: 'MEASUREMENT_FILE_MISSING', status: 503, cause: error,
      });
    }
    throw error;
  }
}

for (const path of ['measurements.ndjson', 'missing.ndjson']) {
  try {
    const text = await readMeasurements(path);
    console.log(`${path}: ${text.trim().split('\n').length} lines`);
  } catch (error) {
    console.log(`${path}: operational = ${isOperational(error)}, code = ${error.code}, status = ${error.status}`);
    console.log(`  root cause: ${error.cause.code}`);
  }
}

// Programmer error: contract violation, never caught and swallowed
try {
  null.value;
} catch (error) {
  console.log(`programmer error: operational = ${isOperational(error)}, type = ${error.constructor.name}`);
}
```

```sh
node try-errors.mjs
```

```
measurements.ndjson: 12 lines
missing.ndjson: operational = true, code = MEASUREMENT_FILE_MISSING, status = 503
  root cause: ENOENT
programmer error: operational = false, type = TypeError
```

Three details matter.

Only the recognized error code is translated; everything else rethrows as is.
Translating every error into a single type hides unknown failures inside a known
category.

The translated error carries its own code. The caller decides without parsing the
message text; the code stays fixed even if the message changes.

The real cause stays reachable — reaching the system-call error at the bottom of
the chain is the shortest path to the failure during diagnosis.

#### The Counterpart in the Service

The error handler in the HTTP Server lesson reported every error as 400. Once the
distinction is established, the correct mapping can be made:

The server imports the `summarizer.mjs` module from the HTTP Server lesson. If it is
not in the working directory, the same module is written like this:

```js
// summarizer.mjs
export function recordFromLine(line) {
  const record = JSON.parse(line);
  if (typeof record.node !== 'string' || typeof record.metric !== 'string') {
    throw new TypeError('node and metric must be strings');
  }
  if (!Number.isFinite(record.value)) {
    throw new TypeError('value must be a finite number');
  }
  return record;
}

export class Summarizer {
  #buckets = new Map();

  add({ node, metric, value }) {
    const key = `${node}/${metric}`;
    const bucket = this.#buckets.get(key) ?? { count: 0, total: 0, max: -Infinity };
    bucket.count += 1;
    bucket.total += value;
    bucket.max = Math.max(bucket.max, value);
    this.#buckets.set(key, bucket);
  }

  summary() {
    return [...this.#buckets].map(([key, { count, total, max }]) => ({
      key,
      count,
      average: Number((total / count).toFixed(2)),
      max,
    }));
  }
}
```

```js
// error-server.mjs
import { createServer } from 'node:http';
import { OperationalError, isOperational } from './error-types.mjs';
import { recordFromLine } from './summarizer.mjs';

function errorResponse(error, response) {
  if (isOperational(error)) {
    response.writeHead(error.status, { 'content-type': 'application/json' });
    response.end(JSON.stringify({ error: error.message, code: error.code }) + '\n');
    return;
  }
  // Programmer error: no detail goes out, the full record stays inside
  process.stderr.write(`programmer error: ${error.stack.split('\n')[0]}\n`);
  response.writeHead(500, { 'content-type': 'application/json' });
  response.end(JSON.stringify({ error: 'internal error', code: 'INTERNAL_ERROR' }) + '\n');
}

const server = createServer(async (request, response) => {
  try {
    if (request.url === '/measurement') {
      const chunks = [];
      for await (const chunk of request) chunks.push(chunk);
      const body = Buffer.concat(chunks).toString('utf8');
      try {
        recordFromLine(body.trim());
      } catch (error) {
        throw new OperationalError('invalid measurement record', {
          code: 'INVALID_RECORD', status: 400, cause: error,
        });
      }
      response.writeHead(201, { 'content-type': 'application/json' });
      response.end('{"received":1}\n');
      return;
    }
    if (request.url === '/crash') {
      null.value;                       // deliberate programmer error
    }
    throw new OperationalError('not found', { code: 'ROUTE_NOT_FOUND', status: 404 });
  } catch (error) {
    errorResponse(error, response);
  }
});

server.listen(8791, '127.0.0.1', () => console.log('listening'));
```

We write a script that starts the server in the background and sends three
requests. The server's standard error goes to a separate file, so it's visible where
the programmer error's internal detail is written:

```sh
#!/bin/sh
# request-test.sh
node error-server.mjs 2> server-error.txt &
serverPid=$!
sleep 1

curl -s -X POST --data-binary 'broken' -w 'status: %{http_code}\n' http://127.0.0.1:8791/measurement
curl -s -w 'status: %{http_code}\n' http://127.0.0.1:8791/missing
curl -s -w 'status: %{http_code}\n' http://127.0.0.1:8791/crash

kill "$serverPid"
wait "$serverPid" 2>/dev/null
echo "server's standard error:"
cat server-error.txt
```

```sh
sh request-test.sh
```

```
listening
{"error":"invalid measurement record","code":"INVALID_RECORD"}
status: 400
{"error":"not found","code":"ROUTE_NOT_FOUND"}
status: 404
{"error":"internal error","code":"INTERNAL_ERROR"}
status: 500
server's standard error:
programmer error: TypeError: Cannot read properties of null (reading 'value')
```

The line written to the server's standard error on the last call is the internal
detail that is not given to the client.

The operational error's message goes to the client, the programmer error's does
not — not a habit of hiding things, a decision resting on two reasons: the stack
trace and internal message leak file paths and code structure; and there is nothing
the client can do about it anyway — the only meaningful response to a 500 is to
retry or report it.

#### Where Asynchronous Errors Escape Catching

`try`/`catch` covers only its own synchronous body and awaited promises. An error
thrown inside a callback falls outside that scope.

```js
// uncaught.mjs
import { readFile } from 'node:fs/promises';

// try/catch does not catch an error deferred to a callback
try {
  setTimeout(() => { throw new Error('thrown inside a callback'); }, 0);
} catch (error) {
  console.log('this line never runs');
}

// Unhandled rejection
readFile('missing.ndjson', 'utf8');

console.log('sync flow done');
```

The details of an uncaught error are written to standard error. We discard them
here and show only standard output and the exit code:

```sh
node uncaught.mjs 2>/dev/null; echo "exit code: $?"
```

```
sync flow done
exit code: 1
```

The output's first line is `sync flow done`; then the uncaught error is written to
standard error and the process ends with a nonzero code. `this line never runs` never prints: `setTimeout`
returned immediately, and the error was thrown much later, in the loop's timers
phase — by then the `try` block had long since finished.

A second problem in the same file is sneakier. Because the `readFile` call is not
awaited, the rejection of the promise it returns is never handled anywhere. This is
an **unhandled rejection**, and its default behavior is to end the process:

```js
// rejection.mjs
import { readFile } from 'node:fs/promises';

readFile('missing.ndjson', 'utf8');     // returned promise is not awaited, its error goes unhandled
console.log('sync flow done');
```

```sh
node rejection.mjs > /dev/null; echo "exit code: $?"
```

```
exit code: 1
```

This looks harsh, but it is correct: a silently swallowed rejection leads to data
being thought written when it was not. Rule: **every call that returns a promise is
either awaited or has its rejection explicitly handled.**

#### Last-Resort Hooks

Two hooks put uncaught errors into the program's own hands:

```js
// last-resort.mjs
process.on('uncaughtException', (error, origin) => {
  process.stderr.write(`fatal: ${error.message} (origin: ${origin})\n`);
  process.exit(1);            // state is considered corrupted: restart
});

process.on('unhandledRejection', (reason) => {
  process.stderr.write(`unhandled rejection: ${reason.message ?? reason}\n`);
  process.exit(1);
});

setTimeout(() => { throw new Error('late error'); }, 0);
console.log('hook installed');
```

```sh
node last-resort.mjs; echo "exit code: $?"
```

```
hook installed
fatal: late error (origin: uncaughtException)
exit code: 1
```

The only legitimate use of these hooks is recording the failure before the process
dies. Installing the hook and continuing to run breaks the runtime's default
behavior and lets it keep going in an unknown state.

No long work happens inside the hook. The process is already in a suspect state;
starting an asynchronous log write there gives no guarantee it will complete. A
short, synchronous line and exiting is safest.

Keeping the service alive after the process dies is not the process's own job — an
external supervisor that restarts it handles that, taken up in the Process
Management and Resilience lesson.

#### Summary

- An operational error is an anticipated running condition and is handled; a
  programmer error is a contract violation and is not.
- Operational errors are marked with their own type and carry a code field; the
  caller does not parse the message text to decide.
- When an error is translated, `cause` preserves the real reason; only recognized
  conditions are translated, the rest is rethrown as is.
- An error thrown inside a callback and the rejection of an unawaited promise fall
  outside `try`/`catch`'s scope; the second's default result is the process ending.
- Last-resort hooks are used only to record and exit; installing the hook and
  continuing means proceeding in an unknown state.

#### Next Step

Recording errors came up a few times, but where and in what form was never
settled. A free-text message is no use to someone searching for the failure —
what's needed is a record format supporting searching, filtering, and counting.
The next lesson builds structured logging and adds level management.

### Logging

Source: https://academia.sh/en/courses/nodejs/logging

Structured records instead of free text, level thresholds, carrying optional fields, writing the log to standard output, and inspecting records with tools.

The Error Handling Strategy lesson mentioned recording failures a few times, but
where and in what form was never settled. A free-text line — `error: could not open
file` — is not enough for someone looking for the failure: which file, which
request, which process, how many times?

This lesson settles the log format. Term choice: the **log** is the sequence of
records a process produces; **logging** is the work of producing those records.

#### Structured Records

A log line addresses two readers at once: the person looking at it during a
failure, and a program that filters, counts, and generates alerts from records.
What satisfies both is each line being a record parsable on its own.

The measurement file's format is useful here too: one JSON object per line. The
line-based tools from the Shell Programming course work directly with this format.

```js
// log.mjs
const LEVELS = { verbose: 20, info: 30, warn: 40, error: 50 };

export function createLogger({
  level = 'info',
  service,
  output = process.stdout,
  clock = () => new Date().toISOString(),   // can be fixed in tests
  fixedFields = {},
} = {}) {
  const threshold = LEVELS[level] ?? LEVELS.info;

  function write(levelName, message, fields = {}) {
    if (LEVELS[levelName] < threshold) return;
    output.write(JSON.stringify({
      time: clock(),
      level: levelName,
      service,
      message,
      ...fixedFields,
      ...fields,
    }) + '\n');
  }

  const logger = {};
  for (const name of Object.keys(LEVELS)) {
    logger[name] = (message, fields) => write(name, message, fields);
  }
  // A child logger that carries its own fields on every line it produces
  logger.child = (extraFields) => createLogger({
    level, service, output, clock,
    fixedFields: { ...fixedFields, ...extraFields },
  });
  return logger;
}
```

```js
// try-log.mjs
import { createLogger } from './log.mjs';

const logger = createLogger({
  level: process.env.MEASUREMENTS_LOG_LEVEL ?? 'info',
  service: 'measurement-collector',
  clock: () => '2024-02-07T09:12:44.000Z',      // fixed so the example output does not change
});

logger.verbose('configuration read', { file: '.env' });
logger.info('request received', { route: '/measurement', method: 'POST', records: 12 });
logger.warn('threshold exceeded', { node: 'edge-03', value: 24.6 });
logger.error('could not open measurement file', { code: 'ENOENT', path: 'missing.ndjson' });

const requestLogger = logger.child({ requestId: 'r-0007' });
requestLogger.info('response written', { status: 201, duration_ms: 3 });
```

```sh
node try-log.mjs
```

```
{"time":"2024-02-07T09:12:44.000Z","level":"info","service":"measurement-collector","message":"request received","route":"/measurement","method":"POST","records":12}
{"time":"2024-02-07T09:12:44.000Z","level":"warn","service":"measurement-collector","message":"threshold exceeded","node":"edge-03","value":24.6}
{"time":"2024-02-07T09:12:44.000Z","level":"error","service":"measurement-collector","message":"could not open measurement file","code":"ENOENT","path":"missing.ndjson"}
{"time":"2024-02-07T09:12:44.000Z","level":"info","service":"measurement-collector","message":"response written","requestId":"r-0007","status":201,"duration_ms":3}
```

The timestamp was fixed in the example; in a real run, every line carries its own
moment. Supplying the clock from outside is not a convenience, it is a testability
requirement — the Test lesson uses this hook.

The message text is fixed, variable information goes into separate fields. This
separation makes searching possible: searching `"message":"request received"` finds
every request; if the file name were embedded in the message, every line would
differ.

#### Level Threshold

Four levels are ordered by numeric weight, and records below the threshold never
get written. The threshold comes from configuration, so log volume adjusts without
changing code.

```sh
MEASUREMENTS_LOG_LEVEL=error node try-log.mjs
```

```
{"time":"2024-02-07T09:12:44.000Z","level":"error","service":"measurement-collector","message":"could not open measurement file","code":"ENOENT","path":"missing.ndjson"}
```

The meaning of each level is fixed by convention; a meaningless order makes the
threshold useless:

- **error** — the operation could not complete, someone needs to look.
- **warn** — the operation completed, but something unexpected happened.
- **info** — milestones in the service's normal flow.
- **verbose** — detail needed for diagnosis; off during normal operation.

The most common mistake is writing everything at the `error` level. A real failure
becomes invisible in the noise, and the level distinction loses its meaning.

#### Carrying Optional Fields

In a concurrently running service, log lines arrive interleaved; collecting the
lines from one request needs a shared field. The `child` method returns a logger
that adds the given fields to every line it produces.

In the output's last line above, `requestId` was added this way. When an id is
generated at the start of a request and a child logger is created from it, every
record from that request carries the same value and is gathered with one search.

Carrying the id in a header from the client makes it possible to trace a request
across multiple services. An incoming id is not trusted directly: its form gets
validated, or it is written alongside a self-generated id as a second field.

#### Writing an Error to the Log

The most important record written to the log is the error record, and it is also
the most often written wrong. Putting the error object directly into a field
creates two problems. First, an `Error` object's `name`, `message`, and `stack`
fields are not enumerable; serialization skips them, leaving an empty object.
Second, spreading the whole object (`...error`) defeats the masking from the
Configuration Management lesson — a connection error might carry the connection
string in one of its fields.

The right way is to explicitly choose which fields get written:

```js
// log-error.mjs
export function errorFields(error, { stackTrace = false } = {}) {
  const fields = { errorName: error.name, errorMessage: error.message };
  if (error.code !== undefined) fields.errorCode = error.code;

  const chain = [];
  let cause = error.cause;
  while (cause instanceof Error && chain.length < 5) {
    chain.push(cause.code ?? cause.name);
    cause = cause.cause;
  }
  if (chain.length > 0) fields.causeChain = chain;

  if (stackTrace && typeof error.stack === 'string') {
    fields.stackTrace = error.stack.split('\n')[0];
  }
  return fields;
}
```

The cause chain built in the Error Handling Strategy lesson pays off here: the
codes underneath a wrapped error are gathered into a single array field.

The experiment relies on the error class from the Error Handling Strategy lesson; if
it is not in the working directory, write it first:

```js
// error-types.mjs
export class OperationalError extends Error {
  constructor(message, { code, status = 500, cause } = {}) {
    super(message, { cause });
    this.name = 'OperationalError';
    this.code = code;
    this.status = status;
    this.operational = true;
  }
}

export function isOperational(error) {
  return error instanceof OperationalError && error.operational === true;
}
```

```js
// try-log-error.mjs
import { readFile } from 'node:fs/promises';
import { createLogger } from './log.mjs';
import { errorFields } from './log-error.mjs';
import { OperationalError } from './error-types.mjs';

const logger = createLogger({
  service: 'measurement-collector',
  clock: () => '2024-02-07T09:12:44.000Z',
});

try {
  await readFile('missing.ndjson', 'utf8');
} catch (underlying) {
  const error = new OperationalError('could not open measurement file', {
    code: 'MEASUREMENT_FILE_MISSING', status: 503, cause: underlying,
  });
  logger.error('request could not be fulfilled', { ...errorFields(error), route: '/summary' });
}

logger.error('unexpected type', { ...errorFields(new TypeError('value must be finite'), { stackTrace: true }) });
```

```sh
node try-log-error.mjs
```

```
{"time":"2024-02-07T09:12:44.000Z","level":"error","service":"measurement-collector","message":"request could not be fulfilled","errorName":"OperationalError","errorMessage":"could not open measurement file","errorCode":"MEASUREMENT_FILE_MISSING","causeChain":["ENOENT"],"route":"/summary"}
{"time":"2024-02-07T09:12:44.000Z","level":"error","service":"measurement-collector","message":"unexpected type","errorName":"TypeError","errorMessage":"value must be finite","stackTrace":"TypeError: value must be finite"}
```

The first record carries the `ENOENT` code from the bottom of the chain; someone
looking for the failure reads the cause all the way down to the file system from a
single line.

The stack trace is not written by default, for two reasons: it spans multiple
lines and breaks the one-record-per-line contract, and it carries file paths and
directory structure into the log. The example takes only the trace's first line.
Where the whole trace is needed, line breaks are escaped and it is written as a
single JSON string — not as raw text.

Operational errors and programmer errors are separated in the log too. An
operational error is an expected condition, and the `warn` level is often enough; a
programmer error is always written at the `error` level with its full trace,
because someone has to look.

#### Where the Log Is Written

The log is written to **standard output**. The process does not open its own file,
does not rotate its own files, does not keep its own archive.

The reasoning is the same principle from the Shell Programming course: where output
goes is decided by whoever calls the command, not the command itself. The
supervisor that starts the process routes the output to a file, a collector, or a
pipeline; the process does not know which. The same build runs unchanged across
different operating setups.

The rule that diagnostic messages go to standard error meets an exception here: the
structured log does not mix with data, because the service's "data" is HTTP
responses, not standard output. For command-line tools, the situation flips — the
tool from the Command-Line Applications lesson writes its log to standard error,
because standard output is the report itself.

One detail: writing is synchronous when standard output connects to a file, and
can run asynchronously when it connects to a pipe. In the second case, a hard exit
with `process.exit` can drop unwritten lines — the concrete counterpart of the
warning from the Process Object lesson.

#### Inspecting Records

The gain from the structured format is that it needs no special tool:

```sh
node try-log.mjs > log.ndjson
grep -o '"level":"[a-z]*"' log.ndjson | sort | uniq -c | sort -rn
```

```
      2 "level":"info"
      1 "level":"warn"
      1 "level":"error"
```

When a field needs extracting, a short filter is enough:

```sh
grep '"level":"error"' log.ndjson | node -e "
const lines = require('node:fs').readFileSync(0, 'utf8').trim().split('\n');
for (const line of lines) { const parsed = JSON.parse(line); console.log(parsed.code, '|', parsed.message); }
"
```

```
ENOENT | could not open measurement file
```

In the second example, `readFileSync(0, 'utf8')` reads file descriptor 0 — standard
input. The descriptor numbers from the Shell Programming course work directly
here.

One last rule: personal data and secrets never go into the log. The admin key
masked in the Configuration Management lesson can leak in as a field on an error
object. Choosing fields explicitly — instead of spreading the whole error
object — prevents this leak.

#### Summary

- A log line addresses both a person and a program; the one-JSON-object-per-line
  format satisfies both readers.
- The message text stays fixed, variable information goes into separate fields;
  this is what makes searching possible.
- Levels are ordered by numeric weight and the threshold comes from
  configuration; writing everything at the error level makes the distinction
  meaningless.
- A child logger adds the given fields to every line it produces, letting a
  request's records be gathered.
- An error object is never written directly: its fields are chosen explicitly,
  the cause chain reduces to codes, and the stack trace is added only when
  requested.
- The log is written to standard output; the file, rotation, and archive are the
  job of the supervisor that starts the process.

#### Next Step

The measurement collector's pieces have multiplied: the summarizer, configuration,
error classes, the log, the server. That these work correctly has been checked by
hand so far, by looking at output. The next lesson automates that check: it writes
unit and integration tests with the runtime's built-in test runner and shows what a
failing test looks like and what the coverage report says.

### Test

Source: https://academia.sh/en/courses/nodejs/test

The built-in test runner, the boundary of a unit test, writing integration tests by separating the server from listening, a failing test's output, and reading the coverage report.

The previous four lessons built the measurement collector's pieces: the
summarizer, configuration, error classes, the log, the server. That these work
correctly has been checked by hand so far, by looking at output.

Manual checking has two flaws: it repeats after every change, and edge cases get
forgotten. This lesson makes the check runnable. The test runner and assertion
module live inside the runtime, considered stable from a certain version onward.

#### Unit Test

A unit test tests a single module on its own, without touching the outside world.
The unit under test has to be written to support this isolation — the reason the
summarizing logic moved into a separate module in the HTTP Server lesson.

```js
// summarizer.mjs
export function recordFromLine(line) {
  const record = JSON.parse(line);
  if (typeof record.node !== 'string' || typeof record.metric !== 'string') {
    throw new TypeError('node and metric must be strings');
  }
  if (!Number.isFinite(record.value)) {
    throw new TypeError('value must be a finite number');
  }
  return record;
}

export class Summarizer {
  #buckets = new Map();

  add({ node, metric, value }) {
    const key = `${node}/${metric}`;
    const bucket = this.#buckets.get(key) ?? { count: 0, total: 0, max: -Infinity };
    bucket.count += 1;
    bucket.total += value;
    bucket.max = Math.max(bucket.max, value);
    this.#buckets.set(key, bucket);
  }

  summary() {
    return [...this.#buckets].map(([key, { count, total, max }]) => ({
      key,
      count,
      average: Number((total / count).toFixed(2)),
      max,
    }));
  }
}
```

```js
// summarizer.test.mjs
import { test, describe } from 'node:test';
import assert from 'node:assert/strict';
import { Summarizer, recordFromLine } from './summarizer.mjs';

describe('recordFromLine', () => {
  test('converts a valid line to a record', () => {
    const record = recordFromLine('{"node":"edge-01","metric":"humidity","value":48}');
    assert.deepEqual(record, { node: 'edge-01', metric: 'humidity', value: 48 });
  });

  test('throws SyntaxError for broken JSON', () => {
    assert.throws(() => recordFromLine('broken'), SyntaxError);
  });

  test('throws TypeError for a non-numeric value', () => {
    assert.throws(
      () => recordFromLine('{"node":"a","metric":"b","value":"twenty"}'),
      { name: 'TypeError' },
    );
  });
});

describe('Summarizer', () => {
  test('merges measurements under the same key', () => {
    const s = new Summarizer();
    s.add({ node: 'edge-01', metric: 'temperature', value: 21.4 });
    s.add({ node: 'edge-01', metric: 'temperature', value: 22.6 });
    assert.deepEqual(s.summary(), [
      { key: 'edge-01/temperature', count: 2, average: 22, max: 22.6 },
    ]);
  });

  test('an empty summarizer returns an empty array', () => {
    assert.deepEqual(new Summarizer().summary(), []);
  });
});
```

```sh
node --test summarizer.test.mjs
```

{/* mask: (?<=\()[0-9.]+(?=ms\)) */}
{/* mask: (?<=duration_ms )[0-9.]+ */}
```
▶ recordFromLine
  ✔ converts a valid line to a record (1.11775ms)
  ✔ throws SyntaxError for broken JSON (0.155333ms)
  ✔ throws TypeError for a non-numeric value (0.074167ms)
✔ recordFromLine (2.002583ms)
▶ Summarizer
  ✔ merges measurements under the same key (0.125584ms)
  ✔ an empty summarizer returns an empty array (0.046625ms)
✔ Summarizer (0.240542ms)
ℹ tests 5
ℹ suites 2
ℹ pass 5
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 34.86225
```

The duration values change on every run; what's meaningful is the counters.

The `node:assert/strict` subpath makes comparisons with strict equality. The
pitfalls of loose equality were covered in the JavaScript Fundamentals course;
loose comparison in a test lets a test that should fail pass instead.

The second argument to `assert.throws` defines the expected error — a constructor,
or an object to test fields against. Saying only "throw some error" is a weak
test: an error thrown for the wrong reason passes it too.

#### Passing the Dependency in from Outside

The logger builder in the Logging lesson took the clock function and the output
stream from outside. That design is exactly for testability. The test imports the
logger module from the Logging lesson; if it is not in the working directory,
write it first:

```js
// log.mjs
const LEVELS = { verbose: 20, info: 30, warn: 40, error: 50 };

export function createLogger({
  level = 'info',
  service,
  output = process.stdout,
  clock = () => new Date().toISOString(),   // can be fixed in tests
  fixedFields = {},
} = {}) {
  const threshold = LEVELS[level] ?? LEVELS.info;

  function write(levelName, message, fields = {}) {
    if (LEVELS[levelName] < threshold) return;
    output.write(JSON.stringify({
      time: clock(),
      level: levelName,
      service,
      message,
      ...fixedFields,
      ...fields,
    }) + '\n');
  }

  const logger = {};
  for (const name of Object.keys(LEVELS)) {
    logger[name] = (message, fields) => write(name, message, fields);
  }
  // A child logger that carries its own fields on every line it produces
  logger.child = (extraFields) => createLogger({
    level, service, output, clock,
    fixedFields: { ...fixedFields, ...extraFields },
  });
  return logger;
}
```

```js
// log.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { Writable } from 'node:stream';
import { createLogger } from './log.mjs';

function capturingOutput() {
  const lines = [];
  const stream = new Writable({
    write(chunk, encoding, callback) { lines.push(chunk.toString('utf8')); callback(); },
  });
  return { stream, lines };
}

test('a record below the threshold is not written', () => {
  const { stream, lines } = capturingOutput();
  const logger = createLogger({ level: 'warn', service: 'test', output: stream, clock: () => 'T' });

  logger.info('should not appear');
  logger.warn('should appear');

  assert.equal(lines.length, 1);
  assert.deepEqual(JSON.parse(lines[0]), {
    time: 'T', level: 'warn', service: 'test', message: 'should appear',
  });
});

test('a child logger adds its fixed fields to every line', () => {
  const { stream, lines } = capturingOutput();
  const logger = createLogger({ service: 'test', output: stream, clock: () => 'T' });

  logger.child({ requestId: 'r-1' }).info('first', { status: 200 });

  assert.equal(JSON.parse(lines[0]).requestId, 'r-1');
});
```

Thanks to the fixed clock and the collecting stream, the test verifies the full
record without touching real time or real output. No need to change a value
inside the module during testing — the dependency is already passed in from
outside.

#### Integration Test

An integration test tests several pieces together. For the HTTP service, the
problem is this: `service.mjs` starts listening the moment it is imported, so it
cannot be controlled from inside a test.

The solution: **separate building the server from starting to listen**. The
module exports a factory function, and the caller decides when to listen.

```js
// build-server.mjs
import { createServer } from 'node:http';
import { Summarizer, recordFromLine } from './summarizer.mjs';

export function buildServer() {
  const summarizer = new Summarizer();

  return createServer(async (request, response) => {
    const url = new URL(request.url, `http://${request.headers.host}`);

    if (request.method === 'POST' && url.pathname === '/measurement') {
      const chunks = [];
      for await (const chunk of request) chunks.push(chunk);
      const body = Buffer.concat(chunks).toString('utf8');
      try {
        let count = 0;
        for (const line of body.split('\n')) {
          if (line.trim() === '') continue;
          summarizer.add(recordFromLine(line));
          count += 1;
        }
        response.writeHead(201, { 'content-type': 'application/json' });
        response.end(JSON.stringify({ received: count }));
      } catch (error) {
        response.writeHead(400, { 'content-type': 'application/json' });
        response.end(JSON.stringify({ error: error.message }));
      }
      return;
    }

    if (request.method === 'GET' && url.pathname === '/summary') {
      response.writeHead(200, { 'content-type': 'application/json' });
      response.end(JSON.stringify(summarizer.summary()));
      return;
    }

    response.writeHead(404, { 'content-type': 'application/json' });
    response.end(JSON.stringify({ error: 'not found' }));
  });
}
```

```js
// server.test.mjs
import { test, before, after, describe } from 'node:test';
import assert from 'node:assert/strict';
import { once } from 'node:events';
import { buildServer } from './build-server.mjs';

describe('measurement service', () => {
  let server;
  let base;

  before(async () => {
    server = buildServer();
    server.listen(0, '127.0.0.1');            // 0: the kernel picks a free port
    await once(server, 'listening');
    base = `http://127.0.0.1:${server.address().port}`;
  });

  after(async () => {
    server.close();
    await once(server, 'close');
  });

  test('posting a measurement returns 201', async () => {
    const response = await fetch(`${base}/measurement`, {
      method: 'POST',
      body: '{"node":"edge-01","metric":"humidity","value":48}\n',
    });
    assert.equal(response.status, 201);
    assert.deepEqual(await response.json(), { received: 1 });
  });

  test('the summary reflects the measurement sent', async () => {
    const response = await fetch(`${base}/summary`);
    assert.equal(response.status, 200);
    assert.deepEqual(await response.json(), [
      { key: 'edge-01/humidity', count: 1, average: 48, max: 48 },
    ]);
  });

  test('an unknown route returns 404', async () => {
    const response = await fetch(`${base}/missing`);
    assert.equal(response.status, 404);
  });
});
```

```sh
node --test server.test.mjs
```

{/* mask: (?<=\()[0-9.]+(?=ms\)) */}
{/* mask: (?<=duration_ms )[0-9.]+ */}
```
▶ measurement service
  ✔ posting a measurement returns 201 (10.196542ms)
  ✔ the summary reflects the measurement sent (1.354708ms)
  ✔ an unknown route returns 404 (0.68975ms)
✔ measurement service (16.061875ms)
ℹ tests 3
ℹ suites 1
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 73.281666
```

Two details make this test reliable.

Zero was given as the port: the kernel picks a free ephemeral port, and the real
number is learned through `address()`. A fixed number would let two tests running
at once collide.

The `after` hook closes the server and waits for the close. A server never closed
keeps the event loop alive, and the test process never ends.

The request during the test was sent with `fetch` — one of the interfaces the host
environment inherited from the web platform. No separate client dependency was
needed.

An end-to-end test goes one step further: it starts the build as a real process,
runs it with real configuration, and sends requests from outside. `spawn` from the
Child Processes lesson does this job. End-to-end tests are slow and brittle; kept
few, limited to critical paths.

#### A Failing Test and Coverage

Seeing what a test prints when it fails is the shortest way to learn how to read
the output. The test below deliberately carries an incomplete expectation:

```js
// failing.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { Summarizer } from './summarizer.mjs';

test('summary fields', () => {
  const s = new Summarizer();
  s.add({ node: 'edge-01', metric: 'humidity', value: 48 });
  assert.deepEqual(s.summary(), [{ key: 'edge-01/humidity', count: 1, average: 48 }]);
});
```

```sh
node --test failing.test.mjs > report.txt; code=$?
head -n 25 report.txt
echo "exit code: $code"
```

{/* mask: (?<=\()[0-9.]+(?=ms\)) */}
{/* mask: (?<=duration_ms )[0-9.]+ */}
```
✖ summary fields (0.978292ms)
ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 33.174375

✖ failing tests:

test at failing.test.mjs:6:1
✖ summary fields (0.978292ms)
  AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
  + actual - expected
  
    [
      {
        average: 48,
        count: 1,
        key: 'edge-01/humidity',
  +     max: 48
      }
    ]
exit code: 1
```

The report was cut to its first 25 lines with `head`; the full output goes on to
print the error object's fields and the call stack. The diff report marks the field present in the actual value but not the expected
one with `+`. A nonzero exit code lets the failure be reported when the test is
called from an automation step.

To see which lines never ran, coverage measurement is turned on:

```sh
node --test --experimental-test-coverage summarizer.test.mjs
```

{/* mask: (?<=\()[0-9.]+(?=ms\)) */}
{/* mask: (?<=duration_ms )[0-9.]+ */}
```
▶ recordFromLine
  ✔ converts a valid line to a record (1.11775ms)
  ✔ throws SyntaxError for broken JSON (0.155333ms)
  ✔ throws TypeError for a non-numeric value (0.074167ms)
✔ recordFromLine (2.002583ms)
▶ Summarizer
  ✔ merges measurements under the same key (0.125584ms)
  ✔ an empty summarizer returns an empty array (0.046625ms)
✔ Summarizer (0.240542ms)
ℹ tests 5
ℹ suites 2
ℹ pass 5
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 34.86225
ℹ start of coverage report
ℹ ---------------------------------------------------------------
ℹ file           | line % | branch % | funcs % | uncovered lines
ℹ ---------------------------------------------------------------
ℹ summarizer.mjs |  93.94 |    90.91 |  100.00 | 5-6
ℹ ---------------------------------------------------------------
ℹ all files      |  93.94 |    90.91 |  100.00 |
ℹ ---------------------------------------------------------------
ℹ end of coverage report
```

The coverage report follows the test results. It says lines 5 and 6 never ran: the branch that runs when `node` is not a
string. Coverage percentage is not a target, it is an indicator — it shows which
behavior was never tested, not that tested behavior was tested correctly.

#### Summary

- A unit test tests a single module in isolation; this isolation requires keeping
  the logic in a module separate from the interface.
- When dependencies (clock, output stream) are passed in from outside, the test
  verifies the full record without touching real time or real output.
- When building the server is separated from starting to listen, the service
  opens up to integration testing; the test asks for port zero and closes the
  server at the end.
- The strict assertion form leaves out the pitfalls of loose equality; throw
  tests also specify the expected error type.
- The coverage report shows which behavior was never tested; it does not show
  that what was tested was tested correctly.

#### Next Step

Tests say expected behavior is preserved; they do not say why unexpected behavior
happens. When the process slows down, memory grows, or it gets stuck somewhere
unexpected, other tools are needed. The next lesson takes up the runtime's
inspector interface, CPU profiling, and heap snapshots.

### Debugging and Profiling

Source: https://academia.sh/en/courses/nodejs/debugging-and-profiling

Opening the inspector protocol, the fields of memory usage, heap snapshots, reading a CPU profile, and in-code duration measurement.

Tests say expected behavior is preserved. When the process slows down, memory
grows, or it gets stuck somewhere unexpected, tests stay silent: these are not
correctness problems, they're behavior problems.

This lesson takes up the tools that make those problems visible, all of them
inside the runtime — no extra component needed.

#### The Inspector Interface

The runtime can open an interface a debugger attaches to. It listens on a port,
and commands for breakpoints, stepping, inspecting variables, and taking profiles
pass through it.

```sh
node --inspect=127.0.0.1:9229 -e "setTimeout(()=>{},200)"
```

{/* mask: (?<=ws://127[.]0[.]0[.]1:9229/)[0-9a-f-]+ */}
```
Debugger listening on ws://127.0.0.1:9229/0dd16a67-741b-482d-86db-82b41709dc92
For help, see: https://nodejs.org/learn/getting-started/debugging
```

The id at the end of the address changes every run; it identifies the session and
makes a random connection harder. The port number can change too.

The `127.0.0.1` part must never be skipped. This interface gives full access to
the process's memory and can run arbitrary code; listening on an address open to
the network means giving up control of the process to the outside. When
diagnostics are needed on a server, the interface opens on the local address, and
the connection tunnels through the secure shell from the Linux Network
Administration and Troubleshooting course.

The `--inspect-brk` flag stops before running the first line — the way to examine
startup problems. A running process can also be attached to later: one that
receives `SIGUSR1` opens the inspector interface.

#### Reading Memory Usage

The `process.memoryUsage()` call returns several separate numbers, and which one
you look at matters:

```sh
node -e "console.log(process.memoryUsage())"
```

{/* mask: (?<=rss: )[0-9]+ */}
{/* mask: (?<=heapTotal: )[0-9]+ */}
{/* mask: (?<=heapUsed: )[0-9]+ */}
{/* mask: (?<=external: )[0-9]+ */}
```
{
  rss: 45137920,
  heapTotal: 5783552,
  heapUsed: 3768872,
  external: 1582649,
  arrayBuffers: 137503
}
```

The numbers are in bytes and change every run; their order of magnitude is what
matters.

- `rss` is the total space the process holds in physical memory, including the
  runtime's own code and the stack.
- `heapTotal` is the heap space allocated for JavaScript objects.
- `heapUsed` is the portion actually in use — the number to watch for a leak.
- `external` is the memory usage of C++ objects bound to JavaScript objects.
  `arrayBuffers` is a subset of it, counting memory for `ArrayBuffer`s and buffers.

The difference shows with a deliberately growing array:

```js
// leak.mjs
import { mkdirSync } from 'node:fs';
import v8 from 'node:v8';

mkdirSync('profile', { recursive: true });

const history = [];

function addMeasurement(record) {
  history.push(record);            // never cleared
}

function heapMB() {
  return (process.memoryUsage().heapUsed / 1024 / 1024).toFixed(0);
}

console.log('heap used at start (MB)        :', heapMB());
for (let i = 0; i < 500000; i += 1) {
  addMeasurement({ node: `edge-${i % 3}`, metric: 'temperature', value: i % 40 });
}
console.log('after 500 thousand records (MB):', heapMB());

const path = v8.writeHeapSnapshot('profile/measurements.heapsnapshot');
console.log('snapshot                       :', path.split('/').pop());
```

```sh
node leak.mjs
```

```
heap used at start (MB)        : 4
after 500 thousand records (MB): 48
snapshot                       : measurements.heapsnapshot
```

The megabyte values shift a few units depending on runtime version and the
garbage collector's state; what matters is the size of the increase.

This is a real risk for the measurement collector: appending every incoming
record to an array grows memory in direct proportion to input volume. This is
why the summarizer's bucket map stores no records, only a count and a total —
memory is bounded by the number of distinct keys.

#### Heap Snapshot

`writeHeapSnapshot` writes every object at that moment and the references between
them to a file. The file is a graph: every node is an object, every edge a
reference — a direct application of the graph representations from the Data
Structures course.

A single snapshot says little on its own. The method is **comparing two
snapshots**: one before the load, one after, checking which object type's count
grew in between. A type that keeps growing and never shrinks leads to the source
of the leak.

Taking a snapshot stops the process and produces a large file; in the example
above, tens of megabytes. Taken in production, it is done accepting that requests
will be delayed, with disk space checked beforehand.

The same signal mechanism works here too: the process can be written to take a
snapshot on a predetermined signal, so one is taken the moment a problem shows
up, without restarting the process.

#### CPU Profile

A CPU profile measures which function the process spends its time in by
sampling: the call stack's snapshot is taken at fixed intervals, and if a
function shows up in most samples, it is spending most of the time there.

Profiling turns on with a flag and writes to a file when the process ends.
The expensive computation from the Cluster and Worker Threads lesson is
profiled below; if the main-thread version is not in the working directory, write
it first:

```js
// no-worker.mjs
const ITERATIONS = 3e8;

function heavySum(iterations) {
  let total = 0;
  for (let i = 0; i < iterations; i += 1) total += Math.sqrt(i) % 1;
  return total;
}

setTimeout(() => console.log('3 timer only ran now'), 0);
console.log('1 computation starting');
console.log('2 result:', Number(heavySum(ITERATIONS).toFixed(3)));
```

```sh
node --cpu-prof --cpu-prof-dir=cpu-profile no-worker.mjs
```

The produced file is in JSON format and can be read directly:

```js
// read-profile.mjs
import { readdirSync, readFileSync } from 'node:fs';
import path from 'node:path';

const dir = process.argv[2] ?? 'cpu-profile';
const file = readdirSync(dir).find((f) => f.endsWith('.cpuprofile'));
const profile = JSON.parse(readFileSync(path.join(dir, file), 'utf8'));

const frames = new Map(profile.nodes.map((d) => [d.id, d.callFrame]));
const counts = new Map();
for (const id of profile.samples) counts.set(id, (counts.get(id) ?? 0) + 1);

const total = profile.samples.length;
console.log(`file: ${file.replace(/\d{8}\.\d{6}\.\d+/, 'DATE.TIME.PID')}`);
console.log(`sample count: ${total}`);
for (const [id, count] of [...counts].sort((a, b) => b[1] - a[1]).slice(0, 1)) {
  const c = frames.get(id);
  const name = c.functionName || '(anonymous)';
  const location = c.url ? `${path.basename(c.url)}:${c.lineNumber + 1}` : '(internal)';
  console.log(`${((count / total) * 100).toFixed(0).padStart(3)}%  ${name.padEnd(14)} ${location}`);
}
```

```sh
node read-profile.mjs cpu-profile
```

{/* mask: (?<=sample count: )[0-9]+ */}
{/* mask: ^ *(9[0-9]|100)(?=%  heavySum) */}
```
file: CPU.DATE.TIME.PID.0.001.cpuprofile
sample count: 1829
100%  heavySum       no-worker.mjs:4
```

The file name carries the date, time, and process id, masked here for
readability. The script prints only the function with the most samples. The sample count changes
run to run, and the percentage can shift a unit or two. Fixed is the result: nearly all the time was spent in one function.

Reading a profile has two rules. First, many lines with a low percentage each
means the bottleneck is not in one place, and a single optimization will not change
the result. Second, sampling only sees the running CPU: time spent waiting for
input/output never shows up. If a request's slowness comes from waiting, the CPU
profile comes back empty, and duration has to be measured some other way.

#### In-Code Duration Measurement

For a measurement that also covers waiting, `node:perf_hooks` is used: marks are
set, intervals are measured, and an observer gathers the measurements.

The experiment needs the summarizer module and the measurement file from the
Command-Line Applications lesson; if they are not in the working directory, write
them first:

```js
// summarizer.mjs
export function recordFromLine(line) {
  const record = JSON.parse(line);
  if (typeof record.node !== 'string' || typeof record.metric !== 'string') {
    throw new TypeError('node and metric must be strings');
  }
  if (!Number.isFinite(record.value)) {
    throw new TypeError('value must be a finite number');
  }
  return record;
}

export class Summarizer {
  #buckets = new Map();

  add({ node, metric, value }) {
    const key = `${node}/${metric}`;
    const bucket = this.#buckets.get(key) ?? { count: 0, total: 0, max: -Infinity };
    bucket.count += 1;
    bucket.total += value;
    bucket.max = Math.max(bucket.max, value);
    this.#buckets.set(key, bucket);
  }

  summary() {
    return [...this.#buckets].map(([key, { count, total, max }]) => ({
      key,
      count,
      average: Number((total / count).toFixed(2)),
      max,
    }));
  }
}
```

```sh
cat > measurements.ndjson <<'SON'
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"temperature","value":21.4}
{"time":"2024-02-07T09:12:44Z","node":"edge-01","metric":"humidity","value":48.0}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"temperature","value":19.8}
{"time":"2024-02-07T09:13:02Z","node":"edge-02","metric":"humidity","value":52.5}
{"time":"2024-02-07T09:14:19Z","node":"edge-01","metric":"temperature","value":21.9}
{"time":"2024-02-07T09:14:19Z","node":"edge-03","metric":"temperature","value":24.1}
{"time":"2024-02-07T09:15:30Z","node":"edge-02","metric":"temperature","value":20.2}
{"time":"2024-02-07T09:15:30Z","node":"edge-03","metric":"humidity","value":41.3}
{"time":"2024-02-07T09:16:04Z","node":"edge-01","metric":"humidity","value":47.2}
{"time":"2024-02-07T09:16:04Z","node":"edge-03","metric":"temperature","value":24.6}
{"time":"2024-02-07T09:17:22Z","node":"edge-02","metric":"temperature","value":20.7}
{"time":"2024-02-07T09:17:22Z","node":"edge-01","metric":"temperature","value":22.3}
SON
```

```js
// measure-duration.mjs
import { performance, PerformanceObserver } from 'node:perf_hooks';
import { readFile } from 'node:fs/promises';
import { Summarizer, recordFromLine } from './summarizer.mjs';

const observer = new PerformanceObserver((list) => {
  for (const entry of list.getEntries()) {
    console.log(`${entry.name.padEnd(12)} ${entry.duration.toFixed(2)} ms`);
  }
});
observer.observe({ entryTypes: ['measure'] });

performance.mark('read-start');
const text = await readFile('measurements.ndjson', 'utf8');
performance.mark('read-end');

const summarizer = new Summarizer();
for (const line of text.split('\n')) {
  if (line.trim() === '') continue;
  summarizer.add(recordFromLine(line));
}
performance.mark('summary-end');

const read = performance.measure('read', 'read-start', 'read-end');
const summarize = performance.measure('summarize', 'read-end', 'summary-end');
console.log('read > summarize:', read.duration > summarize.duration);
```

```sh
node measure-duration.mjs
```

{/* mask: (?<=[ ])[0-9]+[.][0-9]+(?=[ ]ms) */}
```
read > summarize: true
read         0.82 ms
summarize    0.13 ms
```

Durations change every run; a measurement is not taken once and decided on, it is
repeated and its distribution examined. What matters is the comparison between the
two stages; the script measures it itself on every run and prints it on the first
line. In this small file, reading takes longer than summarizing.

Measurement results can be added as a field to the log record from the Logging
lesson. Once records carrying a duration field pile up, which stage a slowdown
comes from shows without taking a profile — why writing a duration field to
request logs is a common habit.

#### Summary

- The inspector interface gives full access to the process's memory; it opens
  on the local address, and remote access is done through tunneling.
- Memory fields carry separate meanings; the used-heap field is watched when
  looking for a leak, the outside-heap fields for binary data.
- A heap snapshot writes objects and references as a graph; diagnosis is done
  by comparing two snapshots.
- A CPU profile works by sampling and sees only the running CPU; waiting time
  does not show up in it.
- Marks and measures cover waiting too; writing a duration field to the log
  shows the source of a slowdown without taking a profile.

#### Next Step

So far, the process has always started by hand and stopped by closing the
terminal. In production, a supervisor starts the process, reopens it when it
crashes, and sends a signal when it needs to shut down. The next lesson builds
the process side of that contract: catching signals, finishing running requests
without taking new connections, and reporting the service's state to the
supervisor.

### Process Management and Resilience

Source: https://academia.sh/en/courses/nodejs/process-management-and-resilience

Catching signals, the phases of graceful shutdown, the distinction between liveness and readiness probes, the restart contract, and bounding shutdown duration.

So far, the process has always started by hand and stopped by closing the
terminal. In production this job belongs to a supervisor: it starts the process,
reopens it when it crashes, stops it during an update.

There's a contract between supervisor and process, and this lesson covers the
process's side of it: catching the shutdown request, finishing running work
without taking new work, and reporting its own state.

#### A Shutdown Request Is a Signal

Signals and traps were covered in the Shell Programming course; the same
mechanism appears here as the process object's events.

The supervisor's shutdown request is the `SIGTERM` signal. Pressing the interrupt
key in a terminal produces `SIGINT`. Both default to ending the process
immediately; once a listener attaches, that default is disabled and the
decision passes to the program.

There's also a signal that cannot be caught. The supervisor uses it if the
process does not shut down within the allowed time after the polite request, and
the process ends without any cleanup. This is why the shutdown needs its own
time limit: one that does not finish before the supervisor's limit counts as
never having happened.

#### The Phases of Graceful Shutdown

**Graceful shutdown** is the process stopping taking new work and ending by
finishing the work it already has. It has four phases, and their order matters.

```js
// resilient-server.mjs
import { createServer } from 'node:http';

const EXIT_DELAY_MS = 500;      // time to tell the supervisor "no longer ready"
const SHUTDOWN_LIMIT_MS = 3000; // longest time given to pending requests

let ready = false;
let shuttingDown = false;

const server = createServer((request, response) => {
  if (request.url === '/live') {                 // is the process up
    response.writeHead(200).end('alive\n');
    return;
  }
  if (request.url === '/ready') {                 // ready to accept requests
    const code = ready && !shuttingDown ? 200 : 503;
    response.writeHead(code).end(code === 200 ? 'ready\n' : 'not ready\n');
    return;
  }
  if (request.url === '/slow') {                 // simulates a pending request
    setTimeout(() => response.writeHead(200).end('done\n'), 800);
    return;
  }
  response.writeHead(404).end('not found\n');
});

server.listen(8791, '127.0.0.1', () => {
  ready = true;
  console.log(`listening, process id ${process.pid}`);
});

function shutdown(signal) {
  if (shuttingDown) return;
  shuttingDown = true;
  console.log(`${signal} received; readiness probe now returns 503`);

  setTimeout(() => {
    console.log('no longer accepting new connections');

    const forced = setTimeout(() => {
      console.log('time is up; cutting remaining connections');
      server.closeAllConnections();
    }, SHUTDOWN_LIMIT_MS);
    forced.unref();                           // must not keep the process alive on its own

    server.close(() => {
      clearTimeout(forced);
      console.log('all requests completed; exiting');
    });
    server.closeIdleConnections();
  }, EXIT_DELAY_MS).unref();
}

process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
```

The trial is run with a script:

```sh
#!/bin/sh
# shutdown-test.sh
node resilient-server.mjs &
serverPid=$!
sleep 1

printf 'before shutdown /ready  -> '; curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8791/ready

# Start a pending request, send the signal while waiting for its response
curl -s http://127.0.0.1:8791/slow > slow-response.txt &
requestPid=$!
sleep 0.1
kill -TERM "$serverPid"

sleep 0.2
printf 'during shutdown /ready -> '; curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8791/ready

wait "$requestPid"
echo "response of the pending request: $(cat slow-response.txt)"
wait "$serverPid"
echo "process exit code: $?"
```

```sh
sh shutdown-test.sh
```

{/* mask: (?<=process id )[0-9]+ */}
```
listening, process id 18312
before shutdown /ready  -> 200
SIGTERM received; readiness probe now returns 503
during shutdown /ready -> 503
no longer accepting new connections
all requests completed; exiting
response of the pending request: done
process exit code: 0
```

The process id changes on every run. Here's the order the output tells.

**Phase one: report not ready.** The moment the signal arrives, the readiness
probe starts returning 503. A load balancer in front of the process needs a
window to take it out of the list; without it, the balancer keeps sending
requests to a process that is shutting down, and those requests get dropped.

**Phase two: stop taking new connections.** When the delay expires, `close`
gets called. This closes the listening socket but does not drop existing
connections. Idle persistent connections are closed separately; otherwise a
client holding a connection open without sending a request would stretch out
the shutdown.

**Phase three: finish running work.** The `close` callback runs once the last
request completes too. In the output above, the pending request gets its `done`
response — the signal had been sent while that request was being processed.

**Phase four: the time limit.** If a client never releases its connection, the
shutdown runs forever. The forcing timer cuts that off. The `unref` call
matters: this timer must not be the sole reason the process stays alive, or it
waits until the limit runs out even after everything else is done.

The exit code is zero. To the supervisor, this means "shut down on request,"
enabling different behavior depending on the restart policy.

There's other work to do at shutdown too: flushing pending log lines, closing
open files, writing the remaining summary to disk — all inside the `close`
callback, before the process ends. As seen in the Process Object lesson,
asynchronous work cannot be done in the `exit` hook.

#### Liveness and Readiness

The two probes answer two separate questions, and mixing them up destabilizes
the service.

The **liveness** question: is the process alive, or should it be killed and
restarted? Its answer looks only at the process's own state; it does not check
its dependencies.

The **readiness** question: can this process accept requests right now? Its
answer looks at transient conditions — has startup finished, has shutdown
begun.

The reasoning: when a dependency is temporarily unreachable, the process is not
ready but it is alive. If the liveness probe checked the dependency, a brief
outage there would restart every process — behavior that both stretches out
the outage and produces a failure of its own.

Probes have to be lightweight: if every check runs a database query, the check
itself turns into load.

#### The Restart Contract

Restarting when the process dies is not the process's job; it cannot restart
itself. That job belongs to the operating system's service supervisor, or to the
primary process from the Cluster and Worker Threads lesson.

Three things fall to the process.

**Giving a meaningful exit code.** Zero means a requested shutdown, nonzero
means a failure. This is why a separate code was chosen for a configuration
error in the Configuration Management lesson: the supervisor can stop instead
of endlessly retrying a failure that clearly will not fix itself.

**Starting up fast and idempotently.** If restarting is a recovery method,
startup itself has to be reliable. Work done at startup must not fail when it
finds traces of a previous run left half-finished — the
write-to-temporary-name-and-replace pattern from the File System lesson
provides that guarantee.

**Accounting for the restart storm.** Reopening a process that crashes
immediately produces hundreds of startups a second and keeps the machine busy.
Supervisors prevent this with increasing delay; the process must not shorten
that delay, for instance by reporting a fake success while unhealthy.

A process trying to restart itself is a separate mistake: the way out of a
corrupted state is abandoning the memory carrying that state — also why
restarting works at all, the reasoning behind limiting the last-resort hook in
the Error Handling Strategy lesson to "record and exit."

#### Summary

- A shutdown request is a signal; once a listener is attached, the default
  termination behavior gives way to the program's decision.
- Graceful shutdown has four phases: report not ready, stop taking new
  connections, finish running work, cut what's left at the time limit.
- The forcing timer is marked with `unref`; otherwise the process stays alive
  until the limit runs out even after everything else is done.
- The liveness probe looks only at the process's own state, the readiness
  probe at transient state; a liveness probe checking a dependency spreads
  outages.
- Restarting is the supervisor's job; what falls to the process is giving a
  meaningful exit code, starting up fast and idempotently, and not feeding the
  storm.

#### Next Step

The measurement collector is up and running: configured, classifying its
errors, writing its log in structured form, tested, observable, and shutting
down cleanly. One question remains — how does its reusable part get carried
into other projects? The final lesson takes up the packaging contract:
declaring entry points, hiding internal files, and the rules of publishing a
version.

### Package Publishing

Source: https://academia.sh/en/courses/nodejs/package-publishing

The export contract of the package manifest, conditional entries, hiding internal paths, the executable entry point, and pre-publish checks.

The previous lesson made the measurement collector run and shut down cleanly. One
question remains: how does the reusable part of this code — the summarizer, the
log, the error classes — get carried into other projects?

The answer is wrapping the code in a package contract. The Modules, Tooling and
the Ecosystem course explained packages from the consuming side; this lesson
moves to the publishing side.

#### The Package Manifest Is a Contract

The manifest file does more than carry the package's name and version: it
defines what the package promises to the outside. The Module Resolution lesson
showed this map's effect from the consumer's side; here, the author's decisions
are taken up.

```
package/
  package.json
  src/
    index.mjs
    index.cjs
    summary.mjs
    internal-format.mjs
```

The manifest file sits at the package root; the files are written to this layout below:

```sh
mkdir -p package/src
cat > package/package.json <<'SON'
{
  "name": "measurement-core",
  "version": "1.2.0",
  "type": "module",
  "engines": { "node": ">=20" },
  "exports": {
    ".": {
      "import": "./src/index.mjs",
      "require": "./src/index.cjs"
    },
    "./summary": "./src/summary.mjs"
  },
  "imports": {
    "#internal/format": "./src/internal-format.mjs"
  },
  "files": ["src/"]
}
SON
```

```js
// package/src/internal-format.mjs
export const format = (n) => Number(n.toFixed(2));
```

```js
// package/src/summary.mjs
import { format } from '#internal/format';
export const average = (values) => format(values.reduce((t, v) => t + v, 0) / values.length);
```

```js
// package/src/index.mjs
export const entry = 'ES module entry';
```

```js
// package/src/index.cjs
module.exports = { entry: 'CommonJS entry' };
```

Each field makes a separate promise.

`exports` fixes which paths can be imported from outside. A file not in the map
is unreachable, even if it is right where it is. This gives the freedom to
reorganize the package's internal layout: as long as the map stays the same,
files can move.

`imports` are the package's own internal aliases. Names starting with `#`
resolve only from the package's own files and are invisible from outside — a
way to shorten long relative paths that carries no risk of opening them to the
outside.

`type` decides which module form files with a `.js` extension are interpreted
as. This field is read from the package's own manifest; the manifest of
whoever uses it has no effect.

`engines` declares which runtime versions the package is tested against. The
runtime does not enforce this field; the package manager warns or refuses at
install time. The field is not documentation, it is a promise: if an interface
used is present from a certain version onward, the lower bound is written to
match.

`files` decides which paths go into the published archive. Tests, examples,
and development configuration are left out of the archive; the user never
downloads them.

#### Conditional Entries

The root entry has two counterparts: one file resolves when reached through
`import`, another when reached through `require`. This makes the same package
usable from either module form.

A directory that consumes the package is set up next. A package manager places
a local package in the dependency directory as a link; the same layout can be
set up by hand:

```sh
mkdir -p package/consumer/node_modules
ln -s ../.. package/consumer/node_modules/measurement-core
printf '{ "name": "consumer", "type": "module" }\n' > package/consumer/package.json
```

```js
// package/consumer/try.mjs
import { entry } from 'measurement-core';
import { average } from 'measurement-core/summary';
import { createRequire } from 'node:module';

const require = createRequire(import.meta.url);

console.log('import condition :', entry);
console.log('require condition:', require('measurement-core').entry);
console.log('subpath          :', average([21.4, 21.9, 22.3]));
```

```sh
node package/consumer/try.mjs
```

```
import condition : ES module entry
require condition: CommonJS entry
subpath          : 21.87
```

The same name resolved to two different files. The third line shows the
subpath working, and that module using another file through its own internal
alias.

Offering both forms together has one trap: a module can be evaluated twice in
the same process; as explained in the Module Resolution lesson, singularity
depends on the resolved address, and two files are two separate addresses. If
the package holds state — a counter, a registry — two copies mean two separate
states. For packages that hold state, offering a single form is safer.

Paths not in the map are closed to access:

```sh
(cd package/consumer && node -e "import('measurement-core/src/internal-format.mjs').catch(e => console.log(e.code))")
```

```
ERR_PACKAGE_PATH_NOT_EXPORTED
```

Internal aliases cannot be resolved from outside either:

```sh
(cd package/consumer && node -e "import('#internal/format').catch(e => console.log(e.code))")
```

```
ERR_PACKAGE_IMPORT_NOT_DEFINED
```

The second error says the alias is not defined in the consuming package's own
manifest. Names starting with `#` are defined separately for each package;
another package's internal names cannot be borrowed.

#### Executable Entry

The tool from the Command-Line Applications lesson can be placed in a package.
The `bin` field in the manifest says which name links to which file when the
package is installed:

```json
{
  "name": "measurement-tools",
  "version": "1.0.0",
  "type": "module",
  "bin": { "measurement-collect": "./src/collect.mjs" },
  "files": ["src/"]
}
```

Two conditions are required. The file's first line has to carry the
interpreter declaration — the `#!/usr/bin/env node` line tells the kernel
which program runs the file. Second, the file has to have execute permission;
the package manager usually sets this at install time, but committing it to
the repository with the right permission is safer.

Combining the library and the tool in the same package is a choice. Separate
packages keep those using only the library from downloading the tool's
dependencies; a single package guarantees version compatibility on its own.

#### Pre-Publish Checks

Publishing is an irreversible operation: a version number gets published once,
and its content cannot be changed. This is why four checks happen before
publishing.

**The archive's contents.** A wrongly written `files` field fails in two
directions: a required file is left out and the package does not work once
installed, or unneeded files get in and the archive bloats. Package managers
offer a dry-run mode that lists the archive's contents without producing it;
this list is read before publishing.

**That entry points work.** Setting up a small directory that consumes the
package — like above — and importing every entry catches a typo in the map
before publishing. A published package with a broken entry stays in users'
caches even if the version is pulled.

**The meaning of the version number.** Semantic versioning rules were
explained in the Modules, Tooling and the Ecosystem course. For the publisher,
the rule reduces to one question: does a user's code break by moving to this
version? If it does, the major version increments. Removing an entry from the
exports map, changing a field's type, and raising the minimum supported
runtime version are all breaking changes.

**Secret leaks.** The env file from the Configuration Management lesson must
never enter the archive. The ignore list from the Introduction to Version
Control course keeps that file out of the repository; the `files` field keeps
it out of the archive. The two checks do not substitute for each other — a file
absent from the repository can still sit in the local working directory and
get into the archive.

The publish action itself is a package manager command and requires access to
a registry account. Protecting that access with a second factor, limiting
publish rights to the smallest possible set of people, and doing the publish
from an automation step triggered by a version tag rather than by hand — these
are common practices.

#### Summary

- The package manifest defines the promise the package makes outward; the
  exports map fixes the public interface and gives freedom to reorganize the
  internal layout.
- Conditional entries make the same package usable from both module forms;
  for packages that hold state, two copies mean two separate states.
- Internal aliases starting with `#` resolve only from their own package;
  they cannot be reached from another package.
- An executable entry depends on three conditions: a name-to-file mapping,
  the interpreter declaration, and execute permission.
- Publishing cannot be undone; archive contents, entry points, the meaning of
  the version number, and secret leaks are all checked before publishing.

#### Course Wrap-Up

The course began with a single question: when the same language runs in a
server process instead of a browser, what changes? The answer was built in
three layers.

**The runtime model.** The host environment concept separated the language
core from access to the outside world. The event loop's phases were measured;
timer delay was shown to be a lower bound, and blocking work was shown to
extend that bound by its own duration. The process object put arguments,
environment, streams, and exit code into the program's hands. Module
resolution settled which file a name corresponds to.

**Built-in modules.** The file system was taken up through its three
interfaces, and the choice between them was tied back to the event loop.
Paths were treated as data; a check was written to prevent escaping the root
directory. Streams were built on the observation that a chunk boundary
differs from a record boundary, and backpressure was measured: a producer
ignoring the return value ran the buffer up to fifty records, while one that
listened never exceeded four. Buffers brought raw bytes, the event emitter
the observer pattern, the HTTP server the streaming nature of request and
response. Child processes and multi-core use separated two ways of moving
work outside the process.

**Writing applications.** The command-line interface, configuration, error
classification, structured logging, testing, profiling, graceful shutdown,
and packaging — all built on one concrete piece of work. The measurement
collector started as a script reading a twelve-line file and turned into a
service that reads through a stream, serves over HTTP, is tested, is
observable, and shuts down on signal by finishing its running requests.

This is the last course in the JavaScript and TypeScript curriculum. Two
directions open up from here.

The **Frontend Development** curriculum returns to the same language's other
host environment: document structure, rendering strategies, component
architecture, accessibility, and performance. The module resolution,
packaging, and event model learned in this course are the foundation of the
build tools there.

The **Backend Development** curriculum continues from where this course
leaves off: the request dispatching, configuration, logging, and error layer
built by hand here get taken up there as an application architecture, with
layers for API contracts, data access, authentication, asynchronous
processing, and observability added on top. The measurement collector's
`/summary` endpoint is the first example of a resource model whose design
gets justified there.

In both directions, the model built in this course stays valid: an
event-driven, single-threaded runtime that works with streams. Frameworks and
libraries hide this model, they do not change it — and a hidden model's
failure can only be solved by someone who knows the model.