Path Operations

Every filename in the previous lesson was written bare: measurements.ndjson, data/2024-02-07. These names are resolved against the working directory, so the same program cannot find the file when invoked from a different directory. Writing the separator character by hand also ties the tool to a single platform’s path rules.

This lesson introduces the built-in module that treats paths as data, and answers two questions: where should the program look for its own files, and how far should a path piece coming from outside be allowed to go?

Joining and Resolving

The node:path module has two core functions, and the difference between them is frequently confused.

join appends the given parts to each other with the platform’s separator and normalizes the result; if given a relative result, it stays relative. resolve instead walks right to left and produces an absolute path; if no part is absolute, it prepends the working directory.

// paths.mjs
import path from 'node:path';

console.log('join      :', path.join('data', '2024-02-07', 'measurements.ndjson'));
console.log('join + .. :', path.join('data', '2024-02-07', '..', 'summary.json'));
console.log('sep       :', JSON.stringify(path.sep));
console.log();

const filePath = 'data/2024-02-07/measurements.ndjson';
console.log('dirname   :', path.dirname(filePath));
console.log('basename  :', path.basename(filePath));
console.log('no ext    :', path.basename(filePath, path.extname(filePath)));
console.log('extname   :', path.extname(filePath));
console.log('parse     :', path.parse(filePath));

node paths.mjs

join      : data/2024-02-07/measurements.ndjson
join + .. : data/summary.json
sep       : "/"

dirname   : data/2024-02-07
basename  : measurements.ndjson
no ext    : measurements
extname   : .ndjson
parse     : {
  root: '',
  dir: 'data/2024-02-07',
  base: 'measurements.ndjson',
  ext: '.ndjson',
  name: 'measurements'
}

The second line shows normalization: a .. part means going up one directory and disappears in the result. The sep value varies by the platform you run on; the forward slash in the example was taken on a Unix derivative.

None of these functions look at the file system. extname does not say a file genuinely is JSON, it says how its name ends; dirname does not say that directory exists. Path operations are pure string operations — they fit the pure function definition from the Programming Fundamentals course, and this is why they are easy to test.

Platform Rules Are Offered Separately

The module applies the rules of the platform it runs on by default, but keeps both rule sets separately accessible too.

// platforms.mjs
import path from 'node:path';

console.log('this platform :', path.join('data', '2024-02-07', 'summary.json'));
console.log('posix         :', path.posix.join('data', '2024-02-07', 'summary.json'));
console.log('win32         :', path.win32.join('data', '2024-02-07', 'summary.json'));
console.log();
console.log('is "C:\\data" absolute by win32 rules:', path.win32.isAbsolute('C:\\data'));
console.log('is "C:\\data" absolute by posix rules:', path.posix.isAbsolute('C:\\data'));
console.log('is "/data"   absolute by posix rules:', path.posix.isAbsolute('/data'));

node platforms.mjs

this platform : data/2024-02-07/summary.json
posix         : data/2024-02-07/summary.json
win32         : data\2024-02-07\summary.json

is "C:\data" absolute by win32 rules: true
is "C:\data" absolute by posix rules: false
is "/data"   absolute by posix rules: true

The first line reflects the rules of whatever platform the example runs on; on a Windows machine it would print the backslash form. The existence of separate rule sets matters for work unrelated to local file paths: parsing an archive entry or an HTTP path should give the same result on every platform, and path.posix is chosen directly for that.

For address paths there is also the URL object, and the two domains should not be mixed up. Character escaping in a request path follows rules the path functions know nothing about; on the server side, path parsing is done with the address object.

Deriving One Path From Another

A frequent job for the measurement collector is producing an output filename from an input filename: measurements.ndjson is read, measurements-summary.json is written next to it. Done through string concatenation, this work rests on an assumption about how many characters the extension is, and breaks between .ndjson and .json.

parse splits a path into its fields, format builds a path from fields. Used together, the derivation is done without counting characters — but the format call has a priority:

// derive.mjs
import path from 'node:path';

const source = 'data/2024-02-07/measurements.ndjson';
const parts = path.parse(source);

// if base is present it overrides ext and name; base is dropped to derive correctly
const destination = path.format({ dir: parts.dir, name: `${parts.name}-summary`, ext: '.json' });
const wrong = path.format({ ...parts, name: `${parts.name}-summary`, ext: '.json' });

console.log('source     :', source);
console.log('destination:', destination);
console.log('base kept  :', wrong);
console.log();

console.log('relative (neighbor):', path.relative('data/2024-02-07', 'data/2024-02-08/measurements.ndjson'));
console.log('relative (same)    :', JSON.stringify(path.relative('data', 'data')));
console.log('relative (outside) :', path.relative('data', 'secret.txt'));

node derive.mjs

source     : data/2024-02-07/measurements.ndjson
destination: data/2024-02-07/measurements-summary.json
base kept  : data/2024-02-07/measurements.ndjson

relative (neighbor): ../2024-02-08/measurements.ndjson
relative (same)    : ""
relative (outside) : ../secret.txt

The third line shows the trap. format ignores the name and ext fields if the given object has a base field; because parse’s output carries a base field, directly spreading it returns the unchanged path unmodified. While deriving, base is either omitted or recomputed.

The output’s second block introduces the relative function: given two paths, it gives the relative step to walk from the first to the second. Three behaviors each mean something separate. The path to a neighboring directory goes up a level and back down. If the two paths are the same, the result is an empty string — not a child item, the path itself is meant. If the target falls outside the base, the result starts with ...

The last behavior is the second notation for the root check in the next section: if the result of path.relative(root, full) does not start with .., the target is under the root. This form does not require adding a separator, but on a case-insensitive file system both forms hit the same limit: string comparison does not know the file system’s name-matching rule.

The Script’s Own Location

A program finding files next to itself should not depend on the working directory. In ES modules the anchor is the import.meta.url value; this is a file address and has to be converted to a file system path.

// safety.mjs
import path from 'node:path';
import { fileURLToPath } from 'node:url';

// The script's own location: a fixed anchor independent of the working directory
const thisFile = fileURLToPath(import.meta.url);
const thisDir = path.dirname(thisFile);
console.log('script directory (last part):', path.basename(thisDir));

const root = path.resolve(thisDir, 'data');

function safeResolve(userPath) {
  const full = path.resolve(root, userPath);
  if (full !== root && !full.startsWith(root + path.sep)) {
    throw new Error(`outside root directory: ${userPath}`);
  }
  return path.relative(root, full);
}

for (const request of ['2024-02-07/measurements.ndjson', './a/../b.json', '../secret.txt']) {
  try {
    console.log(`${request.padEnd(28)} -> ${safeResolve(request)}`);
  } catch (error) {
    console.log(`${request.padEnd(28)} -> REJECTED: ${error.message}`);
  }
}

The same script gives the same result when invoked from two different directories:

node safety.mjs

script directory (last part): lab
2024-02-07/measurements.ndjson -> 2024-02-07/measurements.ndjson
./a/../b.json                -> b.json
../secret.txt                -> REJECTED: outside root directory: ../secret.txt

The directory name on the first line depends on which directory you saved the script in; in the example it was taken from a directory named lab. What matters is that this value does not change whether you invoke the command from the root directory or somewhere else.

The address-to-path conversion cannot be done by hand. A space in a file address is encoded as %20, and on Windows a drive letter follows an extra rule; fileURLToPath takes care of these details. In the reverse direction, converting a path to an address uses pathToFileURL from the same module.

Preventing an Escape Outside the Root Directory

The safeResolve function above already solves a problem the measurement collector will run into later. If a service is going to open a file using a name coming from a request path, it has to be verified that name stays under the designated root directory. Without verification, a request containing a ../ sequence could reach files outside the root.

Verification has three steps, and their order matters:

1. Convert the root directory to an absolute path.
2. Resolve the user path against the root. resolve applies the .. parts at this step.
3. Verify the result equals the root directory, or starts with the root plus a separator.

The separator detail in the third step looks unnecessary but is not. If only full.startsWith(root) is checked, a neighboring directory named data-backup also passes the check for root data; adding the separator closes this leak.

This check operates at the string level and does not account for symbolic links. If there is a symbolic link inside the root directory pointing outside, the resolved path can appear to be under the root while the file that opens is actually outside. The symbolic link behavior introduced in the Introduction to Linux course comes into play here; a realpath call that asks the file system for the real path is used, and the check is repeated on its result.

Summary

• join combines and normalizes parts while preserving relativity; resolve walks right to left to produce an absolute path.
• Path functions do not look at the file system; they speak about the shape of the name, not the file’s existence.
• The module offers both platforms’ rules separately; the rule set is chosen explicitly for paths outside the local file system.
• With parse and format, one path is derived from another without counting characters; format ignores name and ext if the object has a base field.
• In an ES module, the script’s own location is obtained from the import.meta.url address with fileURLToPath; this anchor is independent of the working directory.
• A path coming from outside is verified by resolving it against the absolute root and testing the root-plus-separator prefix; for symbolic links, the real path is also asked for.

Next Step

Up to here, files were loaded into memory all at once. This is fine for a twelve-line file; it is not for a measurement file accumulating over days. The next lesson builds the stream abstraction that moves data piece by piece: readable, writable, and transform streams, the correct way to connect them, and the backpressure that kicks in when the producer is faster than the consumer.