Lesson 13 / 16

Static File Serving

How a file on disk turns into an HTTP response: the document root and path resolution, content type and length, the conditional request enabled by an entity tag, the two classes of cache directive, and the decision to hand the load off to a content delivery network.

Contents

The previous topic built the application’s skeleton: configuration is read from the environment, logs are written, errors turn into a common response shape. The skeleton stands but gives nothing back yet. The library lending service’s first thing to give back is also its simplest: a file sitting on disk. The catalog page’s document, its stylesheet, a script.

This lesson establishes what it takes to serve that file over the network. Reading a file and writing it to a response is the small part of the job; the real decisions are which file is allowed to be read, how the recipient will interpret the content, what happens when the same file is requested a second time, and whether this load should stay on the server at all.

Document Root and Path Resolution

Servable files are collected under a single directory. This directory is called the document root, and it is the only region of the file system the server is authorized to reach. Joining the request path with the document root is called path resolution.

Path resolution alone is not enough: the request path can carry components that point outside the document root. Whether the resulting absolute path still falls under the document root must be checked after the join, because joining normalizes the path and a check performed before it would be misleading.

// server.mjs — file server for the library catalog's static assets
import { createServer } from "node:http";
import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises";
import { extname, join, sep } from "node:path";

const ROOT = join(process.cwd(), "public");

const TYPES = {
  ".html": "text/html; charset=utf-8",
  ".css": "text/css; charset=utf-8",
  ".js": "text/javascript; charset=utf-8",
  ".svg": "image/svg+xml",
  ".json": "application/json; charset=utf-8",
};

// A name carrying a content hash (catalog.7f3a91c2.css) can be cached forever;
// the document must be revalidated on every request.
const HASHED = /\.[0-9a-f]{8}\.(css|js)$/;
const cacheControl = (path) =>
  HASHED.test(path) ? "public, max-age=31536000, immutable" : "no-cache";

// join() concatenates and normalizes the path; the request is rejected if the
// result no longer falls under the root.
const safePath = (requestPath) => {
  const full = join(ROOT, decodeURIComponent(requestPath.split("?")[0]));
  return full === ROOT || full.startsWith(ROOT + sep) ? full : null;
};

createServer(async (request, response) => {
  response.sendDate = false;
  const path = safePath(request.url === "/" ? "/index.html" : request.url);
  if (!path) return response.writeHead(403).end();

  let body;
  try {
    body = await readFile(path);
  } catch {
    return response.writeHead(404).end();
  }

  const etag = `"${createHash("sha256").update(body).digest("hex").slice(0, 16)}"`;
  response.setHeader("ETag", etag);
  response.setHeader("Cache-Control", cacheControl(path));
  response.setHeader("Content-Type", TYPES[extname(path)] ?? "application/octet-stream");

  if (request.headers["if-none-match"] === etag) return response.writeHead(304).end();

  response.setHeader("Content-Length", body.length);
  response.writeHead(200).end(body);
}).listen(8310, "127.0.0.1", () => console.log("listening: 127.0.0.1:8310"));

The files the server expects sit in the public/ directory. The block below produces that directory; it is run in the same place as the server file.

#!/usr/bin/env bash
# Creates the public/ directory and the four assets the server expects.
mkdir -p public
cat > public/index.html <<'END'
<!doctype html>
<html lang="en"><head><meta charset="utf-8"><title>Library Catalog</title>
<link rel="stylesheet" href="/catalog.7f3a91c2.css">
<script src="/catalog.4b8e0d15.js" defer></script></head>
<body><h1>Library Catalog</h1><div id="result"></div></body></html>
END
cat > public/catalog.7f3a91c2.css <<'END'
body { font-family: system-ui, sans-serif; margin: 2rem auto; max-width: 46rem; }
h1 { font-size: 1.5rem; }
.book { border-bottom: 1px solid #ddd; padding: .5rem 0; }
.book .author { color: #555; }
END
cat > public/catalog.4b8e0d15.js <<'END'
const root = document.getElementById("result");
const books = await (await fetch("/books.json")).json();
root.innerHTML = books.map((b) =>
  `<div class="book">${b.title} <span class="author">${b.author}</span></div>`).join("");
END
cat > public/books.json <<'END'
[{"isbn":"9789750718533","title":"Kayip Zamanin Izinde","author":"Marcel Proust"},
 {"isbn":"9789944885300","title":"Tutunamayanlar","author":"Oguz Atay"},
 {"isbn":"9789750726439","title":"Kar","author":"Orhan Pamuk"}]
END

Content Type and Length

What travels over the network is a byte sequence; the bytes themselves do not say what they are. The content type header determines how the recipient will process the content. The mapping from file extension to type is limited to the set of types the server knows, and an unrecognized extension gets the generic binary type. For text types the character encoding is also declared in the header; when it is not, the recipient falls back to a guess, and that guess can conflict with the file’s own encoding.

The content length header gives the body’s byte count. The recipient thereby knows where the response ends without waiting for the connection to close, and can compute download progress. This header can only be written once the whole body is ready in hand: here that is possible because the file is read into memory; when the body is produced piece by piece, chunked transfer takes its place.

Notice that the length header is not written on a 304 response. That response has no body, and the header would report false information on a response with no body.

Validator and Conditional Request

When the same file is requested a second time and the content has not changed, moving the bytes again is wasted work. The way to detect this is a validator: a short value the server derives from the content, one that changes when the content changes. In HTTP this value is carried by the entity tag header.

The server above derives the tag from a digest of the body. Deriving it from the modification time is also possible, but a timestamp has second-level resolution and changes when a file is copied; a tag derived from the content carries neither problem.

The client sends the tag it holds back on the next request. The server compares the tags: if they match, it returns a bodyless 304 response. This exchange is called a conditional request, and it is the server-side counterpart of the mechanism introduced in the How the Internet Works course.

The Two Classes of Cache Directive

A conditional request does not carry the body, but it still brings the request itself all the way to the server. Not making the request at all is achieved with a cache directive: the server states how long the response can be used without being asked for again.

The decision is made by whether the file’s name depends on its content. On the server, the name catalog.7f3a91c2.css carries a content hash: the name changes when the content changes, so the byte sequence under this name never changes. Such an asset can be cached with a long lifetime and a declaration that it will not change. The document’s name, by contrast, is fixed and its content can change; it is granted caching permission, but revalidation is required on every use.

This distinction is static serving’s one real design decision. Giving a long lifetime to a file with a fixed name serves stale content for days; giving a short lifetime to a file with a hashed name produces revalidation requests that were never necessary.

Measurement

The script below brings the server up, tests three behaviors, and stops it. Port 8310 is arbitrary and must be free.

#!/usr/bin/env bash
# Starts server.mjs, measures caching and conditional-request behavior, then stops it.
node server.mjs > /dev/null &
server=$!
sleep 1
A=http://127.0.0.1:8310

echo "--- first request: hashed asset ---"
curl -sS -D - -o /dev/null "$A/catalog.7f3a91c2.css"

echo "--- first request: document ---"
curl -sS -D - -o /dev/null "$A/index.html"

TAG=$(curl -sS -D - -o /dev/null "$A/index.html" | awk '/^ETag:/ {print $2}' | tr -d '\r')
echo "--- conditional request (If-None-Match: $TAG) ---"
curl -sS -D - -o /dev/null -H "If-None-Match: $TAG" "$A/index.html"

echo "--- downloaded bytes: unconditional / conditional ---"
printf 'unconditional  %s B\n' "$(curl -sS -o /dev/null -w '%{size_download}' "$A/index.html")"
printf 'conditional    %s B\n' "$(curl -sS -o /dev/null -w '%{size_download}' -H "If-None-Match: $TAG" "$A/index.html")"

echo "--- attempt to escape the root ---"
curl -sS -o /dev/null -w 'GET /../server.mjs -> %{http_code}\n' --path-as-is "$A/../server.mjs"

kill "$server"

--- first request: hashed asset ---
HTTP/1.1 200 OK
ETag: "a725ac1e3dee13d5"
Cache-Control: public, max-age=31536000, immutable
Content-Type: text/css; charset=utf-8
Content-Length: 198
Connection: keep-alive
Keep-Alive: timeout=5

--- first request: document ---
HTTP/1.1 200 OK
ETag: "9493205a58f55fac"
Cache-Control: no-cache
Content-Type: text/html; charset=utf-8
Content-Length: 270
Connection: keep-alive
Keep-Alive: timeout=5

--- conditional request (If-None-Match: "9493205a58f55fac") ---
HTTP/1.1 304 Not Modified
ETag: "9493205a58f55fac"
Cache-Control: no-cache
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Keep-Alive: timeout=5

--- downloaded bytes: unconditional / conditional ---
unconditional  270 B
conditional    0 B
--- attempt to escape the root ---
GET /../server.mjs -> 403

Tag values are derived from file content; they come out different in a directory where the files were not produced exactly as above. Connection headers depend on the runtime’s default settings.

Three results are worth reading. The hashed asset and the document received different cache directives. The conditional request transferred 0 bytes instead of 270; the gain stays proportionally the same as the file grows, because a 304 response’s body is empty at every size. The request pointing outside the document root was met with a 403: because the path check runs after the join, the path that escapes as a result of normalization is caught.

The Decision to Hand Off to a Content Delivery Network

Past a certain point, serving static assets stops being the application server’s job. Sending the same bytes thousands of times requires no application logic; a layer sitting closer to the user can do it instead. The content delivery network introduced in the How the Internet Works course is exactly this layer, and in this arrangement the application server takes on the name origin server.

The handoff decision is made with two ratios. The script below first learns each asset’s real size and cache directive from the origin server, then computes the same workload as if a caching layer sat in front of it.

// offload.mjs — measures the same workload against the origin server first,
// then computes the remaining load on the origin assuming a shared cache sits in front of it.
const A = "http://127.0.0.1:8310";
const PAGE = ["/index.html", "/catalog.7f3a91c2.css", "/catalog.4b8e0d15.js", "/books.json"];
const VISITS = 500; // number of visitors opening the same page

// 1) Learn the real size and cache directive of each asset from the origin.
const asset = new Map();
for (const path of PAGE) {
  const r = await fetch(A + path);
  const body = await r.arrayBuffer();
  asset.set(path, {
    bytes: body.byteLength,
    cacheable: (r.headers.get("cache-control") ?? "").includes("max-age"),
  });
}

// 2) Origin server alone: every visit pulls every asset from it.
const originRequests = VISITS * PAGE.length;
const originBytes = VISITS * PAGE.reduce((t, path) => t + asset.get(path).bytes, 0);

// 3) A cache layer in front: a cacheable asset is fetched once, the rest is served
//    from the edge. The non-cacheable document is revalidated on every visit; the
//    response is 304, so the request reaches the origin but the body is not transferred.
let edgeRequests = 0, edgeBytes = 0;
for (const path of PAGE) {
  const a = asset.get(path);
  if (a.cacheable) { edgeRequests += 1; edgeBytes += a.bytes; }
  else { edgeRequests += VISITS; edgeBytes += a.bytes; }
}

const ratio = (a, b) => ((1 - a / b) * 100).toFixed(1);
console.log(`workload: ${VISITS} visits x ${PAGE.length} assets`);
console.log(`origin server alone : ${originRequests} requests, ${originBytes} B`);
console.log(`cache layer in front : ${edgeRequests} requests, ${edgeBytes} B`);
console.log(`request hit ratio    : %${ratio(edgeRequests, originRequests)}`);
console.log(`byte hit ratio       : %${ratio(edgeBytes, originBytes)}`);

workload: 500 visits x 4 assets
origin server alone : 2000 requests, 458500 B
cache layer in front : 1002 requests, 917 B
request hit ratio    : %49.9
byte hit ratio       : %99.8

The two ratios are far apart from each other, and that gap is the decision itself. The request hit ratio is the share of requests that never reach the origin; here it is roughly half, because the two non-cacheable assets are revalidated on every visit. The byte hit ratio is the share of bodies that never leave the origin; here it is nearly complete, because revalidation responses carry no body.

The decision therefore looks at which resource is scarce on the server. If the bottleneck is bandwidth, handing off drops the load to nearly zero. If the bottleneck is the connection and processing cost paid per request, the gain is bounded by the share of non-cacheable assets, and improvement comes from revisiting the cache directives. Serving the catalog data with a short-lived cache directive as well pulls the second ratio up directly.

Summary

Static serving begins by joining the request path with the document root and checking, after the join, that the result still falls under the document root.
Content type tells the recipient how to interpret the bytes; content length tells it where the body ends; the length header is not written on bodyless responses.
The entity tag is a validator derived from the content; when a conditional request matches, the response returns without a body, and the measurement transferred 0 bytes instead of 270.
The cache directive splits into two classes: an asset whose name carries a content hash is cached with a long lifetime; a document with a fixed name is revalidated on every use.
The decision to hand off to a content delivery network is made by looking at the request hit ratio and the byte hit ratio; in the measurement these two ratios came out 49.9% and 99.8%, pointing to different conclusions depending on which resource is scarce.

Next Step

Everything served so far already sat ready on disk: the file was read, headers were written, bytes were sent. The catalog page, though, has its script fill in its content — the document arrives empty, and the book list is built in the browser. The server itself could write that same list into the document. That requires a mechanism that merges data with markup, and at the center of that mechanism sits a single question: what happens when a piece of text coming from data is placed inside markup? The next lesson builds this merge by writing its own template engine and measures escaping’s effect on the output.

To keep your progress and take notes, Log in