Local Development Environment

The previous lesson brought out the first persistent data the server writes to disk, and it noted that keeping records in memory was a gap. In a real setup, those records live in a database, and search lives in a separate service. The library lending application is no longer a process running on its own: it needs dependent services alongside it.

This creates a new problem on the developer’s machine. Running the application first requires running other programs, those programs have their own setup work, and it is not obvious when the application can trust them. This lesson answers three questions: in what order do the services start, how is a service’s readiness recognized, and how does the same environment get built identically on two separate machines?

Dependencies Form a Chain

The local environment’s decisions are collected in a single file. This file works as an environment manifest: ports, schema version, the runtime’s minimum version, and the seed data sets.

{
  "runtime": { "name": "node", "min_major_version": 22 },
  "schema_version": 3,
  "seed_set": "small",
  "seed_rows": { "small": 150000, "large": 1000000 },
  "ports": { "database": 8431, "search": 8432, "app": 8433 }
}

The first dependency is the service holding the catalog data. The detail in its behavior is central to this lesson: it opens its port immediately, but it does not accept queries until the schema and the seed data have been written. It also exposes a health endpoint that reports its own state.

// database.mjs — service holding the catalog data; refuses queries until ready
import { createServer } from "node:http";
import { DatabaseSync } from "node:sqlite";
import { readFileSync, rmSync } from "node:fs";

const env = JSON.parse(readFileSync("environment.json", "utf8"));
const set = process.env.SEED_SET ?? env.seed_set;
const rows = env.seed_rows[set];
const port = env.ports.database;

rmSync("library.db", { force: true });
const db = new DatabaseSync("library.db");
let ready = false;

const json = (response, status, body) => {
  const text = JSON.stringify(body);
  response.setHeader("Content-Type", "application/json; charset=utf-8");
  response.setHeader("Content-Length", Buffer.byteLength(text));
  response.writeHead(status).end(text);
};

createServer((request, response) => {
  response.sendDate = false;
  const address = new URL(request.url, "http://local");
  if (address.pathname === "/health")
    return json(response, ready ? 200 : 503, { ready, schema: env.schema_version });
  if (!ready) return json(response, 503, { error: "not_ready" });
  if (address.pathname === "/books") {
    const author = address.searchParams.get("author") ?? "Author 42";
    const records = db.prepare(
      "SELECT isbn, name, author FROM book WHERE author = ? ORDER BY isbn").all(author);
    return json(response, 200, { count: records.length, books: records });
  }
  return json(response, 404, { error: "not_found" });
}).listen(port, "127.0.0.1", () => {
  console.log(`database :${port} opened (not ready)`);
  // Setup work runs in chunks after the port opens: this way a not-yet-ready server
  // does not leave requests unanswered, and instead returns a response reporting its state.
  const started = Date.now();
  db.exec("CREATE TABLE book (isbn TEXT PRIMARY KEY, name TEXT NOT NULL, author TEXT NOT NULL)");
  const insert = db.prepare("INSERT INTO book (isbn, name, author) VALUES (?, ?, ?)");
  db.exec("BEGIN");
  let written = 0;
  const writeChunk = () => {
    for (const end = Math.min(written + 10000, rows); written < end; written++)
      insert.run(`978-${String(written).padStart(10, "0")}`,
               `Book ${written}`, `Author ${written % 500}`);
    if (written < rows) return setImmediate(writeChunk);
    db.exec("COMMIT");
    db.exec("CREATE INDEX ix_author ON book(author)");
    ready = true;
    console.log(`database ready: ${set} seed, ${rows} rows, ${Date.now() - started} ms`);
  };
  setImmediate(writeChunk);
});

The second dependency is the search service, and what forms the chain is the way it starts up: it builds its index by reading from the database, so it cannot start at all if the database is not ready.

// search.mjs — second HTTP service; reads its index from the database at startup
import { createServer } from "node:http";
import { readFileSync } from "node:fs";

const env = JSON.parse(readFileSync("environment.json", "utf8"));
const { database, search } = env.ports;

const response = await fetch(`http://127.0.0.1:${database}/books?author=Author%2042`)
  .catch((e) => { console.error(`search: could not reach the database (${e.cause?.code ?? e.name})`);
                  process.exit(1); });
if (!response.ok) { console.error(`search: database returned ${response.status}, index could not be built`);
                 process.exit(1); }

const { books } = await response.json();
const index = new Map(books.map((b) => [b.name.toLowerCase(), b.isbn]));

createServer((request, reply) => {
  reply.sendDate = false;
  const term = new URL(request.url, "http://local").searchParams.get("q")?.toLowerCase() ?? "";
  const found = [...index].filter(([name]) => name.includes(term)).map(([name, isbn]) => ({ name, isbn }));
  const body = JSON.stringify({ indexSize: index.size, found });
  reply.setHeader("Content-Type", "application/json; charset=utf-8");
  reply.setHeader("Content-Length", Buffer.byteLength(body));
  reply.writeHead(200).end(body);
}).listen(search, "127.0.0.1", () => console.log(`search :${search} ready, index ${index.size} records`));

The chain’s last link is the lending application, which uses both of them.

// app.mjs — lending application; uses both dependent services
import { createServer } from "node:http";
import { readFileSync } from "node:fs";

const env = JSON.parse(readFileSync("environment.json", "utf8"));
const { database, search, app } = env.ports;

createServer(async (request, reply) => {
  reply.sendDate = false;
  const q = new URL(request.url, "http://local").searchParams.get("q") ?? "book 42";
  let status = 200, body;
  try {
    const a = await fetch(`http://127.0.0.1:${search}/?q=${encodeURIComponent(q)}`).then((r) => r.json());
    const v = await fetch(`http://127.0.0.1:${database}/health`).then((r) => r.json());
    body = { query: q, matches: a.found.length, indexSize: a.indexSize, database: v };
  } catch (e) {
    status = 503;
    body = { error: "dependent_service_unavailable", detail: e.cause?.code ?? e.name };
  }
  const text = JSON.stringify(body);
  reply.setHeader("Content-Type", "application/json; charset=utf-8");
  reply.setHeader("Content-Length", Buffer.byteLength(text));
  reply.writeHead(status).end(text);
}).listen(app, "127.0.0.1", () => console.log(`app :${app} ready`));

Having the catalog data live in a process we wrote ourselves is meant to make the lesson’s subject visible; the behavior stays the same once a real database server takes its place. The setup work done at startup stays in the same place too: checking the file format, repairing the log, opening indexes.

Being Ready Is Different from Having Started

Starting a process and that process being able to do work are two separate events, and there is a window between them. A readiness check is a gate that waits out this window: it asks the dependent service by trying it, waits on a negative answer, and tries again. The wait doubles on every failed attempt — the exponential backoff introduced in the Application Architecture course applies here as well: the first attempts come close together, later ones spread out, so a service that starts fast is caught right away, and a service that starts slowly is not polled needlessly.

// wait.mjs — readiness check. Usage: node wait.mjs <port|health> <port-number> [name]
import { connect } from "node:net";
const [mode, port, name = "service"] = process.argv.slice(2);

const isPortOpen = (p) => new Promise((resolve) => {
  const s = connect({ port: Number(p), host: "127.0.0.1" });
  s.once("connect", () => { s.destroy(); resolve(true); });
  s.once("error", () => resolve(false));
});
const isHealthy = async (p) =>
  fetch(`http://127.0.0.1:${p}/health`).then((r) => r.status === 200).catch(() => false);

const check = mode === "port" ? isPortOpen : isHealthy;
const started = Date.now();
let delay = 20;        // first wait, ms; doubles on every failed attempt
let attempt = 0;
while (Date.now() - started < 5000) {
  attempt++;
  if (await check(port)) {
    console.log(`  ${name}: ready (${mode} mode, ${attempt} attempts, ${Date.now() - started} ms)`);
    process.exit(0);
  }
  await new Promise((c) => setTimeout(c, delay));
  delay = Math.min(delay * 2, 500);
}
console.error(`  ${name}: not ready within 5 s`);
process.exit(1);

The difference between the two modes is the measurement’s subject. The port check asks only part of the question: is something listening at this address? The health check asks the real question: can this service do work? The startup script is written so it can compare four separate gates.

#!/usr/bin/env bash
# bring-up.sh — brings dependent services up in order.
# Usage: ./bring-up.sh <none|fixed|port|health>;  SEED_SET=small|large
mode="${1:-health}"; set="${SEED_SET:-small}"
pids=(); cleanup() { kill "${pids[@]}" 2>/dev/null; wait 2>/dev/null; }; trap cleanup EXIT
echo "== gate=$mode  seed=$set =="

SEED_SET="$set" node database.mjs & pids+=($!)
case "$mode" in
  none)   ;;                                                # no gate
  fixed)  sleep 0.5 ;;                                       # fixed wait: a guess
  port)   node wait.mjs port   8431 database || exit 1 ;;   # is it accepting connections
  health) node wait.mjs health 8431 database || exit 1 ;;   # is it able to do work
esac

node search.mjs & pids+=($!)
node wait.mjs port 8432 search || { echo "  -> chain broken: search did not come up"; exit 1; }

node app.mjs & pids+=($!)
node wait.mjs port 8433 app > /dev/null
printf 'request -> '
curl -sS --max-time 5 -w ' (HTTP %{http_code})\n' "http://127.0.0.1:8433/?q=book%2042"

Measurement: Four Gates

for k in none fixed port health; do ./bring-up.sh $k; echo; done

== gate=none  seed=small ==
database :8431 opened (not ready)
search: database returned 503, index could not be built
database ready: small seed, 150000 rows, 133 ms
  search: not ready within 5 s
  -> chain broken: search did not come up

== gate=fixed  seed=small ==
database :8431 opened (not ready)
database ready: small seed, 150000 rows, 132 ms
search :8432 ready, index 300 records
  search: ready (port mode, 3 attempts, 65 ms)
app :8433 ready
request -> {"query":"book 42","matches":3,"indexSize":300,"database":{"ready":true,"schema":3}} (HTTP 200)

== gate=port  seed=small ==
database :8431 opened (not ready)
  database: ready (port mode, 2 attempts, 22 ms)
search: database returned 503, index could not be built
database ready: small seed, 150000 rows, 136 ms
  search: not ready within 5 s
  -> chain broken: search did not come up

== gate=health  seed=small ==
database :8431 opened (not ready)
database ready: small seed, 150000 rows, 136 ms
  database: ready (health mode, 4 attempts, 176 ms)
search :8432 ready, index 300 records
  search: ready (port mode, 2 attempts, 22 ms)
app :8433 ready
request -> {"query":"book 42","matches":3,"indexSize":300,"database":{"ready":true,"schema":3}} (HTTP 200)

The durations are machine-dependent, and the order of the lines written by the three background processes also changes on every run. What does not change is which gate forms the chain.

With no gate, the database’s port had already opened by the time the search service made its request, but the seed data was still being written, so the response was a 503 and the index could never be built. The process exits, and everything after it stays uninstalled. Startup order is not a preference, it is a requirement.

The port check produces the most instructive line. The gate passed in 22 milliseconds, on the second attempt — because the port really was open. But the database was still writing its seed data at that moment and returned 503 to search. The chain breaks again. A listening port does not mean a working service.

The health check does the same job in 176 milliseconds and four attempts: the gate waits for the state the service reports about itself. The chain forms, and the application produces a response using both of its dependencies.

A Fixed Wait Is a Guess

The fixed wait worked with the small seed set. This shows that the guess happened to hold, not that it was correct. When the same script runs with a larger seed set, the guess breaks down.

for k in fixed health; do SEED_SET=large ./bring-up.sh $k; echo; done

== gate=fixed  seed=large ==
database :8431 opened (not ready)
search: database returned 503, index could not be built
database ready: large seed, 1000000 rows, 880 ms
  search: not ready within 5 s
  -> chain broken: search did not come up

== gate=health  seed=large ==
database :8431 opened (not ready)
database ready: large seed, 1000000 rows, 881 ms
  database: ready (health mode, 7 attempts, 1178 ms)
search :8432 ready, index 2000 records
  search: ready (port mode, 3 attempts, 64 ms)
app :8433 ready
request -> {"query":"book 42","matches":23,"indexSize":2000,"database":{"ready":true,"schema":3}} (HTTP 200)

The half-second wait is not enough, because setup took 880 milliseconds. The health check, in the same script with nothing changed, waits 1178 milliseconds and passes. A fixed wait is wrong in both directions: when it is too short it breaks the chain, when it is too long it wastes time on every startup. The check, instead, ties the wait to a measurement.

The Same Environment on Two Machines

Forming the chain is not enough; the same chain has to produce the same result on a different machine. Two things break this: different runtime versions, and seed data that comes out different on every setup.

The first is met with version pinning: the minimum version written in the manifest is compared against the version actually running at startup, and a mismatch is never passed over silently. The second is met with a rule: seed data is input, not something generated. Identifiers, dates, and relationships are written as part of the seed; they are not taken from the moment the code runs or from randomness.

// seed.mjs — writes seed data. Usage: node seed.mjs <file> <fixed|free>
import { DatabaseSync } from "node:sqlite";
import { rmSync } from "node:fs";

const [file, rule = "fixed"] = process.argv.slice(2);
rmSync(file, { force: true });
const db = new DatabaseSync(file);
db.exec(`CREATE TABLE member (id INTEGER PRIMARY KEY, name TEXT NOT NULL, registered TEXT NOT NULL);
         CREATE TABLE loan (id INTEGER PRIMARY KEY, member INTEGER NOT NULL, isbn TEXT NOT NULL)`);

const MEMBERS = ["Alice Kane", "Ben Ortiz", "Clara Diaz", "Derek Voss"];
const ISBNS = ["978-0000000001", "978-0000000002", "978-0000000003"];
const insertMember = db.prepare("INSERT INTO member (id, name, registered) VALUES (?, ?, ?)");
const insertLoan = db.prepare("INSERT INTO loan (id, member, isbn) VALUES (?, ?, ?)");

MEMBERS.forEach((name, i) => {
  // fixed rule: the date is part of the seed. free rule: it comes from the moment it runs.
  const registered = rule === "fixed" ? `2024-01-0${i + 1}` : new Date().toISOString();
  insertMember.run(i + 1, name, registered);
});
ISBNS.forEach((isbn, i) => {
  const member = rule === "fixed"
    ? (i % MEMBERS.length) + 1
    : 1 + Math.floor(Math.random() * MEMBERS.length);
  insertLoan.run(i + 1, member, isbn);
});
db.close();

Whether the rule holds can be measured: the same seed is written twice, and the data content’s checksum is compared. The checksum looks at the table content, not the file layout.

// verify.mjs — checks the runtime version against the declaration, writes the seed checksum
import { execFileSync } from "node:child_process";
import { readFileSync } from "node:fs";

const env = JSON.parse(readFileSync("environment.json", "utf8"));
const majorVersion = Number(process.versions.node.split(".")[0]);
const fit = majorVersion >= env.runtime.min_major_version;
console.log(`runtime: ${env.runtime.name} ${majorVersion}` +
  ` (declared minimum ${env.runtime.min_major_version}) -> ${fit ? "fit" : "NOT FIT"}`);
if (!fit) process.exit(1);

const checksum = (file) =>
  execFileSync("sqlite3", [file, ".sha3sum"], { encoding: "utf8" }).trim().slice(0, 16);
for (const file of process.argv.slice(2)) console.log(`  ${file}: ${checksum(file)}`);

for rule in fixed free; do
  for n in 1 2; do node seed.mjs "$rule-$n.db" "$rule"; done
done
node verify.mjs fixed-1.db fixed-2.db free-1.db free-2.db

runtime: node 24 (declared minimum 22) -> fit
  fixed-1.db: 193ad8ccf56b86f6
  fixed-2.db: 193ad8ccf56b86f6
  free-1.db: c5541004e4a7551c
  free-2.db: 178c62eb7f6888bc

The checksums are written truncated, and the two values the free rule produces are different on every run; the runtime line also depends on the runtime of the machine that executes it. The result to read is this: the fixed rule produced the same checksum across two setups, the free rule did not. The same distinction holds between two machines. In an environment built with the free rule, the sentence “it works on my machine” turns into a claim that cannot be verified, because the data on the two machines is not the same.

Summary

• Dependent services form a chain: each link requires the one before it to be ready; with no gate in place, the first link’s startup delay breaks the whole chain.
• A port being open does not mean being ready; in the measurement the port check passed in 22 milliseconds, and while the service was still doing its setup work at that moment, it returned 503 to the dependent service.
• The readiness check looks at the state the service reports about itself, and after a failed attempt it doubles the wait and retries.
• A fixed wait is a guess: when the seed data grew, the same half second was not enough, while the health check waited exactly as long as it needed to, unchanged.
• Building the same environment on two machines rests on two rules: comparing the runtime version against the manifest, and having seed data come from the seed itself rather than the moment it runs; the second is verified by checksum equality.

Course Wrap-Up

This course built a request’s journey through the server side, start to finish. The question asked at the beginning was “what does the side facing the client take on”; the answer gathered into three responsibilities — persistence, business rules, integration — and measurement showed why these responsibilities cannot be delegated to the client. From there, “the server” stopped being a single program: the web server, the application server, and proxying split into separate roles, and the request lifecycle and the middleware chain gave the ordering within those roles.

The second part brought the application itself up: reading configuration from the environment, separating secrets, structured logging, and a common error layer. The third part took up what the application gives back to the outside: serving a file with a validator and a cache directive, rendering a view from a template, accepting an incoming file within limits, and bringing the services next to it up locally.

One line of reasoning repeated throughout the course: no value coming from the client counts as a decision unless it has passed through the server’s own check. A declared length, a declared content type, a sent file name, a forwarded client address — all of these are claims. The server’s job is to meet these claims with its own measurement.

Up to this point, what the server gave back to the outside was either a file or a document: content meant for a person to read. The library service’s real consumers, though, are programs — the application in the browser, terminals at the branches, reporting jobs. Where programs connect to each other, what determines things is not format but contract: which resources exist, what they are named, which fields they carry, and what happens to the other side when a field is removed? The next course — Web API Design — builds this contract. Its first topic separates the different API styles that answer the same job — resource-based, remote-procedure-call-based, and query-based approaches — and measures the trade-offs between them.