API Gateway

The previous two lessons worked both sides of the boundary: where the boundary should pass, how to call across it, which calls should never be made at all and turn into an event instead. One side has never been addressed — the outside. The client knows every service’s address individually, each service carries its own identity check in its own code, and the same rate-limit rule sits written in more than one place.

The API gateway, backend for frontend, and gatekeeper patterns were built and measured on the traffic side in The Traffic Layer course; that measurement is not repeated here. The question here is on the application side: how many lines in how many services does the same edge responsibility sit as, how many files and endpoints change when it moves to a single place, how many hops are added per request.

The through-line is the loan page: rendering one page needs data from all three of the catalog, membership, and loan services.

The Same Responsibility, Two Layouts

The block below writes the same three services in two forms. The lines carrying edge responsibility are marked in the source; the lean form is produced by stripping those lines from the same files, and the marked lines are written once, in a single gateway process. Both forms are real, running source.

// setup.mjs — writes the same endpoint service in two forms: edge responsibility in the service and in the gateway
import { mkdirSync, writeFileSync } from "node:fs";
const SERVICE = { catalog: 8961, membership: 8962, loan: 8963 }, GATEWAY = 8960;

const service = (name, port) => [
  `// service-${name}.mjs — service producing one part of the loan page`,
  'import { createServer } from "node:http";',
  `const PORT = ${port}, WORK = 25;`,
  "let processed = 0, reset = () => {};",
  "const COUNTER = new Map(), LIMIT = 3; reset = () => COUNTER.clear();        // edge: rate limit",
  "createServer(async (request, response) => {",
  '  if (request.url === "/count") { response.end(String(processed)); return; }',
  '  if (request.url === "/reset") { processed = 0; reset(); response.end("r"); return; }',
  '  const member = request.headers["x-member"];                              // edge: identity',
  '  if (!member) { response.statusCode = 401; response.end("no identity"); return; } // edge: identity',
  "  const n = (COUNTER.get(member) ?? 0) + 1; COUNTER.set(member, n);        // edge: rate limit",
  '  if (n > LIMIT) { response.statusCode = 429; response.end("rate limited"); return; } // edge: rate limit',
  "  processed += 1;",
  "  await new Promise((c) => setTimeout(c, WORK));",
  `  response.end(JSON.stringify({ name: "${name}" }));`,
  "}).listen(PORT);",
];
const gateway = [
  "// gateway.mjs — process collecting the edge responsibilities and merging the responses",
  'import { createServer } from "node:http";',
  `const PORT = ${GATEWAY}, SUCCESSOR = [${Object.values(SERVICE).join(", ")}];`,
  "let reset = () => {};",
  "const COUNTER = new Map(), LIMIT = 3; reset = () => COUNTER.clear();        // edge: rate limit",
  "createServer(async (request, response) => {",
  '  if (request.url === "/reset") { reset(); response.end("r"); return; }',
  '  const member = request.headers["x-member"];                              // edge: identity',
  '  if (!member) { response.statusCode = 401; response.end("no identity"); return; } // edge: identity',
  "  const n = (COUNTER.get(member) ?? 0) + 1; COUNTER.set(member, n);        // edge: rate limit",
  '  if (n > LIMIT) { response.statusCode = 429; response.end("rate limited"); return; } // edge: rate limit',
  "  const parts = await Promise.all(SUCCESSOR.map((p) =>",
  '    fetch(`http://127.0.0.1:${p}/data`).then((y) => y.json())));           // edge: merge',
  "  response.end(JSON.stringify({ merged: parts }));",
  "}).listen(PORT);",
];
const edgeLines = (lines) => lines.filter((s) => s.includes("// edge:")).length;
const write = (dir, file, lines) => {
  mkdirSync(dir, { recursive: true });
  writeFileSync(`${dir}/${file}`, `${lines.join("\n")}\n`);
  return lines.length;
};

let withEdgeLines = 0, withEdgeMarks = 0, leanLines = 0, leanMarks = 0;
for (const [name, port] of Object.entries(SERVICE)) {
  const full = service(name, port), plain = full.filter((s) => !s.includes("// edge:"));
  withEdgeLines += write("with-edge", `service-${name}.mjs`, full);
  withEdgeMarks += edgeLines(full);
  leanLines += write("lean", `service-${name}.mjs`, plain);
  leanMarks += edgeLines(plain);
}
const gatewayLines = write("lean", "gateway.mjs", gateway);
const n = Object.keys(SERVICE).length;
console.log(`${"layout".padEnd(22)}${"files".padStart(7)}${"lines".padStart(7)}` +
  `${"edge lines".padStart(14)}${"touched when rule changes".padStart(28)}${"externally exposed endpoints".padStart(31)}`);
console.log(`${"edge in services".padEnd(22)}${String(n).padStart(7)}${String(withEdgeLines).padStart(7)}` +
  `${String(withEdgeMarks).padStart(14)}${String(n).padStart(28)}${String(n).padStart(31)}`);
console.log(`${"edge in gateway".padEnd(22)}${String(n + 1).padStart(7)}` +
  `${String(leanLines + gatewayLines).padStart(7)}${String(leanMarks + edgeLines(gateway)).padStart(14)}` +
  `${"1".padStart(28)}${"1".padStart(31)}`);
console.log(`edge responsibility had been repeated as ${withEdgeMarks / n} lines across ${n} services; ` +
  `in the gateway it is a single copy in ${edgeLines(gateway)} lines`);

layout                  files  lines    edge lines   touched when rule changes   externally exposed endpoints
edge in services            3     48            15                           3                              3
edge in gateway             4     48             6                           1                              1
edge responsibility had been repeated as 5 lines across 3 services; in the gateway it is a single copy in 6 lines

The total line count is the same in both forms; the moved code did not disappear, it changed location. Three numbers change: the repeated edge lines drop from 15 to 6, the number of files touched when the identity or rate-limit rule changes drops from 3 to 1, the number of externally exposed endpoints drops from 3 to 1. The gateway’s sixth marked line is a responsibility that does not exist in the services at all: merging.

The Difference in the Run

SB4 — the rate limit is three requests per member. Rationale: the value is not a real threshold — it is a measure-sized quantity chosen so that the limit becomes visible within twelve requests. What is measured is not the limit’s value but how many places it is enforced in, and how many requests the same rule grants outward.

// measure.mjs — <measure: round|limit> <mode: direct|gateway>; runs while the processes are up
const WORK = 25, SERVICE = [8961, 8962, 8963], GATEWAY = 8960;
const HEADER = { headers: { "x-member": "7" } };
let calls = 0, serviceCalls = 0;
const endpoints = new Set();
const request = async (p, path) => {
  calls += 1; endpoints.add(p);
  if (SERVICE.includes(p)) serviceCalls += 1;
  const y = await fetch(`http://127.0.0.1:${p}${path}`, HEADER);
  await y.text();
  return y.status;
};
const noIdentity = async (p, path) => {         // request without the x-member header
  const y = await fetch(`http://127.0.0.1:${p}${path}`);
  await y.text();
  return y.status;
};
const reset = async () => {
  for (const p of [...SERVICE, GATEWAY]) await fetch(`http://127.0.0.1:${p}/reset`).catch(() => {});
};
const reached = async () => {
  let t = 0;
  for (const p of SERVICE) t += Number(await (await fetch(`http://127.0.0.1:${p}/count`)).text());
  return t;
};
const [measure, mode] = process.argv.slice(2);
const PATH = {
  "direct-sequential": async () => { for (const p of SERVICE) await request(p, "/data"); },
  "direct-parallel": () => Promise.all(SERVICE.map((p) => request(p, "/data"))),
  gateway: () => request(GATEWAY, "/page"),
};

if (measure === "round") {
  console.log(`${"client path".padEnd(18)}${"client calls".padStart(16)}${"network hops".padStart(13)}` +
    `${"touched endpoints".padStart(19)}${"reached services".padStart(18)}${"measured rounds".padStart(17)}`);
  for (const name of Object.keys(PATH).filter((a) => a.startsWith(mode))) {
    const durations = [];
    for (let i = 0; i < 3; i += 1) {
      await reset();
      const t = performance.now();
      await PATH[name]();
      durations.push(performance.now() - t);
    }
    await reset();
    calls = 0; serviceCalls = 0; endpoints.clear();
    await PATH[name]();
    const u = await reached();
    console.log(`${name.padEnd(18)}${String(calls).padStart(16)}` +
      `${String(calls + u - serviceCalls).padStart(13)}${String(endpoints.size).padStart(19)}` +
      `${String(u).padStart(18)}${String(Math.floor(durations.sort((a, b) => a - b)[1] / WORK)).padStart(17)}`);
  }
} else if (measure === "limit") {
  await reset();
  let accepted = 0, rejected = 0;
  for (let i = 0; i < 12; i += 1) {
    const status = mode === "gateway" ? await request(GATEWAY, "/page") : await request(SERVICE[i % SERVICE.length], "/data");
    if (status === 200) accepted += 1; else rejected += 1;
  }
  console.log(`rate limit (3 per member): 12 external requests -> ${accepted} accepted, ${rejected} rejected, ` +
    `reached services ${await reached()}`);
  await reset();
  const status = mode === "gateway" ? [await noIdentity(GATEWAY, "/page")]
    : await Promise.all(SERVICE.map((p) => noIdentity(p, "/data")));
  console.log(`request without identity: ${status.length} endpoints checked -> ` +
    `${status.filter((d) => d === 401).length} times 401, reached services ${await reached()}`);
}

echo "-- edge responsibility in each service --"
for a in catalog membership loan; do node with-edge/service-$a.mjs & done
sleep 1
node measure.mjs round direct
node measure.mjs limit direct
pkill -f "with-edge/service" ; sleep 0.3

echo
echo "-- edge responsibility in the gateway --"
for a in catalog membership loan; do node lean/service-$a.mjs & done
node lean/gateway.mjs &
sleep 1
node measure.mjs round gateway
node measure.mjs limit gateway
pkill -f "lean/service" ; pkill -f "lean/gateway"

-- edge responsibility in each service --
client path           client calls network hops  touched endpoints  reached services  measured rounds
direct-sequential                3            3                  3                 3                3
direct-parallel                  3            3                  3                 3                1
rate limit (3 per member): 12 external requests -> 9 accepted, 3 rejected, reached services 9
request without identity: 3 endpoints checked -> 3 times 401, reached services 0

-- edge responsibility in the gateway --
client path           client calls network hops  touched endpoints  reached services  measured rounds
gateway                          1            4                  1                 3                1
rate limit (3 per member): 12 external requests -> 3 accepted, 9 rejected, reached services 9
request without identity: 1 endpoints checked -> 1 times 401, reached services 0

Merging and Rounds

The first two rows are the case where the client assembles the loan page itself. The client makes three calls, knows three separate endpoints, and reaches three services. If it makes the calls in sequence, the page renders in three rounds; if in parallel, in one round. The difference is in the client’s hands and cannot be controlled from the server side: the same page’s latency depends on how the client is written.

The third row renders the same page through the gateway. The client makes one call, knows one endpoint, three requests still reach the services, and the page returns in one round. When merging moves to the server side, the worst case is leveled to the best case: the client does not need to know to make the parallel call, the gateway already does. What is gained is not average latency — it is control over latency.

The fourth column gives the cost: network hops per request go from three to four. In the direct layout, the client’s three calls went directly to the services; in the gateway layout there is one outer hop and three inner hops. Because the gateway does no work of its own, this hop did not add a round, but it did take a share of the latency budget.

The Same Rule, Two Different Allowances

The rate-limit rows produce the lesson’s sharpest number. The rule is byte-for-byte identical in both layouts: three requests per member. When twelve requests are sent, the direct layout accepts nine of them, the gateway layout accepts three.

The reason is the limit’s unit. A limit that counts per service grants a member nine requests, not three, in a three-service system; as long as the member spreads their requests across the services, they never hit a limit. A limit that counts at the gateway counts by page request. The last column levels the service load of the two layouts: in both cases, nine requests reach the services. So the same written rule, under the same service load, grants three times as much allowance outward. An edge rule cannot be read without knowing where it is enforced.

The identity row shows the same thing on the verification side. A request without identity is rejected three times, at three separate endpoints, in the direct layout, and once in the gateway layout. Three rejections are three separate copies of the same rule, and it is only a matter of time before the copies drift apart from each other.

The Triple

What it made cheaper. Repeated edge lines dropped from 15 to 6, files touched when the rule changes dropped from 3 to 1, externally exposed endpoints dropped from 3 to 1, client calls dropped from 3 to 1; for a client running sequentially, the page dropped from 3 rounds to 1.

What it made more expensive. Hops per request rose from 3 to 4, and a new deployment unit and a new process were added to the system. The gateway carries the three services’ endpoints in its own source: when a service’s endpoint path or response shape changes, the number of files touched rises back to two. The single file gained on edge responsibility is given back on the contract side.

What new failure mode it created. All outside traffic now passes through a single process. In the previous lesson’s measure, the gateway’s dependency closure is every entry point: when it goes down, the affected request share is one. In the direct layout, when catalog went down the page rendered incomplete; in the gateway layout, when the gateway goes down the page does not render at all. The second failure mode is quieter: because the rate limit and identity check no longer live in the services, a call that bypasses the gateway passes through no check at all. Lean services do not defend themselves; the defense depends on a single place being configured correctly.

Summary

• Edge responsibility had been repeated as 5 lines each across three services (15 total); in the gateway it became a single copy in 6 lines, and the total line count did not change.
• Files touched when the identity or rate-limit rule changes dropped from 3 to 1, externally exposed endpoints from 3 to 1, client calls from 3 to 1.
• When merging moved to the server, the sequential client’s 3 rounds dropped to 1; in exchange, network hops per request rose from 3 to 4 and a new deployment unit was added.
• The same “3 requests per member” rule accepts 9 of 12 requests when counted per service and 3 when counted at the gateway; requests reaching the services are 9 in both cases.
• The gateway is the dependency closure of all outside traffic, and lean services do not defend themselves; verification is tied to the correctness of a single place.

Next Step

The gateway assembled a single page, and that page was assumed to have a single recipient. The library’s loan page, though, is rendered in more than one place: the touchscreen terminal by the shelf wants only copy status, a handheld client wants a small summary, and the staff screen wants the member’s entire history. When all three pass through the same gateway, its merging code has to carry three separate shapes at once, and every client change touches a shared file. The next lesson counts this cost, and measures, on the code side, what building a separate aggregation layer per client — instead of one general gateway — makes cheaper and what it makes more expensive.