API Gateway

The previous topic settled which replica a request lands on. The load balancer is a routing device: it decides at the connection level or by the request’s content, drops a failed replica through the health check, and loses its power to choose once state stays in a replica’s memory — session stickiness priced this as the needed replica count rising from 3 to 4. The decision’s subject was always the same: which peer replica.

The work piling up at the edge goes further. Before a tracking request reaches the application, its identity may need verifying, its rate limiting, its body translating into the shape the service expects, and responses from several services gathering into one body. None of these is the “which replica” question, and each can be placed in one of three spots: the balancer, the application, or a separate layer. This lesson names the third option and reduces the question to a single layout decision: at each service, in one component at the edge, or split between the two.

Which Decision Does the Gateway Make

The API gateway is a system’s single externally facing entry point, and it decides at the transaction level: which service an incoming request belongs to, in what shape it forwards, and which checks it must have already passed. To make this decision the gateway carries a service map — the mapping of paths to services.

This is not the load balancer’s decision. The balancer picks one among replicas doing the same work; it need not know what each service does, and it assumes its pool is made of peers. The reverse proxy, in the form built in the Server-Side Fundamentals course, is the process that takes a request and forwards it to the server behind it; TLS termination and static file serving were handled there. The three names often converge on the same process, which is not a contradiction: the distinction is not the process count but the decision made. For a request, “which replica” is the balancer’s question, “which service and which shape” is the gateway’s.

This distinction also sets what the gateway must not do. When the billing context’s tariff rule or the delivery operations context’s route rule is placed on the gateway, the edge starts knowing both contexts’ domain rules at once, and every change to either context forces a redeploy. Work that belongs at the edge is work decidable from the request’s metadata; work that requires looking at the shipment itself belongs to the service. The topic’s third lesson turns this criterion into a mechanical rule.

Five Responsibilities, Three Layouts

The measurement uses an in-process model: there is no real gateway product, separate process, or network. The model holds three services (tracking, event, fee — the read, write, and batch flows respectively) and five edge responsibilities as a data structure. Token validation was built in the Authentication and Authorization course, rate limiting in the Web API Design course; neither’s algorithm is revisited here, only its placement.

// edge/layout.mjs — three services, five edge responsibilities, three layouts; in-process model
export const SERVICE = { tracking: "read", event: "write", fee: "batch" };

// How many lines each responsibility's implementation takes (model value, written once)
export const RESPONSIBILITY = {
  "route matching": 7,
  "token validation": 12,
  "rate limiting": 9,
  "request logging": 6,
  "format translation": 11,
};

export const LAYOUT = {
  "at each service": { gateway: [], service: Object.keys(RESPONSIBILITY) },
  "at the gateway": { gateway: Object.keys(RESPONSIBILITY), service: [] },
  hybrid: {
    gateway: ["route matching", "rate limiting", "request logging", "token validation"],
    service: ["token validation", "format translation"],
  },
};

export function measure(y) {
  const serviceCount = Object.keys(SERVICE).length;
  const total = (list) => list.reduce((t, s) => t + RESPONSIBILITY[s], 0);
  const hasGateway = y.gateway.length > 0;
  return {
    "addresses the client knows": hasGateway ? 1 : serviceCount,
    "lines collected at the edge": total(y.gateway),
    "lines repeated across services": serviceCount * total(y.service),
    "units carrying the responsibility (average)":
      Number((Object.keys(RESPONSIBILITY).reduce((t, s) =>
        t + (y.gateway.includes(s) ? 1 : 0) + (y.service.includes(s) ? serviceCount : 0), 0)
        / Object.keys(RESPONSIBILITY).length).toFixed(2)),
    "hops a request touches": hasGateway ? 2 : 1,
    "flows the gateway touches": hasGateway ? serviceCount : 0,
    "flows answered when gateway stops": `${hasGateway ? 0 : serviceCount}/${serviceCount}`,
  };
}

The line counts are the model’s input and belong to the assumption class: they were not measured from an implementation, they only carry an order of magnitude. What is read is not the total itself but the difference multiplying it by three produces; even if the five numbers were cut in half, the ratio of lines repeated to lines collected at the edge would stay the same.

In the third layout, token validation appears in both lists at once. This is not a mistake, it is a deliberate choice: the gateway validates the token, and the service validates it too, and the second validation covers the case where the gateway is bypassed. The count shows this as a price paid.

// edge/measure.mjs — measures the three layouts with the same counters and ties them to the K01 calculation
import { LAYOUT, measure } from "./layout.mjs";

const FIELD = ["addresses the client knows", "lines collected at the edge",
  "lines repeated across services", "units carrying the responsibility (average)",
  "hops a request touches", "flows the gateway touches",
  "flows answered when gateway stops"];

const measurement = Object.entries(LAYOUT).map(([name, y]) => ({ name, ...measure(y) }));
console.log(`${"measure".padEnd(46)}${measurement.map((o) => o.name.padStart(18)).join("")}`);
for (const f of FIELD) {
  console.log(`${f.padEnd(46)}${measurement.map((o) => String(o[f]).padStart(18)).join("")}`);
}

// K01 Back-of-the-Envelope Estimation lesson's numbers (computed-value class, taken from there)
const K01 = { readPeak: 416.67, writePeak: 97.22, edgePeak: 513.89 };
const write = (name, value, note = "") => console.log(`${name.padEnd(37)}${value.padStart(10)}  ${note}`);

console.log("");
write("K01 — peak requests/s at the edge", K01.edgePeak.toFixed(2), "read 416.67 + write 97.22");
write("no gateway — heaviest component", K01.readPeak.toFixed(2), "tracking service");
write("with gateway — heaviest component", K01.edgePeak.toFixed(2),
  `gateway, x${(K01.edgePeak / K01.readPeak).toFixed(2)}`);
for (const o of measurement) {
  write(`hop-requests/s — ${o.name}`,
    (K01.edgePeak * o["hops a request touches"]).toFixed(2),
    `hop ${o["hops a request touches"]}`);
}

measure                                          at each service    at the gateway            hybrid
addresses the client knows                                     3                 1                 1
lines collected at the edge                                    0                45                34
lines repeated across services                               135                 0                69
units carrying the responsibility (average)                    3                 1                 2
hops a request touches                                         1                 2                 2
flows the gateway touches                                      0                 3                 3
flows answered when gateway stops                            3/3               0/3               0/3

K01 — peak requests/s at the edge        513.89  read 416.67 + write 97.22
no gateway — heaviest component          416.67  tracking service
with gateway — heaviest component        513.89  gateway, x1.23
hop-requests/s — at each service         513.89  hop 1
hop-requests/s — at the gateway         1027.78  hop 2
hop-requests/s — hybrid                 1027.78  hop 2

Reading the Numbers

The table’s first three rows say what the gateway is built for. Without a gateway the client must know three addresses, and five responsibilities take up 135 lines across three services; with the gateway, addresses drop to 1 and those lines drop to 45 at the edge. The line count carrying the same work does not fall to a third — it rises from zero to 45 while 135 is zeroed out. The difference is removed repetition: the average units carrying a responsibility falls from 3 to 1, so when the rate-limiting rule changes, 1 unit needs updating instead of 3.

The next three rows give the cost. Every request now touches two hops; even with the peak edge rate held constant, hop-requests processed per second rises from 513.89 to 1027.78 — exactly double. This is not latency — no duration was measured — but it shows where latency will grow from: every request now enters one extra component’s queue.

The last row is the most expensive. Flows answered when the gateway stops drop from 3/3 to 0/3. In the gateway-free layout, one service going down drops only that flow; with a gateway, a single component drops all three at once. The hybrid layout does not fix this: validating the token in two places covers the gateway being bypassed, not the gateway itself stopping. Its real gain sits in the middle — lines repeated 69 instead of 135, lines collected at the edge 34 instead of 45 — and so does its cost: average units per responsibility is 2, so half the responsibilities are still watched in two places.

Back to the Calculation

Under the course’s rule, this decision’s counterpart in the K01 calculation gets written down. K01’s Back-of-the-Envelope Estimation lesson put the peak edge request rate at 513.89 per second (416.67 read + 97.22 write), spread across three separate endpoints; that lesson’s heaviest edge was tracking, at 416.67 requests/s. Once the gateway is in place, this spread disappears: a single component carries 513.89 requests/s, 1.23 times the former heaviest component.

More importantly, the flows the gateway touches become 3. In the What Is System Design lesson, this number was 3 for shipment-store alone, and that lesson left a rule: a design’s most fragile point is the component the most flows touch. With the gateway in place, the system has two such components — one in the data layer, one at the edge. For the edge, this is the same fact as flows answered when gateway stops reading 0/3.

What the calculation leaves unchanged matters too: reads behind cache/s at 41.67, read egress Mbit/s at 1.60, and daily data growth of 976 MB do not move. In the form built here, the gateway does not change the body per request; it only changes hop count and responsibility placement. The decisions that change the body are measured in the next two lessons.

Summary

• The API gateway decides at the transaction level and carries a service map; the load balancer picks among peer replicas, the reverse proxy forwards the request. The three can converge in one process; the distinction is the decision made.
• Work that belongs at the edge is decidable from the request’s metadata; a domain rule requiring the shipment itself stays at the service.
• Five responsibilities repeat as 135 lines across three services but drop to 45 at the edge; average units carrying a responsibility falls from 3 to 1.
• Numbers paid: every request touches 2 hops instead of 1 (hop-requests/s 513.89 → 1027.78), and flows answered when the gateway stops drop 3/3 → 0/3.
• The hybrid layout sits in the middle: 69 lines repeated, 34 collected at the edge, but average units per responsibility is 2 and the 0/3 row does not improve.
• Back to K01: 513.89 requests/s at the edge collects from a three-edge spread into one component (1.23 times the former heaviest), and flows the gateway touches becomes 3.

Next Step

This lesson used the gateway’s service map as an assumption: paths were said to map to services, but the mapping itself was not measured. The mapping does two separate jobs. The first is routing — an incoming path translates to a single service, and the client never sees the split behind it. The second does more: if a tracking page wants both a shipment’s status and its fee, the gateway can spread one request across two services and merge the responses into one body. The next lesson builds both with real local processes and measures how far aggregation cuts the client’s request count, and how many bytes it grows the body in exchange.