API Gateway Service

Up to this point, everything that woke a function came from inside: a rule, a queue, a flow step, a clock. The measurement network also has an outward-facing side — the field crew’s app, the branch screens, and the interface subscribers use to query their own consumption. These requests cannot reach the function directly; a layer that maps the route, authenticates identity, and throttles stands in between.

The gateway’s jobs were measured earlier; route mapping, authentication, and throttling were the subject of those lessons and this measurement does not repeat them. This lesson’s question is again narrow: when the gateway is a service, which decision is handed over and what number comes back in exchange?

The Configuration Surface

The handed-over decisions are familiar: the machines that run the gateway itself, their scale, certificate renewal, version upgrades. The constraint that comes back this time is not a limit but a volume. In the gateway service, every endpoint carries its own configuration set: route pattern, method, integration target, authorizer, request mapping, throttle limit, cache key, logging. Eight items, per endpoint.

The measurement network’s outward-facing surface is twelve endpoints. Total configuration items: 12 × 8 + 6 = 102. This number is not a cost but a surface: each of the 102 places can be misconfigured, and none of them show up in function code. It is the same kind of surface as the rule, the flow definition, and the schedule expression in the earlier lessons; the difference is its size.

What makes the surface dangerous is that its items are independent of each other. When a thirteenth endpoint is added, eight more items open up, and one of them — the authorizer — can be left empty. An empty authorizer produces no error; the endpoint works, a response comes back, metrics stay green. The only difference is that endpoint asks for no identity. In the previous lesson the rule’s invisibility was a surface; here that same invisibility touches an access decision directly. The way around the constraint is to generate endpoint definitions from a single template and make the authorizer field mandatory; the cost is that the gateway configuration can no longer be hand-edited, and every change goes through the release pipeline.

• ED25. The daily request mix is 50,000 and comes from three fictional sources: field app 12,000, branch screen 18,000, subscriber portal 20,000.
• ED26. The reason for rejection is authorization — an expired token, a missing scope, a tokenless request; rejection rates range from 0.03 to 0.31 depending on the source.
• ED27. The cost is a fictional unit: 0.002 gateway share per request, 0.010 per function call.
• ED28. Configuration count is 12 endpoints × 8 items plus 6 gateway-level items.
• ED29. Cold start’s extra time is 90 units, the gateway’s own share is 4 units; cold calls are distributed at fixed intervals.
• ED30. A client retry counts as a single occurrence; throttling and response caching are not measured.

Where Authorization Happens

A request rejected at the gateway never wakes the function. This sentence is true, but on its own it is incomplete — the rejection itself requires an operation, and where that operation happens changes the number. The model below runs the same daily request mix through four arrangements and counts how many function calls happen in each.

// gateway.mjs — API gateway service model: where authorization happens, how often the function wakes
// Model: not a real gateway, but the counted effect of authorization's location on call count.
const SOURCES = [                                 // daily request mix from the fictional measurement network
  { name: "field app", requests: 12000, rejectRate: 0.06 },
  { name: "branch screen", requests: 18000, rejectRate: 0.03 },
  { name: "subscriber portal", requests: 20000, rejectRate: 0.31 },
];
const REQUESTS = SOURCES.reduce((t, k) => t + k.requests, 0);
const REJECTED = SOURCES.reduce((t, k) => t + Math.round(k.requests * k.rejectRate), 0);
const PASSED = REQUESTS - REJECTED;
const CACHE_HIT = 0.85;                           // hit rate of the authorizer's cached result

const GATEWAY_RATE = 0.002, CALL_RATE = 0.010;    // fictional unit: per request and per call
const arrangements = [
  ["authorization in function", REQUESTS, 0],
  ["at gateway, uncached", PASSED, REQUESTS],
  ["at gateway, cached", PASSED, Math.round(REQUESTS * (1 - CACHE_HIT))],
  ["at gateway, built-in validation", PASSED, 0],
];

console.log("arrangement".padEnd(34) + "work calls".padStart(12) + "authorizer".padStart(16) +
  "total calls".padStart(14) + "cost".padStart(9));
for (const [name, work, authorizer] of arrangements) {
  const cost = REQUESTS * GATEWAY_RATE + (work + authorizer) * CALL_RATE;
  console.log(name.padEnd(34) + String(work).padStart(12) + String(authorizer).padStart(16) +
    String(work + authorizer).padStart(14) + cost.toFixed(1).padStart(9));
}
console.log(`requests: ${REQUESTS}, rejected: ${REJECTED}, passed: ${PASSED}`);
console.log(`configuration: 12 endpoints x 8 items + gateway-level 6 items = ${12 * 8 + 6}`);

node gateway.mjs

arrangement                         work calls      authorizer   total calls     cost
authorization in function                50000               0         50000    600.0
at gateway, uncached                     42540           50000         92540   1025.4
at gateway, cached                       42540            7500         50040    600.4
at gateway, built-in validation          42540               0         42540    525.4
requests: 50000, rejected: 7460, passed: 42540
configuration: 12 endpoints x 8 items + gateway-level 6 items = 102

The first row is authorization done in the function: all 50,000 requests wake it. Of those, 7,460 wake it only to say “no” — it validates the token, finds it invalid, and returns. No work was done, but a call was made; a slot from the concurrency pool was taken, a chance of cold start was risked.

The second row shows the opposite of what one might expect. When authorization moves to the gateway, function calls fall to 42,540, but because the authorizer function the gateway calls runs once per request, total calls climb to 92,540 and cost to 1,025.4 units. Taking rejection out of the function does not lower the call count if the thing doing the rejecting is itself a function — it raises it.

The third row is the break-even point: once the authorizer’s result is cached at an 85 percent hit rate, total calls come to 50,040 and cost to 600.4 units — almost the same as the first row. The break-even hit rate follows from a simple equation: 50,000 × (1 − h) = 7,460, so h = 85.1 percent. Below this rate, authorizing at the gateway is expensive; above it, it is cheap, and the gain scales directly with the hit rate.

The fourth row shows where the real gain sits: when the gateway can validate the token itself, no function sits in between at all. Total calls come to 42,540, cost to 525.4 — 12.4 percent lower than the first row. The measurement’s result fits one sentence: “a request rejected at the gateway never wakes the function” is a gain only if the rejection itself is not a function call.

The distribution of rejected requests is a finding on its own. Of the 7,460 rejections, 6,200 come from a single source, the subscriber portal, where the rate is 31 percent because browser sessions are short and tokens expire often. The remaining two sources together produce 1,260 rejections. So 83.1 percent of rejections come from a single client’s token-refresh behavior. This is not something the gateway configuration solves; the count drops on its own once the client refreshes its token before it expires. The lesson here is that lowering call count is not always a gateway-side setting.

A detail of the cost is hidden in these same rows. The gateway’s own share is the same in all four arrangements: 50,000 × 0.002 = 100 units. The 7,460 rejected requests, even though they never wake the function, still reach the gateway and are billed 14.9 units. Rejecting at the gateway prevents the call, not the request; a flood of unauthorized requests generates a bill without running a single function, and what stops it is not the gateway itself but the throttle limit in front of it.

There is a further gain beyond call count that does not show up in the numbers. In three of the four arrangements, the work function never sees an unauthorized request at all; authorization code sits in one place, the same way for all twelve endpoints. In the first arrangement, the same check is written separately in twelve functions, and being forgotten in one of them is a silent gap. This is what the 102-item configuration surface buys in exchange.

In the fourth arrangement, the gateway’s own share is 100 out of 525.4, or 19 percent of total cost. This ratio is independent of how much work the request does: the gateway counts requests, not duration. A long-running report query sees the gateway’s share shrink to a small slice; a short endpoint returning a handful of fields can see the gateway cost more than the function itself. The finer-grained the endpoint, the larger the gateway’s share grows, and this is the invisible cost of designing many small endpoints. Consolidating the same work into a single endpoint lowers the share; the cost is that authorization and throttle limits can no longer be separated per endpoint.

The Gateway Does Not Hide Cold Start

The gateway takes a request, routes it, and waits for the response. While waiting it can do only one thing: give up after a while. It cannot hold the request until the function warms up, nor send it elsewhere until it does. So cold start’s delay passes straight through to the client, and the gateway’s timeout threshold decides who gets the error. The scan below runs the same 42,540 calls through four different thresholds; each call’s latency and whether it is cold is the same in every row — the only thing that changes is how long the gateway waits.

// timeout.mjs — gateway service does not hide cold start: the timeout threshold is scanned
const SEED = 20260714;                        // generator seed: run is reproducible
let state = SEED;
const random = () => (state = (state * 1103515245 + 12345) % 2147483648) / 2147483648;

const N = 42540, GATEWAY = 4, COLD_EXTRA = 90;    // fictional unit: gateway's own share and cold extra
const CALLS = [];
for (let i = 0; i < N; i++) {
  const work = random() < 0.88 ? 10 + Math.floor(random() * 31)   // ordinary query: 10-40 units
                               : 60 + Math.floor(random() * 81);  // report query: 60-140 units
  const cold = i % 47 === 3;                                     // cold calls at fixed intervals
  CALLS.push({ latency: GATEWAY + work + (cold ? COLD_EXTRA : 0), cold });
}
const COLD = CALLS.filter((c) => c.cold).length;

console.log("timeout".padEnd(13) + "exceeding requests".padStart(20) + "of these, cold".padStart(17) +
  "if client retries".padStart(25));
for (const T of [60, 120, 200, 300]) {
  const exceeding = CALLS.filter((c) => c.latency > T);
  console.log(String(T).padEnd(13) + String(exceeding.length).padStart(20) +
    String(exceeding.filter((c) => c.cold).length).padStart(17) +
    String(N + exceeding.length).padStart(25));
}
console.log(`calls: ${N}, cold calls: ${COLD}, gateway share: ${GATEWAY} units, seed: ${SEED}`);

node timeout.mjs

timeout        exceeding requests   of these, cold        if client retries
60                           5929              906                    48469
120                          1864              455                    44404
200                            39               39                    42579
300                             0                0                    42540
calls: 42540, cold calls: 906, gateway share: 4 units, seed: 20260714

The bottom row looks reassuring: at a 300-unit timeout, no request exceeds it. The row above gives the real finding. At a 200-unit threshold, 39 requests timed out, and all 39 of them were cold calls. No warm call can ever reach that threshold, because the warm path’s worst case is 144 units; the cold path’s worst case is 234. When the threshold falls between the two, the exact set of clients that gets an error is the set that hit cold start.

The gateway’s own share of 4 units looks small, but where it shows up matters: it is invisible in the function’s own duration metric. The function reports finishing in 30 units while the client waited 34; on a cold call the function still reports 30 while the client waited 124. This gap between what the two sides see is the gateway service’s hidden constraint — when a latency complaint arrives, whoever is looking at function metrics finds no anomaly. The only way to see the gap is to put the gateway’s own access log total side by side with the function’s reported duration; merging the two sources is a separate job, and it is one of the costs of the gateway being a service.

At a 60-unit threshold the table says something else: all 906 cold calls exceed it. Because cold start’s own duration alone exceeds the threshold, how much work that call does no longer matters. Tightening the gateway’s timeout turns cold start from a latency problem into an error problem.

The last column is the second half of the cost. Timeout is the gateway’s decision; the function knows nothing about it and keeps running regardless. When the client gets an error and retries, at a 60-unit threshold call count climbs from 42,540 to 48,469 — 13.9 percent more calls. Worse, the first call already finished its work and wrote it; the second call now lands on a warm instance, finishes fast, and performs the same write a second time. This is the duplicate-effect problem measured in the previous two lessons, showing up on the gateway’s surface, and the fix is the same: the write has to rest on an idempotency key.

Counting what the gateway cannot do here draws its boundary. The gateway does not queue the request — a queue service is a separate thing, and routing through it would mean converting the request to asynchronous first. The gateway does not retry on its own; if it did, it would stack a second call on top of the one that already timed out, and the 48,469 figure in the last column would come from the gateway instead of the client. It does not hold the response until the function warms up, because it does not know when any function will warm. What it does is three things: map the route, validate the token, cut off the wait. Everything outside these three is built either in the client or in the function; the gateway being a service does not widen this boundary, it only lifts the burden of running these three off your hands.

There are three ways around the constraint. Raising the timeout zeroes out the 39 errors but keeps the connection open longer and delays noticing a truly stuck request. Keeping the function warm lowers cold-call count but produces idle calls; the cost of 906 cold calls has to be weighed against the cost of keeping it warm. The third is to convert heavy endpoints to asynchronous: the request is written to a queue, the gateway responds immediately, and the result is queried separately — its cost is a two-step flow on the client side and the question of where the result is held.

In the measurement network the choice is made by endpoint type. The branch screen’s consumption query is short and takes at most 44 units on the warm path; a 300-unit threshold there is more than enough. The report query can reach 234 units on the cold path, and having the client wait that long does not make sense; that endpoint is converted to asynchronous. Same gateway, same configuration surface, two different timeout values. This is also why the 102 items are 102 — items are kept separate per endpoint, because one threshold cannot be right for every endpoint.

Summary

• The gateway service carries 102 configuration items across a twelve-endpoint surface; none of them show up in function code.
• With authorization done in the function, all 50,000 requests woke it and 7,460 calls ran only to reject.
• Moving authorization to the gateway, with the rejection done by an authorizer function, pushed total calls to 92,540; an 85 percent cache hit rate brought it back to 50,040 and matched the first arrangement’s cost. The break-even hit rate is 85.1 percent.
• The gateway validating the token itself is the only arrangement with a net gain: 42,540 calls, 12.4 percent lower cost. “A request rejected at the gateway never wakes the function” is a gain only if the rejection is not itself a function call.
• At a 200-unit timeout, all 39 exceeding requests were cold calls; at 60 units, all 906 cold calls exceeded it. The gateway does not hide cold start, it only turns it from a delay into an error.
• On a timed-out request the function keeps running; when the client retries, call count climbs to 48,469 and the same write happens a second time.

Next Step

The gateway brings outside requests in. The measurement network also has an outward flow it sends: a work notification to the field crew when validation fails, a message to a subscriber when an invoice is cut. These sends are also handed to a service, and that service’s constraint resembles nothing seen so far — the send’s success is not in the sender’s hands, it depends on a decision the receiving side makes. The next lesson measures that constraint: a bounced message and a complaint come back as an event, and what happens if this feedback stream goes unprocessed.