Request Processing and Routing

The previous lesson wrote down what can run at the edge: six of thirteen interface items, four of the fictional network’s eight workloads. But being able to run somewhere is not a placement decision — the same four jobs also run in the region and at the origin. In the fictional measurement network, every request from a branch site goes to the region today; knowing that part of it could be decided at the edge does not say which part should be.

This lesson measures one thing: how many network round trips and milliseconds a request pays depending on where the decision is made. Once trips are counted, placement stops being a matter of preference and becomes arithmetic.

EC7. Trips are counted as full round trips (there and back); one-way latency is not modeled. EC8. Round-trip and processing times are model inputs; they correspond to the fictional network’s edge, region, and origin triad. EC9. A request is processed in at most one place; the same job is never done in two places at once.

Three Places, Three Trips

The path has three legs. The client-to-edge leg is short, because the edge sits close to the branch. The edge-to-region leg is long. The region-to-origin leg is short again, because the two sit in the same region. Wherever a request is processed, every leg up to that point is paid for.

// measurement-network/placement.mjs — the same request processed at three places: trips and latency
// MODEL: leg times are a declared model input; round trips are counted, not one-way hops.

export const LEG = { "client-edge": 8, "edge-region": 60, "region-origin": 20 };
export const PLACE = {
  "edge":   { leg: ["client-edge"], processMs: 2 },
  "region": { leg: ["client-edge", "edge-region"], processMs: 6 },
  "origin": { leg: ["client-edge", "edge-region", "region-origin"], processMs: 15 },
};
export const PLACES = Object.keys(PLACE);
export const trips = (y) => PLACE[y].leg.length;
export const latency = (y) =>
  PLACE[y].leg.reduce((t, b) => t + LEG[b], 0) + PLACE[y].processMs;

// Four job types and where each one can be done.
export const WORK = {
  "routing":            { place: ["edge", "region", "origin"], ends: true },
  "header editing":     { place: ["edge", "region", "origin"], ends: false },
  "authentication":     { place: ["edge", "region", "origin"], ends: true },
  "a/b split":          { place: ["edge", "region", "origin"], ends: true },
  "relational query":   { place: ["origin"], ends: false },
  "transactional write":{ place: ["origin"], ends: false },
  "dataset scan":       { place: ["region", "origin"], ends: false },
};

if (import.meta.main) {
console.log("a request processed at three places (model input)");
console.log("place".padEnd(10) + "trips".padEnd(10) + "net time".padEnd(12) +
  "process".padEnd(10) + "total".padEnd(10) + "vs edge");
for (const y of PLACES) {
  const net = PLACE[y].leg.reduce((t, b) => t + LEG[b], 0);
  console.log(y.padEnd(10) + String(trips(y)).padEnd(10) + `${net} ms`.padEnd(12) +
    `${PLACE[y].processMs} ms`.padEnd(10) + `${latency(y)} ms`.padEnd(10) +
    `+${latency(y) - latency("edge")} ms`);
}
console.log("network time per leg: " +
  Object.entries(LEG).map(([b, v]) => `${b} ${v} ms`).join(", "));

console.log("\nwhere the job type can be done (+ can, - cannot)");
console.log("job type".padEnd(24) + PLACES.map((y) => y.padEnd(10)).join("") + "ends at edge");
for (const [ad, v] of Object.entries(WORK))
  console.log(ad.padEnd(24) + PLACES.map((y) => (v.place.includes(y) ? "+" : "-").padEnd(10)).join("") +
    (v.place.includes("edge") ? (v.ends ? "yes" : "no") : "-"));
const atEdge = Object.values(WORK).filter((v) => v.place.includes("edge")).length;
console.log(`can be done at the edge: ${atEdge}/${Object.keys(WORK).length}`);
}

a request processed at three places (model input)
place     trips     net time    process   total     vs edge
edge      1         8 ms        2 ms      10 ms     +0 ms
region    2         68 ms       6 ms      74 ms     +64 ms
origin    3         88 ms       15 ms     103 ms    +93 ms
network time per leg: client-edge 8 ms, edge-region 60 ms, region-origin 20 ms

where the job type can be done (+ can, - cannot)
job type                edge      region    origin    ends at edge
routing                 +         +         +         yes
header editing          +         +         +         no
authentication          +         +         +         yes
a/b split               +         +         +         yes
relational query        -         -         +         -
transactional write     -         -         +         -
dataset scan            -         +         +         -
can be done at the edge: 4/7

The round-trip count is 1, 2, and 3; latency is 10, 74, and 103 ms. Moving from the edge to the region adds +64 ms, moving from the region to the origin adds +29 ms. The two steps are not equal, and the reason is the leg times: the long leg sits between the edge and the region. The weight of the placement decision is therefore concentrated in the first step — moving a job from the region to the origin is a minor detail, while moving it from the edge to the region is roughly two-thirds of the latency.

Processing times run the opposite way and do not soften the decision. It is 2 ms at the edge, 6 ms in the region, 15 ms at the origin; the farthest place is also the slowest to process, because the work done there is heavier. Across all three items, network time drowns out processing time: at the origin, 88 ms of network against 15 ms of processing.

Counting trips as full round trips (EC7) also determines how the number should be read: a placement’s cost is not a one-way distance but a duration that includes the response coming back. If a decision made at the edge has to ask the origin a question and wait, that wait adds one more row to the same table. What the model leaves out shows up here too: a job that makes two independent data requests in sequence cannot be represented by a single row — it pays two round trips. This is why the job type best suited to the edge is the one that carries everything the decision needs inside the request itself.

The second table places seven job types. Four can be done at all three places, and three of those can close out the request at the edge: routing produces a response, authentication produces a rejection, the A/B split returns a ready copy. Header editing runs at the edge but does not close out the request — the header is written and the request continues on its path. This distinction determines the next count: a job being able to run at the edge, and that job sparing a visit to the origin, are not the same thing.

The Request That Ends at the Edge

Now a request set is run through these four jobs in sequence. The sequence matters: a request that closes out at the routing stage never enters authentication, so the same request is never counted twice.

EC10. The set is 10,000 requests, and the number of requests ending at the edge per stage is a model input. EC11. In the A/B split, the copy ready at the edge is an input; cache hit rate is not re-measured in this lesson. EC12. A request processed at the edge and passed along adds the edge’s own processing time to the path toward the origin.

// measurement-network/filter.mjs — a request set run through a sequential filter at the edge
// MODEL: the number of requests ending at the edge per stage is a declared model input.
import { PLACE, PLACES, WORK, trips, latency } from "./placement.mjs";

const SET = 10_000;
const STAGE = [
  ["routing", 1_240, "a stale address closes out at the edge with a permanent redirect"],
  ["authentication", 860, "an invalid token is rejected at the edge"],
  ["a/b split", 2_100, "the variant is chosen at the edge and a ready copy is returned"],
  ["header editing", 0, "the header is written and the request continues on its path"],
];

console.log(`${SET} requests, sequential filter at the edge (model input)`);
console.log("stage".padEnd(20) + "touched".padEnd(12) + "ends at edge".padEnd(16) +
  "remaining".padEnd(11) + "share of set");
let remaining = SET, ended = 0;
for (const [ad, done, _] of STAGE) {
  const touched = remaining;
  remaining -= done; ended += done;
  console.log(ad.padEnd(20) + String(touched).padEnd(12) + String(done).padEnd(16) +
    String(remaining).padEnd(11) + `${((done / SET) * 100).toFixed(1)}%`);
}
console.log("never reaches origin: " + ended + `/${SET}  ` +
  `${((ended / SET) * 100).toFixed(1)}%   goes to origin: ${remaining}`);

// The weighted latency and total network trips of two scenarios.
const EDGE_EXTRA = PLACE["edge"].processMs;
const scenario = {
  "all at origin": [[SET, latency("origin"), trips("origin")]],
  "four jobs at edge": [[ended, latency("edge"), trips("edge")],
                        [remaining, latency("origin") + EDGE_EXTRA, trips("origin")]],
};
console.log("\nweighted latency and total network trips");
const avg = {};
for (const [ad, row] of Object.entries(scenario)) {
  const ms = row.reduce((t, [n, g]) => t + n * g, 0) / SET;
  const tripCount = row.reduce((t, [n, , k]) => t + n * k, 0);
  avg[ad] = ms;
  console.log("  " + ad.padEnd(20) + `${ms.toFixed(1)} ms/req   ` +
    `${tripCount} trips   ${(tripCount / SET).toFixed(2)} trips/req`);
}
const gap = avg["all at origin"] - avg["four jobs at edge"];
console.log(`  ${"gain".padEnd(20)}${gap.toFixed(1)} ms/req   ` +
  `${((gap / avg["all at origin"]) * 100).toFixed(1)}%`);

// The marginal cost of deciding one step further out instead of at the edge.
console.log("\ncost of moving the same decision one step further out");
for (let i = 1; i < PLACES.length; i++) {
  const [o, y] = [PLACES[i - 1], PLACES[i]];
  console.log("  " + `${o} -> ${y}`.padEnd(20) +
    `+${trips(y) - trips(o)} trips   +${latency(y) - latency(o)} ms/req   ` +
    `+${(((latency(y) - latency(o)) * SET) / 1000).toFixed(1)} s per set`);
}

const REASON = {
  "relational query": "joins many tables; the data is not at the edge",
  "transactional write": "needs strict ordering and rollback; there is no transaction boundary at the edge",
  "dataset scan": "input is larger than the memory baseline; there is no disk at the edge",
};
console.log("\nwork that cannot be done at the edge");
for (const [ad, v] of Object.entries(WORK)) {
  if (v.place.includes("edge")) continue;
  console.log("  " + ad.padEnd(24) + `nearest place: ${v.place[0].padEnd(8)}` + REASON[ad]);
}

10000 requests, sequential filter at the edge (model input)
stage               touched     ends at edge    remaining  share of set
routing             10000       1240            8760       12.4%
authentication      8760        860             7900       8.6%
a/b split           7900        2100            5800       21.0%
header editing      5800        0               5800       0.0%
never reaches origin: 4200/10000  42.0%   goes to origin: 5800

weighted latency and total network trips
  all at origin       103.0 ms/req   30000 trips   3.00 trips/req
  four jobs at edge   65.1 ms/req   21600 trips   2.16 trips/req
  gain                37.9 ms/req   36.8%

cost of moving the same decision one step further out
  edge -> region      +1 trips   +64 ms/req   +640.0 s per set
  region -> origin    +1 trips   +29 ms/req   +290.0 s per set

work that cannot be done at the edge
  relational query        nearest place: origin  joins many tables; the data is not at the edge
  transactional write     nearest place: origin  needs strict ordering and rollback; there is no transaction boundary at the edge
  dataset scan            nearest place: region  input is larger than the memory baseline; there is no disk at the edge

The share of requests that never reaches the origin is read separately for each of the four job types: routing 12.4%, authentication 8.6%, A/B split 21.0%, header editing 0.0%. In total, 4,200 requests never see the origin at all — 42.0% of the set. Header editing producing zero does not make it worthless; the work it does lightens the origin’s code, not its load. But it has no counterpart in the trip count, and the table shows exactly that.

Round trips per request drop from 3.00 to 2.16, weighted latency from 103.0 ms to 65.1 ms; the gain is 37.9 ms, or 36.8%. This number is the direct counterpart of the 42.0%: every request that does not go to the origin moves from the three-trip path to the one-trip path. The 5,800 requests that do go to the origin, however, pay 105 ms, not 103; being processed and passed along at the edge carries a 2 ms cost of its own, and it shows up in the average.

The filter’s order is also a decision, visible in the “touched” column. Authentication sits behind routing, so it looks at 8,760 requests instead of 10,000; the A/B split sits behind both, so it looks at 7,900. Reversing the order would not change the total ending at the edge — 4,200 would still be 4,200 — but the A/B split would then also compute a variant for the 860 requests rejected anyway. What a sequential filter gains is not trips but processing time spent at the edge; the cheap, high-elimination stage goes first.

The marginal table reads the same decision from the other direction. Moving an edge-eligible decision to the region costs +64 ms per request, 640 seconds across ten thousand requests; moving it from the region to the origin costs +29 ms and 290 seconds. In the fictional network, deciding authentication in the region instead of at the edge therefore creates more than ten minutes of total waiting across ten thousand requests — and 860 of those requests will be rejected anyway.

Work That Cannot Be Done at the Edge

The decision handed over in this lesson is placement: the application does not choose which node meets the request. What comes back in exchange is again a number — 3 of the seven job types cannot be done at the edge, and all three are named. The relational query joins multiple tables, and the data to join is not at the edge. The transactional write needs strict ordering and rollback, and no transaction boundary can be established at the edge. The dataset scan’s input is larger than the memory baseline, and there is no disk at the edge. The last two follow directly from the previous lesson’s missing-item list: no local disk persistence, and no durable log for the transactional write to rely on.

The nearest place differs for these three, and the difference is a design lever. The dataset scan can be done in the region; the relational query and the transactional write, only at the origin. So not every job that cannot run at the edge is pushed to the same distance: keeping a job in the region wins back 29 ms per request compared to moving it to the origin. In the fictional network, nightly reconciliation therefore stays in the region while invoice finalization drops all the way to the origin — both leave the edge, but not to the same distance.

How to work around the constraint is clear in this lesson’s own terms, and so is its cost. For the relational query to move to the edge, its result would have to be precomputed and sit somewhere at the edge; the question then stops being a query and becomes a read. For the transactional write to move to the edge, a single writer that guarantees ordering would have to sit at the edge. Both workarounds want the same thing: state that sits at the edge, readable and writable. This lesson’s model never assumed that state existed — all four jobs that ran were stateless.

Summary

• Three placements cost three trip counts: edge 1 trip / 10 ms, region 2 trips / 74 ms, origin 3 trips / 103 ms. The long leg sits between the edge and the region, so the first step is +64 ms, the second +29 ms.
• 4 of the seven job types can be done at the edge; 3 of those close out the request at the edge. Being able to run at the edge and sparing a visit to the origin are not the same thing — header editing is the example.
• The shares of requests that never reach the origin: routing 12.4%, authentication 8.6%, A/B split 21.0%, header editing 0.0%. The total is 4,200/10,000, or 42.0%.
• Round trips per request drop from 3.00 to 2.16, weighted latency from 103.0 ms to 65.1 ms; the gain is 36.8%. A request processed and passed along at the edge pays a 2 ms extra on the path to the origin.
• The three job types that cannot be done at the edge are named: the relational query, the transactional write, the dataset scan. Two of them have the origin as their nearest place, one has the region; the difference between them is 29 ms per request.

Next Step

All four jobs in this lesson were stateless: routing looked at a table, authentication at a signature, the A/B split at a split rule. The three jobs that cannot be done at the edge, however, want state. The question becomes: can a value be read at the edge, can it be written? A read is fast close to the edge — but where does a write go, and when does the written value show up at other edge sites? When two edge sites read the same key, for how long do they answer differently, and how many reads go stale in that window? How many round trips does reaching a large object from the edge cost, and which way does it push the egress cost? The next lesson measures edge storage separately along the read and write axes, scans the propagation window, and counts the store’s size limit as a constraint.