Chatty I/O

The previous lesson shrank the load: the server now sends 25 records instead of 200, and it decides for itself which 25 to show. That decision had a cost, and the cost was never counted. As the user moves through the list, the decision is renewed; opening a row asks for its detail; applying a filter re-fetches the list. Call count grew.

This is the service-call-scale counterpart of the N+1 query problem established in the Data Access Layer and Business Logic course. The mechanics there — query count growing with record count, batch fetching, joining — are not repeated here. The question here is different: by how much do the bytes moved grow in return for cutting call count, and by what measurement is the point where this trade-off reverses found.

Symptom: A Slowdown That Shows Up in No Metric

The panel open is slow end to end. Every call’s server-side duration is small and constant; none crosses an edge threshold. Total useful payload is around nine kilobytes, so transfer cannot be the explanation. On the store side, the boundary ratio is 1.00 — the first lesson’s measurement is clean. The symptom slips through every metric.

Chatty I/O is a job split into a large number of small calls. But naming it is not enough again, because the opposite arrangement produces the same symptom.

Cause A — call count. Every call carries an envelope independent of its body: the request line, identity, trace context, content headers, response headers. The envelope is fixed; it multiplies with the call count. And if calls wait on each other, the round count grows too.

Cause B — over-packing. Call count has been cut to one, but that single call also carries what is not needed. The envelope share is near zero, and most of what moves is never used.

Both produce the symptom “calls are fast but the open is slow,” and their fixes are opposites.

The Measurement Rig

The rig below is a model: a round is an abstract step and duration is not measured. What is measured is bytes, and the envelope’s byte count is not invented — it is actually computed from the header text in the code.

KK3 — calls the client can hold open at once: 6. Rationale: dependent calls do not complete in one round; they complete in as many rounds as this limit divides them into. Its sensitivity shows up in the round column.

Three granularities are compared. G1 makes one call each for the list, the seller, and the zone, plus one detail call for every row opened. G2 fetches the list and all twenty-five details in one call, leaving the seller and the zone separate. G3 fetches everything in one call. The variable $k$ is how many of the twenty-five rows actually have their detail opened.

// calls/granularity.mjs — the same panel open measured at three call granularities. Measure:
// the useful share of bytes moved, and its two components (envelope share, unused-payload share).
// MODEL: a round is an abstract step, duration is not measured; the envelope's bytes are measured
// from the header text below.
const VISIBLE = 25, EVENT = 7, TARIFF = 40, ZONE = 12;   // KK2 = 25 rows, K01 V4 = 7 events
const CONCURRENCY = 6;                                    // KK3: calls the client keeps open at once

const ENVELOPE = ["GET /panel/list HTTP/1.1", "host: edge", "accept: application/json",
  "accept-encoding: gzip", "authorization: Bearer <token>", "traceparent: <context>",
  "HTTP/1.1 200 OK", "content-type: application/json", "cache-control: max-age=30",
  "etag: <version>", "content-length: <n>", "traceparent: <context>"].join("\r\n") + "\r\n\r\n";
const ENV = Buffer.byteLength(ENVELOPE);

const bytes = (x) => Buffer.byteLength(JSON.stringify(x));
const list = () => Array.from({ length: VISIBLE }, (_, i) => ({ no: `TR-${i}`,
  updatedAt: (i * 97) % 100000, state: i % 5, weight: 1 + (i % 30) }));
const detail = (i) => ({ no: `TR-${i}`, events: Array.from({ length: EVENT }, (_, j) => ({
  code: (i * 3 + j) % 5, ts: (i * 97 + j * 13) % 100000, route: `R-${(i + j) % 40}` })) });
const seller = () => ({ name: "seller-10", contract: 0.07,
  tariff: Array.from({ length: TARIFF }, (_, i) => ({ id: i, rate: 12 + (i % 7) * 0.5 })) });
const zone = () => ({ zones: Array.from({ length: ZONE }, (_, i) => ({ id: i, coefficient: 1 + (i % 5) * 0.1 })) });

const L = bytes(list()), S = bytes(seller()), Z = bytes(zone());
const A = Array.from({ length: VISIBLE }, (_, i) => bytes(detail(i)));
const firstK = (k) => A.slice(0, k).reduce((a, x) => a + x, 0);
const ALL_DETAIL = firstK(VISIBLE);

// k: how many of the 25 rows actually have their detail opened
const PLAN = {
  "G1 chatty": (k) => ({ calls: 3 + k, rounds: 1 + Math.ceil(k / CONCURRENCY), payload: L + S + Z + firstK(k) }),
  "G2 batched": (k) => ({ calls: 3, rounds: 1, payload: L + S + Z + ALL_DETAIL }),
  "G3 single call": (k) => ({ calls: 1, rounds: 1, payload: L + S + Z + ALL_DETAIL }),
};
const measure = (name, k) => {
  const p = PLAN[name](k), envelope = p.calls * ENV, moved = p.payload + envelope;
  const used = L + S + Z + firstK(k);
  return { ...p, envelope, moved, used, envelopeShare: envelope / moved,
    unusedShare: (p.payload - used) / moved, usefulShare: used / moved };
};

const NAMES = Object.keys(PLAN), say = (x) => x.toLocaleString("en-US");
console.log(`model: the panel shows ${VISIBLE} rows, envelope ${ENV} bytes (measured from the header ` +
  `text below), concurrent call limit ${CONCURRENCY} (KK3); list ${L}, seller ${S}, zone ${Z}, ` +
  `${VISIBLE} details ${say(ALL_DETAIL)} bytes`);
for (const k of [25, 3]) {
  console.log(`\n${k} of the ${VISIBLE} rows have their detail opened`);
  console.log(`${"plan".padEnd(16)}${"calls".padStart(7)}${"rounds".padStart(8)}${"bytes moved".padStart(14)}` +
    `${"envelope share".padStart(16)}${"unused payload".padStart(16)}${"useful share".padStart(14)}`);
  for (const name of NAMES) {
    const o = measure(name, k);
    console.log(name.padEnd(16) + String(o.calls).padStart(7) + String(o.rounds).padStart(8) +
      say(o.moved).padStart(14) + o.envelopeShare.toFixed(4).padStart(16) +
      o.unusedShare.toFixed(4).padStart(16) + o.usefulShare.toFixed(4).padStart(14));
  }
}

console.log(`\nwhich plan moves less, depending on how many details are opened (cold open)`);
console.log(`${"opened".padStart(7)}${"G1 bytes".padStart(11)}${"G2 bytes".padStart(11)}` +
  `${"G1/G2".padStart(8)}${"G1 rounds".padStart(11)}${"moves least".padStart(17)}`);
for (const k of [1, 3, 6, 12, 13, 18, 25]) {
  const g = NAMES.map((name) => measure(name, k));
  const least = NAMES[g.indexOf(g.reduce((a, b) => (b.moved < a.moved ? b : a)))];
  console.log(String(k).padStart(7) + say(g[0].moved).padStart(11) + say(g[1].moved).padStart(11) +
    (g[0].moved / g[1].moved).toFixed(3).padStart(8) + String(g[0].rounds).padStart(11) + least.padStart(17));
}

const PIECE = { list: 5, detail: 20, seller: 100, zone: 1000 };   // opens between changes, per piece
const CONTENT = { "G1 chatty": [["list"], ["seller"], ["zone"],
    ...Array.from({ length: VISIBLE }, () => ["detail"])],
  "G2 batched": [["list", "detail"], ["seller"], ["zone"]],
  "G3 single call": [["list", "detail", "seller", "zone"]] };
const BODY = { "G1 chatty": [L, S, Z, ...A], "G2 batched": [L + ALL_DETAIL, S, Z],
  "G3 single call": [L + ALL_DETAIL + S + Z] };
const T = 100;
const result = {};
for (const name of NAMES) {
  let hits = 0, moved = 0, envelope = 0;
  for (let t = 1; t <= T; t += 1) CONTENT[name].forEach((pieces, i) => {
    const changed = t === 1 || pieces.some((p) => t % PIECE[p] === 0);
    moved += ENV + (changed ? BODY[name][i] : 0);   // a hit still pays the envelope: a conditional request
    envelope += ENV;
    if (!changed) hits += 1;
  });
  result[name] = { hit: hits / (T * CONTENT[name].length), moved, envelope };
}
console.log(`\nover ${T} opens, all details are opened, every open validates with a conditional request; ` +
  `the piece change interval is list ${PIECE.list} / detail ${PIECE.detail} / ` +
  `seller ${PIECE.seller} / zone ${PIECE.zone} opens`);
console.log(`${"plan".padEnd(16)}${"hit".padStart(9)}${"bytes moved".padStart(14)}` +
  `${"envelope bytes".padStart(16)}${"envelope share".padStart(15)}${"vs. G3".padStart(9)}`);
for (const name of NAMES) {
  const o = result[name];
  console.log(name.padEnd(16) + o.hit.toFixed(4).padStart(9) + say(o.moved).padStart(14) +
    say(o.envelope).padStart(16) + (o.envelope / o.moved).toFixed(4).padStart(15) +
    (o.moved / result["G3 single call"].moved).toFixed(3).padStart(9));
}

model: the panel shows 25 rows, envelope 282 bytes (measured from the header text below), concurrent call limit 6 (KK3); list 1318, seller 871, zone 331, 25 details 6,784 bytes

25 of the 25 rows have their detail opened
plan              calls  rounds   bytes moved  envelope share  unused payload  useful share
G1 chatty            28       6        17,200          0.4591          0.0000        0.5409
G2 batched            3       1        10,150          0.0833          0.0000        0.9167
G3 single call        1       1         9,586          0.0294          0.0000        0.9706

3 of the 25 rows have their detail opened
plan              calls  rounds   bytes moved  envelope share  unused payload  useful share
G1 chatty             6       2         4,989          0.3391          0.0000        0.6609
G2 batched            3       1        10,150          0.0833          0.5918        0.3248
G3 single call        1       1         9,586          0.0294          0.6266        0.3439

which plan moves less, depending on how many details are opened (cold open)
 opened   G1 bytes   G2 bytes   G1/G2  G1 rounds      moves least
      1      3,902     10,150   0.384          2        G1 chatty
      3      4,989     10,150   0.492          2        G1 chatty
      6      6,624     10,150   0.653          2        G1 chatty
     12      9,933     10,150   0.979          3   G3 single call
     13     10,492     10,150   1.034          4   G3 single call
     18     13,287     10,150   1.309          4   G3 single call
     25     17,200     10,150   1.695          6   G3 single call

over 100 opens, all details are opened, every open validates with a conditional request; the piece change interval is list 5 / detail 20 / seller 100 / zone 1000 opens
plan                  hit   bytes moved  envelope bytes envelope share   vs. G3
G1 chatty          0.9379       860,055         789,600         0.9181    3.847
G2 batched         0.9200       256,815          84,600         0.3294    1.149
G3 single call     0.7900       223,584          28,200         0.1261    1.000

Useful Share Names the Cause

The differentiating measurement is the useful share of bytes moved: the bytes actually used, divided by the total bytes moved including the envelope. Where the loss goes shows up in two components, and whichever is bigger names the cause.

When all twenty-five rows are opened, G1’s useful share is 0.5409 — nearly half of what moves is envelope (0.4591). Unused payload is zero: not a single byte moves for nothing, and the plan is still the most expensive. The diagnosis is chattiness.

When three rows are opened, the table changes direction. G3’s useful share is 0.3439, and the source of the loss is not the envelope but unused payload: 0.6266. G1 is best in the same case, at 0.6609. The diagnosis is over-packing.

One measurement, two components, two opposite fixes. If the envelope share is large, calls are merged; if the unused-payload share is large, calls are split. Looking at total bytes does not say this — at k = 3, G3’s 9,586 bytes are the same as its 9,586 bytes at k = 25, yet one is healthy and the other is waste.

Where the Trade-Off Turns

The third table bounds the trade-off with a number. How many of the twenty-five rows have their detail opened decides which plan moves less, and the turning point sits between 12 and 13. At twelve rows, G1 moves 9,933 bytes, G3 moves 9,586; at thirteen rows, G1 climbs to 10,492 and passes G2 by a factor of 1.034.

At one row opened, the ratio is 0.384 — the chatty plan moves less than a third of what G2 moves.

This is the number for the condition under which the antipattern is correct: if the usage ratio of the moved payload is below 12/25 = 0.48, many small calls move less. Same panel, same system, same envelope — the only thing that changes is how many rows the user opens. The pattern is not a prohibition; it is a choice bound to a ratio.

The round column is the second face of the trade-off, and it does not point the same way as bytes. G1 takes 2 rounds at one row, 6 rounds at twenty-five rows; G2 and G3 take 1 round in every case. KK3’s six-call limit shows up here: even if calls go out in parallel, the limit splits them into rounds. The plan that moves fewer bytes can still make the caller wait through more rounds.

A High Hit Rate Does Not Mean Fewer Bytes

The last table breaks an intuition. Over a hundred opens, the chatty plan has the highest cache hit rate: 0.9379, against G3’s 0.7900. The reason is granularity — small calls go stale independently of each other, while a combined response goes stale on whichever of its parts changes most often.

In spite of that, G1 moves 3.85 times more bytes overall: 860,055 against 223,584. The table gives the reason — 789,600 of what moves is envelope, 91.81 percent of it. A call that hits does not move its body, but it still pays its envelope; a conditional request is still a request.

The rule is this: a hit rate is not a granularity measure. A plan with twenty-eight calls can push its hit rate to 0.94 and still move nearly four times what a single-call plan moves. The two numbers have to be read together: hit rate and envelope share.

What Grows in Return

Moving from G1 to G2 cuts bytes moved from 3.85 times down to 1.15 times G3’s baseline. Three things grow in return.

Cache hit rate drops: 0.9379 to 0.9200, down to 0.7900 for the single call. Because the list changes once every five opens, every piece bundled with it now goes stale once every five opens too; the detail’s own twenty-open lifespan is lost.

On the server side, an assembly job is born: the single call has to gather twenty-five details, and that gathering runs on every request at K01’s peak load. The first lesson’s question comes back here — which layer this assembly belongs in is a separate decision.

Failure isolation is lost: in the twenty-eight-call plan, one detail call dropping leaves one row missing; in the single-call plan, the same failure leaves the whole panel blank. Graceful degradation from the Resilience and Reliability course is bound to granularity for exactly this reason; it is not retold here, but it is part of the price paid when moving to a batched call.

Summary

• The symptom does not show up in any call metric: every call is fast, total useful payload is small, and the open is still slow.
• The differentiating measurement is the useful share of bytes moved; the larger loss component names the cause. At k = 25, G1’s envelope share is 0.4591 (chattiness); at k = 3, G3’s unused-payload share is 0.6266 (over-packing).
• The fixes are opposites: if the envelope share is large, calls are merged; if the unused-payload share is large, they are split. Total bytes alone does not separate the two.
• The trade-off’s turning point is measured: below a usage ratio of 12/25 = 0.48, many small calls move less (a ratio of 0.384 at one row); above it, a single call wins (1.695 at twenty-five rows).
• Round count does not point the same way as bytes: G1 takes 6 rounds at twenty-five rows, because of KK3’s six-call concurrency limit.
• A high hit rate does not mean fewer bytes: G1’s hit rate is 0.9379, but the bytes it moves are 3.85 times G3’s, because 91.81 percent of what moves is envelope, and a conditional request pays the envelope too.

Next Step

Batching the call cut what moves down to a third, but nobody looked inside what it carries. Twenty-five details hold 6,784 bytes, and inside those bytes are fields the panel never reads: every event’s carrier code, zone id, full timestamp. The unused-payload share cannot see this, because that measurement works at record granularity; waste happens at field granularity too. The next lesson takes on that layer: what over-fetched fields grow end to end — entries that fit in cache, hit rate, requests reaching the store, fields serialized, and network egress — and which decision hardens in return for trimming the unnecessary bytes.