Improper Instantiation

The four measurements so far counted the data a request carries. Part of a request’s cost, though, is never carried: a request cannot start before the objects that serve it are set up. A store connection, a serializer, a signature validator, a format parser. When these objects get set up has not been asked so far; all of them were assumed to already stand ready.

The connection pool itself was built in the Database Operations and Service Design courses; the pool size’s dependence on the connection limit was measured there. That mechanism is not repeated here. The question here is a diagnosis: which symptom points to object creation, what else gives the same symptom, and which number separates the two.

Symptom: Throughput Does Not Rise With Concurrency

The symptom appears in K01’s write path. The peak request rate reaching the store is 138.89 per second. Throughput does not rise as worker processes are added; each request’s work is constant — the limit ratio is 1.00, the bytes carried are unchanged, the field count is the same. The previous four metrics are all clean.

Cause A — the object is re-created on every request. Setup work gets multiplied by the request count and eats into the processor’s budget for the round. This is improper instantiation.

Cause B — a single shared object exists. It is never re-created, but only one request can use it at a time. Throughput is locked to the object’s hold time.

Both give the “flat throughput” symptom, and their fixes are opposite: one calls for less creation, the other for more. Moving in the wrong direction makes the symptom worse.

Measurement Setup

The run below is a model: a round is an abstract step, not a measured duration. What gets counted is the creation count, work units, waiting request-rounds, and held object-rounds. Arrivals come from a seeded generator, Poisson-distributed.

KK5 — the work that can be done in a round is 100 units; object setup is 60, use is 10 units; the object’s hold time is 3 rounds; the concurrent objects the downstream resource accepts is 24. Rationale: setup includes a handshake, authentication, and buffer allocation; use is preparing a single call; the hold is the downstream system waiting on a response; the last number says each object holds a session on the downstream side, and that this is not unlimited. Its sensitivity is given through the setup cost in the last table. The twenty-four-object limit does not bind in this run.

// object/instantiation.mjs — an expensive client being re-created on every request. MODEL:
// a round is an abstract step, not a duration; what is counted is creation, work units,
// waiting request-rounds, and held object-rounds. Arrivals come from a seeded Poisson generator.
const T = 4000, ROUNDS_PER_SEC = 100;              // 4000 rounds, 100 rounds = 1 s
const WORK = 100, SETUP = 60, USE = 10, HOLD = 3;  // KK5: per-round work budget and object costs
const LIMIT = 24;                                  // KK5: concurrent objects the downstream resource accepts
const PEAK = 138.89, AVG = 46.30;                  // K01: peak and average requests/s reaching the store

function run({ pool, lam }) {                      // pool: null means a new object every request
  let s = 20260801 % 2147483647;
  const rand = () => (s = (s * 48271) % 2147483647) / 2147483647;
  const poisson = (l) => { const L = Math.exp(-l); let k = 0, p = 1;
    do { k += 1; p *= rand(); } while (p > L); return k - 1; };
  const queue = [];                                // number of waiting requests
  let idle = 0, open = 0;                           // objects idle in the pool, objects open at the downstream resource
  const busy = [];                                  // { end } — objects currently in use
  const c = { arrived: 0, done: 0, created: 0, waiting: 0, held: 0,
    setupWork: 0, useWork: 0 };

  for (let t = 0; t < T; t += 1) {
    for (let n = busy.length - 1; n >= 0; n -= 1) if (busy[n].end <= t) {
      busy.splice(n, 1);
      if (pool === null) open -= 1; else idle += 1;       // a poolless setup discards the object
    }
    const arrived = poisson(lam);
    c.arrived += arrived;
    for (let n = 0; n < arrived; n += 1) queue.push(t);
    let budget = WORK;
    while (queue.length > 0) {
      let obj = false;
      if (pool === null) {                                // create on every request
        if (open >= LIMIT) break;
        if (budget < SETUP + USE) break;
        budget -= SETUP; c.setupWork += SETUP; c.created += 1; open += 1; obj = true;
      } else if (idle > 0) {                              // take from the pool
        idle -= 1; obj = true;
      } else if (open < Math.min(pool, LIMIT)) {          // grow the pool
        if (budget < SETUP + USE) break;
        budget -= SETUP; c.setupWork += SETUP; c.created += 1; open += 1; obj = true;
      }
      if (!obj) break;
      if (budget < USE) break;
      budget -= USE; c.useWork += USE;
      queue.shift(); c.done += 1; busy.push({ end: t + HOLD });
    }
    c.waiting += queue.length;
    c.held += open;
  }
  return c;
}

const fmt = (x) => x.toLocaleString("en-US");
const PLAN = [["create per request", null], ["single shared object", 1],
  ["pool 5", 5], ["pool 12", 12], ["pool 24", 24]];
console.log(`model: ${T} rounds (${T / ROUNDS_PER_SEC} s), per-round work budget ${WORK} units, object setup ` +
  `${SETUP} + use ${USE} units, hold ${HOLD} rounds, downstream resource limit ${LIMIT} objects (KK5); ` +
  `arrivals Poisson, seed 20260801`);

for (const [label, lam] of [["average", AVG / ROUNDS_PER_SEC], ["peak", PEAK / ROUNDS_PER_SEC]]) {
  console.log(`\n${label} load: ${(lam * ROUNDS_PER_SEC).toFixed(2)} requests/s (${lam.toFixed(4)} arrivals/round)`);
  console.log(`${"plan".padEnd(22)}${"done".padStart(8)}${"throughput/s".padStart(13)}` +
    `${"created".padStart(9)}${"creation ratio".padStart(15)}${"repeat".padStart(9)}` +
    `${"setup work share".padStart(17)}${"waiting request-rounds".padStart(23)}${"held object-rounds".padStart(19)}`);
  for (const [name, pool] of PLAN) {
    const r = run({ pool, lam });
    const repeat = r.done / Math.max(r.created, 1);
    console.log(name.padEnd(22) + fmt(r.done).padStart(8) + (r.done / (T / ROUNDS_PER_SEC)).toFixed(2).padStart(13) +
      fmt(r.created).padStart(9) + (r.created / Math.max(r.done, 1)).toFixed(4).padStart(15) +
      repeat.toFixed(2).padStart(9) +
      (r.setupWork / Math.max(r.setupWork + r.useWork, 1)).toFixed(4).padStart(17) +
      fmt(r.waiting).padStart(23) + fmt(r.held).padStart(19));
  }
}

const r1 = run({ pool: null, lam: PEAK / ROUNDS_PER_SEC }), r5 = run({ pool: 5, lam: PEAK / ROUNDS_PER_SEC });
const r24 = run({ pool: 24, lam: PEAK / ROUNDS_PER_SEC });
console.log(`\nceilings (computed, full requests per round only): create per request ` +
  `${Math.floor(WORK / (SETUP + USE)) * ROUNDS_PER_SEC} requests/s; warmed pool p gives ` +
  `${(ROUNDS_PER_SEC / HOLD).toFixed(2)} x p, work-budget ceiling ${Math.floor(WORK / USE) * ROUNDS_PER_SEC} requests/s; ` +
  `downstream resource limit ${((LIMIT / HOLD) * ROUNDS_PER_SEC).toFixed(2)} requests/s (for p=5, ${((5 / HOLD) * ROUNDS_PER_SEC).toFixed(2)})`);
console.log(`at peak load: created ${fmt(r1.created)} -> ${fmt(r5.created)} ` +
  `(${(r1.created / Math.max(r5.created, 1)).toFixed(1)}x fewer), setup work ` +
  `${fmt(r1.setupWork)} -> ${fmt(r5.setupWork)} units, waiting request-rounds ` +
  `${fmt(r1.waiting)} -> ${fmt(r5.waiting)}`);
console.log(`the pool's cost: held object-rounds is ${fmt(r5.held)} at pool 5, ${fmt(r24.held)} ` +
  `at pool 24 (${(r24.held / r5.held).toFixed(2)}x); pool 24 throughput ` +
  `${(r24.done / (T / ROUNDS_PER_SEC)).toFixed(2)} requests/s, pool 5 throughput ${(r5.done / (T / ROUNDS_PER_SEC)).toFixed(2)}`);

console.log(`\nhow the setup work share erodes with repeat (computed): share = ${SETUP} / (${SETUP} + repeat x ${USE})`);
console.log(`${"repeat".padStart(8)}${"setup work share".padStart(17)}${"work per request".padStart(17)}`);
for (const repeat of [1, 2, 5, 10, 50, 200])
  console.log(String(repeat).padStart(8) + (SETUP / (SETUP + repeat * USE)).toFixed(4).padStart(17) +
    (SETUP / repeat + USE).toFixed(2).padStart(17));

console.log(`\nhow the pool's gain changes with setup cost (computed, repeat 200)`);
console.log(`${"setup".padStart(8)}${"repeat 1".padStart(10)}${"repeat 200".padStart(12)}` +
  `${"gain factor".padStart(12)}${"units saved".padStart(18)}`);
for (const setup of [5, 10, 30, 60]) {
  const one = setup + USE, two = setup / 200 + USE;
  console.log(String(setup).padStart(8) + one.toFixed(2).padStart(10) + two.toFixed(2).padStart(12) +
    (one / two).toFixed(2).padStart(12) + (one - two).toFixed(2).padStart(18));
}

model: 4000 rounds (40 s), per-round work budget 100 units, object setup 60 + use 10 units, hold 3 rounds, downstream resource limit 24 objects (KK5); arrivals Poisson, seed 20260801

average load: 46.30 requests/s (0.4630 arrivals/round)
plan                      done throughput/s  created creation ratio   repeat setup work share waiting request-rounds held object-rounds
create per request       1,867        46.67    1,867         1.0000     1.00           0.8571                    866              5,598
single shared object     1,333        33.33        1         0.0008  1333.00           0.0045              1,141,381              3,997
pool 5                   1,870        46.75        5         0.0027   374.00           0.0158                     29             19,966
pool 12                  1,870        46.75        7         0.0037   267.14           0.0220                      2             27,439
pool 24                  1,870        46.75        7         0.0037   267.14           0.0220                      2             27,439

peak load: 138.89 requests/s (1.3889 arrivals/round)
plan                      done throughput/s  created creation ratio   repeat setup work share waiting request-rounds held object-rounds
create per request       4,000       100.00    4,000         1.0000     1.00           0.8571              3,205,927             11,997
single shared object     1,334        33.35        1         0.0007  1334.00           0.0045              8,539,260              4,000
pool 5                   5,601       140.03        5         0.0009  1120.20           0.0053                  6,456             19,990
pool 12                  5,608       140.20       12         0.0021   467.33           0.0127                     28             45,257
pool 24                  5,608       140.20       12         0.0021   467.33           0.0127                     28             45,257

ceilings (computed, full requests per round only): create per request 100 requests/s; warmed pool p gives 33.33 x p, work-budget ceiling 1000 requests/s; downstream resource limit 800.00 requests/s (for p=5, 166.67)
at peak load: created 4,000 -> 5 (800.0x fewer), setup work 240,000 -> 300 units, waiting request-rounds 3,205,927 -> 6,456
the pool's cost: held object-rounds is 19,990 at pool 5, 45,257 at pool 24 (2.26x); pool 24 throughput 140.20 requests/s, pool 5 throughput 140.03

how the setup work share erodes with repeat (computed): share = 60 / (60 + repeat x 10)
  repeat setup work share work per request
       1           0.8571            70.00
       2           0.7500            40.00
       5           0.5455            22.00
      10           0.3750            16.00
      50           0.1071            11.20
     200           0.0291            10.30

how the pool's gain changes with setup cost (computed, repeat 200)
   setup  repeat 1  repeat 200 gain factor       units saved
       5     15.00       10.03        1.50              4.97
      10     20.00       10.05        1.99              9.95
      30     40.00       10.15        3.94             29.85
      60     70.00       10.30        6.80             59.70

The run’s numbers are in the measurement class and depend on the seed; the ceilings and the last two tables are computed.

The Creation Ratio Chooses the Direction

The distinguishing measurement is the creation ratio: the number of objects created divided by the number of completed requests. At peak load it is 1.0000 for the create-per-request layout and 0.0007 for the single shared object. Both plans give the same symptom — throughput stays below the incoming 138.89 requests/s, at 100.00 and 33.35 respectively — but the diagnosis is opposite.

If the ratio is close to 1, setup work is being paid again on every request; the fix is to reduce creation. If the ratio is close to zero and throughput is still flat, the object is not being created enough; the fix is to increase creation. One number, two opposite directions.

The setup work share column gives the size of the first cause: in the create-per-request layout, 85.71% of the total work is setup. Of the 70 units of work per request, 60 are preparation that has nothing to do with the request’s own work. In pool 5, the same share drops to 0.53%.

The Pattern Does Not Show at Average Load

The average-load table carries a warning. At 46.30 requests per second, the create-per-request layout gives 46.67 requests/s — nearly the same as pool 5’s 46.75. Waiting request-rounds are 866 against 29 — visible, but small. At average load, the anti-pattern looks absent.

At peak, the table breaks. The create-per-request layout’s ceiling is, by computation, 100 requests/s: a round has 100 units of work, and a request with its setup takes 70 units, so one request fits per round. The incoming 138.89 requests/s is above that ceiling, and the difference is written to the queue: waiting request-rounds climbs from 866 to 3,205,927 — 3702x. In pool 5, the same number climbs from 29 to 6,456, 223x.

The single shared object lands at the same place in both loads: 33.33 and 33.35 requests/s. Its ceiling does not come from the load, it comes from the hold — with 100 rounds per second and a 3-round hold, one object can carry 33.33 requests per second. This plan falls short even at average load; waiting request-rounds is 1,141,381.

The Condition Where Creating Per Request Is Correct

Creating an object on every request is not a mistake by itself; the mistake is creating an expensive object on every request, and the measure of expense is the ratio of setup work to usage work.

The setup-share table gives how this ratio erodes with the repeat count. At repeat 1, the share is 0.8571 and work per request is 70 units; at repeat 10, 0.3750 and 16 units; at repeat 200, 0.0291 and 10.30 units. The gain is bounded not by the repeat count itself but by the size of the setup: the last table shows that at the same repeat, a setup of 60 units gains 6.80x, while a setup of 5 units gains only 1.50x.

The condition can be written with one number: if setup work is less than half of usage work, the most the pool can gain is 1.50x — at this model’s scale, setup is 5 units and usage is 10 units. Since the pool also brings held object-rounds and maintenance load, not building a pool is defensible at this scale. When the same object’s setup rises to 60 units, the same decision means leaving 6.80x on the table.

What Grows in Exchange for the Pool

The pool grows two things at once, and both are countable.

Held object-rounds. Pool 5 holds 19,990 object-rounds at peak load; pool 24 holds 45,257 — 2.26x. These objects hold a session at the downstream resource and eat into KK5’s twenty-four-object limit. This is the cost of growing the pool, and the cost does not show up in throughput: pool 24 completes 140.20 requests per second, pool 5 completes 140.03. The difference is 0.12%.

But it shows up in waiting. Pool 5’s waiting request-rounds is 6,456, pool 12’s is 28. Growing the pool from 5 to 12 grows the resource 2.26x while shrinking waiting 231x. The two numbers move in opposite directions, and the choice depends on which one gives the binding ceiling. Pool 24 gives the exact same result as pool 12 — demand never exceeds twelve, so anything past twelve is only held resource.

A third exchange is not counted but is visible: if an object taken from the pool is never returned, the pool shrinks permanently. In the create-per-request layout, such a leak slows one request and ends there; in the pooled layout, it permanently narrows the system. The pool turns a bug from transient into permanent.

Summary

• The symptom is throughput not rising with concurrency, and two opposite causes give the same symptom: creating per request (100.00 requests/s at peak) and a single shared object (33.35 requests/s).
• The distinguishing measurement is the creation ratio: at 1.0000, creation is reduced; near zero, like 0.0007, creation is increased. One number, two opposite directions.
• The pattern does not show at average load: at 46.30 requests/s, the create-per-request layout gives 46.67, the pool gives 46.75. At peak, waiting request-rounds climbs from 866 to 3,205,927 (3702x).
• Setup work’s share erodes with repeat: 0.8571 and 70 units per request at repeat 1, 0.0291 and 10.30 units at repeat 200. Pool 5 brings creation down from 4,000 to 5 (800x).
• The condition is the ratio of setup to usage: at setup 5 and usage 10 units, the most the pool can gain is 1.50x; at setup 60 units, 6.80x.
• The pool’s cost is held object-rounds: pool 24 holds 2.26x the resource of pool 5 while raising throughput by 0.12%; in exchange, pool 12 brings waiting down from 6,456 to 28.

Next Step

All five patterns were measured in the same place: inside a single process and in front of a single store. The computation’s layer, the client’s work, the call’s granularity, the field’s breadth, and the object’s lifetime — all of them counted a request’s own waste, and every fix stayed inside that process or that response. The assumption shared by all five was never questioned: every one of these requests goes to the same store. The tracking query, the state event, and the end-of-day billing run through the same engine, the same resource pool, the same maintenance window. Why the store is single was never a design decision — it was an assumption. The next lesson tests that assumption starting from a symptom: which measurement reveals the single store’s limit, and what grows in exchange for splitting it.