Cost Awareness

The previous lesson produced one number: how many nodes. Alongside it, it wrote a resource unit too — 1,440 node-hours a month today, 5,760 at ten times the scale. But what those nodes cost, and how every course’s design decisions up to this point changed operating cost, was never asked. What does raising the cache hit ratio reduce, what does splitting the data multiply, which line item does adding a replica grow, where does monitoring’s own cost show up.

This lesson takes up that question and applies the course’s diagnostic rule one last time: a symptom is a number, two causes are set up that give the same symptom, and the measurement that separates them is shown.

Cost Is Written in Resource Units

In this course, operating cost is not written in currency. A price depends on a provider, a contract, and a date; read five years later, it is wrong. A resource unit does not depend on those: processor-seconds, GB-months, node-hours, GB transferred, touches per request. When a design decision’s operating cost is written in these units, the comparison stays valid even if the price changes.

Unit cost in this course is denominated in resources: the resource one unit of work — one request or one record — consumes. All the inputs to the calculation below are the previous lessons’ numbers, and none of them is recomputed; no new assumption is added either.

// cost/unit.mjs — design decisions' operating cost, in resource units. All inputs are the
// previous lessons' numbers and none of them are recomputed. No currency is used.
const QUERY_DAY = 12_000_000;   // K01: daily tracking queries
const EVENT_DAY = 2_800_000;    // K01: daily state events
const RESPONSE_BYTES = 480;     // K01 (V5): tracking response body
const EVENT_BYTES = 220;        // K01 (V6): one state event record
const BATCH_GB = 10.80;         // K01: what the end-of-day job reads
const STORED_GB = 712.48;       // K01: data stored over 730 days
const CPU_S = 0.00085;          // this course's load testing lesson: processor share per request (s)
const PEAK = 513.89;            // K01: peak edge requests/s
const MONITOR_RAW = 239.76;     // this course's instrumentation lesson: raw event, GB-months
const MONITOR_HIST = 2.07;      // same lesson: per-second histogram, GB-months
const DAYS = 30, HOURS = 720;
const b = (x, n = 2) => x.toFixed(n);

const nodes = (peak) => Math.ceil(peak / ((1 / CPU_S) * 0.7)) + 1;   // 05's rule
const requestsMonth = (QUERY_DAY + EVENT_DAY) * DAYS;

const lineItems = (k = 1) => ({
  "processor-seconds": requestsMonth * k * CPU_S,
  "node-hours": nodes(PEAK * k) * HOURS,
  "GB transferred": ((QUERY_DAY * RESPONSE_BYTES + EVENT_DAY * EVENT_BYTES) * DAYS * k) / 1e9 + BATCH_GB * DAYS * k,
  "storage GB-months": STORED_GB * k,
  "monitoring GB-months": MONITOR_HIST,
});

const baseline = lineItems();
console.log(`monthly work volume ${requestsMonth.toLocaleString("en-US")} requests (K01: daily 12,000,000 queries + 2,800,000 events)`);
console.log(`${"line item".padEnd(22)}${"monthly amount".padStart(15)}${"per request".padStart(16)}`);
for (const [name, value] of Object.entries(baseline))
  console.log(`${name.padEnd(22)}${b(value).padStart(15)}${value / requestsMonth < 1e-4
    ? (value / requestsMonth).toExponential(2).padStart(16) : b(value / requestsMonth, 6).padStart(16)}`);

const purchased = nodes(PEAK) * HOURS * 3600;
console.log(`\npurchased node-seconds ${purchased.toLocaleString("en-US")}, ` +
  `working processor-seconds ${Math.round(baseline["processor-seconds"]).toLocaleString("en-US")}`);
console.log(`resource efficiency ${b((100 * baseline["processor-seconds"]) / purchased)}% — the rest is headroom, ` +
  `redundancy, and off-peak hours`);

console.log(`\nas scale grows, which line item grows by how much:`);
console.log(`${"scale".padStart(6)}${"nodes".padStart(7)}${Object.keys(baseline).map((a) => a.padStart(22)).join("")}`);
for (const k of [1, 2, 4, 10]) {
  const y = lineItems(k);
  console.log(`${(k + "x").padStart(6)}${String(nodes(PEAK * k)).padStart(7)}` +
    `${Object.keys(baseline).map((a) => `${b(y[a] / baseline[a])}x`.padStart(22)).join("")}`);
}
console.log(`two line items that do not grow with volume lower the unit cost: at 10x volume, node-hours grow ` +
  `${b(nodes(PEAK * 10) / nodes(PEAK))}x, monitoring 1.00x`);

console.log(`\nsame total growth, two separate causes (the measurement that separates them: processor-seconds per request):`);
console.log(`${"month".padStart(5)}${"A: requests".padStart(16)}${"A: processor-s".padStart(16)}` +
  `${"A: per request".padStart(17)}${"B: requests".padStart(16)}${"B: processor-s".padStart(16)}${"B: per request".padStart(17)}`);
for (const month of [0, 6, 12]) {
  const growth = 1.06 ** month;                    // A: volume grows, unit cost fixed
  const aRequests = requestsMonth * growth, aCpu = aRequests * CPU_S;
  const bRequests = requestsMonth, bCpu = bRequests * CPU_S * growth;   // B: volume fixed, unit cost grows
  console.log(`${String(month).padStart(5)}${Math.round(aRequests).toLocaleString("en-US").padStart(16)}` +
    `${Math.round(aCpu).toLocaleString("en-US").padStart(16)}${(aCpu / aRequests).toExponential(3).padStart(17)}` +
    `${Math.round(bRequests).toLocaleString("en-US").padStart(16)}` +
    `${Math.round(bCpu).toLocaleString("en-US").padStart(16)}${(bCpu / bRequests).toExponential(3).padStart(17)}`);
}

console.log(`\nresource-profile shift — no decision removes cost, each one shifts it between line items:`);
const DECISIONS = [
  ["edge cache (K01 V9: hit ratio 0.90)", `reads reaching the store shrink ${b(416.67 / 41.67)}x`,
    "held memory and the staleness window grow (quantified in K04)"],
  ["splitting off monolithic persistence", "period scan reads 801x fewer records",
    `store writes ${b(194.44 / 97.22)}x, stores in operation 1 -> 3`],
  ["one more node (redundancy)", "a single node's failure no longer causes an outage",
    `node-hours ${b(2160 / 1440)}x (1,440 -> 2,160)`],
  ["per-second histogram instead of raw event", `monitoring holds ${b(MONITOR_RAW / MONITOR_HIST)}x fewer GB-months`,
    "record identity is lost: p99 error rises to 10.0%"],
];
for (const [name, shrinks, grows] of DECISIONS) console.log(`  ${name}\n    shrinks: ${shrinks}\n    grows  : ${grows}`);

monthly work volume 444,000,000 requests (K01: daily 12,000,000 queries + 2,800,000 events)
line item              monthly amount     per request
processor-seconds           377400.00        0.000850
node-hours                    1440.00         3.24e-6
GB transferred                 515.28         1.16e-6
storage GB-months              712.48         1.60e-6
monitoring GB-months             2.07         4.66e-9

purchased node-seconds 5,184,000, working processor-seconds 377,400
resource efficiency 7.28% — the rest is headroom, redundancy, and off-peak hours

as scale grows, which line item grows by how much:
 scale  nodes     processor-seconds            node-hours        GB transferred     storage GB-months  monitoring GB-months
    1x      2                 1.00x                 1.00x                 1.00x                 1.00x                 1.00x
    2x      3                 2.00x                 1.50x                 2.00x                 2.00x                 1.00x
    4x      4                 4.00x                 2.00x                 4.00x                 4.00x                 1.00x
   10x      8                10.00x                 4.00x                10.00x                10.00x                 1.00x
two line items that do not grow with volume lower the unit cost: at 10x volume, node-hours grow 4.00x, monitoring 1.00x

same total growth, two separate causes (the measurement that separates them: processor-seconds per request):
month     A: requests  A: processor-s   A: per request     B: requests  B: processor-s   B: per request
    0     444,000,000         377,400         8.500e-4     444,000,000         377,400         8.500e-4
    6     629,822,486         535,349         8.500e-4     444,000,000         535,349         1.206e-3
   12     893,415,233         759,403         8.500e-4     444,000,000         759,403         1.710e-3

resource-profile shift — no decision removes cost, each one shifts it between line items:
  edge cache (K01 V9: hit ratio 0.90)
    shrinks: reads reaching the store shrink 10.00x
    grows  : held memory and the staleness window grow (quantified in K04)
  splitting off monolithic persistence
    shrinks: period scan reads 801x fewer records
    grows  : store writes 2.00x, stores in operation 1 -> 3
  one more node (redundancy)
    shrinks: a single node's failure no longer causes an outage
    grows  : node-hours 1.50x (1,440 -> 2,160)
  per-second histogram instead of raw event
    shrinks: monitoring holds 115.83x fewer GB-months
    grows  : record identity is lost: p99 error rises to 10.0%

Purchased Resource and Working Resource

The second block gives the lesson’s most uncomfortable number. 5,184,000 node-seconds are purchased per month, and 377,400 of them do work: resource efficiency is 7.28%. The rest has not gone to waste; it was deliberately purchased by three separate decisions. Headroom is held so utilization does not pass the ceiling, and its rationale is the queuing model. Redundancy is held so service continues when a node drops. The third is arithmetic: capacity is bought for the peak, while work is done on average, and K01’s peak factor is three.

Read as a waste measure, this number is misread. The correct reading is this: this is the resource-denominated price of the availability target and the latency target. The only way to raise the 7.28% is to give up one of those two targets; as long as the targets stand, the ratio is a consequence, not a flaw. Cost awareness is not trying to lower this ratio — it is knowing what was bought.

Symptom: Resource Grows Faster Than Volume

Growth in monthly resource consumption is a symptom, and on its own it says nothing. Two causes give the same total. A: volume has grown, unit cost is fixed. B: volume is fixed, unit cost has grown. The third table places the two side by side, and the total columns are identical at month twelve: 759,403 processor-seconds.

The measurement that separates them is resource per unit of work. In A, processor-seconds per request stays fixed at 8.500e-4 across the twelve months; in B, it goes from 8.500e-4 to 1.710e-3 — doubling. Someone looking only at total resource cannot separate the two cases, and will most likely give both the same answer: add a node. In A, that is the right answer. In B, adding a node covers up the symptom and leaves the cause in place.

This course measured a real source of B. The soak test showed a process running at the same load having its steps per maintenance pass grow 61x; accumulation grows the work per request independent of volume. Unit cost’s time series finds, in production, what a soak test finds — and without running a test at all.

Unit Cost Falls With Scale

The second table shows the line items do not grow at the same rate as scale. Processor-seconds, GB transferred, and storage are linear with volume: ten times the work, ten times the resource. Node-hours are not — at ten times the volume, they grow only 4.00x, because the previous lesson’s failure constraint demands one extra node at small scale, and that extra node’s share erodes with scale. The monitoring line item does not grow at all: the per-second histogram holds 2.07 GB-months regardless of request rate.

The conclusion is this: unit cost is not a number independent of scale. Node-hours per request get 2.5 times cheaper at ten times the scale, monitoring per request gets ten times cheaper, processor-seconds per request never change. In an operating-cost discussion, the sentence “this feature costs this much per request” cannot be read without saying at what scale it was measured — the same rule as the course’s first lesson, this time on the cost side.

Resource-Profile Shift

The last block takes four design decisions and shows that none of them removes cost — each only shifts it between line items. An edge cache shrinks reads reaching the store by 10.00x and holds memory in exchange. Splitting off monolithic persistence shrinks the period scan by 801x and raises store writes 2.00x, moving the number of stores in operation from 1 to 3. Adding one more node keeps a single node’s failure from producing an outage and raises node-hours 1.50x. Collecting a histogram instead of the raw event makes monitoring 115.83 times cheaper and erases record identity.

The shape shared by all four is the cost-side counterpart of the course’s diagnostic rule: when a fix is being justified, which line item shrinks and which one grows are written down together. A justification that writes down only what shrank is an incomplete justification, because the resource profile has changed, and the new profile’s bottleneck is somewhere else.

Summary

• Operating cost is written in resource units, not currency — processor-seconds, GB-months, node-hours, GB transferred — because price depends on the provider and the date, and a resource unit does not.
• 5,184,000 node-seconds are purchased per month, and 377,400 of them do work: resource efficiency is 7.28%. The rest is the resource-denominated price of headroom, redundancy, and the peak factor, not waste.
• Two causes give the same resource growth, and the total cannot separate them (both are 759,403 processor-seconds at month twelve); the measurement that separates them is resource per unit of work: fixed at 8.500e-4 in A, 1.710e-3 in B.
• Unit cost changes with scale: at ten times the volume, processor-seconds grow 10.00x, node-hours 4.00x, monitoring 1.00x; cost per request cannot be read without saying at what scale it was measured.
• No fix removes cost, it shifts it between line items: the histogram makes monitoring 115.83 times cheaper and erases record identity, splitting shrinks the scan 801x and raises writes 2.00x.

Course Wrap-Up

This course did one job across sixteen lessons: tying a symptom to a cause. The table below brings together each lesson’s symptom, its distinguishing measurement, and what the fix grows in exchange.

Lesson	Symptom	Distinguishing measurement	What grows in exchange for the fix
Busy Database	App idle while the store is busy	Boundary ratio: work moves up at 1.00, down well above it	App row work 0 → 240,000 records (3333.33 per second)
Busy Front End	Visible time grows while server metrics stay flat	Work per record: 0.95% at load, 16.1% in code	Response 3,871 → 3,896 bytes and the cache key splitting
Chatty I/O	Every call is fast, useful payload small, open still slow	Envelope share 0.4591 / unused-payload share 0.6266	Merging grows unused payload, splitting grows round count (6 rounds)
Extraneous Fetching	Edge hit rate falls below 0.90 while requests hold steady	Working set’s multiplier: bytes 5.19x / variety 4.00x	Narrowing diversifies: hit rate 0.8007 → 0.6315
Improper Instantiation	Throughput does not rise with concurrency	Creation ratio: reduce at 1.0000, increase at 0.0007	Held object-rounds: pool 24 holds 2.26x the resource of pool 5
Monolithic Persistence	Per-write work grows	Share serving its own class: index 0.33, scan 801x	Store writes 97.22 → 194.44/s, store count 1 → 3, atomicity lost
No Caching	Application layer approaches saturation	De-duplication by computation key: read 1.00 / computation 125.00	Memory 960 → 119,365 entries and a wrongness surface (0.5000)
Noisy Neighbor	Small tenants’ wait grows 0.24 → 0.98 rounds	Request share fixed at 0.6143 while work share 0.1702 → 0.2415	S1’s own delay 0.47 → 2.96 rounds, 200 counters held
Synchronous I/O	Throughput hits a ceiling, adding workers does not help	Dependency’s own utilization: 0.9999 / 0.1819	In-flight requests 8 → 46 (5.75x); admission control is needed
Retry Storm	Call rate reaching the dependency grows	Distinct request identity fixed at 4.00, success ratio 0.0400	Throttling the allowance produces dropped requests: 180 at an allowance of 1
Monitoring Categories	All ten diagnoses assume a metric that does not exist	Category coverage: utilization 10/10, performance 7/10	The list of metrics to collect reaches sixteen items
Instrumentation Design	Monitoring’s own write is 5.29x the system’s own write	Information preserved per unit of cost: 93.0% error against 10.0%	Aggregation erases record identity, cannot bound the maximum
Dashboards and Alerts	An alert either fires constantly or never fires	Threshold-window scan and burn rate: 0.5% threshold, rate 7.7	As the window widens, a 2.0 min delay; at 5 min, 3.29 min of budget
Load Testing	Today’s measurement is silent about tomorrow’s load	Three separate tests: median stays put, p95 and p99 close to double	A spike’s cost is bigger than the spike: 1,542 extra requests
Capacity Planning	How many nodes are needed cannot be read from measurement	Load constraint vs. failure constraint: today, n = 2 at every ceiling	Redundancy premium 100%; 1,440 → 2,160 node-hours
Cost Awareness	Monthly resource grows faster than work volume	Resource per unit of work: 8.500e-4 fixed / 1.710e-3 growing	Every fix shifts a line item: 115.83x fewer GB-months, deleted identity

The table’s third column is the course’s rule: a symptom is not a diagnosis. None of the sentences “slow,” “expensive,” “does not scale” points to a cause; the diagnosis is the measurement that separates the two causes giving the same symptom. In none of the sixteen lessons was the symptom sufficient on its own, and in several of them the most-watched metric turned out to be the least distinguishing one — worker utilization could not separate the two causes (0.9562 against 0.9545), total resource could not separate the two causes (both 759,403), and the response-time percentile showed up in six of the ten diagnoses at once.

The fourth column carries the second rule: no pattern is wrong under every condition. Every lesson bounded with a number the condition under which the same shape is correct. Keeping the computation in the store is justified to the extent that it reduces the record count crossing the boundary — in F1, it cuts that count 100x. Carrying an extra field is justified if it is smaller than the 282-byte envelope of the call it eliminates. No caching is the correct design in a stream where the repeat is 1.01; there, memoization would drop to a 0.71x ratio instead of gaining anything. A shared resource is correct while the noise ratio stays close to one and no single request exceeds the per-round capacity. A blocking call is correct while the wait is the same order of magnitude as local work. An antipattern is not a prohibition — it is a choice that turns wrong at a scale, and the measurement in the third column is what gives that scale.

Six courses built a design vocabulary piece by piece: how traffic gets served, how the application gets split, how data gets distributed, how failure gets contained, and how all of it gets made visible. Each course answered its own question with its own measurement. The one thing that was never done is this: using these pieces together, in a single problem, and narrating the choices with their reasoning from beginning to end. In a real design discussion, questions do not line up by course boundary; they are all asked at once, and the trade-offs among them are defended together, not one at a time. The next course, Case Studies, does that work.