--- title: 'Cacheless Design Anti-Pattern' source: 'https://academia.sh/en/courses/scaling-the-data/cacheless-design-anti-pattern' course: 'Scaling the Data Layer' language: en updated: '2026-08-23T07:01:33+00:00' license: 'CC BY-SA 4.0' --- # Cacheless Design Anti-Pattern Deciding whether a cache is needed from the access pattern: a cacheless design loading 513.89 req/s onto the store for the tracking query, the cache missing entirely on the period scan while still costing 2.00 extra touches per request, a 5000x repeat rate in hot writes dropping to a 0.3001 hit ratio, and passing the bulk scan through the same cache cutting the tracking hit ratio by 0.4263 while raising store access by 85,252. The previous lesson's last number left an uncomfortable question. At a hit ratio of 0.8286, the store was carrying 513.89 req/s, as if no cache existed at all. Every lesson up to here assumed a cache exists and argued only over how to arrange it. This lesson removes that assumption and asks the question from the opposite direction: is a cache actually needed. The question has two wrong answers, both from the same flaw. The first is the **cacheless design anti-pattern**: every read going to the store even though the access pattern is full of repetition. The second is its mirror image: putting a cache on a path that carries no repetition. Both are the result of never measuring the access pattern, so this lesson measures it. ## Four Patterns The model is in-process: the same cache runs against four separate streams, counters left explicit. All four streams are derived from K01's three flows — the tracking query, the state event write, and the end-of-day billing scan — with rates from K01's computation: peak read 416.67 req/s, bulk scan 833.33 records/s. The window is again 480 seconds. ```js // cache/patterns.mjs — the same cache run against four access patterns. Cache and store are // modules of their own; counters are explicit. Patterns are derived from K01's three streams. export const K01 = { peakRead: (2_000_000 * 6 / 86_400) * 3, // 416.67 req/s peakWrite: (400_000 * 7 / 86_400) * 3, // 97.22 req/s bulkScanRate: (30 * 400_000) / (4 * 3600), // 833.33 records/s }; export const WINDOW = 480; // seconds function randomGenerator(seed) { return () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648); } // Tracking query: one shipment's queries arrive in bursts of `burst`. export function trackingStream(requests, workingSet = 1000, burst = 10) { const rand = randomGenerator(20260730); let next = 1; const active = Array.from({ length: workingSet }, () => ({ id: next++, remaining: burst })); const output = []; for (let i = 0; i < requests; i++) { const j = Math.floor(rand() * active.length); output.push(["read", `track:${active[j].id}`]); if ((active[j].remaining -= 1) === 0) active[j] = { id: next++, remaining: burst }; } return output; } // Billing scan: every seller-day record is read exactly once, in order. export function scanStream(records) { return Array.from({ length: records }, (_, i) => ["read", `period:${i + 1}`]); } // Hot write: a small key space takes far more writes than reads. export function hotWriteStream(ops, keys = 40, writeShare = 0.7) { const rand = randomGenerator(4242); return Array.from({ length: ops }, () => { const a = `zone:${Math.floor(rand() * keys) + 1}`; return [rand() < writeShare ? "write" : "read", a]; }); } // Interleaves two streams: `everyN` tracking requests, then `thenN` scan requests. export function interleave(a, b, everyN, thenN) { const output = []; let i = 0, j = 0; while (i < a.length || j < b.length) { for (let k = 0; k < everyN && i < a.length; k++) output.push(a[i++]); for (let k = 0; k < thenN && j < b.length; k++) output.push(b[j++]); } return output; } // skip: keys in this namespace never touch the cache at all. export function run(ops, { capacity, skip = null }) { const m = new Map(); const s = { request: 0, read: 0, hit: 0, storeRead: 0, storeWrite: 0, touch: 0, trackingRequest: 0, trackingHit: 0 }; for (const [type, key] of ops) { s.request += 1; const tracking = key.startsWith("track:"); if (tracking) s.trackingRequest += 1; if (skip !== null && key.startsWith(skip)) { if (type === "read") s.storeRead += 1; else s.storeWrite += 1; s.touch += 1; continue; } if (type === "write") { s.storeWrite += 1; s.touch += 2; m.delete(key); continue; } s.read += 1; s.touch += 1; if (m.has(key)) { s.hit += 1; if (tracking) s.trackingHit += 1; const d = m.get(key); m.delete(key); m.set(key, d); continue; } s.storeRead += 1; s.touch += 2; m.set(key, 1); if (m.size > capacity) m.delete(m.keys().next().value); } return s; } ``` ```js // cache/decision.mjs — what the cache gains and costs across four patterns import { K01, WINDOW, trackingStream, scanStream, hotWriteStream, interleave, run } from "./patterns.mjs"; const CAPACITY = 2000; const READS = Math.round(K01.peakRead * WINDOW); const SCAN = Math.round(K01.bulkScanRate * WINDOW); const tracking = trackingStream(READS); const scan = scanStream(SCAN); const hot = hotWriteStream(READS); console.log(`window ${WINDOW} s: tracking query ${READS}, period scan ${SCAN} records, capacity ${CAPACITY}`); console.log(`(K01: peak read ${K01.peakRead.toFixed(2)}/s, bulk scan ${K01.bulkScanRate.toFixed(2)} records/s)\n`); const PATTERNS = [ ["tracking query", tracking], ["period scan", scan], ["hot write", hot], ["tracking + scan", interleave(tracking, scan, 1, 2)], ]; console.log("pattern request read unique keys repeat hit store access cacheless diff extra touch/req"); console.log("--------------- ------- ------ ------------- ------- ------ ------------- ---------- ------ ---------------"); for (const [name, stream] of PATTERNS) { const r = run(stream, { capacity: CAPACITY }); const unique = new Set(stream.map(([, a]) => a)).size; const store = r.storeRead + r.storeWrite; console.log(`${name.padEnd(15)} ${String(r.request).padStart(7)} ${String(r.read).padStart(6)} ${String(unique).padStart(13)} ` + `${(r.request / unique).toFixed(2).padStart(7)} ${(r.hit / r.read).toFixed(4).padStart(6)} ` + `${String(store).padStart(13)} ${String(r.request).padStart(10)} ${(store - r.request).toString().padStart(6)} ` + `${((r.touch - r.request) / r.request).toFixed(2).padStart(15)}`); } console.log("\nwhat happens to the tracking query if the period scan passes through the same cache:"); const together = interleave(tracking, scan, 1, 2); const passing = run(together, { capacity: CAPACITY }); const bypassing = run(together, { capacity: CAPACITY, skip: "period:" }); const alone = run(tracking, { capacity: CAPACITY }); const row = (name, r) => { const store = r.storeRead + r.storeWrite; const trackingRate = K01.peakRead * (1 - r.trackingHit / r.trackingRequest); console.log(`${name.padEnd(28)} ${(r.trackingHit / r.trackingRequest).toFixed(4).padStart(14)} ` + `${String(store).padStart(13)} ${(store / WINDOW).toFixed(2).padStart(15)} ${(trackingRate + K01.peakWrite).toFixed(2).padStart(16)}`); }; console.log("state tracking hit store access store access/s requests reaching the store/s"); console.log("---------------------------- --------------- ------------- --------------- ----------------"); row("no scan", alone); row("scan passes through cache", passing); row("scan bypasses cache", bypassing); console.log(`\ncost of the scan passing through the cache = ${(passing.storeRead + passing.storeWrite) - (bypassing.storeRead + bypassing.storeWrite)} extra store accesses`); console.log(`loss in tracking hit ratio = ${((bypassing.trackingHit / bypassing.trackingRequest) - (passing.trackingHit / passing.trackingRequest)).toFixed(4)}`); ``` ``` window 480 s: tracking query 200000, period scan 400000 records, capacity 2000 (K01: peak read 416.67/s, bulk scan 833.33 records/s) pattern request read unique keys repeat hit store access cacheless diff extra touch/req --------------- ------- ------ ------------- ------- ------ ------------- ---------- ------ --------------- tracking query 200000 200000 20477 9.77 0.8976 20483 200000 -179517 0.20 period scan 400000 400000 400000 1.00 0.0000 400000 400000 0 2.00 hot write 200000 60078 40 5000.00 0.3001 181968 200000 -18032 1.12 tracking + scan 600000 600000 420477 1.43 0.1571 505735 600000 -94265 1.69 what happens to the tracking query if the period scan passes through the same cache: state tracking hit store access store access/s requests reaching the store/s ---------------------------- --------------- ------------- --------------- ---------------- no scan 0.8976 20483 42.67 139.90 scan passes through cache 0.4713 505735 1053.61 317.50 scan bypasses cache 0.8976 420483 876.01 139.90 cost of the scan passing through the cache = 85252 extra store accesses loss in tracking hit ratio = 0.4263 ``` All the numbers belong to the **computed** class: they were counted over deterministic streams. ## The Cost of the Anti-Pattern The first row defines the anti-pattern. The tracking query's repeat rate is 9.77: each tracking number is queried about ten times on average within the window. In a cacheless design, the store sees 200,000 accesses; with a cache, 20,483 — a difference of 179,517 accesses, nine-tenths of it unnecessary. In K01's computation, this is the gap in `requests reaching the store/s` between 513.89 and 139.90: **a 3.67x factor**. A cacheless design requires the store sized for that factor; avoiding it costs 0.20 extra touches per request. The second row is the mirror's other face. The period scan reads 400,000 seller-day records, all of them distinct; the repeat rate is exactly 1.00, the hit ratio 0.0000. Store access is equal with and without a cache — a zero difference. With nothing gained, **2.00 extra touches** are paid per request: one lookup and one placement. In this pattern, a cacheless design is not an anti-pattern — it is correct. The third row shows that repetition alone is not enough. In the hot-write pattern, forty keys see 200,000 operations; the repeat rate is 5000, the highest value in this table. Against that, the hit ratio is only **0.3001**. The reason: 140,000 of those operations are writes, each dropping the entry so the next read misses. A high repeat rate does not justify a cache; what justifies it is repetition that **fits between two writes**. ## One Pattern Breaks Another The fourth row and the second table show the same fact from two angles. When the tracking query and the period scan pass through the same cache, the combined hit ratio drops to 0.1571, and the tracking query's own hit ratio drops from 0.8976 to **0.4713**. The scan's 400,000 distinct keys keep entering and leaving the cache, capacity 2000, pushing tracking entries out on the way in. Here the number changes sign. When the scan bypasses the cache, total store access is 420,483; when it passes through, **505,735**: the 85,252-access gap is load the cache **adds**. `requests reaching the store/s` climbs from 139.90 to 317.50. A cache, opened up to the wrong pattern, breaks the very thing it was protecting. The fix is not to remove the cache but to keep the scan stream from passing through it. The last row measures this: once the `period:` namespace is skipped, the tracking hit ratio stays at 0.8976 with no drop at all, and `requests reaching the store/s` returns to 139.90. The scan still reads 400,000 records — those reads were unavoidable — but no longer evicts anyone's working set. The reason K01 put the end-of-day job in its own four-hour window becomes visible in this table. ## The Decision Rule Read together, the four measurements reduce the cache decision to four conditions, all four countable. **The repeat rate must be greater than one.** If the ratio of requests to unique keys in the window is 1.00, there is nothing to gain; the period scan stays in this condition. **Repetition must fit between two writes.** In the hot-write pattern, repetition is 5000, but the hit ratio stays at 0.3001 because the write share per read is high. What matters is not repetition but how many reads an entry gets over its lifetime. **The working set must fit the capacity.** This topic's first lesson measured a cache with capacity 2000 covering a 1000-shipment working set; below that capacity, layers leak overflow into each other. **No other pattern sharing the same cache may evict the working set.** This condition can break on its own even when every other one holds, turning the cache into a net loss: 85,252 extra store accesses. If the four do not hold, a cacheless design is not an anti-pattern; an unmeasured cache decision always is. ## Summary - The tracking query's repeat rate is 9.77, and a cacheless design loads 200,000 accesses onto the store; with a cache, 20,483. In K01's `requests reaching the store/s` line, this is the 3.67x gap between 513.89 and 139.90. - The period scan's repeat rate is 1.00, hit ratio 0.0000, and store access is 400,000 in both designs; with zero gained, 2.00 extra touches are paid per request. - The hot-write pattern's repeat rate is 5000, yet the hit ratio stays at 0.3001: a high repeat rate does not justify a cache — repetition fitting between two writes does. - When the scan passes through the same cache, the tracking hit ratio drops from 0.8976 to 0.4713 and total store access climbs from 420,483 to 505,735; the cache **adds** 85,252 accesses. - The decision comes down to four conditions: repeat rate greater than one, repetition fitting between two writes, the working set fitting the capacity, and no other pattern evicting it. ## Next Step This lesson separated out the patterns where a cache is needed, but one question stayed open. The value returned from the cache need not match the store's value at that moment; how long the two can drift apart was measured once in this topic's second lesson — 105,794 stale reads once invalidation was turned off — and tied to a parameter in the fourth lesson: the five-second upper-bound lifetime was chosen arbitrarily, and never justified. The next lesson builds that justification. How much staleness is acceptable is not an engineering preference but a number that comes out of the data itself: however often a shipment's state changes, a window shorter than that change does not produce a stale response. The next lesson computes the window from that relationship, counts how many stale responses per second are accepted, and measures how far the hit ratio can climb under the same staleness budget.