--- title: Refresh-Ahead source: 'https://academia.sh/en/courses/scaling-the-data/refresh-ahead' course: 'Scaling the Data Layer' language: en updated: '2026-08-23T07:01:33+00:00' license: 'CC BY-SA 4.0' --- # Refresh-Ahead Measuring an entry's refresh in the background before it expires: the lifetime dropping the hit ratio from 0.9170 to 0.7109, waiting requests falling and wasted refreshes rising as the threshold ratio grows, the store load at threshold 1 equaling the cacheless design's 513.89 req/s while the hit ratio still reads 0.8286, and showing that the real lever is lifetime, not threshold. The previous lesson had writes keep a cache entry alive. A key that never sees a write loses its entry for exactly one reason: it expires. In this design, lifetime is not an optional setting but a safety net — this topic's second lesson measured a lost invalidation message leaving a stale entry with no counter showing it. That is why an entry gets an upper-bound lifetime. That bound has a cost — this lesson's first number. The moment an entry expires, the request asking for that key misses, goes to the store, and **waits**. There is a way to remove the wait: while still valid, the entry is refreshed in the background near the end of its lifetime. This is called **refresh-ahead**. This lesson sweeps the threshold as a parameter and sets two numbers against each other: recovered waiting against wasted refresh. ## The Clock and the Lifetime The model is in-process and represents time with a counter: each read advances the clock one tick. Converting a tick to seconds uses K01's peak read rate — at 416.67 req/s, one second is 416.67 ticks. Real time is never measured; everything measured is a count that never changes between runs. This lesson takes the lifetime as 5 seconds. **B1 (this lesson's assumption): the upper-bound lifetime given to a tracking record's cache entry is 5 seconds.** Rationale: this duration bounds how long a lost invalidation can last, and what it should be is this topic's last lesson's subject; it is held fixed here so threshold is the only variable. Sensitivity is given in the last measurement. Not added to K01's table. ```js // cache/refresh.mjs — the in-process model of refresh-ahead. The clock is a counter: each read // advances one tick, converted to seconds using K01's peak read rate. Lifetime, threshold ratio, // and the selection rule are parameters; time is not measured, operations are counted. export const K01 = { peakRead: (2_000_000 * 6 / 86_400) * 3, peakWrite: (400_000 * 7 / 86_400) * 3, }; export function stream({ reads, ratio, workingSet, burst }) { let seed = 20260730; const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648); let next = 1; const active = Array.from({ length: workingSet }, () => ({ id: next++, remaining: burst })); const output = []; let debt = 0; for (let i = 0; i < reads; i++) { const j = Math.floor(rand() * active.length); const item = active[j]; output.push(["read", `track:${item.id}`]); if ((item.remaining -= 1) === 0) active[j] = { id: next++, remaining: burst }; for (debt += ratio; debt >= 1; debt -= 1) output.push(["write", `track:${active[Math.floor(rand() * active.length)].id}`]); } return output; } // threshold: if an entry's remaining lifetime falls below this ratio of the lifetime, it is // refreshed from the store on read. threshold = 0 means no refresh-ahead. // minSightings: only keys read at least this many times are refreshed (selective refresh). export function run(ops, { capacity, lifetime, threshold, minSightings = 0 }) { const box = new Map(); // key -> { expires, refreshed } const s = { hit: 0, waiting: 0, refresh: 0, wastedRefresh: 0, storeRead: 0, storeWrite: 0 }; const sightings = new Map(); let clock = 0; const evict = (key) => { const g = box.get(key); if (g && g.refreshed) s.wastedRefresh += 1; box.delete(key); }; const put = (key, refreshed) => { box.delete(key); box.set(key, { expires: clock + lifetime, refreshed }); if (box.size > capacity) evict(box.keys().next().value); }; for (const [type, key] of ops) { if (type === "write") { s.storeWrite += 1; if (box.has(key)) put(key, false); continue; } clock += 1; sightings.set(key, (sightings.get(key) ?? 0) + 1); const g = box.get(key); if (g === undefined || g.expires <= clock) { if (g !== undefined) evict(key); // expired entry: wasted if it had been refreshed s.waiting += 1; s.storeRead += 1; put(key, false); continue; } s.hit += 1; if (g.refreshed) g.refreshed = false; // refreshed value read: not wasted if (threshold > 0 && g.expires - clock <= lifetime * threshold && sightings.get(key) >= minSightings) { s.refresh += 1; s.storeRead += 1; put(key, true); } else { box.delete(key); box.set(key, g); // moved to the most-recently-used end } } for (const key of [...box.keys()]) evict(key); return s; } ``` **Wasted refresh** is a refreshed entry dropping without ever being read again. The model counts this with a flag: if the refreshed entry is read, the flag is cleared; if the entry expires or is evicted without being read, the wasted-refresh counter increments. The write path carries over the previous lesson's decision — write-through — so a write does not drop the entry, it restarts its lifetime. ## Sweeping the Threshold ```js // cache/threshold.mjs — sweeping the threshold ratio: recovered waiting versus wasted refresh import { stream, run, K01 } from "./refresh.mjs"; const READS = 200_000, CAPACITY = 2000; const RATIO = K01.peakWrite / K01.peakRead; const LIFETIME_S = 5, LIFETIME = Math.round(LIFETIME_S * K01.peakRead); // lifetime in read ticks const ops = stream({ reads: READS, ratio: RATIO, workingSet: 1000, burst: 10 }); console.log(`reads ${READS}, capacity ${CAPACITY}, lifetime ${LIFETIME_S} s = ${LIFETIME} read ticks`); console.log(`one tick = 1/${K01.peakRead.toFixed(2)} s (K01 peak read rate)\n`); console.log("threshold hit waiting refresh wasted refresh wasted ratio store read reads behind cache/s requests reaching the store/s"); console.log("--------- ------ ------- ------- -------------- ------------ ---------- -------------------- ------------------------------"); for (const threshold of [0, 0.1, 0.25, 0.5, 1]) { const r = run(ops, { capacity: CAPACITY, lifetime: LIFETIME, threshold }); const behind = K01.peakRead * (r.storeRead / READS); console.log(`${threshold.toFixed(2).padStart(9)} ${(r.hit / READS).toFixed(4).padStart(6)} ` + `${String(r.waiting).padStart(7)} ${String(r.refresh).padStart(7)} ${String(r.wastedRefresh).padStart(14)} ` + `${(r.refresh === 0 ? 0 : r.wastedRefresh / r.refresh).toFixed(4).padStart(12)} ` + `${String(r.storeRead).padStart(10)} ${behind.toFixed(2).padStart(20)} ${(behind + K01.peakWrite).toFixed(2).padStart(30)}`); } console.log("\nwasted refresh does not drop by excluding heavily read keys (threshold 0.25):"); console.log("min sightings refresh wasted refresh wasted ratio"); for (const minSeen of [0, 3, 5, 8]) { const r = run(ops, { capacity: CAPACITY, lifetime: LIFETIME, threshold: 0.25, minSightings: minSeen }); console.log(`${String(minSeen).padStart(13)} ${String(r.refresh).padStart(8)} ${String(r.wastedRefresh).padStart(15)} ` + `${(r.wastedRefresh / r.refresh).toFixed(4).padStart(13)}`); } console.log("\nthe real lever is lifetime (threshold 0 next to threshold 0.25):"); console.log("lifetime (s) thr 0 hit thr 0 store thr 0.25 hit thr 0.25 store thr 0.25 waiting requests reaching the store/s"); for (const sec of [2, 5, 15, 60]) { const l = Math.round(sec * K01.peakRead); const a = run(ops, { capacity: CAPACITY, lifetime: l, threshold: 0 }); const b = run(ops, { capacity: CAPACITY, lifetime: l, threshold: 0.25 }); const behind = K01.peakRead * (b.storeRead / READS); console.log(`${String(sec).padStart(12)} ${(a.hit / READS).toFixed(4).padStart(10)} ${String(a.storeRead).padStart(12)} ` + `${(b.hit / READS).toFixed(4).padStart(13)} ${String(b.storeRead).padStart(15)} ${String(b.waiting).padStart(17)} ` + `${(behind + K01.peakWrite).toFixed(2).padStart(29)}`); } ``` ``` reads 200000, capacity 2000, lifetime 5 s = 2083 read ticks one tick = 1/416.67 s (K01 peak read rate) threshold hit waiting refresh wasted refresh wasted ratio store read reads behind cache/s requests reaching the store/s --------- ------ ------- ------- -------------- ------------ ---------- -------------------- ------------------------------ 0.00 0.7109 57815 0 0 0.0000 57815 120.45 217.67 0.10 0.7363 52731 9780 1862 0.1904 62511 130.23 227.45 0.25 0.7660 46806 23922 4120 0.1722 70728 147.35 244.57 0.50 0.7994 40128 50540 8728 0.1727 90668 188.89 286.11 1.00 0.8286 34270 165730 27825 0.1679 200000 416.67 513.89 wasted refresh does not drop by excluding heavily read keys (threshold 0.25): min sightings refresh wasted refresh wasted ratio 0 23922 4120 0.1722 3 22693 4035 0.1778 5 17116 3662 0.2140 8 8524 3075 0.3607 the real lever is lifetime (threshold 0 next to threshold 0.25): lifetime (s) thr 0 hit thr 0 store thr 0.25 hit thr 0.25 store thr 0.25 waiting requests reaching the store/s 2 0.4884 102320 0.5263 116205 94735 339.32 5 0.7109 57815 0.7660 70728 46806 244.57 15 0.8686 26288 0.8884 31562 22315 162.98 60 0.8976 20482 0.8977 20510 20461 139.95 ``` All the numbers belong to the **computed** class: they were counted over a deterministic stream. ## Reading the Numbers The first row hands over the safety net's bill. The previous lesson measured a hit ratio of 0.9170 with write-through; once a five-second upper-bound lifetime is placed on top of that, the hit ratio drops to **0.7109** and `requests reaching the store/s` climbs from 151.04 to 217.67. The bound placed against lost invalidations costs one-fifth of the hit ratio. As the threshold grows, two columns move in opposite directions. Waiting requests fall from 57,815 to 34,270; against that, the refresh count climbs from zero to 165,730, and 27,825 of those go to waste. **The wasted ratio is nearly independent of the threshold**: in four of the five rows it sits between 0.17 and 0.19. The reason is structural — a refresh going to waste means that key is never read again, and this is unavoidable on a burst's last query. No counter here can tell in advance that a query is the last one. The second table confirms this. The rule "do not refresh lightly read keys" should intuitively have cut wasted refresh; the measurement says the opposite. Once the threshold is pulled to eight, the refresh count drops from 23,922 to 8,524, but the wasted ratio climbs from 0.1722 to **0.3607**. The excluded refreshes were the ones already paying off: a heavily read key sits closer to the end of its burst. The waste sits at the end of access, not the beginning, and cannot be separated out by view count. The last row carries this lesson's harshest number. At threshold 1 — meaning every read triggers a refresh — the hit ratio sits at a respectable-looking **0.8286**, but store reads number 200,000, the entirety of all requests. `requests reaching the store/s` is **513.89**: the same number this topic's first lesson computed with no cache at all. The cache is running, the hit counter is filling up, the dashboard looks healthy, and the store is carrying the full load. **A high hit ratio, on its own, says nothing about store load.** ## The Real Lever The third table sets threshold next to lifetime and reverses the ranking. As lifetime grows from 2 seconds to 60 seconds, the threshold-free hit ratio climbs from 0.4884 to 0.8976, while `requests reaching the store/s` drops from 339.32 to 139.95. Over the same range, the threshold's contribution keeps shrinking: it raises the hit ratio by 0.0379 at 2 seconds, and by only 0.0001 at 60 seconds. At a 60-second lifetime, refresh-ahead makes 28 extra store reads and cuts waiting requests from 20,482 to 20,461 — a measurable but meaningless gain. The rule follows from this: **refresh-ahead cuts waiting only when the lifetime stays short relative to the access burst.** If the lifetime is longer than the burst, the entry is already read again before it expires, and there is nothing left to refresh. If the lifetime is short, refreshing pays off, but every waiting request it saves loads an extra read onto the store, and roughly one in six of those reads goes to waste. This makes refresh-ahead a **latency** tool, not a load-reduction tool. K01's `requests reaching the store/s` line went **up** every time in this lesson. A design decision does not have to improve the computation; it has to be measured, its trade-off written down. ## Summary - The five-second upper-bound lifetime placed against lost invalidations dropped the hit ratio from 0.9170 to 0.7109 and raised `requests reaching the store/s` from 151.04 to 217.67. - As the threshold rose from 0 to 1, waiting requests fell from 57,815 to 34,270, refresh climbed to 165,730 with 27,825 of it wasted; the wasted ratio stayed near 0.17 independent of the threshold. - Excluding lightly read keys does not cut wasted refresh — it raises the ratio from 0.1722 to 0.3607: the waste sits at the end of access and cannot be separated out by view count. - At threshold 1, the hit ratio reads 0.8286 while store reads number 200,000 and `requests reaching the store/s` reaches 513.89 — the cacheless design's number. A high hit ratio says nothing about store load. - The real lever is lifetime: as it rises from 2 seconds to 60 seconds, the hit ratio climbs from 0.4884 to 0.8976, `requests reaching the store/s` drops from 339.32 to 139.95, and refresh-ahead's contribution falls to 0.0001. ## Next Step This lesson's last number leaves an uncomfortable question. At a hit ratio of 0.8286, the store was carrying 513.89 req/s, as if no cache existed at all. A running cache, then, can produce the same load as no cache at all. That points to a question this topic has not asked yet, from the opposite direction: is a cache actually needed. Every lesson up to here assumed a cache exists and argued only over how to arrange it. The next lesson removes that assumption: which access patterns leave `requests reaching the store/s` unchanged by adding a cache, which patterns **raise** it, and where the boundary starts at which a cacheless design is a genuine anti-pattern.