Refresh-Ahead

The previous lesson had writes keep a cache entry alive. A key that never sees a write loses its entry for exactly one reason: it expires. In this design, lifetime is not an optional setting but a safety net — this topic’s second lesson measured a lost invalidation message leaving a stale entry with no counter showing it. That is why an entry gets an upper-bound lifetime.

That bound has a cost — this lesson’s first number. The moment an entry expires, the request asking for that key misses, goes to the store, and waits. There is a way to remove the wait: while still valid, the entry is refreshed in the background near the end of its lifetime. This is called refresh-ahead. This lesson sweeps the threshold as a parameter and sets two numbers against each other: recovered waiting against wasted refresh.

The Clock and the Lifetime

The model is in-process and represents time with a counter: each read advances the clock one tick. Converting a tick to seconds uses K01’s peak read rate — at 416.67 req/s, one second is 416.67 ticks. Real time is never measured; everything measured is a count that never changes between runs.

This lesson takes the lifetime as 5 seconds. B1 (this lesson’s assumption): the upper-bound lifetime given to a tracking record’s cache entry is 5 seconds. Rationale: this duration bounds how long a lost invalidation can last, and what it should be is this topic’s last lesson’s subject; it is held fixed here so threshold is the only variable. Sensitivity is given in the last measurement. Not added to K01’s table.

// cache/refresh.mjs — the in-process model of refresh-ahead. The clock is a counter: each read
// advances one tick, converted to seconds using K01's peak read rate. Lifetime, threshold ratio,
// and the selection rule are parameters; time is not measured, operations are counted.
export const K01 = {
  peakRead: (2_000_000 * 6 / 86_400) * 3,
  peakWrite: (400_000 * 7 / 86_400) * 3,
};

export function stream({ reads, ratio, workingSet, burst }) {
  let seed = 20260730;
  const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648);
  let next = 1;
  const active = Array.from({ length: workingSet }, () => ({ id: next++, remaining: burst }));
  const output = [];
  let debt = 0;
  for (let i = 0; i < reads; i++) {
    const j = Math.floor(rand() * active.length);
    const item = active[j];
    output.push(["read", `track:${item.id}`]);
    if ((item.remaining -= 1) === 0) active[j] = { id: next++, remaining: burst };
    for (debt += ratio; debt >= 1; debt -= 1)
      output.push(["write", `track:${active[Math.floor(rand() * active.length)].id}`]);
  }
  return output;
}

// threshold: if an entry's remaining lifetime falls below this ratio of the lifetime, it is
// refreshed from the store on read. threshold = 0 means no refresh-ahead.
// minSightings: only keys read at least this many times are refreshed (selective refresh).
export function run(ops, { capacity, lifetime, threshold, minSightings = 0 }) {
  const box = new Map();                  // key -> { expires, refreshed }
  const s = { hit: 0, waiting: 0, refresh: 0, wastedRefresh: 0, storeRead: 0, storeWrite: 0 };
  const sightings = new Map();
  let clock = 0;
  const evict = (key) => {
    const g = box.get(key);
    if (g && g.refreshed) s.wastedRefresh += 1;
    box.delete(key);
  };
  const put = (key, refreshed) => {
    box.delete(key);
    box.set(key, { expires: clock + lifetime, refreshed });
    if (box.size > capacity) evict(box.keys().next().value);
  };
  for (const [type, key] of ops) {
    if (type === "write") { s.storeWrite += 1; if (box.has(key)) put(key, false); continue; }
    clock += 1;
    sightings.set(key, (sightings.get(key) ?? 0) + 1);
    const g = box.get(key);
    if (g === undefined || g.expires <= clock) {
      if (g !== undefined) evict(key);              // expired entry: wasted if it had been refreshed
      s.waiting += 1; s.storeRead += 1;
      put(key, false);
      continue;
    }
    s.hit += 1;
    if (g.refreshed) g.refreshed = false;            // refreshed value read: not wasted
    if (threshold > 0 && g.expires - clock <= lifetime * threshold && sightings.get(key) >= minSightings) {
      s.refresh += 1; s.storeRead += 1;
      put(key, true);
    } else {
      box.delete(key); box.set(key, g);               // moved to the most-recently-used end
    }
  }
  for (const key of [...box.keys()]) evict(key);
  return s;
}

Wasted refresh is a refreshed entry dropping without ever being read again. The model counts this with a flag: if the refreshed entry is read, the flag is cleared; if the entry expires or is evicted without being read, the wasted-refresh counter increments. The write path carries over the previous lesson’s decision — write-through — so a write does not drop the entry, it restarts its lifetime.

Sweeping the Threshold

// cache/threshold.mjs — sweeping the threshold ratio: recovered waiting versus wasted refresh
import { stream, run, K01 } from "./refresh.mjs";

const READS = 200_000, CAPACITY = 2000;
const RATIO = K01.peakWrite / K01.peakRead;
const LIFETIME_S = 5, LIFETIME = Math.round(LIFETIME_S * K01.peakRead);   // lifetime in read ticks
const ops = stream({ reads: READS, ratio: RATIO, workingSet: 1000, burst: 10 });

console.log(`reads ${READS}, capacity ${CAPACITY}, lifetime ${LIFETIME_S} s = ${LIFETIME} read ticks`);
console.log(`one tick = 1/${K01.peakRead.toFixed(2)} s (K01 peak read rate)\n`);
console.log("threshold    hit waiting refresh wasted refresh wasted ratio store read reads behind cache/s requests reaching the store/s");
console.log("--------- ------ ------- ------- -------------- ------------ ---------- -------------------- ------------------------------");
for (const threshold of [0, 0.1, 0.25, 0.5, 1]) {
  const r = run(ops, { capacity: CAPACITY, lifetime: LIFETIME, threshold });
  const behind = K01.peakRead * (r.storeRead / READS);
  console.log(`${threshold.toFixed(2).padStart(9)} ${(r.hit / READS).toFixed(4).padStart(6)} ` +
    `${String(r.waiting).padStart(7)} ${String(r.refresh).padStart(7)} ${String(r.wastedRefresh).padStart(14)} ` +
    `${(r.refresh === 0 ? 0 : r.wastedRefresh / r.refresh).toFixed(4).padStart(12)} ` +
    `${String(r.storeRead).padStart(10)} ${behind.toFixed(2).padStart(20)} ${(behind + K01.peakWrite).toFixed(2).padStart(30)}`);
}

console.log("\nwasted refresh does not drop by excluding heavily read keys (threshold 0.25):");
console.log("min sightings  refresh  wasted refresh  wasted ratio");
for (const minSeen of [0, 3, 5, 8]) {
  const r = run(ops, { capacity: CAPACITY, lifetime: LIFETIME, threshold: 0.25, minSightings: minSeen });
  console.log(`${String(minSeen).padStart(13)} ${String(r.refresh).padStart(8)} ${String(r.wastedRefresh).padStart(15)} ` +
    `${(r.wastedRefresh / r.refresh).toFixed(4).padStart(13)}`);
}

console.log("\nthe real lever is lifetime (threshold 0 next to threshold 0.25):");
console.log("lifetime (s)  thr 0 hit  thr 0 store  thr 0.25 hit  thr 0.25 store  thr 0.25 waiting  requests reaching the store/s");
for (const sec of [2, 5, 15, 60]) {
  const l = Math.round(sec * K01.peakRead);
  const a = run(ops, { capacity: CAPACITY, lifetime: l, threshold: 0 });
  const b = run(ops, { capacity: CAPACITY, lifetime: l, threshold: 0.25 });
  const behind = K01.peakRead * (b.storeRead / READS);
  console.log(`${String(sec).padStart(12)} ${(a.hit / READS).toFixed(4).padStart(10)} ${String(a.storeRead).padStart(12)} ` +
    `${(b.hit / READS).toFixed(4).padStart(13)} ${String(b.storeRead).padStart(15)} ${String(b.waiting).padStart(17)} ` +
    `${(behind + K01.peakWrite).toFixed(2).padStart(29)}`);
}

reads 200000, capacity 2000, lifetime 5 s = 2083 read ticks
one tick = 1/416.67 s (K01 peak read rate)

threshold    hit waiting refresh wasted refresh wasted ratio store read reads behind cache/s requests reaching the store/s
--------- ------ ------- ------- -------------- ------------ ---------- -------------------- ------------------------------
     0.00 0.7109   57815       0              0       0.0000      57815               120.45                         217.67
     0.10 0.7363   52731    9780           1862       0.1904      62511               130.23                         227.45
     0.25 0.7660   46806   23922           4120       0.1722      70728               147.35                         244.57
     0.50 0.7994   40128   50540           8728       0.1727      90668               188.89                         286.11
     1.00 0.8286   34270  165730          27825       0.1679     200000               416.67                         513.89

wasted refresh does not drop by excluding heavily read keys (threshold 0.25):
min sightings  refresh  wasted refresh  wasted ratio
            0    23922            4120        0.1722
            3    22693            4035        0.1778
            5    17116            3662        0.2140
            8     8524            3075        0.3607

the real lever is lifetime (threshold 0 next to threshold 0.25):
lifetime (s)  thr 0 hit  thr 0 store  thr 0.25 hit  thr 0.25 store  thr 0.25 waiting  requests reaching the store/s
           2     0.4884       102320        0.5263          116205             94735                        339.32
           5     0.7109        57815        0.7660           70728             46806                        244.57
          15     0.8686        26288        0.8884           31562             22315                        162.98
          60     0.8976        20482        0.8977           20510             20461                        139.95

All the numbers belong to the computed class: they were counted over a deterministic stream.

Reading the Numbers

The first row hands over the safety net’s bill. The previous lesson measured a hit ratio of 0.9170 with write-through; once a five-second upper-bound lifetime is placed on top of that, the hit ratio drops to 0.7109 and requests reaching the store/s climbs from 151.04 to 217.67. The bound placed against lost invalidations costs one-fifth of the hit ratio.

As the threshold grows, two columns move in opposite directions. Waiting requests fall from 57,815 to 34,270; against that, the refresh count climbs from zero to 165,730, and 27,825 of those go to waste. The wasted ratio is nearly independent of the threshold: in four of the five rows it sits between 0.17 and 0.19. The reason is structural — a refresh going to waste means that key is never read again, and this is unavoidable on a burst’s last query. No counter here can tell in advance that a query is the last one.

The second table confirms this. The rule “do not refresh lightly read keys” should intuitively have cut wasted refresh; the measurement says the opposite. Once the threshold is pulled to eight, the refresh count drops from 23,922 to 8,524, but the wasted ratio climbs from 0.1722 to 0.3607. The excluded refreshes were the ones already paying off: a heavily read key sits closer to the end of its burst. The waste sits at the end of access, not the beginning, and cannot be separated out by view count.

The last row carries this lesson’s harshest number. At threshold 1 — meaning every read triggers a refresh — the hit ratio sits at a respectable-looking 0.8286, but store reads number 200,000, the entirety of all requests. requests reaching the store/s is 513.89: the same number this topic’s first lesson computed with no cache at all. The cache is running, the hit counter is filling up, the dashboard looks healthy, and the store is carrying the full load. A high hit ratio, on its own, says nothing about store load.

The Real Lever

The third table sets threshold next to lifetime and reverses the ranking. As lifetime grows from 2 seconds to 60 seconds, the threshold-free hit ratio climbs from 0.4884 to 0.8976, while requests reaching the store/s drops from 339.32 to 139.95. Over the same range, the threshold’s contribution keeps shrinking: it raises the hit ratio by 0.0379 at 2 seconds, and by only 0.0001 at 60 seconds. At a 60-second lifetime, refresh-ahead makes 28 extra store reads and cuts waiting requests from 20,482 to 20,461 — a measurable but meaningless gain.

The rule follows from this: refresh-ahead cuts waiting only when the lifetime stays short relative to the access burst. If the lifetime is longer than the burst, the entry is already read again before it expires, and there is nothing left to refresh. If the lifetime is short, refreshing pays off, but every waiting request it saves loads an extra read onto the store, and roughly one in six of those reads goes to waste.

This makes refresh-ahead a latency tool, not a load-reduction tool. K01’s requests reaching the store/s line went up every time in this lesson. A design decision does not have to improve the computation; it has to be measured, its trade-off written down.

Summary

• The five-second upper-bound lifetime placed against lost invalidations dropped the hit ratio from 0.9170 to 0.7109 and raised requests reaching the store/s from 151.04 to 217.67.
• As the threshold rose from 0 to 1, waiting requests fell from 57,815 to 34,270, refresh climbed to 165,730 with 27,825 of it wasted; the wasted ratio stayed near 0.17 independent of the threshold.
• Excluding lightly read keys does not cut wasted refresh — it raises the ratio from 0.1722 to 0.3607: the waste sits at the end of access and cannot be separated out by view count.
• At threshold 1, the hit ratio reads 0.8286 while store reads number 200,000 and requests reaching the store/s reaches 513.89 — the cacheless design’s number. A high hit ratio says nothing about store load.
• The real lever is lifetime: as it rises from 2 seconds to 60 seconds, the hit ratio climbs from 0.4884 to 0.8976, requests reaching the store/s drops from 339.32 to 139.95, and refresh-ahead’s contribution falls to 0.0001.

Next Step

This lesson’s last number leaves an uncomfortable question. At a hit ratio of 0.8286, the store was carrying 513.89 req/s, as if no cache existed at all. A running cache, then, can produce the same load as no cache at all. That points to a question this topic has not asked yet, from the opposite direction: is a cache actually needed. Every lesson up to here assumed a cache exists and argued only over how to arrange it. The next lesson removes that assumption: which access patterns leave requests reaching the store/s unchanged by adding a cache, which patterns raise it, and where the boundary starts at which a cacheless design is a genuine anti-pattern. </content>