Cache Layers

The Data Access Layer and Business Logic course built the full path from a request down to the data: the repository pattern, transaction boundaries, query budgets, domain rules. One assumption remained at the end of the course: every request reaches the origin of the data. That assumption does not always hold. In the library system, a book’s detail page is opened thousands of times a day, and the book’s title and author have not changed in months. If the same query produces the same response, storing the response is cheaper than repeating the query.

This lesson first establishes a measure of that repetition, then lists the places where a response can be stored. Each storage location is a cache layer, and the layers form a chain: the earlier a request stops in the chain, the more cheaply it is served.

Measured Repetition

The measurements run against the library schema used throughout the course: the branch, book, member, and loan relations. The block below builds the database from scratch.

# setup.sh — sets up the library database (same columns as the M16/K04 schema)
rm -f library.db
sqlite3 library.db <<'SQL'
CREATE TABLE branch (branch_id INTEGER PRIMARY KEY, name TEXT NOT NULL, city TEXT NOT NULL);
CREATE TABLE book (book_id INTEGER PRIMARY KEY, title TEXT NOT NULL, author TEXT NOT NULL,
                    publication_year INTEGER, branch_id INTEGER REFERENCES branch(branch_id));
CREATE TABLE member (member_id INTEGER PRIMARY KEY, first_name TEXT NOT NULL, last_name TEXT NOT NULL,
                  email TEXT, registered_at TEXT NOT NULL);
CREATE TABLE loan (loan_id INTEGER PRIMARY KEY,
                    book_id INTEGER NOT NULL REFERENCES book(book_id),
                    member_id INTEGER NOT NULL REFERENCES member(member_id),
                    pickup_date TEXT NOT NULL, return_date TEXT);
INSERT INTO branch VALUES (1,'Central','Ankara'),(2,'Bahcelievler','Ankara'),
  (3,'Kadikoy','Istanbul'),(4,'Konak','Izmir');
INSERT INTO book SELECT n, 'Book ' || n, 'Author ' || (n % 40 + 1), 1950 + n % 70, n % 4 + 1
  FROM (WITH RECURSIVE s(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM s WHERE n < 300) SELECT n FROM s);
INSERT INTO member SELECT n, 'Member' || n, 'Surname' || n, 'member' || n || '@example.test', '2024-01-01'
  FROM (WITH RECURSIVE s(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM s WHERE n < 200) SELECT n FROM s);
INSERT INTO loan SELECT n, n % 300 + 1, n % 200 + 1,
  '2025-' || substr('0' || (n % 12 + 1), -2) || '-' || substr('0' || (n % 28 + 1), -2),
  CASE WHEN n % 5 = 0 THEN NULL ELSE '2025-12-31' END
  FROM (WITH RECURSIVE s(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM s WHERE n < 4000) SELECT n FROM s);
CREATE INDEX loan_member ON loan(member_id);
CREATE INDEX book_branch ON book(branch_id);
SQL
sqlite3 library.db "SELECT 'branch=' || (SELECT count(*) FROM branch) || ' book=' || (SELECT count(*) FROM book) || ' member=' || (SELECT count(*) FROM member) || ' loan=' || (SELECT count(*) FROM loan);"

branch=4 book=300 member=200 loan=4000

The origin exposes three reads: book detail, branch shelf count, and a member’s loan history. Every read is counted — the same approach as the query counter in the Performance Problems topic.

// origin.mjs — the origin of the library data; every read is counted
import { DatabaseSync } from "node:sqlite";

export function openOrigin(file = "library.db") {
  const db = new DatabaseSync(file);
  const counter = { queries: 0 };
  const single = (sql, ...d) => { counter.queries += 1; return db.prepare(sql).get(...d); };
  return {
    counter,
    bookDetail: (id) => single(
      "SELECT book_id, title, author, publication_year, branch_id FROM book WHERE book_id = ?", id),
    branchStock: (branchId) => single(
      `SELECT ? AS branch_id, count(*) AS on_shelf FROM book b
       WHERE b.branch_id = ? AND NOT EXISTS
         (SELECT 1 FROM loan l WHERE l.book_id = b.book_id AND l.return_date IS NULL)`,
      branchId, branchId),
    memberHistory: (memberId) => single(
      "SELECT ? AS member_id, count(*) AS total FROM loan WHERE member_id = ?", memberId, memberId),
    close: () => db.close(),
  };
}

// Request stream: book detail requests concentrate on a handful of titles (deterministic generator).
export function requestStream(count) {
  let seed = 20250729;
  const random = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648);
  const requests = [];
  for (let i = 0; i < count; i++) {
    const r = random();
    if (r < 0.7) requests.push(["book", Math.floor(random() * 20) + 1]);
    else if (r < 0.9) requests.push(["branch", Math.floor(random() * 4) + 1]);
    else requests.push(["member", Math.floor(random() * 50) + 1]);
  }
  return requests;
}

The number that results when a stream of six hundred requests is applied directly to the origin is the measure of the repetition.

// no-cache.mjs — every request goes to the origin: query count equals request count
import { openOrigin, requestStream } from "./origin.mjs";

const origin = openOrigin();
const requests = requestStream(600);
const reader = { book: origin.bookDetail, branch: origin.branchStock, member: origin.memberHistory };

const t = performance.now();
for (const [kind, id] of requests) reader[kind](id);
const duration = performance.now() - t;

const unique = new Set(requests.map(([kind, id]) => `${kind}:${id}`)).size;
console.log(`requests=${requests.length}  unique_keys=${unique}  origin_queries=${origin.counter.queries}`);
console.log(`repeat queries producing the same response = ${origin.counter.queries - unique}`);
console.log(`duration=${duration.toFixed(1)} ms (machine-dependent)`);
origin.close();

requests=600  unique_keys=53  origin_queries=600
repeat queries producing the same response = 547
duration=545.3 ms (machine-dependent)

Six hundred requests produced only 53 distinct responses. The remaining 547 queries ran to reproduce a result the database had already produced. The duration field is machine-dependent and varies on every run; the numbers 600 and 53 do not change, because they are properties of the workload.

The opportunity here is not speeding up the query. The query is already indexed and its plan is sound. The opportunity is not running the query at all.

The Four Layers in the Chain

A response can be stored at four points between the origin and the user.

The client cache sits on the user’s own device: the browser’s HTTP cache or the application’s local storage. Because it never reaches the network, it is the cheapest layer. In exchange, it serves only a single user, and its content is out of the server’s control — the copy placed there stays until the lifetime it was given at write time runs out.

The edge cache is a shared intermediate point geographically close to the user. It was introduced for static files in the Static Hosting lesson of the Frontend Quality course; the same layer also works for API responses. Because it is shared, a copy warmed by one user is used by others. For the same reason, it cannot carry data private to one person: a member’s history placed there could be served to a different member.

The server cache is under the application’s own control. It comes in two forms: in-process memory and a separate store shared by every instance. The in-process form is the fastest, but each instance keeps its own copy; the shared form costs a round trip but gives a single point of truth. This layer lives in the application’s code, so it can apply an invalidation decision instantly.

The database cache is inside the origin itself: the buffer pool and the query plan cache. This layer was already running in the measurement above — all 600 queries ran against warm pages. The result was still 600 queries. The database cache lowers a query’s cost, not the number of queries. The connection, parsing, plan selection, and round trip are paid again on every call.

Measuring the Chain

Setting up the four layers as a chain and counting how many requests each layer stops turns the layer discussion into a number. The setup below implements three cache layers; the fourth is inside the origin. Each layer has a capacity: once it fills, the oldest entry is evicted.

// layers.mjs — the four-layer chain: where does each request stop?
import { openOrigin } from "./origin.mjs";

// A small cache that evicts the oldest entry once it is at capacity.
function smallCache(capacity) {
  const box = new Map();
  return {
    get: (k) => box.get(k),
    has: (k) => box.has(k),
    set(k, v) {
      box.delete(k); box.set(k, v);
      if (box.size > capacity) box.delete(box.keys().next().value);
    },
  };
}

const origin = openOrigin();
const reader = { book: origin.bookDetail, branch: origin.branchStock, member: origin.memberHistory };
const SHARED = new Set(["book", "branch"]);          // resources that return the same thing to every member

const clients = new Map();                          // one cache per user, 4 entries
const edge = smallCache(10);                         // shared, shared resources only
const server = smallCache(200);                      // shared, everything
const counter = { client: 0, edge: 0, server: 0, origin: 0 };

function read(user, kind, id) {
  const key = `${kind}:${id}`;
  if (!clients.has(user)) clients.set(user, smallCache(4));
  const client = clients.get(user);
  const place = (v) => {
    if (SHARED.has(kind)) edge.set(key, v);
    client.set(key, v);
    return v;
  };
  if (client.has(key)) { counter.client += 1; return client.get(key); }
  if (SHARED.has(kind) && edge.has(key)) { counter.edge += 1; return place(edge.get(key)); }
  const serverKey = SHARED.has(kind) ? key : `${key}@${user}`;
  if (server.has(serverKey)) { counter.server += 1; return place(server.get(serverKey)); }
  counter.origin += 1;
  const value = reader[kind](id);
  server.set(serverKey, value);
  return place(value);
}

let seed = 20250729;
const random = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648);
const REQUESTS = 600;
for (let i = 0; i < REQUESTS; i++) {
  const user = Math.floor(random() * 40) + 1;
  const r = random();
  const kind = r < 0.7 ? "book" : r < 0.9 ? "branch" : "member";
  const id = kind === "book" ? Math.floor(random() * 20) + 1
    : kind === "branch" ? Math.floor(random() * 4) + 1 : user;
  read(user, kind, id);
}

console.log(`requests=${REQUESTS}`);
for (const [name, n] of Object.entries(counter)) {
  console.log(`${name.padEnd(8)} ${String(n).padStart(4)}  (%${((n / REQUESTS) * 100).toFixed(1)})`);
}
console.log(`origin queries = ${origin.counter.queries}`);
origin.close();

requests=600
client     98  (%16.3)
edge      189  (%31.5)
server    260  (%43.3)
origin     53  (%8.8)
origin queries = 53

The number of requests reaching the origin fell from 600 to 53; every distinct key in the workload hit the origin exactly once. The rest of the chain is the interesting part. The closest layer stopped only 16% of requests, because its capacity is limited to four entries and each user’s copy is separate. The shared edge layer stopped 31% of them. The server layer did the most work, at 43%: it is the layer with the largest capacity and the broadest scope.

The fact that these numbers depend on capacity is the real lesson of this measurement. If the edge layer’s capacity were increased, its share would grow and the server’s would shrink. The layers do not replace one another; each one picks up what the others leave behind.

Which Layer a Piece of Data Belongs In

The layer is chosen using two criteria.

The first is sharing scope. If a response returns the same thing to every user, it can be placed in shared layers. Book detail and branch shelf count are like this. A member’s loan history is not; placing it in the edge cache means breaching the authorization boundary examined in the Authentication and Authorization course. In the measurement code, this distinction is made with the SHARED set, and records private to one person are keyed with the user’s identity in the server layer.

The second is staleness tolerance. The cached copy stays out of date for a while once the origin changes. For a book’s author, that period can be hours; for a branch’s shelf count, seconds. For the loan transaction itself, it is zero — the result of a write cannot be read from a cache.

A layer’s closeness to the user is inversely proportional to how controllable it is. An entry in the server layer can be deleted by code instantly. Deleting one in the edge layer takes a separate operation. For the copy on the client, there is no way at all: it is used until the lifetime given at write time runs out. For that reason, the data placed in the closest layer is the data given the longest lifetime, and that decision is made knowing it cannot be reversed.

Summary

• A workload of six hundred requests produced only 53 distinct responses; 547 queries reproduced a result already produced before. The opportunity in caching is not speeding up the query but not running it at all.
• A response can be stored at four points: the client, edge, server, and database caches. The first three lower the query count; the database cache only lowers the cost of each query.
• When the chain was measured, requests reaching the origin fell from 600 to 53; the layers’ shares were distributed according to their capacities. The layers do not replace one another — they pick up what the others leave behind.
• Only data that returns the same thing to every user belongs in shared layers; data private to one person breaches the authorization boundary in a shared cache.
• A layer gets cheaper and less controllable as it moves closer to the user; the copy on the client cannot be revoked until the lifetime given at write time runs out.

Next Step

In this lesson, the cache was placed by hand inside the read path: look it up, fetch it from the origin if it is missing, store it. This pattern has a name, and it is not the only option. A cache that sits beside the read path and one that every read passes through set up different divisions of responsibility; on the write path, the question of when the cache gets updated arises — at write time, or later. The next lesson runs these three strategies against the same workload and compares the number of reads and writes reaching the origin, along with the inconsistency window.