--- title: 'HTTP Caching' source: 'https://academia.sh/en/courses/application-protocols/http-caching' course: 'Application Layer Protocols' language: en updated: '2026-08-17T18:06:59+00:00' license: 'CC BY-SA 4.0' --- # HTTP Caching Freshness and validation are separate questions, and the validator has a resolution: it catches 9 of the 12 stale copies across forty exchanges, misses the tag change on 3, and the intermediary calls them fresh — three of the five wrong decisions in the open regime come from here. The previous lesson measured how framing determines which field the intermediary can read. But being able to read a field is not always enough. Four of the five decisions look at a single field: the method says retryability, the flag says shareability. The fifth is different. **Freshness** does not come from one field's value; it comes from **comparing** the copy on hand against the message that just arrived. Comparison brings a new problem. Any mechanism comparing two values has a **resolution**: a difference smaller than what it can distinguish is no difference to it. This is what this lesson measures — how often the intermediary notices its stored copy has gone stale, and what it says when it does not. ## Freshness and Validation: Two Separate Questions There are two separate questions for a copy in the intermediary's cache. **The freshness question:** can I give out this copy **without asking**? The answer is the **freshness lifetime** the server declares. Within that lifetime, the intermediary hands out the copy directly, never returning to the server. **The validation question:** once the lifetime has run out, is the copy still valid? The answer comes from the **validator**: a tag the server attaches to the representation. The intermediary compares its held tag against the server's; a match means the copy is valid. Confusing the two loses the measure. Freshness is a **duration** decision and spends no round trip. Validation is an **identity** decision and gives the truth. Caching's whole design rests on the trade-off between the two: stretch the duration and round trips fall, but the odds of handing out a stale copy rise. A third question comes before these two, established in the previous two lessons: is this response **storable**, and if stored, is it **shareable**? The intermediary stores a copy only if the method is safe, and hands it to someone else only if there is no personalization flag. Freshness follows these two decisions; there is no freshness for a copy never stored. The measurement preserves this order: all three regimes give the same `storable` and `shareable` decisions, differing only in freshness. What this course measures is not the directive list or the body caching saves; those belong to the **Caching, Queues and Asynchronous Processing** course. The only thing measured here is the **validator's resolution** — which changes the tag sees, and which it does not. ## The Validator's Resolution HTTP defines two kinds of validator, and their resolutions are not the same. The **timestamp** validator carries the representation's last moment of change. Its resolution is the time unit the tag is written in: a representation changing twice within the same second carries the same tag after both changes, and the intermediary sees no difference. The **entity tag** validator is a string derived from the representation's content and is not tied to a time unit. It has a **strong** and a **weak** form. A strong tag carries byte-level distinction. A weak tag deliberately disregards changes considered semantically unimportant — a validator whose **resolution is lowered on purpose.** ```text # teaching dump, not run # validators the server carries in the response HTTP/1.1 200 OK Last-Modified: Sun, 09 Aug 2026 11:04:07 GMT ETag: "olcum-7a" Cache-Control: max-age=60 Age: 12 # the conditional request the intermediary builds to test its copy GET /olcum/kuzey HTTP/1.1 Host: station.example If-None-Match: "olcum-7a" If-Modified-Since: Sun, 09 Aug 2026 11:04:07 GMT # if the tag is unchanged HTTP/1.1 304 Not Modified ETag: "olcum-7a" ``` `Cache-Control` declares the freshness lifetime, `Age` declares how long the copy has spent in cache; together they answer the freshness question. `If-None-Match` and `If-Modified-Since` build the **conditional request**, and `304` is the status code declaring the tag stayed the same. The measurement reduces all these fields to one declaration: does the tag in the message differ from the one the intermediary holds? `Age` has a separate function: the intermediary declares its stored copy's age through this field, so the next intermediary in the chain can do the same accounting. With more than one intermediary, age accumulates as it travels, and the freshness lifetime is **a single budget** for the whole chain; each stop spends its own share. The tag itself is nothing more than what is written in the message. **Nothing verifies the validator:** if the tag has not changed, the intermediary believes the representation has not changed, because it has no other basis to go on. - **HA55** — The same forty exchanges and oracle are used; the oracle knows whether the intermediary's stored copy is actually stale. - **HA56** — The intermediary sits on the path and sees passing responses; it compares its held tag against the one arriving in the message. This comparison **needs no extra round trip**, since the message already passes in front of it. - **HA57** — The tag has a resolution: if a change is smaller than what the tag can distinguish, the tag stays the same and the intermediary cannot see the change. - **HA58** — Three regimes are measured. In `duration only`, the message declares a freshness lifetime and no validator. In `tag`, the validator is declared. In `always ask`, neither is declared. - **HA59** — The freshness lifetime does not run out within the measurement window; in `duration only`, the intermediary counts every copy as fresh. - **HA60** — An intermediary that cannot make the freshness decision sends a conditional request; the server knows the truth, so catching after the round trip is complete. - **HA61** — The byte count is a count of the message's fields; the body is not counted. ## The Measurement ```python """HTTP caching: freshness, validation, and the validator's resolution. Part 1 - how many stale copies the validator catches. Part 2 - three regimes: duration only, tag, always ask. """ SEED = 20260809 SAFE = {"GET", "HEAD"} IDEMPOTENT = {"GET", "HEAD", "PUT", "DELETE"} METHOD_POOL = ["GET", "GET", "GET", "HEAD", "POST", "PUT", "DELETE"] PATH_POOL = ["/olcum/kuzey", "/olcum/yamac", "/ozet", "/kayit", "/oturum"] DECISIONS = ("storable", "shareable", "fresh", "repeatable", "redirectable") def generator(seed): d = seed % 2147483646 + 1 def r(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return r def exchanges(count=40): r, out = generator(SEED), [] for i in range(count): method, path = METHOD_POOL[r(7)], PATH_POOL[r(5)] private = path == "/oturum" or r(5) == 0 stale = r(3) == 0 out.append({"num": i + 1, "method": method, "path": path, "private": private, "stale": stale, "private_marker": private and r(4) != 0, "tag_missed": stale and r(6) == 0}) return out def oracle(a): return {"storable": a["method"] in SAFE and not a["private"], "shareable": not a["private"], "fresh": not a["stale"], "repeatable": a["method"] in IDEMPOTENT, "redirectable": True} def message(a, visible): full = {"method": a["method"], "path": a["path"], "host": "station.example", "private_marker": a["private_marker"], "validator": a["stale"] and not a["tag_missed"], "freshness_window": True} return {k: v for k, v in full.items() if k in visible} def byte_count(i): return sum(len(k) + len(str(v)) + 4 for k, v in i.items()) def infer(i): k = {} if "method" in i: k["repeatable"] = i["method"] in IDEMPOTENT if "private_marker" in i: k["shareable"] = not i["private_marker"] if "method" in i: k["storable"] = i["method"] in SAFE and not i["private_marker"] if "validator" in i: k["fresh"] = not i["validator"] elif "freshness_window" in i: k["fresh"] = i["freshness_window"] if "host" in i: k["redirectable"] = True return k COMMON = ("method", "path", "host", "private_marker") VISIBLE = {"duration only": COMMON + ("freshness_window",), "tag": COMMON + ("validator",), "always ask": COMMON} batch = exchanges() STALE = [a for a in batch if a["stale"]] MISSED = [a for a in STALE if a["tag_missed"]] print(f"exchanges {len(batch)} | decisions {len(batch) * len(DECISIONS)} | " f"stale copy {len(STALE)} | missed tag change {len(MISSED)}") print("exchanges the tag missed:", [(a["num"], a["path"]) for a in MISSED]) print() print(f" {'regime':18s}{'caught':>8s}{'missed':>8s}{'resolution':>12s}" f"{'rounds asked':>14s}{'after round trip':>18s}") for name in VISIBLE: caught = sum(1 for a in STALE if infer(message(a, VISIBLE[name])).get("fresh") is False) asked = sum(1 for a in batch if "fresh" not in infer(message(a, VISIBLE[name]))) print(f" {name:18s}{caught:>8d}{len(STALE) - caught:>8d}" f"{caught / len(STALE):>12.4f}{asked:>14d}" f"{len(STALE) if asked else caught:>18d}") print() print(f" {'regime':18s}{'decision':9s}{'correct':>8s}{'wrong':>8s}" f"{'unavailable':>13s}{'extra round':>14s}{'bytes':>7s}") for name in VISIBLE: fd = fy = fe = d = y = e = t = b = 0 for a in batch: truth, given = oracle(a), infer(message(a, VISIBLE[name])) missing = False for decision in DECISIONS: if decision not in given: e += 1 missing = True fe += decision == "fresh" elif given[decision] == truth[decision]: d += 1 fd += decision == "fresh" else: y += 1 fy += decision == "fresh" t += 1 if missing else 0 b += byte_count(message(a, VISIBLE[name])) print(f" {name:18s}{'freshness':9s}{fd:>8d}{fy:>8d}{fe:>13d}") print(f" {name:18s}{'all':9s}{d:>8d}{y:>8d}{e:>13d}{t:>14d}{b:>7d}") ``` ``` exchanges 40 | decisions 200 | stale copy 12 | missed tag change 3 exchanges the tag missed: [(24, '/ozet'), (25, '/oturum'), (37, '/olcum/yamac')] regime caught missed resolution rounds asked after round trip duration only 0 12 0.0000 0 0 tag 9 3 0.7500 0 9 always ask 0 12 0.0000 40 12 regime decision correct wrong unavailable extra round bytes duration only freshness 28 12 0 duration only all 186 14 0 0 3973 tag freshness 37 3 0 tag all 195 5 0 0 3724 always ask freshness 0 0 40 always ask all 158 2 40 40 3013 ``` ## Resolution: Three-Quarters The upper table's middle row is this lesson's number. Across forty exchanges, the intermediary's stored copy is stale **12** times. The validator sees the tag changed in **9** of those and counts the copy stale. In **3**, the tag has not changed; the intermediary sees the two tags as identical and calls the copy **fresh**. Resolution **0.7500**. The three missed exchanges have different paths: `/ozet`, `/oturum`, and `/olcum/yamac`. The miss depends not on a path or a method but on the change being smaller than what the tag can distinguish. This follows from the definition of resolution and is not a fixable bug; it is a range narrowed only by changing the tag. None of these three misses show any sign: the intermediary followed the rule, compared the tags correctly, and handed out the copy for a valid reason. Only the oracle knows the decision was wrong, and it exists only because we generated the setup ourselves. In a real deployment, no such oracle exists: **a stale copy looks identical to a fresh one.** One more way to drop a stored copy costs no round trip: seeing a `PUT` or `DELETE` on its stored path, the intermediary infers that copy is no longer valid and drops it — a decision drawn from the message, independent of the tag's resolution. Its limit: the intermediary sees only the request that **passes in front of it**. A request changing the same representation through another path does not, and the copy silently goes stale — the same outcome as the three exchanges the tag missed. The upper table's first row gives the case with no validator. The intermediary only looks at the freshness lifetime, which never runs out: it catches **none** of the **12** stale copies. Resolution **0.0000**. Duration is not even a resolution measure, since it never tests for change — it only places a bet. The last row does the opposite: the intermediary asks on every exchange, **40** round trips, and catching is complete, **12/12**. But forty questions were asked to catch those twelve copies; twenty-eight were spent on copies already fresh. ## The Decision Cost of Three Regimes The lower table gives the same three regimes across the full decision set. In **`duration only`**, the freshness decision is **28** correct, **12** wrong; the full set is **186**/**14**/**0**. The wrong count peaks here: without asking once, without leaving a single decision unavailable, the intermediary is wrong fourteen times. Bytes: **3973**, the highest of the three regimes. In **`tag`**, freshness is **37** correct, **3** wrong; the full set is **195**/**5**/**0** and **0** round trips. This is exactly the shared setup's open regime, and **three of its five wrong decisions come from this row**. The remaining two come from the personalization flag the server forgot to set and have nothing to do with freshness. In **`always ask`**, freshness is never decided on its own: **40** unavailable. The full set is **158**/**2**/**40** and **40** round trips. Wrong drops from **5** to **2**; the remaining two are again from the personalization flag — asking does not fix that, since the question asked concerns freshness. The three rows give the course's axis once more. **An unavailable decision is a round-trip debt:** forty round trips for forty decisions. **A decision that is made can still be wrong:** the tag regime takes all forty freshness decisions with zero round trips and is wrong on three. The duration regime also spends zero round trips but is wrong on twelve — the same cost, three times the wrong count. ## What Ranking Depends On There is no single "best" among the three regimes; ranking changes with what is expensive. If round trips are expensive, `tag` is chosen: zero round trips, three wrong. Three wrong decisions, in a set of two hundred, is a share of **0.0150**; since the smallest measurable difference here is **1/200 = 0.0050**, three wrong decisions sits inside the band and is defensible. If handing out a stale copy is unacceptable, `always ask` is chosen and forty round trips are paid. `duration only` is defensible only for representations not expected to change; with a stale rate of **12/40** here, it is not defensible. Raising the tag's resolution is a fourth path, absent from the table since the measurement takes the tag as given. Still, the direction is clear: an entity tag over a timestamp, a strong tag over a weak one, narrow resolution rather than widen it. Each narrowing reduces the three misses, and none adds a round trip. One last observation: `always ask`'s bytes are **3013**, `tag`'s are **3724**. Asking shrinks the message, since a field not carried takes no bytes — but the smaller message is nothing next to the forty round trips owed. **Bytes are cheap, round trips are expensive** — the course's measurement axis has said exactly this all along. ## Summary - Freshness and validation are separate questions: freshness answers whether the copy can be given out **without asking**; validation answers whether it is still valid. - The validator has a resolution; a timestamp is limited by its time unit, an entity tag by content, and a weak tag deliberately lowers resolution. - **9** of the **12** stale copies across forty exchanges are caught, **3** miss the tag change and the intermediary calls them **fresh**; resolution **0.7500**. - **3** of the open regime's **5** wrong decisions come from these three copies; the remaining two come from the personalization flag, unrelated to freshness. - Three regimes: `duration only` **186**/**14**/**0** and 0 round trips, `tag` **195**/**5**/**0** and 0 round trips, `always ask` **158**/**2**/**40** and 40 round trips. Asking lowers wrong decisions and creates a round-trip debt. ## Next Step Throughout this topic, the intermediary made its decisions for the same reason every time: **it could read what was written in the message.** It read the method and decided retryability, read the flag and decided shareability, read the tag and decided freshness. Even where it was wrong, it could still read; the error came not from failing to read but from what it read being incomplete. The next topic removes this condition. **What if the message cannot be read?** If the two endpoints turn the message between them into something only they can understand, what is left in the intermediary's hands? The question connects to the course's third claim: the endpoints are not the only ones making decisions, and closing off the message moves decisions from the middle out to the endpoints. The next lesson takes up how that closing off is built, and who pays what for it.