--- title: 'Context Compression' source: 'https://academia.sh/en/courses/ai-agents/context-compression' course: 'AI Agents' language: en updated: '2026-10-21T05:02:26+00:00' license: 'CC BY-SA 4.0' objectives: - 'Shows that the carried state is a token item and builds two compaction policies with the standard library: summarization and selective recall.' - 'Sweeps the compaction ratio and measures the tokens gained next to the number of tasks that need the dropped field.' - 'Relates the choice of content on the same context budget to the hit rate and ties it to the irrelevant-context finding.' - 'Explains with the error and step counts the point where compaction raises tokens above the baseline.' --- # Context Compression Sweeping two compaction policies over the same 24 tasks: when the carried state is paid for as well, the baseline gives 21913 tokens and 0.9583 task success. When selective recall cuts the kept fields to three, tokens fall to 16569 and task success does not change at all, but the number of tasks that need the dropped field rises from 0 to 8. Summarization falls to 0.7917 on the same budget and at the tightest setting raises tokens to 33566, 1.53 times the baseline. This is the point where compaction makes the job worse. The previous lesson treated memory between tasks: what was written determined the steps saved in later tasks. Within a task the problem reverses. The context is not erased, it **grows**: every observation is added to the state and carried again at every later step. The token count of the shared loop sees only half of this: it counts the step trace, it does not count the carried state itself. This lesson first adds that missing item, then sweeps two policies of **context compaction**: **summarization** drops the oldest fields and puts a single summary entry in their place; **selective recall** keeps the fields that the remaining steps declare. What will be measured is two numbers side by side: as the compaction ratio rises, the carried tokens fall, but **the number of tasks that need the dropped field** grows. This lesson is an instance of the course's second claim; at the tightest setting compaction raises tokens **above** the baseline. - **TM67.** The code is a **simulator**; no real model endpoint or tool server is called. The environment, the tools, and the tasks are **fictional**, seed **20260218**. - **TM68.** The carried state is a token item: each field is paid 12 tokens for its name and format and its own token count for its value, and this cost is paid **again at every step**. - **TM69. The compaction ratio is given by the `keep` setting:** at most `keep` fields are carried in the state and the excess is dropped. Compaction is applied at the end of every step. - **TM70. Summarization** drops the oldest fields and puts a single summary entry in their place; the entry takes up the place of a field but carries no usable value. - **TM71. Selective recall** keeps, on the same `keep` budget, the fields that the remaining steps of the plan **declare**; the newest fields fill the leftover room. - **TM72.** When a tool asks for a field that was dropped earlier, a **missing field** error returns. A task in which this error is seen at least once counts as a **task that needs the dropped field**. - **TM73.** The task set is 24, the step budget is 8, and the baseline is the **loop without compaction**. ## The Cost of the Carried State The block below sets up the shared definition and prints the baseline. The tool and field names in the code are the `read_meter`, `consumption`, `tariff`, `search_docs`, and `write_record` tools of the previous lessons; they are counted in tokens here, so they stay Turkish. ```python # FICTIONAL environment and FICTIONAL agent. No real model endpoint, tool server, or memory # service is called. The subscriber records, documents, and tasks are fictional, seed 20260218. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 def tokenize(m): b = [] for s in str(m).lower().replace(",", " , ").split(): while len(s) > 5: b.append(s[:5]) s = s[5:] b.append(s) return b def trace_id(m): h = 2166136261 for c in str(m): h = ((h ^ ord(c)) * 16777619) & M32 return h # ---- FICTIONAL environment: records of the measurement network # wire words stay Turkish: they are counted in tokens (same rule as the MCP course) # onceki/simdiki/tip/bolge = previous/current/type/region; mesken/isyeri = residential/commercial REGION = ["dogu", "bati", "kuzey", "guney", "merkez"] TYPE = ["mesken", "isyeri"] SUBSCRIBERS = {} for i in range(60): r = rng(SEED + 31 * i) previous = 10000 + int(r() * 80000) SUBSCRIBERS[f"A{i + 1:03d}"] = {"bolge": REGION[int(r() * 5)], "tip": TYPE[int(r() * 2)], "onceki": previous, "simdiki": previous + int(r() * 70)} TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16 COMMERCIAL_FACTOR = 2 DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu", "TR02": "isyeri abonesinde birim ucret iki katidir", "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir", "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider", "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"} CALLS = [0] # tool call counter def _charge(m3, type_): remaining, lower, total = m3, 0, 0 for upper, unit in TIERS: band = min(remaining, upper - lower) if band <= 0: break total += band * unit remaining -= band lower = upper total += remaining * ABOVE return total * (COMMERCIAL_FACTOR if type_ == "isyeri" else 1) def t_read_meter(abone=None, **_): a = SUBSCRIBERS.get(abone) if a is None: return {"hata": "abone yok"} return {"onceki": a["onceki"], "simdiki": a["simdiki"], "tip": a["tip"], "bolge": a["bolge"]} def t_consumption(onceki=None, simdiki=None, **_): if onceki is None or simdiki is None: return {"hata": "eksik alan"} f = simdiki - onceki return {"m3": f + 100000 if f < 0 else f} def t_tariff(m3=None, tip="mesken", **_): if m3 is None: return {"hata": "eksik alan"} return {"tutar": _charge(m3, tip)} def t_document(anahtar=None, **_): if not anahtar: return {"hata": "eksik alan"} q = set(tokenize(anahtar)) best, code = -1, None for k, m in DOCUMENTS.items(): o = len(q & set(tokenize(m))) if o > best: best, code = o, k return {"kod": code, "metin": DOCUMENTS[code]} if best > 0 else {"hata": "bulunamadi"} def t_record(abone=None, sutun=None, deger=None, **_): if not (abone and sutun and deger is not None): return {"hata": "eksik alan"} return {"yazildi": True, "abone": abone, "sutun": sutun, "deger": deger} TOOLS = { "sayac_oku": {"function": t_read_meter, "fields": ("abone",), "side_effect": False}, "tuketim": {"function": t_consumption, "fields": ("onceki", "simdiki"), "side_effect": False}, "tarife": {"function": t_tariff, "fields": ("m3", "tip"), "side_effect": False}, "belge_ara": {"function": t_document, "fields": ("anahtar",), "side_effect": False}, "kayit_yaz": {"function": t_record, "fields": ("abone", "sutun", "deger"), "side_effect": True}, } TOOL_NAMES = list(TOOLS) def call_tool(name, state): """Fields outside the schema are filtered out; a missing field is the tool's own error.""" CALLS[0] += 1 if name not in TOOLS: return {"hata": "arac yok"} kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state} return TOOLS[name]["function"](**kw) # ---- FICTIONAL tasks: each one is solved by a known tool sequence TASKS = [] for i in range(24): r = rng(SEED + 9973 * i) sub = f"A{1 + int(r() * 60):03d}" t = pick(r(), [0.30, 0.22, 0.20, 0.28]) if t == 0: TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub, "request": f"{sub} abonesinin bu donem bedeli nedir", "sequence": ("sayac_oku", "tuketim", "tarife")}) elif t == 1: TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub, "request": f"{sub} abonesinin bu donem tuketimi kac metrekup", "sequence": ("sayac_oku", "tuketim")}) elif t == 2: TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub, "request": "sayac donusunde endeks farki nasil hesaplanir", "sequence": ("belge_ara",)}) else: TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub, "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz", "sequence": ("sayac_oku", "tuketim", "tarife", "kayit_yaz")}) def correct_answer(g): a = SUBSCRIBERS[g["subscriber"]] f = a["simdiki"] - a["onceki"] m3 = f + 100000 if f < 0 else f if g["kind"] == "consumption": return m3 if g["kind"] == "document": return "SY01" return _charge(m3, a["tip"]) # ---- FICTIONAL agent: at each step it picks a tool or finishes def selection(d): return max(0.34, 0.80 - 0.09 * (d - 1)) # share of picking the right tool, falls with depth def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False): """FICTIONAL agent loop. No real model is called; tool selection comes from a single draw that depends on the task's sequence and the depth.""" r = rng(seed ^ trace_id(g["name"])) state = {"abone": g["subscriber"], "anahtar": g["request"], "sutun": "bedel"} trace, step, tok, errors = [], 0, 0, 0 seq, k = g["sequence"], 0 while step < max_steps: step += 1 tok += 60 + 22 * len(trace) target = seq[k] if k < len(seq) else None if target is None: # the job is done , the agent should stop if r() < 0.88: return {"done": True, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("sonuc"), "approvals": state.get("onay", 0)} choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] if TOOLS[choice]["side_effect"] and approval: state["onay"] = state.get("onay", 0) + 1 if choice == "kayit_yaz": state["deger"] = state.get("sonuc") s = call_tool(choice, state) trace.append((choice, "hata" in s)) tok += 30 if "hata" in s: errors += 1 if self_critique: tok += 40 continue state.update({kk: vv for kk, vv in s.items() if kk != "hata"}) if choice == "tuketim": state["sonuc"] = s["m3"] if choice == "tarife": state["sonuc"] = s["tutar"] if choice == "belge_ara": state["sonuc"] = s["kod"] if choice == target: k += 1 return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("sonuc"), "approvals": state.get("onay", 0)} CALLS[0] = 0 t = [agent(g) for g in TASKS] print("baseline: steps", sum(x["steps"] for x in t), ", calls", CALLS[0], ", trace tokens", sum(x["tokens"] for x in t), ", errors", sum(x["errors"] for x in t), ", success", round(sum(x["answer"] == correct_answer(g) for x, g in zip(t, TASKS)) / 24, 4)) ``` ``` baseline: steps 97 , calls 74 , trace tokens 11978 , errors 8 , success 0.9583 ``` The shared loop finishes the 24 tasks in 97 steps and counts 11978 tokens. This number covers **only the trace**: at every step the tool names and observation headings up to that point. Yet the prompt that goes to the model carries the state as well: the index readings that were read, the subscriber type, the document text that was found. Every field is paid again at every step, and as the state grows this item grows faster than the trace. ## Sweeping the Compaction Ratio The block below completes the cost and sweeps the two policies over four settings. ```python NAME_COST = 12 # name and format cost of a carried field RESULT_KEY = {"tuketim": "m3", "tarife": "tutar", "belge_ara": "kod"} def state_tokens(state): return sum(NAME_COST + len(tokenize(v)) for v in state.values()) def compact(state, remaining, keep, policy): """Anything beyond keep fields is dropped. Summary drops the oldest and puts a single summary entry in its place; selective keeps the fields the remaining steps declare.""" if len(state) <= keep: return state, () keys = list(state) if policy == "selective": needed = {a for name in remaining for a in TOOLS[name]["fields"]} | {"sonuc"} keys.sort(key=lambda k: k in needed) dropped = tuple(keys[:len(state) - keep]) new = {k: v for k, v in state.items() if k not in dropped} if policy == "summary": new["ozet"] = f"{len(dropped)} alan dusuruldu" return new, dropped def agent_c(g, keep=99, policy="summary", seed=SEED, max_steps=8): """The shared loop, with context compaction at the end of every step.""" r = rng(seed ^ trace_id(g["name"])) state = {"abone": g["subscriber"], "anahtar": g["request"], "sutun": "bedel"} trace, step, tok, errors, dropped, needed = [], 0, 0, 0, set(), 0 seq, k = g["sequence"], 0 while step < max_steps: step += 1 tok += 60 + 22 * len(trace) + state_tokens(state) target = seq[k] if k < len(seq) else None if target is None: if r() < 0.88: break choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] if choice == "kayit_yaz": state["deger"] = state.get("sonuc") missing = [a for a in TOOLS[choice]["fields"] if a not in state and a in dropped] s = call_tool(choice, state) trace.append(choice) tok += 30 if "hata" in s: errors, needed = errors + 1, needed + bool(missing) continue state.update({kk: vv for kk, vv in s.items() if kk != "hata"}) if choice in RESULT_KEY: state["sonuc"] = s[RESULT_KEY[choice]] if choice == target: k += 1 state, dr = compact(state, seq[k:], keep, policy) dropped |= set(dr) return {"steps": step, "tokens": tok, "errors": errors, "needed": needed, "answer": state.get("sonuc")} def run(keep, policy, seed=SEED): CALLS[0] = 0 st = tok = errors = needed = correct = 0 for g in TASKS: s = agent_c(g, keep, policy, seed) st, tok, errors = st + s["steps"], tok + s["tokens"], errors + s["errors"] needed, correct = needed + (s["needed"] > 0), correct + (s["answer"] == correct_answer(g)) return st, CALLS[0], tok, errors, needed, correct / len(TASKS) SWEEP = [(99, "uncompacted")] + [(t, p) for p in ("summary", "selective") for t in (6, 5, 4, 3)] print("keep policy steps calls tokens errors needed success | 2nd run") for keep, pol in SWEEP: a = run(keep, "summary" if keep == 99 else pol) b = run(keep, "summary" if keep == 99 else pol, SEED + 1) print(f"{keep if keep != 99 else '--':>4} {pol:<14} {a[0]:>5} {a[1]:>5} {a[2]:>8}" f" {a[3]:>6} {a[4]:>6} {a[5]:>8.4f} | {b[2]:>6} {b[5]:.4f}") ``` ``` keep policy steps calls tokens errors needed success | 2nd run -- uncompacted 97 74 21913 8 0 0.9583 | 21998 0.9583 6 summary 102 82 21281 29 5 0.9167 | 23749 0.9167 5 summary 98 77 19027 24 6 0.9583 | 21458 0.9583 4 summary 98 77 18086 24 6 0.7917 | 20420 0.7500 3 summary 156 149 33566 125 18 0.2917 | 34523 0.2500 6 selective 97 74 19023 15 3 0.9583 | 19233 1.0000 5 selective 97 74 18004 18 5 0.9583 | 18184 1.0000 4 selective 98 77 17475 24 6 0.9583 | 18846 1.0000 3 selective 98 77 16569 24 8 0.9583 | 17861 1.0000 ``` Once the state item is added, the baseline rises from 11978 to **21913 tokens**: the carried state holds more than the trace itself. This is where compaction belongs. The selective rows advance as expected. As `keep` goes from six to three, tokens fall from 19023 to **16569**, that is, by **0.2439** relative to the baseline. The cost shows in two columns: tool errors climb from 8 in the uncompacted row to 24, and the **number of tasks that need the dropped field** goes from 0 to **8**, that is, climbs to a third of the twenty-four tasks. Task success stays fixed at 0.9583 in the first run and is 1.0000 in the second. The dropped fields break the next step, but the agent gets the field back by calling the same tool again; the cost is paid not in steps but in errors. The summary rows start in the same direction and then break. At `keep` four the tokens, 18086, are a little above the selective ones, but task success is **0.7917**, and 0.7500 in the second run. At `keep` three the table turns around: tokens are **33566**, that is, **1.53 times** the uncompacted baseline; steps climb from 97 to 156, tool calls from 74 to 149, tool errors from 8 to **125**, and task success falls to 0.2917. Here compaction is not a saving but a multiplier: a dropped field leads to an error, the error to a new step, and the new step to a context that is carried again. ## Same Budget, Different Content The most telling place in the table is the `keep` four rows. The two policies carry **the same number of fields**; the token difference is 611, less than four percent. The task success difference, however, is 0.7917 against 0.9583, and in the second run 0.7500 against 1.0000. With the largest difference between runs at 0.0417, this gap is four and six times the band, that is, **measured**. What decides is not the **size** of the context but **which fields remain**. This is the memory-side counterpart of the finding measured in the Building Applications with Model APIs course: there, in an editing task, the expected acceptance was 0.6921 when all files were put into the context and **0.8233** when only the relevant files were selected. The selection there was over files, the selection here is over the agent's own observations; the result points the same way. Irrelevant context is not only expensive, it also takes the place of the relevant. ## Summary - The carried state is paid again at every step: the trace tokens are 11978, and with the state the baseline rises to 21913 tokens. - Selective recall cuts tokens to 16569 at `keep` three, by 0.2439 relative to the baseline, and does not change task success. - In the same sweep the number of tasks that need the dropped field rises from 0 to 8 and tool errors from 8 to 24: the cost of the tokens gained is written here. - On the same `keep` budget summarization gives 0.7917 and selective recall 0.9583; the gap is four and six times the run-to-run difference. - At `keep` three summarization raises tokens to 33566, 1.53 times the baseline; at this point compaction is not a saving but a multiplier. ## Next Step In these two lessons memory was treated from two directions: what to write between tasks, what to carry within a task. In both, the records were assumed to be **correct**. Yet a meter is read again, a tariff changes, a document is renewed; the record in long-term memory stays where it is and **silently turns wrong**. Dropping a field does not solve this, because the problem is not the narrowness of the place but the age of the record. The next lesson sweeps the expiry threshold and puts two numbers side by side: as the threshold tightens, stale records drop, but the share of records that are discarded although they are still valid grows.