--- title: 'Memory Decay' source: 'https://academia.sh/en/courses/ai-agents/memory-decay' course: 'AI Agents' language: en updated: '2026-10-21T05:02:27+00:00' license: 'CC BY-SA 4.0' objectives: - 'Sweeps the expiry threshold and measures the stale share, the lost share, the steps, and task success in the same stream.' - 'Shows with the share of valid records dropped that age is only a proxy for validity.' - 'Determines with the run-to-run difference the point where the gain of tightening the threshold falls below the measurement band.' --- # Memory Decay Sweeping the expiry threshold over seven settings: in memory without a threshold 0.4528 of the reads are stale and task success is 0.6389. As the threshold tightens, the stale share falls to 0.0000 and task success rises to 0.9306, but the share of dropped records that are still valid climbs from 0.0000 to 0.9701 and steps rise from 138 to 315. The task success gained by tightening the threshold below eight stays as large as the run-to-run difference itself, that is, it counts as unmeasured. The previous two lessons treated memory from two directions: what to write between tasks, what to carry within a task. In both, the records were assumed to be **correct**. Yet the measurement network moves: a meter is read again in the new period, the index difference turns into another number, the charge changes. The record in memory stays where it is and **turns wrong** without giving any error. The remedy for this is not dropping a field; the problem is not the narrowness of the place but the **age** of the record. **Memory expiry** puts an age limit on a record: a record that exceeds the limit is dropped. This lesson sweeps that limit and prints two numbers side by side. As the threshold tightens, the share of **stale records** falls, but the share of dropped records that are **still valid** grows; the two move in opposite directions with the same setting. - **TM74.** The code is a **simulator**; no real memory service or tool server is called. The environment, the tools, and the tasks are **fictional**, seed **20260218**. - **TM75. The environment moves:** once every six tasks, twelve subscribers get **a new period reading** (the previous index is replaced by the current one). The document vocabulary does not change. - **TM76.** Memory is written with the selective policy: the observations that enter the plan are merged into a single record, the record is keyed by the **subscriber code** (document records by the question text), and it carries **the task order in which it was written**. - **TM77. The expiry threshold** is the greatest age a record may carry; a record whose age is greater than the threshold is dropped at the start of every task. - **TM78. A stale record** is a record whose fields do not coincide with the environment's value at that moment; the **stale share** is counted per read. - **TM79. A lost record** is a record that the threshold dropped but that was **still valid** at the moment it was dropped; the **lost share** is counted per dropped record. - **TM80.** The stream is 72 tasks, the step budget is 8; the baseline is **memory without a threshold**. ## The Record Stays, the Value Changes The block below sets up the shared definition and shows staleness on a single subscriber. ```python # FICTIONAL environment and FICTIONAL agent. No real model endpoint, tool server, or memory # service is called. The subscriber records, documents, and tasks are fictional, seed 20260218. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 def tokenize(m): b = [] for s in str(m).lower().replace(",", " , ").split(): while len(s) > 5: b.append(s[:5]) s = s[5:] b.append(s) return b def trace_id(m): h = 2166136261 for c in str(m): h = ((h ^ ord(c)) * 16777619) & M32 return h # ---- FICTIONAL environment: records of the measurement network REGION = ["east", "west", "north", "south", "central"] TYPE = ["residential", "commercial"] SUBSCRIBERS = {} for i in range(60): r = rng(SEED + 31 * i) previous = 10000 + int(r() * 80000) SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)], "previous": previous, "current": previous + int(r() * 70)} TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16 COMMERCIAL_FACTOR = 2 DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu", "TR02": "isyeri abonesinde birim ucret iki katidir", "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir", "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider", "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"} CALLS = [0] # tool call counter def _charge(m3, type_): remaining, lower, total = m3, 0, 0 for upper, unit in TIERS: band = min(remaining, upper - lower) if band <= 0: break total += band * unit remaining -= band lower = upper total += remaining * ABOVE return total * (COMMERCIAL_FACTOR if type_ == "commercial" else 1) def t_read_meter(subscriber=None, **_): a = SUBSCRIBERS.get(subscriber) if a is None: return {"error": "no such subscriber"} return {"previous": a["previous"], "current": a["current"], "type": a["type"], "region": a["region"]} def t_consumption(previous=None, current=None, **_): if previous is None or current is None: return {"error": "missing field"} f = current - previous return {"m3": f + 100000 if f < 0 else f} def t_tariff(m3=None, type="residential", **_): if m3 is None: return {"error": "missing field"} return {"amount": _charge(m3, type)} def t_document(query=None, **_): if not query: return {"error": "missing field"} q = set(tokenize(query)) best, code = -1, None for k, m in DOCUMENTS.items(): o = len(q & set(tokenize(m))) if o > best: best, code = o, k return {"code": code, "text": DOCUMENTS[code]} if best > 0 else {"error": "not found"} def t_record(subscriber=None, column=None, value=None, **_): if not (subscriber and column and value is not None): return {"error": "missing field"} return {"written": True, "subscriber": subscriber, "column": column, "value": value} TOOLS = { "read_meter": {"function": t_read_meter, "fields": ("subscriber",), "side_effect": False}, "consumption": {"function": t_consumption, "fields": ("previous", "current"), "side_effect": False}, "tariff": {"function": t_tariff, "fields": ("m3", "type"), "side_effect": False}, "search_docs": {"function": t_document, "fields": ("query",), "side_effect": False}, "write_record": {"function": t_record, "fields": ("subscriber", "column", "value"), "side_effect": True}, } TOOL_NAMES = list(TOOLS) def call_tool(name, state): """Fields outside the schema are filtered out; a missing field is the tool's own error.""" CALLS[0] += 1 if name not in TOOLS: return {"error": "no such tool"} kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state} return TOOLS[name]["function"](**kw) # ---- FICTIONAL tasks: each one is solved by a known tool sequence TASKS = [] for i in range(24): r = rng(SEED + 9973 * i) sub = f"A{1 + int(r() * 60):03d}" t = pick(r(), [0.30, 0.22, 0.20, 0.28]) if t == 0: TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub, "request": f"{sub} abonesinin bu donem bedeli nedir", "sequence": ("read_meter", "consumption", "tariff")}) elif t == 1: TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub, "request": f"{sub} abonesinin bu donem tuketimi kac metrekup", "sequence": ("read_meter", "consumption")}) elif t == 2: TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub, "request": "sayac donusunde endeks farki nasil hesaplanir", "sequence": ("search_docs",)}) else: TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub, "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz", "sequence": ("read_meter", "consumption", "tariff", "write_record")}) def correct_answer(g): a = SUBSCRIBERS[g["subscriber"]] f = a["current"] - a["previous"] m3 = f + 100000 if f < 0 else f if g["kind"] == "consumption": return m3 if g["kind"] == "document": return "SY01" return _charge(m3, a["type"]) # ---- FICTIONAL agent: at each step it picks a tool or finishes def selection(d): return max(0.34, 0.80 - 0.09 * (d - 1)) # share of picking the right tool, falls with depth def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False): """FICTIONAL agent loop. No real model is called; tool selection comes from a single draw that depends on the task's sequence and the depth.""" r = rng(seed ^ trace_id(g["name"])) state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"} trace, step, tok, errors = [], 0, 0, 0 seq, k = g["sequence"], 0 while step < max_steps: step += 1 tok += 60 + 22 * len(trace) target = seq[k] if k < len(seq) else None if target is None: # the job is done , the agent should stop if r() < 0.88: return {"done": True, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("result"), "approvals": state.get("approvals", 0)} choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] if TOOLS[choice]["side_effect"] and approval: state["approvals"] = state.get("approvals", 0) + 1 if choice == "write_record": state["value"] = state.get("result") s = call_tool(choice, state) trace.append((choice, "error" in s)) tok += 30 if "error" in s: errors += 1 if self_critique: tok += 40 continue state.update({kk: vv for kk, vv in s.items() if kk != "error"}) if choice == "consumption": state["result"] = s["m3"] if choice == "tariff": state["result"] = s["amount"] if choice == "search_docs": state["result"] = s["code"] if choice == target: k += 1 return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("result"), "approvals": state.get("approvals", 0)} sub = "A026" old = dict(SUBSCRIBERS[sub]) record = t_read_meter(subscriber=sub) SUBSCRIBERS[sub]["previous"] = SUBSCRIBERS[sub]["current"] SUBSCRIBERS[sub]["current"] += 31 # new period reading print("record in memory :", record["previous"], record["current"], "-> consumption", t_consumption(previous=record["previous"], current=record["current"])["m3"]) print("value in environment:", SUBSCRIBERS[sub]["previous"], SUBSCRIBERS[sub]["current"], "-> consumption", t_consumption(previous=SUBSCRIBERS[sub]["previous"], current=SUBSCRIBERS[sub]["current"])["m3"]) SUBSCRIBERS[sub].update(old) ``` ``` record in memory : 88422 88477 -> consumption 55 value in environment: 88477 88508 -> consumption 31 ``` The record did not change, the environment did. The index pair that comes back from memory gives 55 cubic meters, the meter's value at that moment is 31. The agent gets no error, the tool returns no error, the loop gets shorter, and the answer is wrong: a stale record **cannot be told apart** from a correct one. ## Tightening the Threshold The block below sets up the stream, the period renewals, and the threshold sweep. ```python # ---- Expiry: age and validity of a record PLAN = {"charge": (("read_meter", "consumption", "tariff"), "{sub} abonesinin bu donem bedeli nedir"), "consumption": (("read_meter", "consumption"), "{sub} abonesinin bu donem tuketimi kac metrekup"), "document": (("search_docs",), "sayac donusunde endeks farki nasil hesaplanir"), "record": (("read_meter", "consumption", "tariff", "write_record"), "{sub} abonesinin bedelini hesapla ve kayda yaz")} OUTPUTS = {"read_meter": ("previous", "current", "type", "region"), "consumption": ("m3",), "tariff": ("amount",), "search_docs": ("code",)} RESULT_KEY = {"consumption": "m3", "tariff": "amount", "search_docs": "code"} INITIAL = {k: dict(v) for k, v in SUBSCRIBERS.items()} def stream(n=72): """Task stream of n questions asked to the subscribers of the first set.""" k = [] for i in range(n): r = rng(SEED + 7919 * (i + 101)) sub = TASKS[int(r() * 24)]["subscriber"] t = list(PLAN)[pick(r(), [0.30, 0.22, 0.20, 0.28])] k.append({"name": f"H{i + 1:02d}", "kind": t, "subscriber": sub, "sequence": PLAN[t][0], "request": PLAN[t][1].format(sub=sub)}) return k def new_period(i): """Once every six tasks, twelve subscribers get a new period reading.""" r = rng(SEED + 61 * i) for _ in range(12): a = SUBSCRIBERS[f"A{1 + int(r() * 60):03d}"] a["previous"] = a["current"] a["current"] = a["previous"] + int(r() * 70) def is_valid(rec): """Do the record's fields still coincide with the environment's values at this moment.""" if rec["subscriber"] is None: return True a = SUBSCRIBERS[rec["subscriber"]] f = a["current"] - a["previous"] m3 = f + 100000 if f < 0 else f now = dict(a, m3=m3, amount=_charge(m3, a["type"])) return all(now.get(x) == v for x, v in rec["fields"].items()) def agent_e(g, rec, seed=SEED, max_steps=8): """The shared loop; the initial state comes from the record in memory.""" r = rng(seed ^ trace_id(g["name"])) state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"} state.update(rec["fields"] if rec else {}) step, k, seq, on_path = 0, 0, g["sequence"], {} while k < len(seq) and seq[k] in OUTPUTS and all(x in state for x in OUTPUTS[seq[k]]): state["result"] = state.get(RESULT_KEY.get(seq[k]), state.get("result")) k += 1 if k == len(seq): return {"steps": 0, "answer": state.get("result"), "on_path": {}} while step < max_steps: step += 1 target = seq[k] if k < len(seq) else None if target is None: if r() < 0.88: break choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] if choice == "write_record": state["value"] = state.get("result") s = call_tool(choice, state) if "error" in s: continue state.update({x: v for x, v in s.items() if x != "error"}) if choice in RESULT_KEY: state["result"] = s[RESULT_KEY[choice]] if choice == target: on_path.update({x: s[x] for x in OUTPUTS.get(choice, ()) if x in s}) k += 1 return {"steps": step, "answer": state.get("result"), "on_path": on_path} def run(threshold, seed=SEED, n=72): """Expiry threshold: a record whose age is greater than the threshold is dropped.""" for x, v in INITIAL.items(): SUBSCRIBERS[x].update(v) CALLS[0] = 0 memory = {} s = dict.fromkeys(("reads", "stale", "dropped", "lost", "steps", "correct"), 0) for i, g in enumerate(stream(n)): if i and i % 6 == 0: new_period(i) for key in [x for x, k in memory.items() if i - k["time"] > threshold]: s["dropped"] += 1 s["lost"] += is_valid(memory.pop(key)) key = g["request"] if g["kind"] == "document" else g["subscriber"] rec = memory.get(key) if rec: s["reads"] += 1 s["stale"] += not is_valid(rec) c = agent_e(g, rec, seed) s["steps"] += c["steps"] s["correct"] += c["answer"] == correct_answer(g) if c["on_path"]: memory[key] = {"fields": c["on_path"], "time": i, "subscriber": None if g["kind"] == "document" else g["subscriber"]} return (s["reads"], s["stale"] / max(s["reads"], 1), s["dropped"], s["lost"] / max(s["dropped"], 1), s["steps"], CALLS[0], s["correct"] / n) print("threshold reads stale dropped lost steps calls success | 2nd run") for e in (10 ** 6, 32, 24, 16, 8, 4, 1): a, b = run(e), run(e, SEED + 1) print(f"{e if e < 100 else '--':>9} {a[0]:>6} {a[1]:>7.4f} {a[2]:>9} {a[3]:>7.4f}" f" {a[4]:>7} {a[5]:>7} {a[6]:>9.4f} | {b[6]:.4f}") ``` ``` threshold reads stale dropped lost steps calls success | 2nd run -- 53 0.4528 0 0.0000 138 108 0.6389 | 0.5972 32 42 0.3333 18 0.3333 176 139 0.7778 | 0.7361 24 36 0.2222 23 0.4783 201 158 0.8194 | 0.7778 16 31 0.0968 32 0.6250 220 171 0.8750 | 0.8611 8 23 0.0870 41 0.7805 246 193 0.8889 | 0.8611 4 13 0.0000 55 0.8909 283 221 0.9028 | 0.8750 1 3 0.0000 67 0.9701 315 246 0.9306 | 0.8889 ``` The row without a threshold is the most efficient and the most wrong state of memory: 53 reads, 138 steps, but **0.4528 of the reads are stale** and task success is **0.6389**; in more than a third of the tasks the user is told last period's number. As the threshold tightens, two columns move in opposite directions. The stale share falls from 0.4528 to 0.3333, to 0.2222, to 0.0968, and to **0.0000**, and task success rises from 0.6389 to **0.9306**. In the same rows the lost share climbs from 0.0000 to 0.6250 and to **0.9701**: when the threshold falls to one, 65 of the 67 dropped records were **still valid** at the moment they were dropped. The cost is written in steps as well: 138 steps go to 315, and tool calls from 108 to 246. ## Age Is a Proxy for Validity The lost share column alone explains what expiry is. The threshold does not ask whether a record **is valid**; it asks **how many tasks ago it was written**. Age is a proxy for validity, and in this stream it is a weak one: only twelve subscribers are renewed every six tasks, so staleness is a probability, not a certainty determined by age. The two runs say where to stop. The row without a threshold gives 0.6389 and 0.5972 in the two runs; the band is **0.0417**. The gain of tightening the threshold to eight is 0.2500, six times the band, and is **measured**. The gain of tightening from eight to one, however, lies between 0.8889 and 0.9306, **exactly the band itself**; in the second run the same gain falls to 0.0278. This difference **cannot be declared** a finding: lowering the threshold below eight makes one pay 69 more steps and 53 more tool calls and gives in return a gain that is **unmeasured**. Whoever wants to measure it should enlarge the number of tasks or sweep the renewal frequency as a setting. The stale and lost shares come out exactly the same in the two runs: these two numbers depend not on the tool-selection draw but on the relation between the renewal calendar and the threshold. The expiry threshold cannot be chosen independently of how often the data changes. ## Summary - A record in memory turns wrong without giving an error and cannot be told apart from a correct record. - In memory without a threshold 0.4528 of the reads are stale and task success is 0.6389; when the threshold falls to one, the stale share is 0.0000 and task success is 0.9306. - In the same sweep the share of dropped records that are still valid rises from 0.0000 to 0.9701 and steps climb from 138 to 315. - The threshold asks for the age of a record, not its validity; age is a proxy, and on this renewal calendar the price of zeroing the stale share is throwing away nearly all of the valid records. - The task success gained by tightening the threshold below eight is as large as the run-to-run difference itself and counts as unmeasured; going from eight to one makes one pay 69 more steps. ## Next Step This topic equipped the agent: the tool definition, execution, code execution, the file and terminal surface, outside information, short-term and long-term memory, compaction, and expiry. Together with the previous topic, an agent was built in sixteen lessons, where it would stop was written down, checkpoints were placed, and tools were put in its hands. In every lesson the number and the cost of one mechanism were printed, but each was read with a measure that looked only at the mechanism of that lesson. The question that looks at the whole was never asked: **how do we know this agent works**. Whether a run that gives the correct answer followed the correct **path**, whether a defect of a tool that is not visible when it is tested alone appears inside the flow, and which record can speak about a loop that runs in production were not addressed. The next topic opens with this question.