Memory Decay

The previous two lessons treated memory from two directions: what to write between tasks, what to carry within a task. In both, the records were assumed to be correct. Yet the measurement network moves: a meter is read again in the new period, the index difference turns into another number, the charge changes. The record in memory stays where it is and turns wrong without giving any error.

The remedy for this is not dropping a field; the problem is not the narrowness of the place but the age of the record. Memory expiry puts an age limit on a record: a record that exceeds the limit is dropped. This lesson sweeps that limit and prints two numbers side by side. As the threshold tightens, the share of stale records falls, but the share of dropped records that are still valid grows; the two move in opposite directions with the same setting.

• TM74. The code is a simulator; no real memory service or tool server is called. The environment, the tools, and the tasks are fictional, seed 20260218.
• TM75. The environment moves: once every six tasks, twelve subscribers get a new period reading (the previous index is replaced by the current one). The document vocabulary does not change.
• TM76. Memory is written with the selective policy: the observations that enter the plan are merged into a single record, the record is keyed by the subscriber code (document records by the question text), and it carries the task order in which it was written.
• TM77. The expiry threshold is the greatest age a record may carry; a record whose age is greater than the threshold is dropped at the start of every task.
• TM78. A stale record is a record whose fields do not coincide with the environment’s value at that moment; the stale share is counted per read.
• TM79. A lost record is a record that the threshold dropped but that was still valid at the moment it was dropped; the lost share is counted per dropped record.
• TM80. The stream is 72 tasks, the step budget is 8; the baseline is memory without a threshold.

The Record Stays, the Value Changes

The block below sets up the shared definition and shows staleness on a single subscriber.

# FICTIONAL environment and FICTIONAL agent. No real model endpoint, tool server, or memory
# service is called. The subscriber records, documents, and tasks are fictional, seed 20260218.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL environment: records of the measurement network
REGION = ["east", "west", "north", "south", "central"]
TYPE = ["residential", "commercial"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)],
                                    "previous": previous, "current": previous + int(r() * 70)}
TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16
COMMERCIAL_FACTOR = 2
DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu",
             "TR02": "isyeri abonesinde birim ucret iki katidir",
             "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir",
             "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider",
             "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"}

CALLS = [0]                              # tool call counter


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    total += remaining * ABOVE
    return total * (COMMERCIAL_FACTOR if type_ == "commercial" else 1)


def t_read_meter(subscriber=None, **_):
    a = SUBSCRIBERS.get(subscriber)
    if a is None:
        return {"error": "no such subscriber"}
    return {"previous": a["previous"], "current": a["current"], "type": a["type"],
            "region": a["region"]}


def t_consumption(previous=None, current=None, **_):
    if previous is None or current is None:
        return {"error": "missing field"}
    f = current - previous
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, type="residential", **_):
    if m3 is None:
        return {"error": "missing field"}
    return {"amount": _charge(m3, type)}


def t_document(query=None, **_):
    if not query:
        return {"error": "missing field"}
    q = set(tokenize(query))
    best, code = -1, None
    for k, m in DOCUMENTS.items():
        o = len(q & set(tokenize(m)))
        if o > best:
            best, code = o, k
    return {"code": code, "text": DOCUMENTS[code]} if best > 0 else {"error": "not found"}


def t_record(subscriber=None, column=None, value=None, **_):
    if not (subscriber and column and value is not None):
        return {"error": "missing field"}
    return {"written": True, "subscriber": subscriber, "column": column, "value": value}


TOOLS = {
    "read_meter":    {"function": t_read_meter, "fields": ("subscriber",), "side_effect": False},
    "consumption":   {"function": t_consumption, "fields": ("previous", "current"),
                      "side_effect": False},
    "tariff":        {"function": t_tariff, "fields": ("m3", "type"), "side_effect": False},
    "search_docs":   {"function": t_document, "fields": ("query",), "side_effect": False},
    "write_record":  {"function": t_record, "fields": ("subscriber", "column", "value"),
                      "side_effect": True},
}
TOOL_NAMES = list(TOOLS)


def call_tool(name, state):
    """Fields outside the schema are filtered out; a missing field is the tool's own error."""
    CALLS[0] += 1
    if name not in TOOLS:
        return {"error": "no such tool"}
    kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state}
    return TOOLS[name]["function"](**kw)


# ---- FICTIONAL tasks: each one is solved by a known tool sequence
TASKS = []
for i in range(24):
    r = rng(SEED + 9973 * i)
    sub = f"A{1 + int(r() * 60):03d}"
    t = pick(r(), [0.30, 0.22, 0.20, 0.28])
    if t == 0:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem bedeli nedir",
                      "sequence": ("read_meter", "consumption", "tariff")})
    elif t == 1:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem tuketimi kac metrekup",
                      "sequence": ("read_meter", "consumption")})
    elif t == 2:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub,
                      "request": "sayac donusunde endeks farki nasil hesaplanir",
                      "sequence": ("search_docs",)})
    else:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub,
                      "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz",
                      "sequence": ("read_meter", "consumption", "tariff", "write_record")})


def correct_answer(g):
    a = SUBSCRIBERS[g["subscriber"]]
    f = a["current"] - a["previous"]
    m3 = f + 100000 if f < 0 else f
    if g["kind"] == "consumption":
        return m3
    if g["kind"] == "document":
        return "SY01"
    return _charge(m3, a["type"])


# ---- FICTIONAL agent: at each step it picks a tool or finishes
def selection(d):
    return max(0.34, 0.80 - 0.09 * (d - 1))       # share of picking the right tool, falls with depth


def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False):
    """FICTIONAL agent loop. No real model is called; tool selection
    comes from a single draw that depends on the task's sequence and the depth."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"}
    trace, step, tok, errors = [], 0, 0, 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace)
        target = seq[k] if k < len(seq) else None
        if target is None:                         # the job is done , the agent should stop
            if r() < 0.88:
                return {"done": True, "steps": step, "trace": trace, "tokens": tok,
                        "errors": errors, "answer": state.get("result"),
                        "approvals": state.get("approvals", 0)}
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if TOOLS[choice]["side_effect"] and approval:
            state["approvals"] = state.get("approvals", 0) + 1
        if choice == "write_record":
            state["value"] = state.get("result")
        s = call_tool(choice, state)
        trace.append((choice, "error" in s))
        tok += 30
        if "error" in s:
            errors += 1
            if self_critique:
                tok += 40
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "error"})
        if choice == "consumption":
            state["result"] = s["m3"]
        if choice == "tariff":
            state["result"] = s["amount"]
        if choice == "search_docs":
            state["result"] = s["code"]
        if choice == target:
            k += 1
    return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors,
            "answer": state.get("result"), "approvals": state.get("approvals", 0)}


sub = "A026"
old = dict(SUBSCRIBERS[sub])
record = t_read_meter(subscriber=sub)
SUBSCRIBERS[sub]["previous"] = SUBSCRIBERS[sub]["current"]
SUBSCRIBERS[sub]["current"] += 31                    # new period reading
print("record in memory    :", record["previous"], record["current"], "-> consumption",
      t_consumption(previous=record["previous"], current=record["current"])["m3"])
print("value in environment:", SUBSCRIBERS[sub]["previous"], SUBSCRIBERS[sub]["current"], "-> consumption",
      t_consumption(previous=SUBSCRIBERS[sub]["previous"],
                    current=SUBSCRIBERS[sub]["current"])["m3"])
SUBSCRIBERS[sub].update(old)

record in memory    : 88422 88477 -> consumption 55
value in environment: 88477 88508 -> consumption 31

The record did not change, the environment did. The index pair that comes back from memory gives 55 cubic meters, the meter’s value at that moment is 31. The agent gets no error, the tool returns no error, the loop gets shorter, and the answer is wrong: a stale record cannot be told apart from a correct one.

Tightening the Threshold

The block below sets up the stream, the period renewals, and the threshold sweep.

# ---- Expiry: age and validity of a record
PLAN = {"charge": (("read_meter", "consumption", "tariff"),
                   "{sub} abonesinin bu donem bedeli nedir"),
        "consumption": (("read_meter", "consumption"),
                        "{sub} abonesinin bu donem tuketimi kac metrekup"),
        "document": (("search_docs",), "sayac donusunde endeks farki nasil hesaplanir"),
        "record": (("read_meter", "consumption", "tariff", "write_record"),
                   "{sub} abonesinin bedelini hesapla ve kayda yaz")}
OUTPUTS = {"read_meter": ("previous", "current", "type", "region"), "consumption": ("m3",),
           "tariff": ("amount",), "search_docs": ("code",)}
RESULT_KEY = {"consumption": "m3", "tariff": "amount", "search_docs": "code"}
INITIAL = {k: dict(v) for k, v in SUBSCRIBERS.items()}


def stream(n=72):
    """Task stream of n questions asked to the subscribers of the first set."""
    k = []
    for i in range(n):
        r = rng(SEED + 7919 * (i + 101))
        sub = TASKS[int(r() * 24)]["subscriber"]
        t = list(PLAN)[pick(r(), [0.30, 0.22, 0.20, 0.28])]
        k.append({"name": f"H{i + 1:02d}", "kind": t, "subscriber": sub,
                  "sequence": PLAN[t][0], "request": PLAN[t][1].format(sub=sub)})
    return k


def new_period(i):
    """Once every six tasks, twelve subscribers get a new period reading."""
    r = rng(SEED + 61 * i)
    for _ in range(12):
        a = SUBSCRIBERS[f"A{1 + int(r() * 60):03d}"]
        a["previous"] = a["current"]
        a["current"] = a["previous"] + int(r() * 70)


def is_valid(rec):
    """Do the record's fields still coincide with the environment's values at this moment."""
    if rec["subscriber"] is None:
        return True
    a = SUBSCRIBERS[rec["subscriber"]]
    f = a["current"] - a["previous"]
    m3 = f + 100000 if f < 0 else f
    now = dict(a, m3=m3, amount=_charge(m3, a["type"]))
    return all(now.get(x) == v for x, v in rec["fields"].items())


def agent_e(g, rec, seed=SEED, max_steps=8):
    """The shared loop; the initial state comes from the record in memory."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"}
    state.update(rec["fields"] if rec else {})
    step, k, seq, on_path = 0, 0, g["sequence"], {}
    while k < len(seq) and seq[k] in OUTPUTS and all(x in state for x in OUTPUTS[seq[k]]):
        state["result"] = state.get(RESULT_KEY.get(seq[k]), state.get("result"))
        k += 1
    if k == len(seq):
        return {"steps": 0, "answer": state.get("result"), "on_path": {}}
    while step < max_steps:
        step += 1
        target = seq[k] if k < len(seq) else None
        if target is None:
            if r() < 0.88:
                break
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if choice == "write_record":
            state["value"] = state.get("result")
        s = call_tool(choice, state)
        if "error" in s:
            continue
        state.update({x: v for x, v in s.items() if x != "error"})
        if choice in RESULT_KEY:
            state["result"] = s[RESULT_KEY[choice]]
        if choice == target:
            on_path.update({x: s[x] for x in OUTPUTS.get(choice, ()) if x in s})
            k += 1
    return {"steps": step, "answer": state.get("result"), "on_path": on_path}


def run(threshold, seed=SEED, n=72):
    """Expiry threshold: a record whose age is greater than the threshold is dropped."""
    for x, v in INITIAL.items():
        SUBSCRIBERS[x].update(v)
    CALLS[0] = 0
    memory = {}
    s = dict.fromkeys(("reads", "stale", "dropped", "lost", "steps", "correct"), 0)
    for i, g in enumerate(stream(n)):
        if i and i % 6 == 0:
            new_period(i)
        for key in [x for x, k in memory.items() if i - k["time"] > threshold]:
            s["dropped"] += 1
            s["lost"] += is_valid(memory.pop(key))
        key = g["request"] if g["kind"] == "document" else g["subscriber"]
        rec = memory.get(key)
        if rec:
            s["reads"] += 1
            s["stale"] += not is_valid(rec)
        c = agent_e(g, rec, seed)
        s["steps"] += c["steps"]
        s["correct"] += c["answer"] == correct_answer(g)
        if c["on_path"]:
            memory[key] = {"fields": c["on_path"], "time": i,
                           "subscriber": None if g["kind"] == "document" else g["subscriber"]}
    return (s["reads"], s["stale"] / max(s["reads"], 1), s["dropped"],
            s["lost"] / max(s["dropped"], 1), s["steps"], CALLS[0], s["correct"] / n)


print("threshold  reads   stale   dropped    lost   steps   calls   success | 2nd run")
for e in (10 ** 6, 32, 24, 16, 8, 4, 1):
    a, b = run(e), run(e, SEED + 1)
    print(f"{e if e < 100 else '--':>9} {a[0]:>6} {a[1]:>7.4f} {a[2]:>9} {a[3]:>7.4f}"
          f" {a[4]:>7} {a[5]:>7} {a[6]:>9.4f} | {b[6]:.4f}")

threshold  reads   stale   dropped    lost   steps   calls   success | 2nd run
       --     53  0.4528         0  0.0000     138     108    0.6389 | 0.5972
       32     42  0.3333        18  0.3333     176     139    0.7778 | 0.7361
       24     36  0.2222        23  0.4783     201     158    0.8194 | 0.7778
       16     31  0.0968        32  0.6250     220     171    0.8750 | 0.8611
        8     23  0.0870        41  0.7805     246     193    0.8889 | 0.8611
        4     13  0.0000        55  0.8909     283     221    0.9028 | 0.8750
        1      3  0.0000        67  0.9701     315     246    0.9306 | 0.8889

The row without a threshold is the most efficient and the most wrong state of memory: 53 reads, 138 steps, but 0.4528 of the reads are stale and task success is 0.6389; in more than a third of the tasks the user is told last period’s number.

As the threshold tightens, two columns move in opposite directions. The stale share falls from 0.4528 to 0.3333, to 0.2222, to 0.0968, and to 0.0000, and task success rises from 0.6389 to 0.9306. In the same rows the lost share climbs from 0.0000 to 0.6250 and to 0.9701: when the threshold falls to one, 65 of the 67 dropped records were still valid at the moment they were dropped. The cost is written in steps as well: 138 steps go to 315, and tool calls from 108 to 246.

Age Is a Proxy for Validity

The lost share column alone explains what expiry is. The threshold does not ask whether a record is valid; it asks how many tasks ago it was written. Age is a proxy for validity, and in this stream it is a weak one: only twelve subscribers are renewed every six tasks, so staleness is a probability, not a certainty determined by age.

The two runs say where to stop. The row without a threshold gives 0.6389 and 0.5972 in the two runs; the band is 0.0417. The gain of tightening the threshold to eight is 0.2500, six times the band, and is measured. The gain of tightening from eight to one, however, lies between 0.8889 and 0.9306, exactly the band itself; in the second run the same gain falls to 0.0278. This difference cannot be declared a finding: lowering the threshold below eight makes one pay 69 more steps and 53 more tool calls and gives in return a gain that is unmeasured. Whoever wants to measure it should enlarge the number of tasks or sweep the renewal frequency as a setting.

The stale and lost shares come out exactly the same in the two runs: these two numbers depend not on the tool-selection draw but on the relation between the renewal calendar and the threshold. The expiry threshold cannot be chosen independently of how often the data changes.

Summary

• A record in memory turns wrong without giving an error and cannot be told apart from a correct record.
• In memory without a threshold 0.4528 of the reads are stale and task success is 0.6389; when the threshold falls to one, the stale share is 0.0000 and task success is 0.9306.
• In the same sweep the share of dropped records that are still valid rises from 0.0000 to 0.9701 and steps climb from 138 to 315.
• The threshold asks for the age of a record, not its validity; age is a proxy, and on this renewal calendar the price of zeroing the stale share is throwing away nearly all of the valid records.
• The task success gained by tightening the threshold below eight is as large as the run-to-run difference itself and counts as unmeasured; going from eight to one makes one pay 69 more steps.

Next Step

This topic equipped the agent: the tool definition, execution, code execution, the file and terminal surface, outside information, short-term and long-term memory, compaction, and expiry. Together with the previous topic, an agent was built in sixteen lessons, where it would stop was written down, checkpoints were placed, and tools were put in its hands. In every lesson the number and the cost of one mechanism were printed, but each was read with a measure that looked only at the mechanism of that lesson. The question that looks at the whole was never asked: how do we know this agent works. Whether a run that gives the correct answer followed the correct path, whether a defect of a tool that is not visible when it is tested alone appears inside the flow, and which record can speak about a loop that runs in production were not addressed. The next topic opens with this question.