Short-Term Memory

The previous lesson counted that a wrong record, after it enters the state, is carried for an average of three to four steps. What carries it is called short-term memory: the fields the agent holds during that task and the trace of the steps that produced those fields. As the loop lengthens this load grows, and at some point it hits a limit.

The limit was named in the Introduction to AI Engineering course; that the conversation state is resent with every request, and what this holds per turn, was measured in the Building Applications with Model APIs course. The same limit was counted in the retrieval pipeline too: in the Embeddings and Retrieval-Augmented Generation course an average of 11.3 chunks fit in the window. None of the three routines is repeated. The only thing this lesson asks is specific to the loop: how many tokens are carried per step, how many steps fit in the window, and which field drops from the state when the window fills.

• TM44. The code is a simulator; the environment, the tasks, and the tokenizer are fictional, seed 20260218. The window values are fictional too; what is measured is not the value of the threshold but the effect of narrowing.
• TM45. The load carried has two parts: the state load, the tokens of the fields the agent holds; and the step trace load, the room the steps so far take up in the loop. The second is the shared loop’s own count and grows by 22 tokens per step; in the text it is called briefly the trace load.
• TM46. The window is placed on the sum of the state and the trace. If the total exceeds the threshold, the field that entered the state first drops, and the dropping continues; the trace load cannot be dropped.
• TM47. A dropped field is silent. The agent does not see that a field is gone; the loss shows up only when the tool that asks for that field returns missing field.
• TM48. Fitting steps is the number of calls made until the first drop happens; in a task with no drop, all calls are counted.
• TM49. The baseline is the unlimited window — the loop in which the limit never binds. The cost is written as total tokens, steps, tool errors, and dropped fields.
• TM50. The task set is the course’s shared twenty-four tasks; the measure is again the answer in the final state.

The Shared Loop

The environment, the five basic tools, and the agent loop are the same throughout the course. The tool and field names in the code are the read_meter, consumption, tariff, search_docs, and write_record tools of the previous lessons; they are counted in tokens here, so they stay Turkish.

SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL environment: records of the measurement network
# wire words stay Turkish: they are counted in tokens (same rule as the MCP course)
# onceki/simdiki/tip/bolge = previous/current/type/region; mesken/isyeri = residential/commercial
REGION = ["dogu", "bati", "kuzey", "guney", "merkez"]
TYPE = ["mesken", "isyeri"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"bolge": REGION[int(r() * 5)], "tip": TYPE[int(r() * 2)],
                                    "onceki": previous, "simdiki": previous + int(r() * 70)}
TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16
COMMERCIAL_FACTOR = 2
DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu",
             "TR02": "isyeri abonesinde birim ucret iki katidir",
             "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir",
             "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider",
             "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"}

CALLS = [0]                              # tool call counter


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    total += remaining * ABOVE
    return total * (COMMERCIAL_FACTOR if type_ == "isyeri" else 1)


def t_read_meter(abone=None, **_):
    a = SUBSCRIBERS.get(abone)
    if a is None:
        return {"hata": "abone yok"}
    return {"onceki": a["onceki"], "simdiki": a["simdiki"], "tip": a["tip"],
            "bolge": a["bolge"]}


def t_consumption(onceki=None, simdiki=None, **_):
    if onceki is None or simdiki is None:
        return {"hata": "eksik alan"}
    f = simdiki - onceki
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, tip="mesken", **_):
    if m3 is None:
        return {"hata": "eksik alan"}
    return {"tutar": _charge(m3, tip)}


def t_document(anahtar=None, **_):
    if not anahtar:
        return {"hata": "eksik alan"}
    q = set(tokenize(anahtar))
    best, code = -1, None
    for k, m in DOCUMENTS.items():
        o = len(q & set(tokenize(m)))
        if o > best:
            best, code = o, k
    return {"kod": code, "metin": DOCUMENTS[code]} if best > 0 else {"hata": "bulunamadi"}


def t_record(abone=None, sutun=None, deger=None, **_):
    if not (abone and sutun and deger is not None):
        return {"hata": "eksik alan"}
    return {"yazildi": True, "abone": abone, "sutun": sutun, "deger": deger}


TOOLS = {
    "sayac_oku":  {"function": t_read_meter, "fields": ("abone",), "side_effect": False},
    "tuketim":    {"function": t_consumption, "fields": ("onceki", "simdiki"),
                      "side_effect": False},
    "tarife":     {"function": t_tariff, "fields": ("m3", "tip"), "side_effect": False},
    "belge_ara":  {"function": t_document, "fields": ("anahtar",), "side_effect": False},
    "kayit_yaz":  {"function": t_record, "fields": ("abone", "sutun", "deger"),
                      "side_effect": True},
}
TOOL_NAMES = list(TOOLS)


def call_tool(name, state):
    """Fields outside the schema are filtered out; a missing field is the tool's own error."""
    CALLS[0] += 1
    if name not in TOOLS:
        return {"hata": "arac yok"}
    kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state}
    return TOOLS[name]["function"](**kw)


# ---- FICTIONAL tasks: each one is solved by a known tool sequence
TASKS = []
for i in range(24):
    r = rng(SEED + 9973 * i)
    sub = f"A{1 + int(r() * 60):03d}"
    t = pick(r(), [0.30, 0.22, 0.20, 0.28])
    if t == 0:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem bedeli nedir",
                      "sequence": ("sayac_oku", "tuketim", "tarife")})
    elif t == 1:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem tuketimi kac metrekup",
                      "sequence": ("sayac_oku", "tuketim")})
    elif t == 2:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub,
                      "request": "sayac donusunde endeks farki nasil hesaplanir",
                      "sequence": ("belge_ara",)})
    else:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub,
                      "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz",
                      "sequence": ("sayac_oku", "tuketim", "tarife", "kayit_yaz")})


def correct_answer(g):
    a = SUBSCRIBERS[g["subscriber"]]
    f = a["simdiki"] - a["onceki"]
    m3 = f + 100000 if f < 0 else f
    if g["kind"] == "consumption":
        return m3
    if g["kind"] == "document":
        return "SY01"
    return _charge(m3, a["tip"])


# ---- FICTIONAL agent: at each step it picks a tool or finishes
def selection(d):
    return max(0.34, 0.80 - 0.09 * (d - 1))       # share of picking the right tool, falls with depth


def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False):
    """FICTIONAL agent loop. No real model is called; tool selection
    comes from a single draw that depends on the task's sequence and the depth."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"abone": g["subscriber"], "anahtar": g["request"], "sutun": "bedel"}
    trace, step, tok, errors = [], 0, 0, 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace)
        target = seq[k] if k < len(seq) else None
        if target is None:                         # the job is done , the agent should stop
            if r() < 0.88:
                return {"done": True, "steps": step, "trace": trace, "tokens": tok,
                        "errors": errors, "answer": state.get("sonuc"),
                        "approvals": state.get("onay", 0)}
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if TOOLS[choice]["side_effect"] and approval:
            state["onay"] = state.get("onay", 0) + 1
        if choice == "kayit_yaz":
            state["deger"] = state.get("sonuc")
        s = call_tool(choice, state)
        trace.append((choice, "hata" in s))
        tok += 30
        if "hata" in s:
            errors += 1
            if self_critique:
                tok += 40
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "hata"})
        if choice == "tuketim":
            state["sonuc"] = s["m3"]
        if choice == "tarife":
            state["sonuc"] = s["tutar"]
        if choice == "belge_ara":
            state["sonuc"] = s["kod"]
        if choice == target:
            k += 1
    return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors,
            "answer": state.get("sonuc"), "approvals": state.get("onay", 0)}


print("shared environment:", len(SUBSCRIBERS), "subscribers ,", len(DOCUMENTS), "documents ,",
      len(TOOLS), "tools ,", len(TASKS), "tasks")

shared environment: 60 subscribers , 5 documents , 5 tools , 24 tasks

Tokens Carried per Step

The measure of short-term memory is not a single number but a curve: the load carried at each step is larger than at the previous one. The first block prints this step by step in a single task; the second spreads the same growth over twenty-four tasks at five window values.

# ---- Short-term memory: state + trace, which must fit into a window
MEMORY = {"window": 10 ** 9, "calls": 0, "tok": 0, "dropped": {},
          "fits": -1, "profile": None}
_execute = call_tool


def state_tokens(d):
    return sum(len(tokenize(f"{k} : {v}")) for k, v in d.items())


def call_tool(name, state):
    """If the window is exceeded, the field that entered the state first drops; the trace load cannot be dropped."""
    b = MEMORY
    b["calls"] += 1
    trace_load = 60 + 22 * (b["calls"] - 1)
    while state_tokens(state) + trace_load > b["window"] and len(state) > 1:
        k = next(iter(state))
        b["dropped"][k] = b["dropped"].get(k, 0) + 1
        if b["fits"] < 0:
            b["fits"] = b["calls"] - 1
        del state[k]
    d = state_tokens(state)
    b["tok"] += d + trace_load
    if b["profile"] is not None:
        b["profile"].append((b["calls"], name, len(state), d, trace_load))
    return _execute(name, state)


def run(window, seed=SEED):
    MEMORY.update(window=window, tok=0, dropped={})
    correct = steps = errors = calls = fits = 0
    for g in TASKS:
        MEMORY.update(calls=0, fits=-1)
        s = agent(g, seed=seed)
        correct += s["answer"] == correct_answer(g)
        steps, errors = steps + s["steps"], errors + s["errors"]
        calls += MEMORY["calls"]
        fits += MEMORY["calls"] if MEMORY["fits"] < 0 else MEMORY["fits"]
    return (correct, errors, steps, MEMORY["tok"], MEMORY["tok"] / calls, fits / 24,
            sum(MEMORY["dropped"].values()), MEMORY["dropped"])


sample = next(g for g in TASKS if g["kind"] == "record")
MEMORY.update(window=10 ** 9, calls=0, tok=0, dropped={}, fits=-1, profile=[])
agent(sample, seed=SEED)
print(f"single task ({sample['kind']}) , unlimited window: tokens carried per step")
print("step  tool         fields  state  trace load  total")
for c, name, n, d, y in MEMORY["profile"]:
    print(f"{c:>4}  {name:<12} {n:>6}  {d:>5}  {y:>10}  {d + y:>6}")
MEMORY["profile"] = None

print("\nwindow    correct A  correct B  steps  total tokens  tokens/call  fits  errors A"
      "  errors B  dropped")
for w in (10 ** 9, 260, 200, 150, 120):
    a, b = run(w), run(w, SEED + 1)
    print(f"{'unlimited' if w > 10 ** 6 else w:<9} {a[0] / 24:>9.4f}  {b[0] / 24:>9.4f}"
          f"  {a[2]:>5}  {a[3]:>12}  {a[4]:>11.1f}  {a[5]:>4.2f}  {a[1]:>8}  {b[1]:>8}"
          f"  {a[6]:>7}")
most = sorted(run(120)[7].items(), key=lambda x: (-x[1], x[0]))
print("\nwindow 120 , most dropped fields:", most[:6])

single task (record) , unlimited window: tokens carried per step
step  tool         fields  state  trace load  total
   1  sayac_oku         3     19          60      79
   2  belge_ara         7     34          82     116
   3  belge_ara        10     54         104     158
   4  belge_ara        10     54         126     180
   5  tuketim          10     54         148     202
   6  tarife           11     57         170     227
   7  kayit_yaz        13     63         192     255

window    correct A  correct B  steps  total tokens  tokens/call  fits  errors A  errors B  dropped
unlimited    0.9583     0.9583     97          9316        125.9  3.08         8         7        0
260          0.9583     0.9583     97          9297        125.6  3.04         9         7        3
200          0.8750     0.9167     98          9681        125.7  2.75        15        10       34
150          0.8333     0.7500    100          9926        124.1  2.42        28        35       40
120          0.7083     0.6250    112         12026        126.6  1.83        50        58       85

window 120 , most dropped fields: [('abone', 16), ('anahtar', 16), ('sutun', 15), ('onceki', 10), ('simdiki', 10), ('bolge', 5)]

The Field That Drops When the Window Fills

The first output says where the growth comes from. The state load is 19 tokens at the first step and 63 at the seventh; the number of fields rises from three to thirteen. The trace load goes from 60 to 192 and at the seventh step is three times the state. The total rises from 79 to 255. Most of the growth comes not from the information the agent has collected, but from how many steps it has taken. This distinction shapes the rest of the lesson, because when the window narrows the only part that can be dropped is the state.

The second table binds the limit. With an unlimited window 0.9583 of the twenty-four tasks are correct, a total of 9316 tokens is carried, no field drops, and tool errors are 8. When the window goes down to 260, three fields drop and performance does not move. At 200 the dropped fields rise to 34, tool errors to 15, and the correct answer falls to 0.8750. At 150, dropped 40, errors 28, correct answer 0.8333. At 120, dropped 85, errors 50, correct answer 0.7083. The fits column reads the narrowing directly: from 3.08 to 1.83, that is, in the narrowest window a field is gone before even two calls are completed on average.

Which field goes is in the last line and comes from the layout itself. The ones dropped most are the fields that entered the state first: abone (subscriber) 16, anahtar (query) 16, sutun (column) 15; that is, the identity information of the task. Then onceki (previous) and simdiki (current) drop ten times each. These explain the outcome: when abone is gone, sayac_oku (read meter) is called without a subscriber, and when onceki is gone, tuketim (consumption) returns a missing field. The rise of tool errors from 8 to 50 is the count of a single mechanism, and it moves in the same direction in both runs (7, 7, 10, 35, 58).

The performance column must be read more carefully. The largest two run-to-run differences are between 0.8333 and 0.7500, and the band is 0.0833. The drop between the unlimited window and 200 is also exactly 0.0833: not below the band but not above it either, so it does not count as a finding; the mechanism is visible in the dropped fields and tool errors columns, and its effect on task performance has not been measured. The 0.1250 at 150 and the 0.2500 at 120 are above the band and are measured.

The steps and total tokens columns are the most counterintuitive result of the lesson. Narrowing the window does not lower the tokens carried: the load per call goes from 125.9 to 126.6, that is, it does not change at all. The total, however, rises from 9316 to 12026, by 29%, because the dropped fields produce tool errors and the loop takes 112 steps instead of 97 to finish the same work. The constraint itself is not a saving mechanism: the part it shrinks is the state load, and that is already the smaller part of the load; the part that grows, the trace load, is not affected by the constraint at all.

Summary

• The load carried per step has two parts: while the state load rises from 19 to 63 tokens in a single task, the trace load rises from 60 to 192 and at the seventh step is three times the state.
• With an unlimited window the correct answer is 0.9583, the total is 9316 tokens, dropped fields are 0, and tool errors are 8.
• As the window narrows, the dropped fields rise from 0 to 85, tool errors from 8 to 50, and the fitting calls fall from 3.08 to 1.83.
• The dropped fields are the identity fields that entered the state first (abone 16, anahtar 16, sutun 15); the drop is silent and shows up only in the error of the tool that asks for that field.
• Narrowing the window does not lower the tokens carried: while the load per call stays constant from 125.9 to 126.6, the total rises from 9316 to 12026 because steps rise from 97 to 112.
• The measurement band is 0.0833; the 0.0833 drop at the window of 200 does not count as a finding, and the 0.1250 and 0.2500 at 150 and 120 are measured.

Next Step

In this lesson memory lived only as long as a single task. The window filled, a field dropped, a tool gave an error, and the loop ended; when it ended, nothing was left behind. The next task has to learn the same subscriber, the same document code, and the same tariff band from scratch. What short-term memory holds is the working set of the task, and that set is thrown away when the task ends. The next lesson asks about what is lasting: which information to write outside the loop, how many records the decision to write accumulates, and how many steps those records save in later tasks.