Context Compression

The previous lesson treated memory between tasks: what was written determined the steps saved in later tasks. Within a task the problem reverses. The context is not erased, it grows: every observation is added to the state and carried again at every later step. The token count of the shared loop sees only half of this: it counts the step trace, it does not count the carried state itself.

This lesson first adds that missing item, then sweeps two policies of context compaction: summarization drops the oldest fields and puts a single summary entry in their place; selective recall keeps the fields that the remaining steps declare. What will be measured is two numbers side by side: as the compaction ratio rises, the carried tokens fall, but the number of tasks that need the dropped field grows. This lesson is an instance of the course’s second claim; at the tightest setting compaction raises tokens above the baseline.

• TM67. The code is a simulator; no real model endpoint or tool server is called. The environment, the tools, and the tasks are fictional, seed 20260218.
• TM68. The carried state is a token item: each field is paid 12 tokens for its name and format and its own token count for its value, and this cost is paid again at every step.
• TM69. The compaction ratio is given by the keep setting: at most keep fields are carried in the state and the excess is dropped. Compaction is applied at the end of every step.
• TM70. Summarization drops the oldest fields and puts a single summary entry in their place; the entry takes up the place of a field but carries no usable value.
• TM71. Selective recall keeps, on the same keep budget, the fields that the remaining steps of the plan declare; the newest fields fill the leftover room.
• TM72. When a tool asks for a field that was dropped earlier, a missing field error returns. A task in which this error is seen at least once counts as a task that needs the dropped field.
• TM73. The task set is 24, the step budget is 8, and the baseline is the loop without compaction.

The Cost of the Carried State

The block below sets up the shared definition and prints the baseline. The tool and field names in the code are the read_meter, consumption, tariff, search_docs, and write_record tools of the previous lessons; they are counted in tokens here, so they stay Turkish.

# FICTIONAL environment and FICTIONAL agent. No real model endpoint, tool server, or memory
# service is called. The subscriber records, documents, and tasks are fictional, seed 20260218.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL environment: records of the measurement network
# wire words stay Turkish: they are counted in tokens (same rule as the MCP course)
# onceki/simdiki/tip/bolge = previous/current/type/region; mesken/isyeri = residential/commercial
REGION = ["dogu", "bati", "kuzey", "guney", "merkez"]
TYPE = ["mesken", "isyeri"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"bolge": REGION[int(r() * 5)], "tip": TYPE[int(r() * 2)],
                                    "onceki": previous, "simdiki": previous + int(r() * 70)}
TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16
COMMERCIAL_FACTOR = 2
DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu",
             "TR02": "isyeri abonesinde birim ucret iki katidir",
             "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir",
             "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider",
             "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"}

CALLS = [0]                              # tool call counter


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    total += remaining * ABOVE
    return total * (COMMERCIAL_FACTOR if type_ == "isyeri" else 1)


def t_read_meter(abone=None, **_):
    a = SUBSCRIBERS.get(abone)
    if a is None:
        return {"hata": "abone yok"}
    return {"onceki": a["onceki"], "simdiki": a["simdiki"], "tip": a["tip"],
            "bolge": a["bolge"]}


def t_consumption(onceki=None, simdiki=None, **_):
    if onceki is None or simdiki is None:
        return {"hata": "eksik alan"}
    f = simdiki - onceki
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, tip="mesken", **_):
    if m3 is None:
        return {"hata": "eksik alan"}
    return {"tutar": _charge(m3, tip)}


def t_document(anahtar=None, **_):
    if not anahtar:
        return {"hata": "eksik alan"}
    q = set(tokenize(anahtar))
    best, code = -1, None
    for k, m in DOCUMENTS.items():
        o = len(q & set(tokenize(m)))
        if o > best:
            best, code = o, k
    return {"kod": code, "metin": DOCUMENTS[code]} if best > 0 else {"hata": "bulunamadi"}


def t_record(abone=None, sutun=None, deger=None, **_):
    if not (abone and sutun and deger is not None):
        return {"hata": "eksik alan"}
    return {"yazildi": True, "abone": abone, "sutun": sutun, "deger": deger}


TOOLS = {
    "sayac_oku":  {"function": t_read_meter, "fields": ("abone",), "side_effect": False},
    "tuketim":    {"function": t_consumption, "fields": ("onceki", "simdiki"),
                      "side_effect": False},
    "tarife":     {"function": t_tariff, "fields": ("m3", "tip"), "side_effect": False},
    "belge_ara":  {"function": t_document, "fields": ("anahtar",), "side_effect": False},
    "kayit_yaz":  {"function": t_record, "fields": ("abone", "sutun", "deger"),
                      "side_effect": True},
}
TOOL_NAMES = list(TOOLS)


def call_tool(name, state):
    """Fields outside the schema are filtered out; a missing field is the tool's own error."""
    CALLS[0] += 1
    if name not in TOOLS:
        return {"hata": "arac yok"}
    kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state}
    return TOOLS[name]["function"](**kw)


# ---- FICTIONAL tasks: each one is solved by a known tool sequence
TASKS = []
for i in range(24):
    r = rng(SEED + 9973 * i)
    sub = f"A{1 + int(r() * 60):03d}"
    t = pick(r(), [0.30, 0.22, 0.20, 0.28])
    if t == 0:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem bedeli nedir",
                      "sequence": ("sayac_oku", "tuketim", "tarife")})
    elif t == 1:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem tuketimi kac metrekup",
                      "sequence": ("sayac_oku", "tuketim")})
    elif t == 2:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub,
                      "request": "sayac donusunde endeks farki nasil hesaplanir",
                      "sequence": ("belge_ara",)})
    else:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub,
                      "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz",
                      "sequence": ("sayac_oku", "tuketim", "tarife", "kayit_yaz")})


def correct_answer(g):
    a = SUBSCRIBERS[g["subscriber"]]
    f = a["simdiki"] - a["onceki"]
    m3 = f + 100000 if f < 0 else f
    if g["kind"] == "consumption":
        return m3
    if g["kind"] == "document":
        return "SY01"
    return _charge(m3, a["tip"])


# ---- FICTIONAL agent: at each step it picks a tool or finishes
def selection(d):
    return max(0.34, 0.80 - 0.09 * (d - 1))       # share of picking the right tool, falls with depth


def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False):
    """FICTIONAL agent loop. No real model is called; tool selection
    comes from a single draw that depends on the task's sequence and the depth."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"abone": g["subscriber"], "anahtar": g["request"], "sutun": "bedel"}
    trace, step, tok, errors = [], 0, 0, 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace)
        target = seq[k] if k < len(seq) else None
        if target is None:                         # the job is done , the agent should stop
            if r() < 0.88:
                return {"done": True, "steps": step, "trace": trace, "tokens": tok,
                        "errors": errors, "answer": state.get("sonuc"),
                        "approvals": state.get("onay", 0)}
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if TOOLS[choice]["side_effect"] and approval:
            state["onay"] = state.get("onay", 0) + 1
        if choice == "kayit_yaz":
            state["deger"] = state.get("sonuc")
        s = call_tool(choice, state)
        trace.append((choice, "hata" in s))
        tok += 30
        if "hata" in s:
            errors += 1
            if self_critique:
                tok += 40
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "hata"})
        if choice == "tuketim":
            state["sonuc"] = s["m3"]
        if choice == "tarife":
            state["sonuc"] = s["tutar"]
        if choice == "belge_ara":
            state["sonuc"] = s["kod"]
        if choice == target:
            k += 1
    return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors,
            "answer": state.get("sonuc"), "approvals": state.get("onay", 0)}


CALLS[0] = 0
t = [agent(g) for g in TASKS]
print("baseline: steps", sum(x["steps"] for x in t), ", calls", CALLS[0], ", trace tokens",
      sum(x["tokens"] for x in t), ", errors", sum(x["errors"] for x in t), ", success",
      round(sum(x["answer"] == correct_answer(g) for x, g in zip(t, TASKS)) / 24, 4))

baseline: steps 97 , calls 74 , trace tokens 11978 , errors 8 , success 0.9583

The shared loop finishes the 24 tasks in 97 steps and counts 11978 tokens. This number covers only the trace: at every step the tool names and observation headings up to that point. Yet the prompt that goes to the model carries the state as well: the index readings that were read, the subscriber type, the document text that was found. Every field is paid again at every step, and as the state grows this item grows faster than the trace.

Sweeping the Compaction Ratio

The block below completes the cost and sweeps the two policies over four settings.

NAME_COST = 12                      # name and format cost of a carried field
RESULT_KEY = {"tuketim": "m3", "tarife": "tutar", "belge_ara": "kod"}


def state_tokens(state):
    return sum(NAME_COST + len(tokenize(v)) for v in state.values())


def compact(state, remaining, keep, policy):
    """Anything beyond keep fields is dropped. Summary drops the oldest and puts a
    single summary entry in its place; selective keeps the fields the remaining steps declare."""
    if len(state) <= keep:
        return state, ()
    keys = list(state)
    if policy == "selective":
        needed = {a for name in remaining for a in TOOLS[name]["fields"]} | {"sonuc"}
        keys.sort(key=lambda k: k in needed)
    dropped = tuple(keys[:len(state) - keep])
    new = {k: v for k, v in state.items() if k not in dropped}
    if policy == "summary":
        new["ozet"] = f"{len(dropped)} alan dusuruldu"
    return new, dropped


def agent_c(g, keep=99, policy="summary", seed=SEED, max_steps=8):
    """The shared loop, with context compaction at the end of every step."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"abone": g["subscriber"], "anahtar": g["request"], "sutun": "bedel"}
    trace, step, tok, errors, dropped, needed = [], 0, 0, 0, set(), 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace) + state_tokens(state)
        target = seq[k] if k < len(seq) else None
        if target is None:
            if r() < 0.88:
                break
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if choice == "kayit_yaz":
            state["deger"] = state.get("sonuc")
        missing = [a for a in TOOLS[choice]["fields"] if a not in state and a in dropped]
        s = call_tool(choice, state)
        trace.append(choice)
        tok += 30
        if "hata" in s:
            errors, needed = errors + 1, needed + bool(missing)
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "hata"})
        if choice in RESULT_KEY:
            state["sonuc"] = s[RESULT_KEY[choice]]
        if choice == target:
            k += 1
        state, dr = compact(state, seq[k:], keep, policy)
        dropped |= set(dr)
    return {"steps": step, "tokens": tok, "errors": errors, "needed": needed,
            "answer": state.get("sonuc")}


def run(keep, policy, seed=SEED):
    CALLS[0] = 0
    st = tok = errors = needed = correct = 0
    for g in TASKS:
        s = agent_c(g, keep, policy, seed)
        st, tok, errors = st + s["steps"], tok + s["tokens"], errors + s["errors"]
        needed, correct = needed + (s["needed"] > 0), correct + (s["answer"] == correct_answer(g))
    return st, CALLS[0], tok, errors, needed, correct / len(TASKS)


SWEEP = [(99, "uncompacted")] + [(t, p) for p in ("summary", "selective")
                                 for t in (6, 5, 4, 3)]
print("keep policy         steps calls   tokens errors needed  success | 2nd run")
for keep, pol in SWEEP:
    a = run(keep, "summary" if keep == 99 else pol)
    b = run(keep, "summary" if keep == 99 else pol, SEED + 1)
    print(f"{keep if keep != 99 else '--':>4} {pol:<14} {a[0]:>5} {a[1]:>5} {a[2]:>8}"
          f" {a[3]:>6} {a[4]:>6} {a[5]:>8.4f} | {b[2]:>6} {b[5]:.4f}")

keep policy         steps calls   tokens errors needed  success | 2nd run
  -- uncompacted       97    74    21913      8      0   0.9583 |  21998 0.9583
   6 summary          102    82    21281     29      5   0.9167 |  23749 0.9167
   5 summary           98    77    19027     24      6   0.9583 |  21458 0.9583
   4 summary           98    77    18086     24      6   0.7917 |  20420 0.7500
   3 summary          156   149    33566    125     18   0.2917 |  34523 0.2500
   6 selective         97    74    19023     15      3   0.9583 |  19233 1.0000
   5 selective         97    74    18004     18      5   0.9583 |  18184 1.0000
   4 selective         98    77    17475     24      6   0.9583 |  18846 1.0000
   3 selective         98    77    16569     24      8   0.9583 |  17861 1.0000

Once the state item is added, the baseline rises from 11978 to 21913 tokens: the carried state holds more than the trace itself. This is where compaction belongs.

The selective rows advance as expected. As keep goes from six to three, tokens fall from 19023 to 16569, that is, by 0.2439 relative to the baseline. The cost shows in two columns: tool errors climb from 8 in the uncompacted row to 24, and the number of tasks that need the dropped field goes from 0 to 8, that is, climbs to a third of the twenty-four tasks. Task success stays fixed at 0.9583 in the first run and is 1.0000 in the second. The dropped fields break the next step, but the agent gets the field back by calling the same tool again; the cost is paid not in steps but in errors.

The summary rows start in the same direction and then break. At keep four the tokens, 18086, are a little above the selective ones, but task success is 0.7917, and 0.7500 in the second run. At keep three the table turns around: tokens are 33566, that is, 1.53 times the uncompacted baseline; steps climb from 97 to 156, tool calls from 74 to 149, tool errors from 8 to 125, and task success falls to 0.2917. Here compaction is not a saving but a multiplier: a dropped field leads to an error, the error to a new step, and the new step to a context that is carried again.

Same Budget, Different Content

The most telling place in the table is the keep four rows. The two policies carry the same number of fields; the token difference is 611, less than four percent. The task success difference, however, is 0.7917 against 0.9583, and in the second run 0.7500 against 1.0000. With the largest difference between runs at 0.0417, this gap is four and six times the band, that is, measured. What decides is not the size of the context but which fields remain.

This is the memory-side counterpart of the finding measured in the Building Applications with Model APIs course: there, in an editing task, the expected acceptance was 0.6921 when all files were put into the context and 0.8233 when only the relevant files were selected. The selection there was over files, the selection here is over the agent’s own observations; the result points the same way. Irrelevant context is not only expensive, it also takes the place of the relevant.

Summary

• The carried state is paid again at every step: the trace tokens are 11978, and with the state the baseline rises to 21913 tokens.
• Selective recall cuts tokens to 16569 at keep three, by 0.2439 relative to the baseline, and does not change task success.
• In the same sweep the number of tasks that need the dropped field rises from 0 to 8 and tool errors from 8 to 24: the cost of the tokens gained is written here.
• On the same keep budget summarization gives 0.7917 and selective recall 0.9583; the gap is four and six times the run-to-run difference.
• At keep three summarization raises tokens to 33566, 1.53 times the baseline; at this point compaction is not a saving but a multiplier.

Next Step

In these two lessons memory was treated from two directions: what to write between tasks, what to carry within a task. In both, the records were assumed to be correct. Yet a meter is read again, a tariff changes, a document is renewed; the record in long-term memory stays where it is and silently turns wrong. Dropping a field does not solve this, because the problem is not the narrowness of the place but the age of the record. The next lesson sweeps the expiry threshold and puts two numbers side by side: as the threshold tightens, stale records drop, but the share of records that are discarded although they are still valid grows.