--- title: 'Executing Tool Calls' source: 'https://academia.sh/en/courses/ai-agents/executing-tool-calls' course: 'AI Agents' language: en updated: '2026-10-21T05:02:26+00:00' license: 'CC BY-SA 4.0' objectives: - 'Writes a layer that turns a tool result into raw, truncated, and summarized observation forms.' - 'Compares task success, tool errors, steps, and accumulated tokens across the four observation forms.' - 'Tells apart with numbers that truncation drops fields by position while a summary drops them by declaration.' - 'Shows that in an accumulating context an early observation is costlier than a late one.' --- # Executing Tool Calls Measuring in which form the result a tool returns is given back to the model: on 24 fictional tasks the raw observation gives a task success of 0.9583 with 770 observation tokens and 10103 total tokens. Truncating the observation to seven words drops the tokens to 9160, but because it drops the subscriber type, four commercial invoices come out at exactly half and task success falls to 0.7917. Truncating to four words breaks the chain: tool errors jump from 8 to 116, steps from 97 to 156, total tokens to 15945, and task success stays at 0.2917; making it cheaper makes it more expensive. The summary that declares the fields to carry keeps the task success of the raw observation in both runs while dropping the observation tokens to 401 and the tokens per correct answer from 439 to 401. When the subscriber type is removed from the same summary, task success falls to the truncation level, 0.7917: a summary is only as good as its declaration. In the previous lesson the tool definition was split into four levels, and at the highest level the valid call rate was 0.8919 and task success 0.9583. That measurement never questioned the **return** of the call: whatever the tool returned was written to the state with all its fields and carried to the next step. In a loop this is a shaky assumption. A tool result is an **observation** given back to the model, and that observation sits in the prompt again at every step. The full text a document tool returns, the four fields a reading tool returns, the confirmation of a write tool: all of them accumulate as the loop grows. The question of this lesson is this: in what form should the observation be given back, what happens to task success, and how many tokens accumulate across the loop. - **TM9.** The environment, the tool set, the task set, and the agent loop are **fictional**; no real model endpoint, tool server, or runtime is called. 60 fictional subscribers, 5 fictional documents, 5 tools, and 24 tasks each solved by a known tool sequence. Seed **20260218**, second run **20260219**. - **TM10.** A tool returns a dictionary; the **observation** is that dictionary turned into text. The agent updates its state not from the dictionary but **by reading back from the observation text**. - **TM11.** There are three observation forms. The **raw** observation carries all the fields. The **truncated** observation cuts the text at a given number of words; a field whose value is left incomplete at the cut cannot be read back. The **summarized** observation carries the fields **declared** per tool. - **TM12.** Truncation is measured at two thresholds: seven words and four words. - **TM13.** The accumulated observation is sent again in the prompt at every step. The shared loop counts this accumulation with a constant coefficient per step; this lesson puts **the token count of the observation actually carried** in place of that coefficient. Tokens are counted with the fictional tokenizer. - **TM14.** The observation of a call that returns an error also accumulates; an error text is also an observation. - **TM15.** The baseline is the **raw observation**; this is the loop without scaffolding. Truncation and summarizing are the scaffolding added. The step budget is eight in all four forms. - **TM16.** Task success is the result the loop leaves being equal to the correct answer. ## The Three Forms of Observation The code below builds the shared environment, the tool set, and the task set, then adds the observation layer: a writer that turns the dictionary into text, a parser that reads the fields back from the text, and the three forms. The tool and field names in the code are the `read_meter`, `consumption`, `tariff`, `search_docs`, and `write_record` tools of the previous lessons; they are counted in tokens here, so they stay Turkish. ```python SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 def tokenize(m): b = [] for s in str(m).lower().replace(",", " , ").split(): while len(s) > 5: b.append(s[:5]) s = s[5:] b.append(s) return b def trace_id(m): h = 2166136261 for c in str(m): h = ((h ^ ord(c)) * 16777619) & M32 return h # ---- FICTIONAL environment: records of the measurement network # wire words stay Turkish: they are counted in tokens (same rule as the MCP course) # onceki/simdiki/tip/bolge = previous/current/type/region; mesken/isyeri = residential/commercial REGION = ["dogu", "bati", "kuzey", "guney", "merkez"] TYPE = ["mesken", "isyeri"] SUBSCRIBERS = {} for i in range(60): r = rng(SEED + 31 * i) previous = 10000 + int(r() * 80000) SUBSCRIBERS[f"A{i + 1:03d}"] = {"bolge": REGION[int(r() * 5)], "tip": TYPE[int(r() * 2)], "onceki": previous, "simdiki": previous + int(r() * 70)} TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16 COMMERCIAL_FACTOR = 2 DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu", "TR02": "isyeri abonesinde birim ucret iki katidir", "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir", "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider", "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"} CALLS = [0] # tool call counter def _charge(m3, type_): remaining, lower, total = m3, 0, 0 for upper, unit in TIERS: band = min(remaining, upper - lower) if band <= 0: break total += band * unit remaining -= band lower = upper total += remaining * ABOVE return total * (COMMERCIAL_FACTOR if type_ == "isyeri" else 1) def t_read_meter(abone=None, **_): a = SUBSCRIBERS.get(abone) if a is None: return {"hata": "abone yok"} return {"onceki": a["onceki"], "simdiki": a["simdiki"], "tip": a["tip"], "bolge": a["bolge"]} def t_consumption(onceki=None, simdiki=None, **_): if onceki is None or simdiki is None: return {"hata": "eksik alan"} f = simdiki - onceki return {"m3": f + 100000 if f < 0 else f} def t_tariff(m3=None, tip="mesken", **_): if m3 is None: return {"hata": "eksik alan"} return {"tutar": _charge(m3, tip)} def t_document(anahtar=None, **_): if not anahtar: return {"hata": "eksik alan"} q = set(tokenize(anahtar)) best, code = -1, None for k, m in DOCUMENTS.items(): o = len(q & set(tokenize(m))) if o > best: best, code = o, k return {"kod": code, "metin": DOCUMENTS[code]} if best > 0 else {"hata": "bulunamadi"} def t_record(abone=None, sutun=None, deger=None, **_): if not (abone and sutun and deger is not None): return {"hata": "eksik alan"} return {"yazildi": True, "abone": abone, "sutun": sutun, "deger": deger} TOOLS = { "sayac_oku": {"function": t_read_meter, "fields": ("abone",), "side_effect": False}, "tuketim": {"function": t_consumption, "fields": ("onceki", "simdiki"), "side_effect": False}, "tarife": {"function": t_tariff, "fields": ("m3", "tip"), "side_effect": False}, "belge_ara": {"function": t_document, "fields": ("anahtar",), "side_effect": False}, "kayit_yaz": {"function": t_record, "fields": ("abone", "sutun", "deger"), "side_effect": True}, } TOOL_NAMES = list(TOOLS) def call_tool(name, state): """Fields outside the schema are filtered out; a missing field is the tool's own error.""" CALLS[0] += 1 if name not in TOOLS: return {"hata": "arac yok"} kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state} return TOOLS[name]["function"](**kw) # ---- FICTIONAL tasks: each one is solved by a known tool sequence TASKS = [] for i in range(24): r = rng(SEED + 9973 * i) sub = f"A{1 + int(r() * 60):03d}" t = pick(r(), [0.30, 0.22, 0.20, 0.28]) if t == 0: TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub, "request": f"{sub} abonesinin bu donem bedeli nedir", "sequence": ("sayac_oku", "tuketim", "tarife")}) elif t == 1: TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub, "request": f"{sub} abonesinin bu donem tuketimi kac metrekup", "sequence": ("sayac_oku", "tuketim")}) elif t == 2: TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub, "request": "sayac donusunde endeks farki nasil hesaplanir", "sequence": ("belge_ara",)}) else: TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub, "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz", "sequence": ("sayac_oku", "tuketim", "tarife", "kayit_yaz")}) def correct_answer(g): a = SUBSCRIBERS[g["subscriber"]] f = a["simdiki"] - a["onceki"] m3 = f + 100000 if f < 0 else f if g["kind"] == "consumption": return m3 if g["kind"] == "document": return "SY01" return _charge(m3, a["tip"]) # ---- FICTIONAL agent: at each step it picks a tool or finishes def selection(d): return max(0.34, 0.80 - 0.09 * (d - 1)) # share of picking the right tool, falls with depth # The dictionary a tool returns is turned into text before it is given back to the model. SUMMARY_FIELDS = {"sayac_oku": ("onceki", "simdiki", "tip"), "tuketim": ("m3",), "tarife": ("tutar",), "belge_ara": ("kod",), "kayit_yaz": ("yazildi",)} FORMAT = {"raw": None, "truncated 7": 7, "truncated 4": 4, "summarized": None} def to_text(s): return " , ".join(f"{k} = {v}" for k, v in s.items()) def parse(text): """Reads the fields back from the observation text; a field whose value was cut off is dropped.""" d = {} for p in text.split(" , "): a, _, v = p.partition(" = ") v = v.strip(" ,") if v: d[a.strip(" ,")] = int(v) if v.lstrip("-").isdigit() else v return d def observe(name, result, form): """Raw , truncated , or summarized observation text.""" if form == "summarized": result = {k: v for k, v in result.items() if k in SUMMARY_FIELDS.get(name, ())} text = to_text(result) if FORMAT[form]: text = " ".join(text.split()[:FORMAT[form]]) return text def agent_observation(g, form="raw", max_steps=8, seed=SEED): """The shared agent loop; the only differences are that the observation is turned into text and read back, and that the accumulated observation tokens are sent again at every step.""" r = rng(seed ^ trace_id(g["name"])) state = {"abone": g["subscriber"], "anahtar": g["request"], "sutun": "bedel"} trace, step, tok, errors, accum = [], 0, 0, 0, 0 seq, k = g["sequence"], 0 while step < max_steps: step += 1 tok += 60 + accum # the accumulated observation is sent again at every step target = seq[k] if k < len(seq) else None if target is None: if r() < 0.88: break choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] if choice == "kayit_yaz": state["deger"] = state.get("sonuc") s = call_tool(choice, state) trace.append((choice, "hata" in s)) tok += 30 text = observe(choice, s, form) accum += len(tokenize(text)) if "hata" in s: errors += 1 continue d = parse(text) state.update(d) if choice == "tuketim" and "m3" in d: state["sonuc"] = d["m3"] if choice == "tarife" and "tutar" in d: state["sonuc"] = d["tutar"] if choice == "belge_ara" and "kod" in d: state["sonuc"] = d["kod"] if choice == target: k += 1 return {"steps": step, "tokens": tok, "errors": errors, "observation": accum, "answer": state.get("sonuc")} for name in ("sayac_oku", "belge_ara"): sample = TOOLS[name]["function"](**{"abone": "A007", "anahtar": "endeks farki"}) for form in ("raw", "truncated 7", "summarized"): m = observe(name, sample, form) print(f"{name:<10} {form:<11} {len(tokenize(m)):>2} tokens {m}") print(f"{name:<10} truncated 7 fields read back :", " , ".join(parse(observe(name, sample, "truncated 7")))) ``` ``` sayac_oku raw 19 tokens onceki = 78189 , simdiki = 78241 , tip = mesken , bolge = merkez sayac_oku truncated 7 9 tokens onceki = 78189 , simdiki = 78241 sayac_oku summarized 14 tokens onceki = 78189 , simdiki = 78241 , tip = mesken sayac_oku truncated 7 fields read back : onceki , simdiki belge_ara raw 19 tokens kod = SY01 , metin = endeks farki tuketimi verir sayac donusu 100000 eklenir belge_ara truncated 7 8 tokens kod = SY01 , metin = endeks belge_ara summarized 3 tokens kod = SY01 belge_ara truncated 7 fields read back : kod , metin ``` Both tools carry the same raw length, nineteen tokens, and the two forms cut them in very different places. Truncation looks at **position**: in the `sayac_oku` (read meter) observation the last two fields, `tip` (type) and `bolge` (region), drop because they remain after the seventh word. The summary looks at **declaration**: in the same tool the type stays because it is declared, and the region goes because it is not. In the document tool the order is reversed; there truncation keeps the code because it comes first, while the summary throws away the whole text and drops the nineteen tokens to three. ## The Numbers of the Four Forms The same 24 fictional tasks run in four forms. Observation tokens are the total length of the observation texts produced across the loop; total tokens also include their **being sent again**. ```python def measure(form, seed=SEED): CALLS[0] = 0 ok = steps = tok = errors = obs = 0 for g in TASKS: s = agent_observation(g, form, seed=seed) ok += s["answer"] == correct_answer(g) steps += s["steps"] tok += s["tokens"] errors += s["errors"] obs += s["observation"] return ok / len(TASKS), CALLS[0], errors, steps, obs, tok print("observation form calls errors steps observation tokens total tokens" " task success A task success B tokens per answer") for form in FORMAT: a, c = measure(form), measure(form, SEED + 1) print(f"{form:<16} {a[1]:>6} {a[2]:>6} {a[3]:>5} {a[4]:>18} {a[5]:>12}" f" {a[0]:>14.4f} {c[0]:>14.4f} {a[5] / (a[0] * 24):>17.0f}") SUMMARY_FIELDS["sayac_oku"] = ("onceki", "simdiki") # type dropped from the declaration a = measure("summarized") print(f"narrow summary ( type not declared ) : task success {a[0]:.4f} , observation tokens" f" {a[4]} , tool errors {a[2]}") ``` ``` observation form calls errors steps observation tokens total tokens task success A task success B tokens per answer raw 74 8 97 770 10103 0.9583 0.9583 439 truncated 7 74 8 97 428 9160 0.7917 0.7500 482 truncated 4 149 116 156 616 15945 0.2917 0.2917 2278 summarized 74 8 97 401 9229 0.9583 0.9583 401 narrow summary ( type not declared ) : task success 0.7917 , observation tokens 311 , tool errors 8 ``` The baseline is the first row: the raw observation produces 770 observation tokens, spends 10103 total tokens, and holds task success at **0.9583**. The second row puts side by side what truncation adds and what it drops. Observation tokens fall to **428**, total tokens to **9160**, but task success becomes **0.7917**. Calls, steps, and tool errors do not change at all; the four tasks that drop are all commercial subscribers, and in all four the invoice comes out exactly **half**. When the type drops, the tariff tool uses the declared default, that is residential. Truncation produced no error; it produced a silent wrong answer. The third row is a clear example of the course's second claim. Squeezing the truncation to four words cuts the second index in the `sayac_oku` observation as well; the `tuketim` (consumption) tool returns with a missing field, the chain breaks, and the agent retries the same step. Tool errors rise **from 8 to 116**, calls from 74 to **149**, steps from 97 to **156**. The result: observation tokens are still lower than in the raw form (616), but **total tokens jump to 15945**, one and a half times the raw form. Making the observation cheaper made the loop more expensive. ## Declaration and Position The last row is the number of the summary and it stands alone in the table: task success is **0.9583** in both runs, the same as the raw observation; observation tokens are **401**, less than half of the raw; tokens per correct answer fall **from 439 to 401**. The summary is both the cheapest and the most accurate form, because it reads what to throw away not from position but from **declaration**. This advantage is not free, and the last output line writes its cost. When only the `tip` field is dropped from the summary's declaration, task success falls to **0.7917**, to where truncation dropped it. A summary is only as good as its declaration; whoever knows which field will be needed at the next step must declare it, and if one does not know this, a summary is no safer than truncation. There is one row in the table that looks reversed: the summary's observation tokens are lower than those of truncated 7 (401 against 428) but its total tokens are higher (9229 against 9160). The reason is the arithmetic of accumulation. An observation is sent again at every step that comes after it; therefore **an early observation is costlier than a late one**. The summary keeps the `sayac_oku` observation at 14 tokens (truncation at 9), and that observation is in the **first** step of most tasks. In contrast, the summary's big gain is in `belge_ara` (search docs; from 19 to 3), and that observation is the **last** step of single-step tasks, so it is never sent again. ## Summary - A tool result is not written to the state directly: it is turned into an observation text, read back, and sent again in the prompt at every step. - The raw observation gives 770 observation tokens, 10103 total tokens, and a task success of 0.9583; this is the baseline. - Truncating to seven words drops the tokens to 9160 but, because it drops the subscriber type, halves four commercial invoices and pulls task success to 0.7917; the tool errors do not rise at all, the wrong answer is silent. - Truncating to four words breaks the chain: tool errors rise to 116, steps to 156, total tokens to 15945, and task success falls to 0.2917; making the observation cheaper makes the loop more expensive. - The summary with a declaration keeps the task success of the raw observation while dropping the observation tokens to 401 and the tokens per correct answer to 401; when the type is removed from the same summary, task success falls to 0.7917. - In an accumulating context an early observation is costlier than a late one: an observation is sent again at every step after it. ## Next Step In these two lessons the tools were defined, called, and their returns formatted. All of them had one thing in common: each tool did a **fixed job** and returned. A reading is a fixed number of fields, a computation a fixed number of operations, a search a fixed text. That is why how much a call would cost was never asked. In a tool that runs a program the agent **wrote itself**, this assumption lifts: how many steps it will take, how many cells it will occupy, and how many outputs it will produce are not written in the definition and are seen only while it runs. The next lesson builds such a tool inside a sandbox with resource limits, counts the calls that exceed the declared limit, and writes separately how many tasks stopping it drops.