--- title: 'The Agent Loop' source: 'https://academia.sh/en/courses/ai-agents/agent-loop' course: 'AI Agents' language: en updated: '2026-10-21T05:02:26+00:00' license: 'CC BY-SA 4.0' objectives: - 'Splits an agent step into the phases of perception, reasoning, action and observation and shows what each phase corresponds to in the loop code.' - 'Counts the share of picking the right tool by step depth and compares it with the share the mechanism predicts.' - 'Tells which differences between depths exceed the run-to-run band and which do not.' - 'Reads the same number separately across four task kinds and measures how steps per task grow with sequence length.' --- # The Agent Loop Opening one step of the loop into perception, reasoning, action and observation and counting the share of picking the right tool by step depth: on 24 fictional tasks the share is 0.8000 at depth 1 and 0.5000 at depth 4; on the second run 0.7742 and 0.5000. The fall of 0.3000 between them stands far above the widest run-to-run band (0.0644) and is measured, but the differences of 0.0917 and 0.0160 between neighboring depths stay inside the band and count as unmeasured. As depth grows the measurement itself thins out: the number of selections falls from 30 to 6. Read across four task kinds, the success of tasks of one, two and three steps is 1.0000 on both runs and 0.6667 on four-step write tasks; steps per task rise from 2.86 to 7.67. A wrong selection does not only spend a step, it can also overwrite the accumulated result. The previous lesson measured the loop as a box: twenty-four tasks going in, the answers coming out and the total cost in between. The inside of the box showed up in a single trace only, that trace held two wrong tool selections, and the last row by kind said that success falls on four-step tasks. Where the fall comes from stayed open. This lesson opens the box. It first writes the phases a step goes through, then counts one number **by step depth**: whether the agent was able to pick the **right tool** at that step. The baseline of this lesson is not the scaffolding-free loop but **the first step of the same loop**; as depth grows, what the same measure becomes is counted. - **AA13.** The environment, the tool set, the task set and the loop are the same definition as in the previous lesson and are **fictional**. The seed is **20260218**, the second run **20260219**. - **AA14. A step is four phases:** reading the fields in the state (**perception**), determining the next target (**reasoning**), calling a tool (**action**) and writing the returned result into the state (**observation**). - **AA15. Depth is which step of the task's sequence one is at** — not the number of steps spent. A target missed three times consumes three steps but does not change the depth. - **AA16. The share of picking the right tool** is how many of the selections made at that depth were the target tool. Because a wrong selection is a random one of five tools, it **can also land on the target by chance**; the share the mechanism predicts is therefore `selection(d) + (1 - selection(d)) / 5`. - **AA17. Steps after the job is done do not count as selections**; they have no target. - **AA18. A wasted call** is a selection that does not hit the target. It spends a step, spends tokens and can write an **unwanted field** into the state. - **AA19.** The step budget is fixed at **eight** in this lesson; the budget is the subject of the fourth lesson. - **AA20.** As depth grows, fewer tasks reach it; **the sample thins out too** and is written on every row. - **AA21.** A difference below the run-to-run difference **counts as unmeasured**. ## Four Phases and a Breakdown of One Task Every step of the loop goes through the same four phases. **Perception** is what the state dictionary holds at that moment: which fields are filled, which observations have accumulated. **Reasoning** is deciding what the next target is. **Action** is calling a tool. **Observation** is writing the fields the tool returned into the state — or it is a **tool error**, because the call did not find the fields the tool declares. The code below builds the definition and prints the breakdown of a four-step write task. ```python # SIMULATOR -- FICTIONAL. No real model endpoint, tool server or runtime # is called; the environment, the tools and the tasks are all fictional. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 def tokenize(m): b = [] for s in str(m).lower().replace(",", " , ").split(): while len(s) > 5: b.append(s[:5]) s = s[5:] b.append(s) return b def trace_id(m): h = 2166136261 for c in str(m): h = ((h ^ ord(c)) * 16777619) & M32 return h # ---- FICTIONAL environment: records of the measurement network REGION = ["east", "west", "north", "south", "central"] TYPE = ["residential", "commercial"] SUBSCRIBERS = {} for i in range(60): r = rng(SEED + 31 * i) previous = 10000 + int(r() * 80000) SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)], "previous": previous, "current": previous + int(r() * 70)} TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16 COMMERCIAL_FACTOR = 2 DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu", "TR02": "isyeri abonesinde birim ucret iki katidir", "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir", "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider", "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"} CALLS = [0] # tool call counter def _charge(m3, type_): remaining, lower, total = m3, 0, 0 for upper, unit in TIERS: band = min(remaining, upper - lower) if band <= 0: break total += band * unit remaining -= band lower = upper total += remaining * ABOVE return total * (COMMERCIAL_FACTOR if type_ == "commercial" else 1) def t_read_meter(subscriber=None, **_): a = SUBSCRIBERS.get(subscriber) if a is None: return {"error": "no such subscriber"} return {"previous": a["previous"], "current": a["current"], "type": a["type"], "region": a["region"]} def t_consumption(previous=None, current=None, **_): if previous is None or current is None: return {"error": "missing field"} f = current - previous return {"m3": f + 100000 if f < 0 else f} def t_tariff(m3=None, type="residential", **_): if m3 is None: return {"error": "missing field"} return {"amount": _charge(m3, type)} def t_document(query=None, **_): if not query: return {"error": "missing field"} q = set(tokenize(query)) best, code = -1, None for k, m in DOCUMENTS.items(): o = len(q & set(tokenize(m))) if o > best: best, code = o, k return {"code": code, "text": DOCUMENTS[code]} if best > 0 else {"error": "not found"} def t_record(subscriber=None, column=None, value=None, **_): if not (subscriber and column and value is not None): return {"error": "missing field"} return {"written": True, "subscriber": subscriber, "column": column, "value": value} TOOLS = { "read_meter": {"function": t_read_meter, "fields": ("subscriber",), "side_effect": False}, "consumption": {"function": t_consumption, "fields": ("previous", "current"), "side_effect": False}, "tariff": {"function": t_tariff, "fields": ("m3", "type"), "side_effect": False}, "search_docs": {"function": t_document, "fields": ("query",), "side_effect": False}, "write_record": {"function": t_record, "fields": ("subscriber", "column", "value"), "side_effect": True}, } TOOL_NAMES = list(TOOLS) def call_tool(name, state): """Fields outside the schema are filtered out; a missing field is the tool's own error.""" CALLS[0] += 1 if name not in TOOLS: return {"error": "no such tool"} kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state} return TOOLS[name]["function"](**kw) # ---- FICTIONAL tasks: each one is solved by a known tool sequence TASKS = [] for i in range(24): r = rng(SEED + 9973 * i) sub = f"A{1 + int(r() * 60):03d}" t = pick(r(), [0.30, 0.22, 0.20, 0.28]) if t == 0: TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub, "request": f"{sub} abonesinin bu donem bedeli nedir", "sequence": ("read_meter", "consumption", "tariff")}) elif t == 1: TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub, "request": f"{sub} abonesinin bu donem tuketimi kac metrekup", "sequence": ("read_meter", "consumption")}) elif t == 2: TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub, "request": "sayac donusunde endeks farki nasil hesaplanir", "sequence": ("search_docs",)}) else: TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub, "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz", "sequence": ("read_meter", "consumption", "tariff", "write_record")}) def correct_answer(g): a = SUBSCRIBERS[g["subscriber"]] f = a["current"] - a["previous"] m3 = f + 100000 if f < 0 else f if g["kind"] == "consumption": return m3 if g["kind"] == "document": return "SY01" return _charge(m3, a["type"]) # ---- FICTIONAL agent: at each step it picks a tool or finishes def selection(d): return max(0.34, 0.80 - 0.09 * (d - 1)) # share of picking the right tool, falls with depth def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False): """FICTIONAL agent loop. No real model is called; tool selection comes from a single draw that depends on the task's sequence and the depth.""" r = rng(seed ^ trace_id(g["name"])) state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"} trace, step, tok, errors = [], 0, 0, 0 seq, k = g["sequence"], 0 while step < max_steps: step += 1 tok += 60 + 22 * len(trace) target = seq[k] if k < len(seq) else None if target is None: # the job is done , the agent should stop if r() < 0.88: return {"done": True, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("result"), "approvals": state.get("approvals", 0)} choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] if TOOLS[choice]["side_effect"] and approval: state["approvals"] = state.get("approvals", 0) + 1 if choice == "write_record": state["value"] = state.get("result") s = call_tool(choice, state) trace.append((choice, "error" in s)) tok += 30 if "error" in s: errors += 1 if self_critique: tok += 40 continue state.update({kk: vv for kk, vv in s.items() if kk != "error"}) if choice == "consumption": state["result"] = s["m3"] if choice == "tariff": state["result"] = s["amount"] if choice == "search_docs": state["result"] = s["code"] if choice == target: k += 1 return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("result"), "approvals": state.get("approvals", 0)} def read_trace(g, seed=SEED): """Rereads the trace against the task's sequence: at each step the target of reasoning, the tool chosen as action and what the observation brought.""" s = agent(g, seed=seed) k, pairs, lines = 0, [], [] for i, (choice, failed) in enumerate(s["trace"], 1): target = g["sequence"][k] if k < len(g["sequence"]) else "none , must stop" if k < len(g["sequence"]): pairs.append((k + 1, choice == target)) lines.append(f" {i}. depth {min(k + 1, len(g['sequence'])):<2}" f" target {target:<17} action {choice:<12}" f" observation {'tool error' if failed else 'fields arrived'}") if choice == target and not failed: k += 1 return s, pairs, lines g = [x for x in TASKS if x["name"] == "G23"][0] s, _, lines = read_trace(g) print(f"{g['name']} ({g['kind']}) : {g['request']}") print("\n".join(lines)) print(f" step {s['steps']} answer {s['answer']} correct answer {correct_answer(g)}") ``` ``` G23 (record) : A006 abonesinin bedelini hesapla ve kayda yaz 1. depth 1 target read_meter action read_meter observation fields arrived 2. depth 2 target consumption action consumption observation fields arrived 3. depth 3 target tariff action search_docs observation fields arrived 4. depth 3 target tariff action consumption observation fields arrived 5. depth 3 target tariff action tariff observation fields arrived 6. depth 4 target write_record action tariff observation fields arrived 7. depth 4 target write_record action search_docs observation fields arrived 8. depth 4 target write_record action write_record observation fields arrived step 8 answer TR01 correct answer 1132 ``` This breakdown shows **two separate** costs of a wrong selection. The `search_docs` and `consumption` calls at the third and fourth steps miss the target; neither gives an error, because the fields they expect are in the state. A step is spent and the depth does not advance. The seventh step is heavier: `search_docs` returns successfully and **overwrites the accumulated result**; the state now holds a document code instead of 1132. At the eighth step the agent finally hits the target, but `write_record` puts that corrupted value into the record. A wasted call that returns successfully does not only spend a step, it **corrupts the state**. ## The Share of Picking the Right Tool as Depth Grows The same number is added up over the whole task set, depth by depth. The share the mechanism predicts is written next to it, because a wrong selection is a random tool and can also land on the target by chance. ```python print("depth expected A: picks right share B: picks right share") for d in (1, 2, 3, 4): row = [f"{d:>5} {selection(d) + (1 - selection(d)) / 5:>12.4f}"] for seed in (SEED, SEED + 1): n = k = 0 for task in TASKS: for depth, ok in read_trace(task, seed)[1]: if depth == d: n, k = n + 1, k + ok row.append(f"{n:>10} {k:>6} {k / n:>6.4f}") print(" ".join(row)) print() print("kind seq tasks A: steps/task wasted errors success" " B: steps/task wasted errors success") for kind, length in (("document", 1), ("consumption", 2), ("charge", 3), ("record", 4)): subset = [x for x in TASKS if x["kind"] == kind] row = [f"{kind:<11} {length:>4} {len(subset):>6}"] for seed in (SEED, SEED + 1): steps = wasted = errors = hits = 0 for task in subset: res, pairs, _ = read_trace(task, seed) steps, errors = steps + res["steps"], errors + res["errors"] wasted += sum(1 for _, ok in pairs if not ok) hits += res["answer"] == correct_answer(task) row.append(f"{steps / len(subset):>13.2f} {wasted:>6} {errors:>6} {hits / len(subset):>8.4f}") print(" ".join(row)) ``` ``` depth expected A: picks right share B: picks right share 1 0.8400 30 24 0.8000 31 24 0.7742 2 0.7680 24 17 0.7083 22 17 0.7727 3 0.6960 13 9 0.6923 12 9 0.7500 4 0.6240 6 3 0.5000 6 3 0.5000 kind seq tasks A: steps/task wasted errors success B: steps/task wasted errors success document 1 7 2.86 5 5 1.0000 2.86 3 2 1.0000 consumption 2 8 3.50 4 3 1.0000 3.62 5 1 1.0000 charge 3 6 4.33 2 0 1.0000 5.00 6 3 1.0000 record 4 3 7.67 9 0 0.6667 6.33 4 1 0.6667 ``` The first table says two things at once, and they must be read separately. The end-to-end difference is **measured**: the share is **0.8000** at depth 1 and **0.5000** at depth 4; on the second run **0.7742** and **0.5000**. The fall is 0.3000 and 0.2742. The run-to-run difference of the same depth is **0.0644** at its widest (depth 2), so the end-to-end fall is more than four times the band. As the plan grows longer, the selection deteriorates. The differences between neighboring depths tell a different story. On the first run the fall from 1 to 2 is 0.0917 and from 2 to 3 is 0.0160; on the second run the first is 0.0015 and the second is 0.0227. All of these are inside the band or at its edge and **count as unmeasured**. Only the move from 3 to 4 exceeds the band on both runs: 0.1923 and 0.2500. A measurement that wants to read the share of single steps needs a larger task set. The reason is written in the selection columns on the right. There are **30** selections at depth 1 and only **6** at depth 4. The number of tasks with a four-step sequence is three, and the number of runs that reach there is even smaller. **As depth grows, the measurement itself thins out**: the place where the agent is weakest is the place where we have the fewest numbers. ## Reading Across Four Task Kinds The second table shows the same fall by kind. Steps per task grow with sequence length: **2.86**, **3.50**, **4.33**, **7.67**. The minimum steps needed are 2, 3, 4 and 5 respectively — one step always goes to stopping. On the four-step kind the realized steps are **1.53 times** the minimum needed, and this ratio falls to 1.27 on the second run; the difference between the two cannot be measured in a subset of three tasks. What can be measured is success. The one-, two- and three-step kinds give **1.0000** on both runs and the four-step kind gives **0.6667** on both runs. The same agent, the same tools, the same budget; the only thing that changes is the length of the sequence. The tool error column, on the other hand, swings from 5 to 2 and from 0 to 3 between the two runs: at this scale error counts are noise. ## Summary - An agent step is four phases: reading the state, determining the target, calling the tool and writing the returned result into the state. - A wrong selection pays two costs: the step spent and the unwanted field written into the state; a successful wrong call can overwrite the accumulated result. - The share of picking the right tool is 0.8000 at depth 1 and 0.5000 at depth 4; on the second run 0.7742 and 0.5000. The end-to-end fall is more than four times the widest run-to-run band and is measured. - The differences of 0.0917 and 0.0160 between neighboring depths are inside the band and count as unmeasured; only the move from the third depth to the fourth exceeds the band on both runs. - As depth grows the sample thins out: the number of selections falls from 30 to 6, so the place where the agent is weakest is the place where we have the fewest numbers. - By kind, steps per task rise from 2.86 to 7.67; success is 1.0000 on the three short kinds and 0.6667 on both runs on the four-step kind. ## Next Step In this lesson a single loop did all the work itself: it determined the target and it called the tool. The halving of the share of hitting the target at the fourth step of the sequence is the cost of that combination, because the same call has to carry both the whole plan and the next action. The first solution that comes to mind is to split the job: let one side write a plan and let the other side only carry out the steps of that plan. The next lesson builds this split and asks two questions together — whether the split raises success, and how much the split job costs in steps and tokens.