The Agent Loop

The previous lesson measured the loop as a box: twenty-four tasks going in, the answers coming out and the total cost in between. The inside of the box showed up in a single trace only, that trace held two wrong tool selections, and the last row by kind said that success falls on four-step tasks. Where the fall comes from stayed open.

This lesson opens the box. It first writes the phases a step goes through, then counts one number by step depth: whether the agent was able to pick the right tool at that step. The baseline of this lesson is not the scaffolding-free loop but the first step of the same loop; as depth grows, what the same measure becomes is counted.

• AA13. The environment, the tool set, the task set and the loop are the same definition as in the previous lesson and are fictional. The seed is 20260218, the second run 20260219.
• AA14. A step is four phases: reading the fields in the state (perception), determining the next target (reasoning), calling a tool (action) and writing the returned result into the state (observation).
• AA15. Depth is which step of the task’s sequence one is at — not the number of steps spent. A target missed three times consumes three steps but does not change the depth.
• AA16. The share of picking the right tool is how many of the selections made at that depth were the target tool. Because a wrong selection is a random one of five tools, it can also land on the target by chance; the share the mechanism predicts is therefore selection(d) + (1 - selection(d)) / 5.
• AA17. Steps after the job is done do not count as selections; they have no target.
• AA18. A wasted call is a selection that does not hit the target. It spends a step, spends tokens and can write an unwanted field into the state.
• AA19. The step budget is fixed at eight in this lesson; the budget is the subject of the fourth lesson.
• AA20. As depth grows, fewer tasks reach it; the sample thins out too and is written on every row.
• AA21. A difference below the run-to-run difference counts as unmeasured.

Four Phases and a Breakdown of One Task

Every step of the loop goes through the same four phases. Perception is what the state dictionary holds at that moment: which fields are filled, which observations have accumulated. Reasoning is deciding what the next target is. Action is calling a tool. Observation is writing the fields the tool returned into the state — or it is a tool error, because the call did not find the fields the tool declares.

The code below builds the definition and prints the breakdown of a four-step write task.

# SIMULATOR -- FICTIONAL. No real model endpoint, tool server or runtime
# is called; the environment, the tools and the tasks are all fictional.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL environment: records of the measurement network
REGION = ["east", "west", "north", "south", "central"]
TYPE = ["residential", "commercial"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)],
                                    "previous": previous, "current": previous + int(r() * 70)}
TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16
COMMERCIAL_FACTOR = 2
DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu",
             "TR02": "isyeri abonesinde birim ucret iki katidir",
             "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir",
             "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider",
             "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"}

CALLS = [0]                              # tool call counter


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    total += remaining * ABOVE
    return total * (COMMERCIAL_FACTOR if type_ == "commercial" else 1)


def t_read_meter(subscriber=None, **_):
    a = SUBSCRIBERS.get(subscriber)
    if a is None:
        return {"error": "no such subscriber"}
    return {"previous": a["previous"], "current": a["current"], "type": a["type"],
            "region": a["region"]}


def t_consumption(previous=None, current=None, **_):
    if previous is None or current is None:
        return {"error": "missing field"}
    f = current - previous
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, type="residential", **_):
    if m3 is None:
        return {"error": "missing field"}
    return {"amount": _charge(m3, type)}


def t_document(query=None, **_):
    if not query:
        return {"error": "missing field"}
    q = set(tokenize(query))
    best, code = -1, None
    for k, m in DOCUMENTS.items():
        o = len(q & set(tokenize(m)))
        if o > best:
            best, code = o, k
    return {"code": code, "text": DOCUMENTS[code]} if best > 0 else {"error": "not found"}


def t_record(subscriber=None, column=None, value=None, **_):
    if not (subscriber and column and value is not None):
        return {"error": "missing field"}
    return {"written": True, "subscriber": subscriber, "column": column, "value": value}


TOOLS = {
    "read_meter":    {"function": t_read_meter, "fields": ("subscriber",), "side_effect": False},
    "consumption":   {"function": t_consumption, "fields": ("previous", "current"),
                      "side_effect": False},
    "tariff":        {"function": t_tariff, "fields": ("m3", "type"), "side_effect": False},
    "search_docs":   {"function": t_document, "fields": ("query",), "side_effect": False},
    "write_record":  {"function": t_record, "fields": ("subscriber", "column", "value"),
                      "side_effect": True},
}
TOOL_NAMES = list(TOOLS)


def call_tool(name, state):
    """Fields outside the schema are filtered out; a missing field is the tool's own error."""
    CALLS[0] += 1
    if name not in TOOLS:
        return {"error": "no such tool"}
    kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state}
    return TOOLS[name]["function"](**kw)


# ---- FICTIONAL tasks: each one is solved by a known tool sequence
TASKS = []
for i in range(24):
    r = rng(SEED + 9973 * i)
    sub = f"A{1 + int(r() * 60):03d}"
    t = pick(r(), [0.30, 0.22, 0.20, 0.28])
    if t == 0:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem bedeli nedir",
                      "sequence": ("read_meter", "consumption", "tariff")})
    elif t == 1:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem tuketimi kac metrekup",
                      "sequence": ("read_meter", "consumption")})
    elif t == 2:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub,
                      "request": "sayac donusunde endeks farki nasil hesaplanir",
                      "sequence": ("search_docs",)})
    else:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub,
                      "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz",
                      "sequence": ("read_meter", "consumption", "tariff", "write_record")})


def correct_answer(g):
    a = SUBSCRIBERS[g["subscriber"]]
    f = a["current"] - a["previous"]
    m3 = f + 100000 if f < 0 else f
    if g["kind"] == "consumption":
        return m3
    if g["kind"] == "document":
        return "SY01"
    return _charge(m3, a["type"])


# ---- FICTIONAL agent: at each step it picks a tool or finishes
def selection(d):
    return max(0.34, 0.80 - 0.09 * (d - 1))       # share of picking the right tool, falls with depth


def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False):
    """FICTIONAL agent loop. No real model is called; tool selection
    comes from a single draw that depends on the task's sequence and the depth."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"}
    trace, step, tok, errors = [], 0, 0, 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace)
        target = seq[k] if k < len(seq) else None
        if target is None:                         # the job is done , the agent should stop
            if r() < 0.88:
                return {"done": True, "steps": step, "trace": trace, "tokens": tok,
                        "errors": errors, "answer": state.get("result"),
                        "approvals": state.get("approvals", 0)}
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if TOOLS[choice]["side_effect"] and approval:
            state["approvals"] = state.get("approvals", 0) + 1
        if choice == "write_record":
            state["value"] = state.get("result")
        s = call_tool(choice, state)
        trace.append((choice, "error" in s))
        tok += 30
        if "error" in s:
            errors += 1
            if self_critique:
                tok += 40
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "error"})
        if choice == "consumption":
            state["result"] = s["m3"]
        if choice == "tariff":
            state["result"] = s["amount"]
        if choice == "search_docs":
            state["result"] = s["code"]
        if choice == target:
            k += 1
    return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors,
            "answer": state.get("result"), "approvals": state.get("approvals", 0)}



def read_trace(g, seed=SEED):
    """Rereads the trace against the task's sequence: at each step the target of
    reasoning, the tool chosen as action and what the observation brought."""
    s = agent(g, seed=seed)
    k, pairs, lines = 0, [], []
    for i, (choice, failed) in enumerate(s["trace"], 1):
        target = g["sequence"][k] if k < len(g["sequence"]) else "none , must stop"
        if k < len(g["sequence"]):
            pairs.append((k + 1, choice == target))
        lines.append(f"  {i}. depth {min(k + 1, len(g['sequence'])):<2}"
                     f" target {target:<17} action {choice:<12}"
                     f" observation {'tool error' if failed else 'fields arrived'}")
        if choice == target and not failed:
            k += 1
    return s, pairs, lines


g = [x for x in TASKS if x["name"] == "G23"][0]
s, _, lines = read_trace(g)
print(f"{g['name']} ({g['kind']}) : {g['request']}")
print("\n".join(lines))
print(f"  step {s['steps']}  answer {s['answer']}  correct answer {correct_answer(g)}")

G23 (record) : A006 abonesinin bedelini hesapla ve kayda yaz
  1. depth 1  target read_meter        action read_meter   observation fields arrived
  2. depth 2  target consumption       action consumption  observation fields arrived
  3. depth 3  target tariff            action search_docs  observation fields arrived
  4. depth 3  target tariff            action consumption  observation fields arrived
  5. depth 3  target tariff            action tariff       observation fields arrived
  6. depth 4  target write_record      action tariff       observation fields arrived
  7. depth 4  target write_record      action search_docs  observation fields arrived
  8. depth 4  target write_record      action write_record observation fields arrived
  step 8  answer TR01  correct answer 1132

This breakdown shows two separate costs of a wrong selection. The search_docs and consumption calls at the third and fourth steps miss the target; neither gives an error, because the fields they expect are in the state. A step is spent and the depth does not advance. The seventh step is heavier: search_docs returns successfully and overwrites the accumulated result; the state now holds a document code instead of 1132. At the eighth step the agent finally hits the target, but write_record puts that corrupted value into the record. A wasted call that returns successfully does not only spend a step, it corrupts the state.

The Share of Picking the Right Tool as Depth Grows

The same number is added up over the whole task set, depth by depth. The share the mechanism predicts is written next to it, because a wrong selection is a random tool and can also land on the target by chance.

print("depth     expected   A: picks  right  share   B: picks  right  share")
for d in (1, 2, 3, 4):
    row = [f"{d:>5} {selection(d) + (1 - selection(d)) / 5:>12.4f}"]
    for seed in (SEED, SEED + 1):
        n = k = 0
        for task in TASKS:
            for depth, ok in read_trace(task, seed)[1]:
                if depth == d:
                    n, k = n + 1, k + ok
        row.append(f"{n:>10} {k:>6} {k / n:>6.4f}")
    print(" ".join(row))

print()
print("kind         seq  tasks A: steps/task wasted errors  success"
      " B: steps/task wasted errors  success")
for kind, length in (("document", 1), ("consumption", 2), ("charge", 3), ("record", 4)):
    subset = [x for x in TASKS if x["kind"] == kind]
    row = [f"{kind:<11} {length:>4} {len(subset):>6}"]
    for seed in (SEED, SEED + 1):
        steps = wasted = errors = hits = 0
        for task in subset:
            res, pairs, _ = read_trace(task, seed)
            steps, errors = steps + res["steps"], errors + res["errors"]
            wasted += sum(1 for _, ok in pairs if not ok)
            hits += res["answer"] == correct_answer(task)
        row.append(f"{steps / len(subset):>13.2f} {wasted:>6} {errors:>6} {hits / len(subset):>8.4f}")
    print(" ".join(row))

depth     expected   A: picks  right  share   B: picks  right  share
    1       0.8400         30     24 0.8000         31     24 0.7742
    2       0.7680         24     17 0.7083         22     17 0.7727
    3       0.6960         13      9 0.6923         12      9 0.7500
    4       0.6240          6      3 0.5000          6      3 0.5000

kind         seq  tasks A: steps/task wasted errors  success B: steps/task wasted errors  success
document       1      7          2.86      5      5   1.0000          2.86      3      2   1.0000
consumption    2      8          3.50      4      3   1.0000          3.62      5      1   1.0000
charge         3      6          4.33      2      0   1.0000          5.00      6      3   1.0000
record         4      3          7.67      9      0   0.6667          6.33      4      1   0.6667

The first table says two things at once, and they must be read separately. The end-to-end difference is measured: the share is 0.8000 at depth 1 and 0.5000 at depth 4; on the second run 0.7742 and 0.5000. The fall is 0.3000 and 0.2742. The run-to-run difference of the same depth is 0.0644 at its widest (depth 2), so the end-to-end fall is more than four times the band. As the plan grows longer, the selection deteriorates.

The differences between neighboring depths tell a different story. On the first run the fall from 1 to 2 is 0.0917 and from 2 to 3 is 0.0160; on the second run the first is 0.0015 and the second is 0.0227. All of these are inside the band or at its edge and count as unmeasured. Only the move from 3 to 4 exceeds the band on both runs: 0.1923 and 0.2500. A measurement that wants to read the share of single steps needs a larger task set.

The reason is written in the selection columns on the right. There are 30 selections at depth 1 and only 6 at depth 4. The number of tasks with a four-step sequence is three, and the number of runs that reach there is even smaller. As depth grows, the measurement itself thins out: the place where the agent is weakest is the place where we have the fewest numbers.

Reading Across Four Task Kinds

The second table shows the same fall by kind. Steps per task grow with sequence length: 2.86, 3.50, 4.33, 7.67. The minimum steps needed are 2, 3, 4 and 5 respectively — one step always goes to stopping. On the four-step kind the realized steps are 1.53 times the minimum needed, and this ratio falls to 1.27 on the second run; the difference between the two cannot be measured in a subset of three tasks.

What can be measured is success. The one-, two- and three-step kinds give 1.0000 on both runs and the four-step kind gives 0.6667 on both runs. The same agent, the same tools, the same budget; the only thing that changes is the length of the sequence. The tool error column, on the other hand, swings from 5 to 2 and from 0 to 3 between the two runs: at this scale error counts are noise.

Summary

• An agent step is four phases: reading the state, determining the target, calling the tool and writing the returned result into the state.
• A wrong selection pays two costs: the step spent and the unwanted field written into the state; a successful wrong call can overwrite the accumulated result.
• The share of picking the right tool is 0.8000 at depth 1 and 0.5000 at depth 4; on the second run 0.7742 and 0.5000. The end-to-end fall is more than four times the widest run-to-run band and is measured.
• The differences of 0.0917 and 0.0160 between neighboring depths are inside the band and count as unmeasured; only the move from the third depth to the fourth exceeds the band on both runs.
• As depth grows the sample thins out: the number of selections falls from 30 to 6, so the place where the agent is weakest is the place where we have the fewest numbers.
• By kind, steps per task rise from 2.86 to 7.67; success is 1.0000 on the three short kinds and 0.6667 on both runs on the four-step kind.

Next Step

In this lesson a single loop did all the work itself: it determined the target and it called the tool. The halving of the share of hitting the target at the fourth step of the sequence is the cost of that combination, because the same call has to carry both the whole plan and the next action. The first solution that comes to mind is to split the job: let one side write a plan and let the other side only carry out the steps of that plan. The next lesson builds this split and asks two questions together — whether the split raises success, and how much the split job costs in steps and tokens.