The Planner and Executor Split

The previous lesson measured how a single loop deteriorates with depth: at the fourth step of the sequence the share of picking the right tool was cut in half. One reason is structural — the same call carries both the plan of the whole job and the single next action.

The first arrangement that comes to mind is to split the job in two. Let one side put out the plan of the job in one go, and let the other side execute the steps of that plan and not pick tools by itself. This lesson builds the split and asks two questions together: does the split raise success, and what does it make one pay. The baseline is the scaffolding-free loop — the single loop of the previous lesson.

• AA22. The environment, the tool set, the task set and the single loop are the same definition as in the previous lessons and are fictional. The seed is 20260218, the second run 20260219.
• AA23. The split arrangement is two parts: the planner writes the whole plan in a single call, and the executor runs the plan in order and does not pick tools by itself.
• AA24. The planner sees no observation. It picks all positions at the same time, with the same information; so the depth penalty does not apply to it and it picks every position with the flat share selection(1).
• AA25. Planning is a step and is deducted from the budget. Both setups are run with the same budget of eight steps.
• AA26. The executor runs the plan blindly: a wrong position is executed too.
• AA27. If the plan ends and the job is not done, the remaining positions are tried by the rule of the single loop; correction starts only after the plan runs out.
• AA28. Tokens are counted separately in the two setups. A step of the single loop is 60 + 22 x trace length, the planner’s call is 60 + 25 x plan length, a step of the executor is 40 + 10 x trace length; every observation is 30 tokens. The executor carries the plan, not the reasoning trace.
• AA29. Plan length is the sequence length of the task; the planner does not know the sequence, it knows its length.
• AA30. The counted units are success, steps, tool calls, tokens and tool errors; a difference below the run-to-run difference counts as unmeasured.

The Split: One Plan, One Execution

In the split arrangement the decision is made once. The planner reads the request and writes a sequence such as read_meter -> consumption -> tariff. The executor runs that sequence in order, and because it makes no selection its step is cheap: it carries the plan and the accumulated state, not the reasoning trace. The cost of the split is written in the same sentence — because the plan is written before the observation, a wrong position is not understood to be wrong until it has been executed.

# SIMULATOR -- FICTIONAL. No real model endpoint, tool server or runtime
# is called; the environment, the tools and the tasks are all fictional.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL environment: records of the measurement network
REGION = ["east", "west", "north", "south", "central"]
TYPE = ["residential", "commercial"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)],
                                    "previous": previous, "current": previous + int(r() * 70)}
TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16
COMMERCIAL_FACTOR = 2
DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu",
             "TR02": "isyeri abonesinde birim ucret iki katidir",
             "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir",
             "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider",
             "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"}

CALLS = [0]                              # tool call counter


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    total += remaining * ABOVE
    return total * (COMMERCIAL_FACTOR if type_ == "commercial" else 1)


def t_read_meter(subscriber=None, **_):
    a = SUBSCRIBERS.get(subscriber)
    if a is None:
        return {"error": "no such subscriber"}
    return {"previous": a["previous"], "current": a["current"], "type": a["type"],
            "region": a["region"]}


def t_consumption(previous=None, current=None, **_):
    if previous is None or current is None:
        return {"error": "missing field"}
    f = current - previous
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, type="residential", **_):
    if m3 is None:
        return {"error": "missing field"}
    return {"amount": _charge(m3, type)}


def t_document(query=None, **_):
    if not query:
        return {"error": "missing field"}
    q = set(tokenize(query))
    best, code = -1, None
    for k, m in DOCUMENTS.items():
        o = len(q & set(tokenize(m)))
        if o > best:
            best, code = o, k
    return {"code": code, "text": DOCUMENTS[code]} if best > 0 else {"error": "not found"}


def t_record(subscriber=None, column=None, value=None, **_):
    if not (subscriber and column and value is not None):
        return {"error": "missing field"}
    return {"written": True, "subscriber": subscriber, "column": column, "value": value}


TOOLS = {
    "read_meter":    {"function": t_read_meter, "fields": ("subscriber",), "side_effect": False},
    "consumption":   {"function": t_consumption, "fields": ("previous", "current"),
                      "side_effect": False},
    "tariff":        {"function": t_tariff, "fields": ("m3", "type"), "side_effect": False},
    "search_docs":   {"function": t_document, "fields": ("query",), "side_effect": False},
    "write_record":  {"function": t_record, "fields": ("subscriber", "column", "value"),
                      "side_effect": True},
}
TOOL_NAMES = list(TOOLS)


def call_tool(name, state):
    """Fields outside the schema are filtered out; a missing field is the tool's own error."""
    CALLS[0] += 1
    if name not in TOOLS:
        return {"error": "no such tool"}
    kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state}
    return TOOLS[name]["function"](**kw)


# ---- FICTIONAL tasks: each one is solved by a known tool sequence
TASKS = []
for i in range(24):
    r = rng(SEED + 9973 * i)
    sub = f"A{1 + int(r() * 60):03d}"
    t = pick(r(), [0.30, 0.22, 0.20, 0.28])
    if t == 0:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem bedeli nedir",
                      "sequence": ("read_meter", "consumption", "tariff")})
    elif t == 1:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem tuketimi kac metrekup",
                      "sequence": ("read_meter", "consumption")})
    elif t == 2:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub,
                      "request": "sayac donusunde endeks farki nasil hesaplanir",
                      "sequence": ("search_docs",)})
    else:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub,
                      "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz",
                      "sequence": ("read_meter", "consumption", "tariff", "write_record")})


def correct_answer(g):
    a = SUBSCRIBERS[g["subscriber"]]
    f = a["current"] - a["previous"]
    m3 = f + 100000 if f < 0 else f
    if g["kind"] == "consumption":
        return m3
    if g["kind"] == "document":
        return "SY01"
    return _charge(m3, a["type"])


# ---- FICTIONAL agent: at each step it picks a tool or finishes
def selection(d):
    return max(0.34, 0.80 - 0.09 * (d - 1))       # share of picking the right tool, falls with depth


def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False):
    """FICTIONAL agent loop. No real model is called; tool selection
    comes from a single draw that depends on the task's sequence and the depth."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"}
    trace, step, tok, errors = [], 0, 0, 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace)
        target = seq[k] if k < len(seq) else None
        if target is None:                         # the job is done , the agent should stop
            if r() < 0.88:
                return {"done": True, "steps": step, "trace": trace, "tokens": tok,
                        "errors": errors, "answer": state.get("result"),
                        "approvals": state.get("approvals", 0)}
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if TOOLS[choice]["side_effect"] and approval:
            state["approvals"] = state.get("approvals", 0) + 1
        if choice == "write_record":
            state["value"] = state.get("result")
        s = call_tool(choice, state)
        trace.append((choice, "error" in s))
        tok += 30
        if "error" in s:
            errors += 1
            if self_critique:
                tok += 40
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "error"})
        if choice == "consumption":
            state["result"] = s["m3"]
        if choice == "tariff":
            state["result"] = s["amount"]
        if choice == "search_docs":
            state["result"] = s["code"]
        if choice == target:
            k += 1
    return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors,
            "answer": state.get("result"), "approvals": state.get("approvals", 0)}



def plan_and_execute(g, max_steps=8, seed=SEED):
    """SPLIT arrangement: the PLANNER writes the whole plan in a single call; since
    it sees no observation it picks every position with the flat selection(1) share.
    The EXECUTOR runs the plan in order and makes no selection; if the plan ends
    and the job is not done, the remaining positions are tried by the single
    loop's rule."""
    r = rng(seed ^ trace_id("plan" + g["name"]))
    seq = g["sequence"]
    plan = [d if r() < selection(1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] for d in seq]
    state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"}
    trace, step, errors, k = [], 1, 0, 0       # planning is a step
    tok = 60 + 25 * len(plan)                  # the planner's call
    while step < max_steps:
        step += 1
        tok += 40 + 10 * len(trace)            # the executor carries the plan, not the trace
        target = seq[k] if k < len(seq) else None
        if target is None:
            if r() < 0.88:
                break
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        elif len(trace) < len(plan):
            choice = plan[len(trace)]          # the executor follows the plan's order
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if choice == "write_record":
            state["value"] = state.get("result")
        s = call_tool(choice, state)
        trace.append((choice, "error" in s))
        tok += 30
        if "error" in s:
            errors += 1
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "error"})
        for a, b in (("consumption", "m3"), ("tariff", "amount"), ("search_docs", "code")):
            if choice == a:
                state["result"] = s[b]
        if choice == target:
            k += 1
    return {"steps": step, "trace": trace, "tokens": tok, "errors": errors, "plan": plan,
            "answer": state.get("result")}


g = [x for x in TASKS if x["name"] == "G05"][0]
s = plan_and_execute(g)
print(f"{g['name']} ({g['kind']}) correct sequence :", " -> ".join(g["sequence"]))
print("  plan     :", " -> ".join(s["plan"]))
print("  executed :", " -> ".join(f"{a}{' [error]' if h else ''}" for a, h in s["trace"]))
print(f"  steps {s['steps']}  tokens {s['tokens']}  tool errors {s['errors']}"
      f"  answer {s['answer']}  correct {correct_answer(g)}")

G05 (charge) correct sequence : read_meter -> consumption -> tariff
  plan     : tariff -> consumption -> tariff
  executed : tariff [error] -> consumption [error] -> tariff [error] -> read_meter -> consumption -> tariff
  steps 8  tokens 805  tool errors 3  answer 550  correct 550

The first position of the plan is wrong, and this single mistake wastes the whole plan. tariff gives an error because it finds no m3 in the state, consumption gives an error because the meter readings were not read, and the third position is tariff again and an error again. Since the executor is bound to the plan, it has to run all three; correction starts only when the plan runs out. The correct answer comes out in the end, but only after three tool errors were paid.

What the Split Adds and What It Costs

The same twenty-four tasks are run in two setups, with two seeds and with the same budget of eight steps.

def measure(build, seed=SEED):
    CALLS[0] = 0
    success = steps = tok = errors = 0
    for t in TASKS:
        y = build(t, seed=seed)
        steps, tok, errors = steps + y["steps"], tok + y["tokens"], errors + y["errors"]
        success += y["answer"] == correct_answer(t)
    return success / len(TASKS), steps, CALLS[0], tok, errors


print("setup                    run  success   step  calls  tokens error")
for name, build in (("single loop", agent), ("planner + executor", plan_and_execute)):
    for label, seed in (("A", SEED), ("B", SEED + 1)):
        m = measure(build, seed)
        print(f"{name:<22} {label:>5} {m[0]:>8.4f} {m[1]:>6} {m[2]:>6} {m[3]:>9} {m[4]:>5}")
total = sum(len(t["sequence"]) for t in TASKS)
for label, seed in (("A", SEED), ("B", SEED + 1)):
    plans = [plan_and_execute(t, seed=seed)["plan"] for t in TASKS]
    right = sum(a == b for pl, t in zip(plans, TASKS) for a, b in zip(pl, t["sequence"]))
    whole = sum(list(pl) == list(t["sequence"]) for pl, t in zip(plans, TASKS))
    print(f"plan {label} : right positions {right}/{total} = {right / total:.4f} ;"
          f" fully right plans {whole}/{len(TASKS)}")

setup                    run  success   step  calls  tokens error
single loop                A   0.9583     97     74     11978     8
single loop                B   0.9583     98     74     11994     7
planner + executor         A   0.8750    118     71     10345    10
planner + executor         B   0.9583    116     70     10135    12
plan A : right positions 47/53 = 0.8868 ; fully right plans 18/24
plan B : right positions 47/53 = 0.8868 ; fully right plans 18/24

The success column yields no finding. The single loop is 0.9583 on both runs; the split arrangement is 0.8750 and 0.9583. The split arrangement’s own run-to-run difference is 0.0833; the difference between the two setups is exactly 0.0833 on the first run and zero on the second. The difference is inside the band, and the effect of the split on success counts as unmeasured. A difference that can be seen with the eye is not a measured difference.

The difference in the step column is measured, and where it comes from can be calculated exactly. The split arrangement adds one planning step to every task, that is twenty-four steps; 97 plus 24 is 121, and the realized figure is 118. The counterpart of the three missing steps is in the neighboring column: tool calls fall from 74 to 71. The plan protects the executor from a number of wasted calls, but planning itself takes back eight times that gain.

The token column is where the split adds: against 11978 and 11994 stand 10345 and 10135, a fall of 13.6 percent and 15.5 percent. The two setups’ own bands are 16 and 210 tokens; the difference stands far above the band and is measured. The reason is in the definition — in the single loop every step carries the growing reasoning trace again, while in the executor a step carries a fixed plan.

Tool errors go the other way: against 8 and 7 stand 10 and 12. The direction is the same on both runs and its mechanism is clear; a blindly executed wrong position calls a tool whose fields are missing from the state. But with bands of 1 and 2, the difference on the first run is 2: we know the direction from the mechanism, we cannot measure the size at this scale.

The last two lines give the limit of the split. 0.8868 of the plan’s positions are right, but only 18 of the twenty-four plans are right from start to finish. As the plan grows longer, the chance of all of it being right falls, and a single wrong position can waste the whole plan as in G05. Splitting gathers the decision at a single point; it also gathers the cost of an error at that point at a single point.

Summary

• In the split arrangement the planner writes the whole plan before the observation and the executor runs it in order and blindly; correction starts only after the plan runs out.
• Success is 0.9583 and 0.9583 in the single loop and 0.8750 and 0.9583 in the split arrangement; the difference equals the split arrangement’s own run-to-run difference and counts as unmeasured.
• The step the split adds is exactly one planning step per task: 24 is added to 97 steps, and since tool calls fall from 74 to 71, the total stays at 118.
• Tokens fall from 11978 to 10345 and from 11994 to 10135; the fall stands far above the band, because the executor carries the plan, not the reasoning trace.
• Tool errors are high in the split arrangement on both runs (10 and 12 against 8 and 7); the direction is clear from the mechanism, the size cannot be measured at this scale.
• 0.8868 of the plan’s positions are right, but the plans that are fully right are eighteen of twenty-four; a single wrong position can waste the whole plan.

Next Step

Both setups stumble in the same place, and that place has not been asked about for three lessons: where will the loop stop. Eight steps was said and eight steps was written, but that number was a decision and its reason was not written. After the plan ran out, G05 needed three more steps to correct itself; with a budget of six it would have given a wrong answer. On the other hand, an agent that has finished its job and cannot manage to stop can corrupt the correct result it holds. The next lesson sweeps the step budget over the values two, four, six, eight and twelve, and writes the stopping criterion as the rule of the course.