Keeping the Human in the Loop

In the previous two lessons the agent looked at its own trace, then a second agent stepped in; in both, the last word stayed with the scaffolding. Of the five tools of the fictional environment, four read and one writes: write_record leaves a trace that cannot be undone and until now was treated the same as the other four.

This lesson puts an approval point in front of that tool: how many records are written with approval on and off, how many of them are wrong and how many times the human is asked. Placing the stops according to the step interval was measured in the Building Applications with Model APIs course — stopping at every step gets 478 of 480 steps right, stopping once every eight steps gets 355. That procedure is not repeated; here the stop is placed not according to the interval but according to the side effect.

• AA53. The setup is shared: the fictional subscriber records of the measurement network, the fictional document corpus, five fictional tools, twenty-four fictional tasks and an agent simulator written inside the lesson. No real model endpoint or tool server is called.
• AA54. The baseline is the loop without scaffolding: approval off, a budget of eight steps.
• AA55. A write is wrong in two cases: if the request does not ask for writing (unintended side effect) or if the written value is not the correct answer (wrong value).
• AA56. At the approval point the proposed record is shown. Two checks are made by rule (does the request ask for writing, is there a value); the third is recalculation and is made with a share of 0.60.
• AA57. A rejected call is not executed at all; the agent spends its step and continues. Each question also carries a 35 token state summary.
• AA58. The side-effect scope asks only before write_record; the every-step scope asks before every step but can still stop only the call with a side effect.
• AA59. The second run has seed 20260219; the difference between the two runs is one task in this set (0.0417) and every difference below it counts as unmeasured.

Two Writes, Two Results

First, two tasks are run in three scopes. G09 is a document task and does not ask for writing, G23 is a record task.

# COMMON FICTIONAL SETUP -- the 60 fictional subscribers of the measurement network,
# 5 fictional documents, 5 fictional tools, 24 fictional tasks and the agent simulator
# written inside the lesson. No real model endpoint, tool server or runtime is called.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL environment: records of the measurement network
REGION = ["east", "west", "north", "south", "central"]
TYPE = ["residential", "commercial"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)],
                                    "previous": previous, "current": previous + int(r() * 70)}
TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16
COMMERCIAL_FACTOR = 2
DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu",
             "TR02": "isyeri abonesinde birim ucret iki katidir",
             "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir",
             "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider",
             "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"}

CALLS = [0]                              # tool call counter


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    total += remaining * ABOVE
    return total * (COMMERCIAL_FACTOR if type_ == "commercial" else 1)


def t_read_meter(subscriber=None, **_):
    a = SUBSCRIBERS.get(subscriber)
    if a is None:
        return {"error": "no such subscriber"}
    return {"previous": a["previous"], "current": a["current"], "type": a["type"],
            "region": a["region"]}


def t_consumption(previous=None, current=None, **_):
    if previous is None or current is None:
        return {"error": "missing field"}
    f = current - previous
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, type="residential", **_):
    if m3 is None:
        return {"error": "missing field"}
    return {"amount": _charge(m3, type)}


def t_document(query=None, **_):
    if not query:
        return {"error": "missing field"}
    q = set(tokenize(query))
    best, code = -1, None
    for k, m in DOCUMENTS.items():
        o = len(q & set(tokenize(m)))
        if o > best:
            best, code = o, k
    return {"code": code, "text": DOCUMENTS[code]} if best > 0 else {"error": "not found"}


def t_record(subscriber=None, column=None, value=None, **_):
    if not (subscriber and column and value is not None):
        return {"error": "missing field"}
    return {"written": True, "subscriber": subscriber, "column": column, "value": value}


TOOLS = {
    "read_meter":    {"function": t_read_meter, "fields": ("subscriber",), "side_effect": False},
    "consumption":   {"function": t_consumption, "fields": ("previous", "current"),
                      "side_effect": False},
    "tariff":        {"function": t_tariff, "fields": ("m3", "type"), "side_effect": False},
    "search_docs":   {"function": t_document, "fields": ("query",), "side_effect": False},
    "write_record":  {"function": t_record, "fields": ("subscriber", "column", "value"),
                      "side_effect": True},
}
TOOL_NAMES = list(TOOLS)


def call_tool(name, state):
    """Fields outside the schema are filtered out; a missing field is the tool's own error."""
    CALLS[0] += 1
    if name not in TOOLS:
        return {"error": "no such tool"}
    kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state}
    return TOOLS[name]["function"](**kw)


# ---- FICTIONAL tasks: each one is solved by a known tool sequence
TASKS = []
for i in range(24):
    r = rng(SEED + 9973 * i)
    sub = f"A{1 + int(r() * 60):03d}"
    t = pick(r(), [0.30, 0.22, 0.20, 0.28])
    if t == 0:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem bedeli nedir",
                      "sequence": ("read_meter", "consumption", "tariff")})
    elif t == 1:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem tuketimi kac metrekup",
                      "sequence": ("read_meter", "consumption")})
    elif t == 2:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub,
                      "request": "sayac donusunde endeks farki nasil hesaplanir",
                      "sequence": ("search_docs",)})
    else:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub,
                      "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz",
                      "sequence": ("read_meter", "consumption", "tariff", "write_record")})


def correct_answer(g):
    a = SUBSCRIBERS[g["subscriber"]]
    f = a["current"] - a["previous"]
    m3 = f + 100000 if f < 0 else f
    if g["kind"] == "consumption":
        return m3
    if g["kind"] == "document":
        return "SY01"
    return _charge(m3, a["type"])


# ---- FICTIONAL agent: at each step it picks a tool or finishes
def selection(d):
    return max(0.34, 0.80 - 0.09 * (d - 1))       # share of picking the right tool, falls with depth


def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False):
    """FICTIONAL agent loop. No real model is called; tool selection
    comes from a single draw that depends on the task's sequence and the depth."""
    r = rng(seed ^ trace_id(g["name"]))
    state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"}
    trace, step, tok, errors = [], 0, 0, 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace)
        target = seq[k] if k < len(seq) else None
        if target is None:                         # the job is done , the agent should stop
            if r() < 0.88:
                return {"done": True, "steps": step, "trace": trace, "tokens": tok,
                        "errors": errors, "answer": state.get("result"),
                        "approvals": state.get("approvals", 0)}
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if TOOLS[choice]["side_effect"] and approval:
            state["approvals"] = state.get("approvals", 0) + 1
        if choice == "write_record":
            state["value"] = state.get("result")
        s = call_tool(choice, state)
        trace.append((choice, "error" in s))
        tok += 30
        if "error" in s:
            errors += 1
            if self_critique:
                tok += 40
            continue
        state.update({kk: vv for kk, vv in s.items() if kk != "error"})
        if choice == "consumption":
            state["result"] = s["m3"]
        if choice == "tariff":
            state["result"] = s["amount"]
        if choice == "search_docs":
            state["result"] = s["code"]
        if choice == target:
            k += 1
    return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors,
            "answer": state.get("result"), "approvals": state.get("approvals", 0)}




# ---- Part added in this lesson: approval point in front of the tool with a side effect

RECHECK_SHARE = 0.60


def is_wrong(g, value):
    """A write is wrong: the request does not ask for writing or the value is not the correct answer."""
    return "yaz" not in g["request"] or value != correct_answer(g)


def give_approval(g, value, r):
    """At the approval point the proposed record is shown. Two checks are made by rule,
    the third by recalculation; a human who does not recalculate approves."""
    if "yaz" not in g["request"]:
        return False
    if value is None:
        return False
    if r() < RECHECK_SHARE:
        return value == correct_answer(g)
    return True


def approved_agent(g, max_steps, seed, scope="off"):
    """Shared loop. If scope is 'side_effect' the human is asked only before the call with a
    side effect, if 'every_step' before every step; blocking happens only on the side effect."""
    r = rng(seed ^ trace_id(g["name"]))
    ri = rng(seed ^ trace_id("insan|" + g["name"]))
    state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"}
    trace, step, tok, k = [], 0, 0, 0
    out = {"writes": 0, "wrong_writes": 0, "blocked": 0,
           "blocked_wrong": 0, "questions": 0, "written": None}
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace)
        target = g["sequence"][k] if k < len(g["sequence"]) else None
        if target is None:
            if r() < 0.88:
                break
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        side = TOOLS[choice]["side_effect"]
        if scope == "every_step" or (scope == "side_effect" and side):
            out["questions"] += 1
            tok += 35
            if side and not give_approval(g, state.get("result"), ri):
                out["blocked"] += 1
                out["blocked_wrong"] += is_wrong(g, state.get("result"))
                trace.append(choice)
                continue
        if choice == "write_record":
            state["value"] = state.get("result")
        res = call_tool(choice, state)
        trace.append(choice)
        tok += 30
        if res.get("written"):
            out["writes"] += 1
            out["written"] = res["value"]
            out["wrong_writes"] += is_wrong(g, state.get("value"))
        if "error" in res:
            continue
        state.update({key: val for key, val in res.items() if key != "error"})
        if choice == "consumption":
            state["result"] = res["m3"]
        if choice == "tariff":
            state["result"] = res["amount"]
        if choice == "search_docs":
            state["result"] = res["code"]
        if choice == target:
            k += 1
    out.update(steps=step, tok=tok, answer=state.get("result"))
    return out


print("task  kind      scope         questions  writes  wrong  blocked   written value")
for i in (8, 22):
    g = TASKS[i]
    print(f"-- {g['name']} correct answer {correct_answer(g)}")
    for name in ("off", "side_effect", "every_step"):
        s = approved_agent(g, 8, SEED, name)
        print(f"{g['name']:<6}{g['kind']:<10}{name:<13}{s['questions']:>10}{s['writes']:>8}"
              f"{s['wrong_writes']:>7}{s['blocked']:>9}   {s['written']}")

task  kind      scope         questions  writes  wrong  blocked   written value
-- G09 correct answer SY01
G09   document  off                   0       1      1        0   SY01
G09   document  side_effect           1       0      0        1   None
G09   document  every_step            2       0      0        1   None
-- G23 correct answer 1132
G23   record    off                   0       1      1        0   TR01
G23   record    side_effect           1       1      1        0   TR01
G23   record    every_step            8       1      1        0   TR01

The two blocks show two different wrongs. G09 is a document question; after finding the correct answer the agent wanders and calls write_record, leaving a record nobody asked for. The written value is right, what is wrong is the writing itself. The approval point stops this by rule: if the request does not ask for writing, the proposal is rejected and no calculation is needed.

G23 is the other one. The request asks for writing and there is a value, so both rule checks pass. But before writing, the agent wandered and called search_docs and overwrote the accumulated result with a document code: TR01 is written to the record instead of 1132. The rule check looks at the presence of the value, not its type; the only way to see it is recalculation, and the human does not recalculate in this run. The record is written in all three scopes. A rule violation is caught for free; a wrong value is caught only by recalculation.

The cost of the scope is in the questions column: in G23 asking at every step makes eight questions, the side-effect scope asks once, and the result is the same.

What the Approval Stops and What It Costs

Now the whole set is run in three scopes.

def run(seed, scope, max_steps=8):
    CALLS[0] = 0
    d = {"correct": 0, "steps": 0, "tok": 0, "questions": 0, "writes": 0,
         "wrong_writes": 0, "blocked": 0}
    for g in TASKS:
        s = approved_agent(g, max_steps, seed, scope)
        d["correct"] += s["answer"] == correct_answer(g)
        d["steps"] += s["steps"]
        d["tok"] += s["tok"]
        for a in ("questions", "writes", "wrong_writes", "blocked"):
            d[a] += s[a]
    d["calls"] = CALLS[0]
    return d


for t in (SEED, SEED + 1):
    print(f"\nseed {t}")
    print("scope        success  steps  calls    tokens  questions  writes  wrong  blocked")
    for scope in ("off", "side_effect", "every_step"):
        d = run(t, scope)
        print(f"{scope:<13}{d['correct']:>4}/24{d['steps']:>7}{d['calls']:>7}"
              f"{d['tok']:>10}{d['questions']:>11}{d['writes']:>8}"
              f"{d['wrong_writes']:>7}{d['blocked']:>9}")

seed 20260218
scope        success  steps  calls    tokens  questions  writes  wrong  blocked
off            23/24     97     74     11978          0       5      3        0
side_effect    23/24     97     70     12103          7       3      1        4
every_step     23/24     97     70     14448         74       3      1        4

seed 20260219
scope        success  steps  calls    tokens  questions  writes  wrong  blocked
off            23/24     98     74     11994          0       8      6        0
side_effect    23/24    100     68     12605         11       3      1        8
every_step     23/24    100     68     14880         76       3      1        8

The first row is the baseline: with approval off the set leaves five records and three are wrong; on the second run eight records, six wrong. In the same row task success is 23/24. Side by side, the two numbers make the lesson clear: task success does not measure the side effect. While giving the correct answer the agent leaves three wrong records in the environment and success does not see it.

When the side-effect scope is opened, the written records fall to three and the wrong records to one; on the second run also to three and one. The wrong writes blocked are 2 and 5 on the two runs, both above the measurement band. The cost is seven and eleven questions — 3.5 and 2.2 questions per blocked wrong write. Tokens go from 11978 to 12103, calls fall from 74 to 70.

The difference between the blocked column and the writes column matters. In the first run the approval point blocks four calls but the records it prevents are two; the other two calls would already have fallen into the tool’s error because there was no value. A blocked call is not a prevented record.

The third row is an example of the course’s second claim. Asking at every step does not change the written records, the wrong records and the blocked calls at all — all three columns are exactly the same as in the side-effect scope. The only thing that changes is the cost: questions from 7 to 74, tokens from 12103 to 14448. Ten times the questions, zero gain. The only kind of call that can be blocked is the one with a side effect; in the remaining sixty-seven questions the only answer the human can give is approval.

Summary

• A tool with a side effect has to be handled separately: write_record leaves a trace that cannot be undone the moment it is called.
• With approval off, the set leaves five records and three are wrong (eight and six on the second run) — but in the same run task success is 23/24. Task success does not measure the side effect.
• When approval is opened on the call with a side effect, the written records fall to three and the wrong records to one; the wrong writes blocked are 2 and 5 on the two runs, at a cost of seven and eleven questions.
• A rule violation is caught for free; a wrong value is caught only by recalculation. The number of blocked calls is also larger than the number of prevented records (four against two).
• Spreading the approval scope to every step does not change the blocked writes at all; it raises the questions from 7 to 74 and the tokens from 12103 to 14448.

Next Step

In seven lessons an agent was built: the loop, step depth, splitting the job, the stopping criterion, a round that looks at its own trace, a second agent and the approval point in front of the side effect. In each lesson the cost was written down and the places where the added step made the job worse were shown with numbers.

Under all these measurements there is a silent assumption: the tool was ready and correct. read_meter always returned the same fields, tariff gave a number even when it fell to its default, write_record wrote the moment it was asked. How the agent could select these tools was never asked: what the name, description and schema said, in what form the result was given back to the model, whether the place where the tool ran had a limit. The next topic looks at the tool itself and the first question is the definition: how much the name, description and schema of a tool move the valid call rate.