--- title: 'Keeping the Human in the Loop' source: 'https://academia.sh/en/courses/ai-agents/keeping-the-human-in-the-loop' course: 'AI Agents' language: en updated: '2026-10-21T05:02:26+00:00' license: 'CC BY-SA 4.0' objectives: - 'Places an approval point in front of a tool with a side effect and defines which checks the approval decision passes through.' - 'Compares the numbers of written records, wrong records, blocked calls and calls asked of the human with approval on and off.' - 'Shows that widening the approval scope from the call with a side effect to every step does not change the blocked writes and only raises the number of questions and tokens.' - 'Argues that task success does not measure the side effect and that the remaining records must be counted separately.' --- # Keeping the Human in the Loop Measuring the approval point placed in front of the tool with a side effect: with approval off, five records are written in twenty-four fictional tasks and three are wrong, and on the second run eight records are written and six are wrong. When approval is opened only on the call with a side effect, the written records fall to three and the wrong records to one; the cost is seven and eleven questions. When approval is spread to every step, the blocked writes do not change at all, but the questions rise to 74 and 76 and the tokens from 12103 to 14448. Task success stays fixed at 23/24 in all three scopes, so success does not measure the side effect. The number of blocked calls is larger than the number of prevented records, because some of the blocked calls would have failed anyway with a missing field. In the previous two lessons the agent looked at its own trace, then a second agent stepped in; in both, the last word stayed with the scaffolding. Of the five tools of the fictional environment, four **read** and one **writes**: `write_record` leaves a trace that cannot be undone and until now was treated the same as the other four. This lesson puts an **approval point** in front of that tool: how many records are written with approval on and off, how many of them are wrong and how many times the human is asked. Placing the stops according to the **step interval** was measured in the Building Applications with Model APIs course — stopping at every step gets 478 of 480 steps right, stopping once every eight steps gets 355. That procedure is not repeated; here the stop is placed not according to the interval but according to the **side effect**. - **AA53.** The setup is shared: the **fictional** subscriber records of the measurement network, the **fictional** document corpus, five **fictional** tools, twenty-four **fictional** tasks and an **agent simulator** written inside the lesson. No real model endpoint or tool server is called. - **AA54.** The baseline is the **loop without scaffolding**: approval off, a budget of eight steps. - **AA55.** A write is **wrong** in two cases: if the request does not ask for writing (**unintended side effect**) or if the written value is not the correct answer (**wrong value**). - **AA56.** At the approval point the proposed record is shown. Two checks are made by **rule** (does the request ask for writing, is there a value); the third is **recalculation** and is made with a **share of 0.60**. - **AA57.** A rejected call is **not executed at all**; the agent spends its step and continues. Each question also carries a **35 token** state summary. - **AA58.** The **side-effect scope** asks only before `write_record`; the **every-step scope** asks before every step but can still stop only the call with a side effect. - **AA59.** The second run has seed **20260219**; the difference between the two runs is **one task** in this set (0.0417) and every difference below it **counts as unmeasured**. ## Two Writes, Two Results First, two tasks are run in three scopes. `G09` is a document task and does not ask for writing, `G23` is a record task. ```python # COMMON FICTIONAL SETUP -- the 60 fictional subscribers of the measurement network, # 5 fictional documents, 5 fictional tools, 24 fictional tasks and the agent simulator # written inside the lesson. No real model endpoint, tool server or runtime is called. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 def tokenize(m): b = [] for s in str(m).lower().replace(",", " , ").split(): while len(s) > 5: b.append(s[:5]) s = s[5:] b.append(s) return b def trace_id(m): h = 2166136261 for c in str(m): h = ((h ^ ord(c)) * 16777619) & M32 return h # ---- FICTIONAL environment: records of the measurement network REGION = ["east", "west", "north", "south", "central"] TYPE = ["residential", "commercial"] SUBSCRIBERS = {} for i in range(60): r = rng(SEED + 31 * i) previous = 10000 + int(r() * 80000) SUBSCRIBERS[f"A{i + 1:03d}"] = {"region": REGION[int(r() * 5)], "type": TYPE[int(r() * 2)], "previous": previous, "current": previous + int(r() * 70)} TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16 COMMERCIAL_FACTOR = 2 DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu", "TR02": "isyeri abonesinde birim ucret iki katidir", "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir", "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider", "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"} CALLS = [0] # tool call counter def _charge(m3, type_): remaining, lower, total = m3, 0, 0 for upper, unit in TIERS: band = min(remaining, upper - lower) if band <= 0: break total += band * unit remaining -= band lower = upper total += remaining * ABOVE return total * (COMMERCIAL_FACTOR if type_ == "commercial" else 1) def t_read_meter(subscriber=None, **_): a = SUBSCRIBERS.get(subscriber) if a is None: return {"error": "no such subscriber"} return {"previous": a["previous"], "current": a["current"], "type": a["type"], "region": a["region"]} def t_consumption(previous=None, current=None, **_): if previous is None or current is None: return {"error": "missing field"} f = current - previous return {"m3": f + 100000 if f < 0 else f} def t_tariff(m3=None, type="residential", **_): if m3 is None: return {"error": "missing field"} return {"amount": _charge(m3, type)} def t_document(query=None, **_): if not query: return {"error": "missing field"} q = set(tokenize(query)) best, code = -1, None for k, m in DOCUMENTS.items(): o = len(q & set(tokenize(m))) if o > best: best, code = o, k return {"code": code, "text": DOCUMENTS[code]} if best > 0 else {"error": "not found"} def t_record(subscriber=None, column=None, value=None, **_): if not (subscriber and column and value is not None): return {"error": "missing field"} return {"written": True, "subscriber": subscriber, "column": column, "value": value} TOOLS = { "read_meter": {"function": t_read_meter, "fields": ("subscriber",), "side_effect": False}, "consumption": {"function": t_consumption, "fields": ("previous", "current"), "side_effect": False}, "tariff": {"function": t_tariff, "fields": ("m3", "type"), "side_effect": False}, "search_docs": {"function": t_document, "fields": ("query",), "side_effect": False}, "write_record": {"function": t_record, "fields": ("subscriber", "column", "value"), "side_effect": True}, } TOOL_NAMES = list(TOOLS) def call_tool(name, state): """Fields outside the schema are filtered out; a missing field is the tool's own error.""" CALLS[0] += 1 if name not in TOOLS: return {"error": "no such tool"} kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state} return TOOLS[name]["function"](**kw) # ---- FICTIONAL tasks: each one is solved by a known tool sequence TASKS = [] for i in range(24): r = rng(SEED + 9973 * i) sub = f"A{1 + int(r() * 60):03d}" t = pick(r(), [0.30, 0.22, 0.20, 0.28]) if t == 0: TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub, "request": f"{sub} abonesinin bu donem bedeli nedir", "sequence": ("read_meter", "consumption", "tariff")}) elif t == 1: TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub, "request": f"{sub} abonesinin bu donem tuketimi kac metrekup", "sequence": ("read_meter", "consumption")}) elif t == 2: TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub, "request": "sayac donusunde endeks farki nasil hesaplanir", "sequence": ("search_docs",)}) else: TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub, "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz", "sequence": ("read_meter", "consumption", "tariff", "write_record")}) def correct_answer(g): a = SUBSCRIBERS[g["subscriber"]] f = a["current"] - a["previous"] m3 = f + 100000 if f < 0 else f if g["kind"] == "consumption": return m3 if g["kind"] == "document": return "SY01" return _charge(m3, a["type"]) # ---- FICTIONAL agent: at each step it picks a tool or finishes def selection(d): return max(0.34, 0.80 - 0.09 * (d - 1)) # share of picking the right tool, falls with depth def agent(g, max_steps=8, seed=SEED, self_critique=False, approval=False): """FICTIONAL agent loop. No real model is called; tool selection comes from a single draw that depends on the task's sequence and the depth.""" r = rng(seed ^ trace_id(g["name"])) state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"} trace, step, tok, errors = [], 0, 0, 0 seq, k = g["sequence"], 0 while step < max_steps: step += 1 tok += 60 + 22 * len(trace) target = seq[k] if k < len(seq) else None if target is None: # the job is done , the agent should stop if r() < 0.88: return {"done": True, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("result"), "approvals": state.get("approvals", 0)} choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] if TOOLS[choice]["side_effect"] and approval: state["approvals"] = state.get("approvals", 0) + 1 if choice == "write_record": state["value"] = state.get("result") s = call_tool(choice, state) trace.append((choice, "error" in s)) tok += 30 if "error" in s: errors += 1 if self_critique: tok += 40 continue state.update({kk: vv for kk, vv in s.items() if kk != "error"}) if choice == "consumption": state["result"] = s["m3"] if choice == "tariff": state["result"] = s["amount"] if choice == "search_docs": state["result"] = s["code"] if choice == target: k += 1 return {"done": False, "steps": step, "trace": trace, "tokens": tok, "errors": errors, "answer": state.get("result"), "approvals": state.get("approvals", 0)} # ---- Part added in this lesson: approval point in front of the tool with a side effect RECHECK_SHARE = 0.60 def is_wrong(g, value): """A write is wrong: the request does not ask for writing or the value is not the correct answer.""" return "yaz" not in g["request"] or value != correct_answer(g) def give_approval(g, value, r): """At the approval point the proposed record is shown. Two checks are made by rule, the third by recalculation; a human who does not recalculate approves.""" if "yaz" not in g["request"]: return False if value is None: return False if r() < RECHECK_SHARE: return value == correct_answer(g) return True def approved_agent(g, max_steps, seed, scope="off"): """Shared loop. If scope is 'side_effect' the human is asked only before the call with a side effect, if 'every_step' before every step; blocking happens only on the side effect.""" r = rng(seed ^ trace_id(g["name"])) ri = rng(seed ^ trace_id("insan|" + g["name"])) state = {"subscriber": g["subscriber"], "query": g["request"], "column": "charge"} trace, step, tok, k = [], 0, 0, 0 out = {"writes": 0, "wrong_writes": 0, "blocked": 0, "blocked_wrong": 0, "questions": 0, "written": None} while step < max_steps: step += 1 tok += 60 + 22 * len(trace) target = g["sequence"][k] if k < len(g["sequence"]) else None if target is None: if r() < 0.88: break choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))] else: choice = target if r() < selection(k + 1) else TOOL_NAMES[int(r() * len(TOOL_NAMES))] side = TOOLS[choice]["side_effect"] if scope == "every_step" or (scope == "side_effect" and side): out["questions"] += 1 tok += 35 if side and not give_approval(g, state.get("result"), ri): out["blocked"] += 1 out["blocked_wrong"] += is_wrong(g, state.get("result")) trace.append(choice) continue if choice == "write_record": state["value"] = state.get("result") res = call_tool(choice, state) trace.append(choice) tok += 30 if res.get("written"): out["writes"] += 1 out["written"] = res["value"] out["wrong_writes"] += is_wrong(g, state.get("value")) if "error" in res: continue state.update({key: val for key, val in res.items() if key != "error"}) if choice == "consumption": state["result"] = res["m3"] if choice == "tariff": state["result"] = res["amount"] if choice == "search_docs": state["result"] = res["code"] if choice == target: k += 1 out.update(steps=step, tok=tok, answer=state.get("result")) return out print("task kind scope questions writes wrong blocked written value") for i in (8, 22): g = TASKS[i] print(f"-- {g['name']} correct answer {correct_answer(g)}") for name in ("off", "side_effect", "every_step"): s = approved_agent(g, 8, SEED, name) print(f"{g['name']:<6}{g['kind']:<10}{name:<13}{s['questions']:>10}{s['writes']:>8}" f"{s['wrong_writes']:>7}{s['blocked']:>9} {s['written']}") ``` ``` task kind scope questions writes wrong blocked written value -- G09 correct answer SY01 G09 document off 0 1 1 0 SY01 G09 document side_effect 1 0 0 1 None G09 document every_step 2 0 0 1 None -- G23 correct answer 1132 G23 record off 0 1 1 0 TR01 G23 record side_effect 1 1 1 0 TR01 G23 record every_step 8 1 1 0 TR01 ``` The two blocks show two different wrongs. `G09` is a document question; after finding the correct answer the agent wanders and calls `write_record`, leaving **a record nobody asked for**. The written value is right, what is wrong is the writing itself. The approval point stops this by **rule**: if the request does not ask for writing, the proposal is rejected and no calculation is needed. `G23` is the other one. The request asks for writing and there is a value, so both rule checks pass. But before writing, the agent wandered and called `search_docs` and overwrote the accumulated result with a document code: `TR01` is written to the record instead of 1132. The rule check looks at the **presence** of the value, not its **type**; the only way to see it is recalculation, and the human does not recalculate in this run. The record is written in all three scopes. **A rule violation is caught for free; a wrong value is caught only by recalculation.** The cost of the scope is in the questions column: in `G23` asking at every step makes **eight questions**, the side-effect scope asks **once**, and the result is the same. ## What the Approval Stops and What It Costs Now the whole set is run in three scopes. ```python def run(seed, scope, max_steps=8): CALLS[0] = 0 d = {"correct": 0, "steps": 0, "tok": 0, "questions": 0, "writes": 0, "wrong_writes": 0, "blocked": 0} for g in TASKS: s = approved_agent(g, max_steps, seed, scope) d["correct"] += s["answer"] == correct_answer(g) d["steps"] += s["steps"] d["tok"] += s["tok"] for a in ("questions", "writes", "wrong_writes", "blocked"): d[a] += s[a] d["calls"] = CALLS[0] return d for t in (SEED, SEED + 1): print(f"\nseed {t}") print("scope success steps calls tokens questions writes wrong blocked") for scope in ("off", "side_effect", "every_step"): d = run(t, scope) print(f"{scope:<13}{d['correct']:>4}/24{d['steps']:>7}{d['calls']:>7}" f"{d['tok']:>10}{d['questions']:>11}{d['writes']:>8}" f"{d['wrong_writes']:>7}{d['blocked']:>9}") ``` ``` seed 20260218 scope success steps calls tokens questions writes wrong blocked off 23/24 97 74 11978 0 5 3 0 side_effect 23/24 97 70 12103 7 3 1 4 every_step 23/24 97 70 14448 74 3 1 4 seed 20260219 scope success steps calls tokens questions writes wrong blocked off 23/24 98 74 11994 0 8 6 0 side_effect 23/24 100 68 12605 11 3 1 8 every_step 23/24 100 68 14880 76 3 1 8 ``` The first row is the baseline: with approval off the set leaves **five records** and **three are wrong**; on the second run eight records, six wrong. In the same row task success is **23/24**. Side by side, the two numbers make the lesson clear: **task success does not measure the side effect.** While giving the correct answer the agent leaves three wrong records in the environment and success does not see it. When the side-effect scope is opened, the written records fall to **three** and the wrong records to **one**; on the second run also to three and one. The wrong writes blocked are **2** and **5** on the two runs, both above the measurement band. The cost is seven and eleven questions — **3.5** and **2.2** questions per blocked wrong write. Tokens go from 11978 to 12103, calls fall from 74 to 70. The difference between the blocked column and the writes column matters. In the first run the approval point blocks **four** calls but the records it prevents are **two**; the other two calls would already have fallen into the tool's error because there was no value. **A blocked call is not a prevented record.** The third row is an example of the course's second claim. Asking at every step does **not change** the written records, the wrong records and the blocked calls at all — all three columns are exactly the same as in the side-effect scope. The only thing that changes is the cost: questions from 7 to **74**, tokens from 12103 to **14448**. Ten times the questions, zero gain. The only kind of call that can be blocked is the one with a side effect; in the remaining sixty-seven questions the only answer the human can give is approval. ## Summary - A tool with a side effect has to be handled separately: `write_record` leaves a trace that cannot be undone the moment it is called. - With approval off, the set leaves five records and three are wrong (eight and six on the second run) — but in the same run task success is 23/24. **Task success does not measure the side effect.** - When approval is opened on the call with a side effect, the written records fall to three and the wrong records to one; the wrong writes blocked are 2 and 5 on the two runs, at a cost of seven and eleven questions. - A rule violation is caught for free; a wrong value is caught only by **recalculation**. The number of blocked calls is also larger than the number of prevented records (four against two). - Spreading the approval scope to every step does not change the blocked writes at all; it raises the questions from 7 to 74 and the tokens from 12103 to 14448. ## Next Step In seven lessons an agent was built: the loop, step depth, splitting the job, the stopping criterion, a round that looks at its own trace, a second agent and the approval point in front of the side effect. In each lesson the cost was written down and the places where the added step made the job worse were shown with numbers. Under all these measurements there is a silent assumption: **the tool was ready and correct.** `read_meter` always returned the same fields, `tariff` gave a number even when it fell to its default, `write_record` wrote the moment it was asked. How the agent **could select** these tools was never asked: what the name, description and schema said, in what form the result was given back to the model, whether the place where the tool ran had a limit. The next topic looks at **the tool itself** and the first question is the definition: how much the name, description and schema of a tool move the valid call rate.