Tool Definition

The previous seven lessons built the agent loop: perception, reasoning, the sequence of action and observation, the separation of the planner and the executor, the step budget, the stopping criterion, the self-critique round, the multi-agent arrangement, and the approval point on a tool with a side effect. One thing was silently assumed throughout: the tool was ready and correct.

This lesson removes that assumption: what does a tool’s definition add to the loop? The Building Applications with Model APIs course measured the form of a single call: with no catalog given the valid call rate was 0.0500, and with names and parameters given together it was 0.6917. That measurement was about parsing a text and is not repeated; the numbers are taken as input. The call here crosses a function boundary, and what is measured is what the definition adds across the loop.

• TM1. The environment, the tool set, the task set, and the agent loop are fictional; no real model endpoint, tool server, or runtime is called. 60 fictional subscribers, 5 fictional documents, 5 tools, and 24 tasks each solved by a known tool sequence. Seed 20260218, second run 20260219.
• TM2. The tool definition is given at four levels: name only; name and description; name, description, and schema; schema and example call. Each level includes the one before it.
• TM3. The definition determines two things: the multiplier on the share of picking the right tool and the fill share of each declared field. The name is a weak hint; the description says what the tool does, and the schema says which fields it asks for.
• TM4. The schema gives the field names; it does not say which value a field carries. Two fields of the same type (onceki (previous) and simdiki (current)) swap with a share of 0.30; the example call zeroes this share.
• TM5. A tool error is an error the tool itself returns: a missing field or a nonexistent tool. Fields outside the schema are filtered at the call boundary; a valid call is one that returns no error.
• TM6. The catalog stays in the prompt at every step; its cost is the token count of the level.
• TM7. The baseline is the name only level; this is the loop without scaffolding. The step budget is eight at all four levels.
• TM8. Task success is the result the loop leaves being equal to the correct answer; it does not depend on whether the agent stopped on its own.

The Four Levels of Definition

The code below builds the environment, the tool set, the task set, and the agent loop, then adds the definition layer on top of the loop. The call boundary still filters fields outside the schema, but whether a declared field is filled now depends on the level of the definition. The tool names in the code are the read_meter, consumption, tariff, search_docs, and write_record tools of the course’s earlier lessons; they are counted in tokens here, so they stay Turkish.

SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def trace_id(m):
    h = 2166136261
    for c in str(m):
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


# ---- FICTIONAL environment: records of the measurement network
# wire words stay Turkish: they are counted in tokens (same rule as the MCP course)
# onceki/simdiki/tip/bolge = previous/current/type/region; mesken/isyeri = residential/commercial
REGION = ["dogu", "bati", "kuzey", "guney", "merkez"]
TYPE = ["mesken", "isyeri"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"bolge": REGION[int(r() * 5)], "tip": TYPE[int(r() * 2)],
                                    "onceki": previous, "simdiki": previous + int(r() * 70)}
TIERS, ABOVE = ((10, 4), (25, 7), (40, 11)), 16
COMMERCIAL_FACTOR = 2
DOCUMENTS = {"TR01": "tarife basamaklari 10 25 40 metrekup ve ustu",
             "TR02": "isyeri abonesinde birim ucret iki katidir",
             "SY01": "endeks farki tuketimi verir sayac donusu 100000 eklenir",
             "AR01": "ariza kaydi acildiginda saha ekibi iki gun icinde gider",
             "KS01": "planli kesinti duyurusu bir hafta once yayimlanir"}

CALLS = [0]                              # tool call counter


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    total += remaining * ABOVE
    return total * (COMMERCIAL_FACTOR if type_ == "isyeri" else 1)


def t_read_meter(abone=None, **_):
    a = SUBSCRIBERS.get(abone)
    if a is None:
        return {"hata": "abone yok"}
    return {"onceki": a["onceki"], "simdiki": a["simdiki"], "tip": a["tip"],
            "bolge": a["bolge"]}


def t_consumption(onceki=None, simdiki=None, **_):
    if onceki is None or simdiki is None:
        return {"hata": "eksik alan"}
    f = simdiki - onceki
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, tip="mesken", **_):
    if m3 is None:
        return {"hata": "eksik alan"}
    return {"tutar": _charge(m3, tip)}


def t_document(anahtar=None, **_):
    if not anahtar:
        return {"hata": "eksik alan"}
    q = set(tokenize(anahtar))
    best, code = -1, None
    for k, m in DOCUMENTS.items():
        o = len(q & set(tokenize(m)))
        if o > best:
            best, code = o, k
    return {"kod": code, "metin": DOCUMENTS[code]} if best > 0 else {"hata": "bulunamadi"}


def t_record(abone=None, sutun=None, deger=None, **_):
    if not (abone and sutun and deger is not None):
        return {"hata": "eksik alan"}
    return {"yazildi": True, "abone": abone, "sutun": sutun, "deger": deger}


TOOLS = {
    "sayac_oku":  {"function": t_read_meter, "fields": ("abone",), "side_effect": False},
    "tuketim":    {"function": t_consumption, "fields": ("onceki", "simdiki"),
                   "side_effect": False},
    "tarife":     {"function": t_tariff, "fields": ("m3", "tip"), "side_effect": False},
    "belge_ara":  {"function": t_document, "fields": ("anahtar",), "side_effect": False},
    "kayit_yaz":  {"function": t_record, "fields": ("abone", "sutun", "deger"),
                   "side_effect": True},
}
TOOL_NAMES = list(TOOLS)

# ---- FICTIONAL tasks: each one is solved by a known tool sequence
TASKS = []
for i in range(24):
    r = rng(SEED + 9973 * i)
    sub = f"A{1 + int(r() * 60):03d}"
    t = pick(r(), [0.30, 0.22, 0.20, 0.28])
    if t == 0:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "charge", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem bedeli nedir",
                      "sequence": ("sayac_oku", "tuketim", "tarife")})
    elif t == 1:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "consumption", "subscriber": sub,
                      "request": f"{sub} abonesinin bu donem tuketimi kac metrekup",
                      "sequence": ("sayac_oku", "tuketim")})
    elif t == 2:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "document", "subscriber": sub,
                      "request": "sayac donusunde endeks farki nasil hesaplanir",
                      "sequence": ("belge_ara",)})
    else:
        TASKS.append({"name": f"G{i + 1:02d}", "kind": "record", "subscriber": sub,
                      "request": f"{sub} abonesinin bedelini hesapla ve kayda yaz",
                      "sequence": ("sayac_oku", "tuketim", "tarife", "kayit_yaz")})


def correct_answer(g):
    a = SUBSCRIBERS[g["subscriber"]]
    f = a["simdiki"] - a["onceki"]
    m3 = f + 100000 if f < 0 else f
    if g["kind"] == "consumption":
        return m3
    if g["kind"] == "document":
        return "SY01"
    return _charge(m3, a["tip"])


# ---- FICTIONAL agent: at each step it picks a tool or finishes
def selection(d):
    return max(0.34, 0.80 - 0.09 * (d - 1))       # share of picking the right tool, falls with depth


DESCRIPTION = {
    "sayac_oku": "abonenin onceki ve simdiki sayac endeksini dondurur",
    "tuketim":   "iki endeks arasindaki farki metrekup olarak verir",
    "tarife":    "metrekup ve abone tipinden donem bedelini hesaplar",
    "belge_ara": "anahtar metne en yakin kurgu belgeyi getirir",
    "kayit_yaz": "bir abonenin bir sutununa deger yazar ; yan etkilidir"}
EXAMPLE = {"sayac_oku": "sayac_oku ( abone = A007 )",
           "tuketim":   "tuketim ( onceki = 41200 , simdiki = 41255 )",
           "tarife":    "tarife ( m3 = 55 , tip = mesken )",
           "belge_ara": "belge_ara ( anahtar = endeks farki )",
           "kayit_yaz": "kayit_yaz ( abone = A007 , sutun = bedel , deger = 470 )"}
# level : name , selection factor , field fill share , swap share
DEFINITION = {1: ("name only", 0.62, 0.45, 0.30),
              2: ("name + description", 0.86, 0.72, 0.30),
              3: ("name + description + schema", 1.00, 1.00, 0.30),
              4: ("schema + example call", 1.00, 1.00, 0.00)}


def catalog(level, name):
    s = name
    if level >= 2:
        s += " : " + DESCRIPTION[name]
    if level >= 3:
        s += " ( " + " , ".join(TOOLS[name]["fields"]) + " )"
    if level >= 4:
        s += " ornek : " + EXAMPLE[name]
    return s


CATALOG_TOKENS = {d: sum(len(tokenize(catalog(d, a))) for a in TOOLS) for d in DEFINITION}


def call_defined(name, state, level, rd):
    """The defined variant of the shared call boundary: whether a declared
    field is filled depends on the level of the definition."""
    CALLS[0] += 1
    if name not in TOOLS:
        return {"hata": "arac yok"}
    _, _, share, swap = DEFINITION[level]
    kw = {a: state[a] for a in TOOLS[name]["fields"] if a in state and rd() < share}
    if {"onceki", "simdiki"} <= set(kw) and rd() < swap:
        kw["onceki"], kw["simdiki"] = kw["simdiki"], kw["onceki"]
    return TOOLS[name]["function"](**kw)


def agent_defined(g, level=4, max_steps=8, seed=SEED):
    """The shared agent loop; the only differences are that the call passes
    through the definition layer and the catalog stays in the prompt at every step."""
    r = rng(seed ^ trace_id(g["name"]))
    rd = rng(seed ^ trace_id(g["name"] + "tanim"))
    factor = DEFINITION[level][1]
    state = {"abone": g["subscriber"], "anahtar": g["request"], "sutun": "bedel"}
    trace, step, tok, errors, foreign = [], 0, 0, 0, 0
    seq, k = g["sequence"], 0
    while step < max_steps:
        step += 1
        tok += 60 + 22 * len(trace) + CATALOG_TOKENS[level]
        target = seq[k] if k < len(seq) else None
        if target is None:
            if r() < 0.88:
                break
            choice = TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        else:
            choice = target if r() < selection(k + 1) * factor else TOOL_NAMES[int(r() * len(TOOL_NAMES))]
        if choice == "kayit_yaz":
            state["deger"] = state.get("sonuc")
        s = call_defined(choice, state, level, rd)
        trace.append((choice, "hata" in s))
        tok += 30
        if "hata" in s:
            errors += 1
            continue
        foreign += choice not in seq
        state.update({kk: vv for kk, vv in s.items() if kk != "hata"})
        if choice == "tuketim":
            state["sonuc"] = s["m3"]
        if choice == "tarife":
            state["sonuc"] = s["tutar"]
        if choice == "belge_ara":
            state["sonuc"] = s["kod"]
        if choice == target:
            k += 1
    return {"steps": step, "tokens": tok, "errors": errors, "foreign": foreign,
            "answer": state.get("sonuc")}


for d in DEFINITION:
    print(f"level {d}  {DEFINITION[d][0]:<27} catalog {CATALOG_TOKENS[d]:>3} tokens")
print("  level 3 line   :", catalog(3, "tuketim"))
print("  level 4 suffix : ornek :", EXAMPLE["tuketim"])
g = TASKS[2]
for d in (3, 4):
    s = agent_defined(g, d)
    print(f"  {g['name']} {g['kind']:<11} level {d}  answer {s['answer']:>6}  correct"
          f" {correct_answer(g)}  tool errors {s['errors']}")

level 1  name only                   catalog  10 tokens
level 2  name + description          catalog  71 tokens
level 3  name + description + schema catalog  97 tokens
level 4  schema + example call       catalog 164 tokens
  level 3 line   : tuketim : iki endeks arasindaki farki metrekup olarak verir ( onceki , simdiki )
  level 4 suffix : ornek : tuketim ( onceki = 41200 , simdiki = 41255 )
  G03 consumption level 3  answer  99948  correct 52  tool errors 1
  G03 consumption level 4  answer     52  correct 52  tool errors 1

The last two lines are the core of this lesson. The same task, the same seed, the same tool sequence, and the same number of tool errors. At the third level the agent called the tuketim (consumption) tool with the two fields the schema declares, the call passed the schema, the tool returned no error, and the result came out 99948 instead of 52. The schema declares the names onceki and simdiki; it does not declare which one is smaller. When the fields swap, the difference goes negative, the meter rollover rule kicks in, and the tool returns a consumption close to a hundred thousand. This call is valid and wrong.

The Numbers of the Four Levels

The same 24 fictional tasks run at four definition levels; task success is printed separately for the two seeds.

def measure(level, seed=SEED):
    CALLS[0] = 0
    ok = steps = tok = errors = foreign = 0
    for g in TASKS:
        s = agent_defined(g, level, seed=seed)
        ok += s["answer"] == correct_answer(g)
        steps += s["steps"]
        tok += s["tokens"]
        errors += s["errors"]
        foreign += s["foreign"]
    return ok / len(TASKS), CALLS[0], errors, foreign, steps, tok


print("definition level            calls errors   valid  task success A  task success B"
      "  steps  tokens  tokens per answer")
for d in DEFINITION:
    a, b = measure(d), measure(d, SEED + 1)
    print(f"{DEFINITION[d][0]:<27} {a[1]:>5} {a[2]:>6}  {(a[1] - a[2]) / a[1]:>6.4f}"
          f"  {a[0]:>14.4f}  {b[0]:>14.4f}  {a[4]:>5}  {a[5]:>6}  {a[5] / (a[0] * 24):>17.0f}")
a = measure(4)
print(f"highest level : {a[3]} of {a[1] - a[2]} valid calls call a tool"
      f" outside the task's sequence")

definition level            calls errors   valid  task success A  task success B  steps  tokens  tokens per answer
name only                     157    113  0.2803          0.3333          0.4167    167   27950               3494
name + description            111     56  0.4955          0.6667          0.5833    129   27489               1718
name + description + schema    74      8  0.8919          0.7917          0.7917     97   21387               1126
schema + example call          74      8  0.8919          0.9583          0.9583     97   27886               1212
highest level : 8 of 66 valid calls call a tool outside the task's sequence

The baseline is the first row: when only the name is known, 113 of 157 calls come back as tool errors, the valid call rate is 0.2803, and task success is 0.3333. Most of the errors are missing fields. The name does not say what the tool wants; the agent cannot know which key to pass from the state it holds. The cost shows in the steps column: although the budget is eight, the total step count is 167, more than seven per task. A failed call spends a step and does not advance the sequence.

When the description is added, the errors fall to 56, the calls to 111, and task success becomes 0.6667. The description does two jobs: it raises the share of picking the right tool and it hints at what information the fields ask for. When the schema is added, the calls fall to 74, the errors to 8, the valid rate becomes 0.8919, and the steps fall to 97. The remaining eight errors are unrelated to the schema: a wrongly chosen tool fails its own precondition.

What the Schema Does Not Say

The third and fourth rows carry the same calls, the same errors, and the same steps; their task success separates as 0.7917 and 0.9583. The 0.1666 between them is what the example call adds, and it arrives without adding a single step to the loop. The run-to-run difference is 0.0834 at the first two levels and zero at the last two; 0.1666 is above twice this band and counts as measured. The example call supplies the one piece of information the schema cannot: which value the field carries.

The cost is in the tokens column. The catalog grows from 97 tokens to 164 tokens, and because it stays in the prompt at every step, the total tokens rise from 21387 to 27886. Tokens per correct answer rise from 1126 to 1212: while the example call raises task success, at this scale it raises the unit cost as well. A scaffolding whose cost is not written down counts as unmeasured; the cost here is sixty-seven tokens.

The last line is the outermost limit of the schema. Even at the highest level, eight of the 66 valid calls call a tool that is not in the task’s sequence. In the Building Applications with Model APIs course the same residue was fourteen wrong tool names; there the call was a text, here it crossed a function boundary and the tool ran correctly. A definition says what can be called; it does not say what should be called.

Summary

• The tool definition splits into four levels: name only, name and description, schema, and schema and example call.
• When only the name is given, 113 of 157 calls come back as tool errors, the valid rate is 0.2803, and task success is 0.3333; a failed call spends a step and raises the total steps to 167.
• The description brings the errors down to 56 and the schema to 8; with the schema the calls fall to 74, the steps to 97, and the valid rate becomes 0.8919.
• The schema gives the field names and not which value a field carries: two swapped indexes return 99948 instead of 52 in a call that passes the schema.
• The example call raises task success from 0.7917 to 0.9583 and changes the steps, the calls, and the errors not at all; its cost is the catalog growing to 164 tokens and the tokens per correct answer rising from 1126 to 1212.
• Even at the highest level, eight of the 66 valid calls call a tool outside the task’s sequence; being valid is not being correct.

Next Step

In this lesson the tool was called and the dictionary it returned was written to the state as it was; every field was carried to the next step and stayed in the prompt again at every step. This is one reason the tokens column rises above a thousand tokens per task. Carrying the result as it is is not the only option: it can be truncated, it can be summarized, or only the needed field can be kept. Each lowers the tokens, and each risks dropping something. The next lesson measures in which form the result is given back to the model and prints task success side by side with the tokens that accumulate across the loop.