Reason and Act

The first four lessons of this topic asked for the same thing in four different ways: let the model take more steps inside itself before giving the response. The steps became a chain, were duplicated, branched and abstracted; in all four, everything happened inside the model. Yet one of the steps that solve a ticket is not thinking but looking outside — searching a document, fetching a past record, opening a new record. This lesson measures how that step is asked for in the prompt: when a thought field and an action field are requested together in the same prompt, how many responses produce a valid action field, and what it costs.

What is measured is not the action itself. No action is run in this lesson; the tool scaffolding, the memory and the loop are left to this curriculum’s AI Agents course.

• RT41. The code does not call a real model endpoint; it calls a language model simulator written with the standard library. The support tickets are fictional too: 120 records, five topics.
• RT42. Seed 20260218. The same prompt gives the same response on the same seed; uncertainty is measured by changing the seed, and the second run is 20260219.
• RT43. The action names are fictional and there are three of them; none is run, the response text is read and left there.
• RT44. Three numbers are counted separately: label accuracy, the share of responses carrying an action field and the share of valid action fields. To count as valid, the name must come from the closed set and the argument must be closed with a parenthesis.
• RT45. The simulator’s format branch depends on the prompt: when an action field is requested, field patterns are produced; when only a thought is requested, thought patterns are produced; all the thought patterns carry a label.
• RT46. Writing the action names into the prompt changes the pattern weights. It is the action-set counterpart of writing the label set into the prompt.
• RT47. The largest output is 24 tokens. When a long thought and the action field do not fit into the same response, the stop condition cuts the field in the middle and the action name is corrupted in the truncated response.
• RT48. The four prompts differ only in text. The model, the task, the set and the seed are fixed.

Two Fields in the Same Prompt

The task did not change: each of 120 fictional tickets will be assigned to one of five topics. The only thing that changes is the text of the prompt. P1 is where this topic left off: think first, then write the label. P2 adds an action field to the same prompt. P3 states the action set explicitly. P4 adds one more sentence to this: keep the thought short.

# SIMULATOR. No real model endpoint is called; it is a language model simulator written with the
# standard library. The support tickets and the action set are FICTIONAL; no action runs.
import math

SEED, M32 = 20260218, 0xFFFFFFFF
WINDOW, MAX_OUT_TOKENS = 240, 24


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(text):
    out = []
    for piece in text.lower().replace(",", " , ").split():
        while len(piece) > 5:
            out.append(piece[:5])
            piece = piece[5:]
        out.append(piece)
    return out


def trace_id(text):
    h = 2166136261
    for c in text:
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


TOPICS = ["fatura", "ariza", "sayac", "tarife", "kesinti"]
PATTERNS = {
    "fatura": ["bu ayki fatura tutari gecen aya gore cok yuksek geldi",
               "faturami odedim ama sistemde borc gorunuyor",
               "fatura ayrintisini nereden gorebilirim"],
    "ariza": ["evde su akmiyor bir ariza var galiba",
              "baglanti bozuk oldu ve hic calismiyor",
              "ariza kaydi actim ama ekip gelmedi"],
    "sayac": ["sayac okumasi yanlis girilmis endeks tutmuyor",
              "sayac degisimi icin randevu istiyorum",
              "sayac uzerindeki endeks ile fatura ayni degil"],
    "tarife": ["hangi tarife basamagindayim ogrenmek istiyorum",
               "tarife degisikligi icin ne yapmam gerekiyor",
               "birim ucret abone tipine gore degisiyor mu"],
    "kesinti": ["mahallede kesinti var ne zaman gelecek",
                "planli kesinti duyurusu nereden takip edilir",
                "aksamdan beri kesinti yasiyoruz"],
}
EXTRAS = ["ayrica gecen ay da ayni sorun olmustu", "bu konuda memnun degilim",
          "acil bilgi almam gerekiyor", "kayit numaram elimde yok"]

TICKETS = []
for i in range(120):
    r = rng(SEED + i)
    k = TOPICS[pick(r(), [0.24, 0.22, 0.18, 0.20, 0.16])]
    text = PATTERNS[k][int(r() * 3)]
    if r() < 0.34:
        text += " " + EXTRAS[int(r() * 4)]
    TICKETS.append({"id": 4001 + i, "text": text, "topic": k})

CONCEPTS = TOPICS + ["odeme", "sikayet", "bilgi", "genel"]
CUE = {"fatur": "fatura", "tutar": "fatura", "ayrin": "fatura",
       "ariza": "ariza", "bozuk": "ariza", "calis": "ariza", "akmiy": "ariza",
       "sayac": "sayac", "endek": "sayac", "okuma": "sayac",
       "tarif": "tarife", "birim": "tarife", "abone": "tarife", "basam": "tarife",
       "kesin": "kesinti", "duyur": "kesinti", "mahal": "kesinti",
       "borc": "odeme", "odedi": "odeme", "ucret": "odeme",
       "memnu": "sikayet", "sorun": "sikayet", "acil": "sikayet",
       "bilgi": "bilgi", "ogren": "bilgi", "nered": "bilgi"}
TRAP = {"degis": "tarife", "kayit": "ariza", "yukse": "tarife", "gorun": "fatura",
        "ekip": "kesinti", "takip": "tarife", "giril": "fatura"}
FREE_FORM = ["talebiniz {k} konusuyla ilgili gorunuyor",
             "bu kayit {k} basligina girer , ilgili ekibe yonlendirildi",
             "kaydiniz alindi , konuyu inceleyip donus yapacagiz"]
COUNTER = [0]

ACTIONS = ["belge_ara", "gecmis_getir", "kayit_ac"]      # FICTIONAL action names
THOUGHTS = ["dusunce : metinde {k} isaretleri var | etiket : {k}",
            "dusunce : once konu belirlendi sonra etiket yazildi | etiket : {k}",
            "dusunce : kaydiniz alindi , {k} basligina girer"]
FIELDS = ["dusunce : metinde {k} isaretleri var ve gecmis kayit kontrol edilmeli "
          "| eylem : belge_ara ( {k} )",
          "dusunce : {k} konusunda once gecmis kayit gerekli | eylem : gecmis_getir ( {k} )",
          "dusunce : bu kayit {k} basligina girer , dogrudan yanit yazilabilir"]
FIELDS_SHORT = ["dusunce : {k} isareti var | eylem : belge_ara ( {k} )",
                "dusunce : gecmis gerekli | eylem : gecmis_getir ( {k} )",
                "dusunce : {k} basligina girer"]


def call(prompt, seed=SEED, temperature=0.6, max_out=MAX_OUT_TOKENS):
    COUNTER[0] += 1
    toks = tokenize(prompt)
    if len(toks) > WINDOW:
        return {"status": "context_overflow", "input": len(toks), "output": 0,
                "response": "", "truncated": 0}
    if COUNTER[0] % 37 == 0:
        return {"status": "rate_limit", "input": len(toks), "output": 0,
                "response": "", "truncated": 0}
    r = rng(seed ^ trace_id(prompt))
    if len(toks) > 90 and r() < 0.04:
        return {"status": "timeout", "input": len(toks), "output": 0,
                "response": "", "truncated": 0}
    candidates = [k for k in CONCEPTS if k[:5] in toks]
    if len(candidates) < 3:
        candidates = CONCEPTS
    p = {k: 0.15 for k in candidates}
    for t in toks:
        for table, w in ((CUE, 1.0), (TRAP, 1.3)):
            if table.get(t) in p:
                p[table[t]] += w
    if temperature <= 0.0:
        k = max(sorted(p), key=lambda a: p[a])
    else:
        e = [math.exp(p[a] / temperature) for a in sorted(p)]
        k = sorted(p)[pick(r(), [v / sum(e) for v in e])]
    if "eylem" in toks:                                  # thought + action field
        explicit = all(a[:5] in toks for a in ACTIONS)   # is the action set written
        w = [0.46, 0.38, 0.16] if explicit else [0.30, 0.22, 0.48]
        y = (FIELDS_SHORT if "kisa" in toks else FIELDS)[pick(r(), w)].format(k=k)
    elif "dusun" in toks:                               # thought field only
        y = THOUGHTS[pick(r(), [0.42, 0.26, 0.32])].format(k=k)
    elif "tek" in toks:
        y = k
    else:
        y = FREE_FORM[pick(r(), [0.42, 0.26, 0.32])].format(k=k)
    out_toks, cut = tokenize(y), 0
    if len(out_toks) > max_out:                          # stop condition
        out_toks, cut = out_toks[:max_out], 1
        y = " ".join(out_toks)
    return {"status": "done", "input": len(toks), "output": len(out_toks),
            "response": y, "truncated": cut}


def read_label(y):
    for s in y.lower().split():
        if s in TOPICS:
            return s
    return None


def read_action(y):                   # the action field is read, no action is run
    if "eylem :" not in y:
        return None
    p = y.split("eylem :", 1)[1].split()
    if len(p) < 3 or p[0] not in ACTIONS or p[1] != "(" or ")" not in p[2:]:
        return None
    return p[0]


LABEL_PROMPT = "Talebi su etiketlerden birine ata: fatura , ariza , sayac , tarife , kesinti . "
ACT_PROMPT = "Once dusun , sonra yapilacak eylemi yaz . "
ACTION_SET = "Eylem su kumeden secilir: belge_ara , gecmis_getir , kayit_ac . "
P1 = LABEL_PROMPT + "Once dusun , sonra etiketi yaz . Talep: "
P2 = LABEL_PROMPT + ACT_PROMPT + "Talep: "
P3 = LABEL_PROMPT + ACT_PROMPT + ACTION_SET + "Talep: "
P4 = LABEL_PROMPT + ACT_PROMPT + ACTION_SET + "Dusunceyi kisa tut . Talep: "

t = TICKETS[6]
print(f"FICTIONAL ticket {t['id']} ({t['topic']}): {t['text']}")
for name, prefix in (("P1 thought only", P1), ("P2 action field", P2),
                     ("P3 set written", P3), ("P4 thought short", P4)):
    s = call(prefix + t["text"])
    print(f"\n{name}: input {s['input']} tok, output {s['output']} tok, "
          f"truncated {s['truncated']}, label {read_label(s['response'])}, "
          f"valid action {read_action(s['response'])}")
    print(f"  {s['response']}")

FICTIONAL ticket 4007 (sayac): sayac uzerindeki endeks ile fatura ayni degil

P1 thought only: input 42 tok, output 14 tok, truncated 0, label sayac, valid action None
  dusunce : metinde sayac isaretleri var | etiket : sayac

P2 action field: input 44 tok, output 16 tok, truncated 0, label sayac, valid action None
  dusunce : bu kayit sayac basligina girer , dogrudan yanit yazilabilir

P3 set written: input 60 tok, output 16 tok, truncated 0, label ariza, valid action None
  dusunce : bu kayit ariza basligina girer , dogrudan yanit yazilabilir

P4 thought short: input 65 tok, output 16 tok, truncated 0, label sayac, valid action gecmis_getir
  dusunce : gecmis gerekli | eylem : gecmis_getir ( sayac )

On a single ticket four prompts gave four different results. P2 and P3 asked for an action field, and the response returned a sentence of reasoning instead; on top of that, in P3 the label slipped too. Only P4 produced both the right label and a readable action field. A single ticket is not a measurement, but it says where the measurement should look: the presence of the action field and its validity are two separate numbers.

In How Many Responses a Valid Action Field Appears

The second block runs the four prompts on all 120 tickets and measures each prompt with two seeds.

def measure(prefix, seed=SEED):
    COUNTER[0] = 0
    tin = tout = hits = with_field = valid = cut = 0
    for x in TICKETS:
        s = call(prefix + x["text"], seed=seed)
        tin, tout, cut = tin + s["input"], tout + s["output"], cut + s["truncated"]
        hits += read_label(s["response"]) == x["topic"]
        with_field += "eylem :" in s["response"]
        valid += read_action(s["response"]) is not None
    n = len(TICKETS)
    return {"accuracy": hits / n, "with_field": with_field / n, "valid": valid / n,
            "truncated": cut / n, "input": tin, "output": tout, "calls": COUNTER[0],
            "valid_n": valid}


NAMES = ["P1 thought", "P2 +action", "P3 +set", "P4 +short"]
PREFIXES = [P1, P2, P3, P4]
R1 = [measure(p) for p in PREFIXES]
R2 = [measure(p, seed=SEED + 1) for p in PREFIXES]
print(f"{'measure (seed 20260218)':<24}" + "".join(f"{a:>12}" for a in NAMES))
for name, k, fmt in (("label accuracy", "accuracy", "{:.4f}"),
                     ("carrying action field", "with_field", "{:.4f}"),
                     ("valid action field", "valid", "{:.4f}"),
                     ("truncated response share", "truncated", "{:.4f}"),
                     ("input tokens", "input", "{:d}"),
                     ("output tokens", "output", "{:d}")):
    print(f"{name:<24}" + "".join(fmt.format(x[k]).rjust(12) for x in R1))
print(f"{'total tokens':<24}" + "".join(
    str(x["input"] + x["output"]).rjust(12) for x in R1))
print(f"{'per valid action field':<24}" + "".join(
    ("-" if x["valid_n"] == 0 else
     f"{(x['input'] + x['output']) / x['valid_n']:.1f}").rjust(12) for x in R1))
print(f"\n{'second run (20260219)':<24}" + "".join(f"{a:>12}" for a in NAMES))
for name, k in (("label accuracy", "accuracy"), ("valid action field", "valid")):
    print(f"{name:<24}" + "".join(f"{x[k]:.4f}".rjust(12) for x in R2))
largest = max(abs(a["valid"] - b["valid"]) for a, b in zip(R1, R2))
print(f"\nlargest two-run difference (valid action field): {largest:.4f}")
for i in range(3):
    d1 = R1[i + 1]["valid"] - R1[i]["valid"]
    d2 = R2[i + 1]["valid"] - R2[i]["valid"]
    verdict = "measured" if min(abs(d1), abs(d2)) > largest else "unmeasured"
    print(f"  {NAMES[i]} -> {NAMES[i + 1]}: diff {d1:+.4f} / {d2:+.4f}  {verdict}")

measure (seed 20260218)   P1 thought  P2 +action     P3 +set   P4 +short
label accuracy                0.6750      0.5917      0.4833      0.6750
carrying action field         0.0000      0.3167      0.8833      0.8000
valid action field            0.0000      0.1000      0.5250      0.8000
truncated response share      0.0000      0.2167      0.3583      0.0000
input tokens                    5300        5540        7460        8060
output tokens                   1729        2210        2584        1745
total tokens                    7029        7750       10044        9805
per valid action field             -       645.8       159.4       102.1

second run (20260219)     P1 thought  P2 +action     P3 +set   P4 +short
label accuracy                0.8083      0.4500      0.5917      0.7333
valid action field            0.0000      0.3250      0.3417      0.9333

largest two-run difference (valid action field): 0.2250
  P1 thought -> P2 +action: diff +0.1000 / +0.3250  unmeasured
  P2 +action -> P3 +set: diff +0.4250 / +0.0167  unmeasured
  P3 +set -> P4 +short: diff +0.2750 / +0.5917  measured

The second and third rows are the real distinction of this lesson. In P3, 0.8833 of the responses carry an action field, but only 0.5250 are valid. The 0.3583 in between stands word for word in the row right below: truncated response share 0.3583. The source of the loss is not the text of the prompt but the output length limit. The thought and the action field share the 24-token budget of the same response; when the thought runs long, the stop condition falls right on the action field and what is left behind is a remnant such as eylem : belge _ara ( (act : find_doc ( ), whose format looks right but whose argument is not closed. A field looking present and its being readable are not the same thing.

P4 closes this loss: a single sentence that cuts the thought to one sentence carries the truncated response share from 0.3583 to 0.0000 and the valid action field from 0.5250 to 0.8000. What changed in the text is not even about the action, it is about the length of the thought.

Cost and the Unmeasurable Difference

Input tokens went from 5300 to 8060; writing the action set and the keep-it-short sentence at the head of every ticket brought the prompt to 1.52 times. Output tokens rose to 2584 in P3 and fell to 1745 in P4: the short thought reduced both the generated text and the truncation. The number of calls is 120 in all four prompts, so the whole cost is tokens. The real comparison is in the last row: from 645.8 tokens to 102.1 tokens per valid action field.

The last block applies the curriculum’s measurement rule. The largest difference between two runs in the valid action field is 0.2250. Writing the action set into the prompt added +0.4250 in the first run and +0.0167 in the second; since the two runs do not agree, this difference counts as unmeasured. It was known that writing the label set into the prompt works, but that the same move works for the action set as well could not be shown by this measurement. Shortening the thought stayed above the band in both runs: +0.2750 and +0.5917. In label accuracy no change produced a consistent difference; the two-run difference is at most 0.1417, and the strongest candidate, the move from P3 to P4, stays right at the edge of the band with +0.1917 and +0.1416. Asking for an action field did not move accuracy; it gained a machine-readable action field, and the cost was paid in tokens.

Summary

• Asking for thought and action together in the same prompt cannot be measured with a single number: the share of responses carrying an action field and the share of valid action fields are counted separately.
• When the action field was first requested, only 0.1000 of the responses produced a valid field; it rose to 0.5250 when the action set was written and to 0.8000 when the thought was shortened.
• The 0.3583 gap between presence and validity in P3 is exactly the truncated response share: the two fields share the same 24-token output budget and the limit falls on the field.
• Cost: input tokens 5300 → 8060, 645.8 → 102.1 tokens per valid action field. The number of calls did not change.
• When the two-run rule is applied, the difference of writing the action set stayed unmeasurable (+0.4250 and +0.0167); only the difference of shortening the thought was measured.

Next Step

In this lesson and in the previous four, every extra step the model took was visible: the thought field was inside the response, its tokens were counted, and what happened when it was cut could be read. The decision to shorten the step could be made only because the step was seen. If a class of models produces these steps inside the call without putting them into the response text at all, what is left: the steps cannot be read and their length cannot be limited, but their tokens are still paid for. The next lesson measures this class by its structural difference — two classes on the same task, side by side in accuracy and total tokens — and reports on which task the invisible token gained anything and on which the difference stayed unmeasurable.