Reasoning Models

In the previous lesson every extra step the model took was visible: the thought field was inside the response, its tokens were counted, what happened when it was cut could be read, and the decision to shorten the thought could be made only for that reason. The distinguishing feature of the class measured in this lesson is exactly the absence of this. The class produces intermediate steps before writing the response; the text of these steps never returns to the caller, their length cannot be limited by the prompt and the stop condition does not touch them — but they are paid for as output tokens. This is what will be measured: the cost of the invisible token.

This is not a product comparison. No model, version or provider is named; the two classes are defined only by a structural difference. The single-pass class reads the prompt and writes the response; the reasoning class inserts steps we do not see. As in self-consistency, the number of calls does not increase — both classes make one call per record, and what is paid for there is not visible samples but invisible tokens.

• RT49. The code does not call a real model endpoint; it calls a language model simulator written with the standard library. The support tickets and the document codes are fictional.
• RT50. Seed 20260218, second run 20260219. The same prompt gives the same response on the same seed.
• RT51. The reasoning class produces one intermediate step per field. In the single-field task the second step is a verification pass: the same candidate set is scored a second time and the response is written from the result of the last step.
• RT52. The tokens of the intermediate steps are added to the output tokens but do not enter the response text. The stop condition cuts only the visible response; the invisible steps cannot be limited.
• RT53. The single-pass class fills the two fields with two separate draws: the second field is not tied to the result of the first. In the reasoning class the second intermediate step makes a mapping from the result of the first step.
• RT54. Both tasks use the same 120 fictional tickets. Task A asks for one field (the label), task B asks for two fields (the label and the document code).
• RT55. The document codes come from a fictional document pool and each topic has a single code; for a correct response both fields must be correct.
• RT56. The response format is the same in both classes and is imposed. The only measured difference is the class itself.

The Structural Difference Between the Two Classes

The first block builds the simulator and separates the two classes with two functions. call_standard reads the prompt and writes the response. call_reasoning inserts intermediate steps, accumulates the tokens of the steps in the hidden field and writes only the result into the response text.

# SIMULATOR. No real model endpoint is called; it is a language model simulator written with the
# standard library. The support tickets and the document codes are FICTIONAL.
import math

SEED, M32 = 20260218, 0xFFFFFFFF
WINDOW, MAX_OUT_TOKENS = 240, 24


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(text):
    out = []
    for piece in text.lower().replace(",", " , ").split():
        while len(piece) > 5:
            out.append(piece[:5])
            piece = piece[5:]
        out.append(piece)
    return out


def trace_id(text):
    h = 2166136261
    for c in text:
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


TOPICS = ["fatura", "ariza", "sayac", "tarife", "kesinti"]
CODES = {"fatura": "FT", "ariza": "AR", "sayac": "SY", "tarife": "TR", "kesinti": "KS"}
PATTERNS = {
    "fatura": ["bu ayki fatura tutari gecen aya gore cok yuksek geldi",
               "faturami odedim ama sistemde borc gorunuyor",
               "fatura ayrintisini nereden gorebilirim"],
    "ariza": ["evde su akmiyor bir ariza var galiba",
              "baglanti bozuk oldu ve hic calismiyor",
              "ariza kaydi actim ama ekip gelmedi"],
    "sayac": ["sayac okumasi yanlis girilmis endeks tutmuyor",
              "sayac degisimi icin randevu istiyorum",
              "sayac uzerindeki endeks ile fatura ayni degil"],
    "tarife": ["hangi tarife basamagindayim ogrenmek istiyorum",
               "tarife degisikligi icin ne yapmam gerekiyor",
               "birim ucret abone tipine gore degisiyor mu"],
    "kesinti": ["mahallede kesinti var ne zaman gelecek",
                "planli kesinti duyurusu nereden takip edilir",
                "aksamdan beri kesinti yasiyoruz"],
}
EXTRAS = ["ayrica gecen ay da ayni sorun olmustu", "bu konuda memnun degilim",
          "acil bilgi almam gerekiyor", "kayit numaram elimde yok"]

TICKETS = []
for i in range(120):
    r = rng(SEED + i)
    k = TOPICS[pick(r(), [0.24, 0.22, 0.18, 0.20, 0.16])]
    text = PATTERNS[k][int(r() * 3)]
    if r() < 0.34:
        text += " " + EXTRAS[int(r() * 4)]
    TICKETS.append({"id": 4001 + i, "text": text, "topic": k, "code": CODES[k]})

CONCEPTS = TOPICS + ["odeme", "sikayet", "bilgi", "genel"]
CUE = {"fatur": "fatura", "tutar": "fatura", "ayrin": "fatura",
       "ariza": "ariza", "bozuk": "ariza", "calis": "ariza", "akmiy": "ariza",
       "sayac": "sayac", "endek": "sayac", "okuma": "sayac",
       "tarif": "tarife", "birim": "tarife", "abone": "tarife", "basam": "tarife",
       "kesin": "kesinti", "duyur": "kesinti", "mahal": "kesinti",
       "borc": "odeme", "odedi": "odeme", "ucret": "odeme",
       "memnu": "sikayet", "sorun": "sikayet", "acil": "sikayet",
       "bilgi": "bilgi", "ogren": "bilgi", "nered": "bilgi"}
TRAP = {"degis": "tarife", "kayit": "ariza", "yukse": "tarife", "gorun": "fatura",
        "ekip": "kesinti", "takip": "tarife", "giril": "fatura"}
STEPS = ["ara adim : metinde {k} izleri var , konu buna gore belirlendi",
         "ara adim : {k} konusunun belge kodu {d} olarak eslendi"]
COUNTER = [0]


def score(toks, pool):
    candidates = [k for k in pool if k[:5] in toks]
    if len(candidates) < 3:
        candidates = list(pool)
    p = {k: 0.15 for k in candidates}
    for t in toks:
        for table, w in ((CUE, 1.0), (TRAP, 1.3)):
            if table.get(t) in p:
                p[table[t]] += w
    return p


def choose(p, r, temperature):
    if temperature <= 0.0:
        return max(sorted(p), key=lambda a: p[a])
    e = [math.exp(p[a] / temperature) for a in sorted(p)]
    return sorted(p)[pick(r(), [v / sum(e) for v in e])]


def limit_check(toks, r):
    if len(toks) > WINDOW:
        return "context_overflow"
    if COUNTER[0] % 37 == 0:
        return "rate_limit"
    if len(toks) > 90 and r() < 0.04:
        return "timeout"
    return "done"


def reply(status, toks, y, hidden=0):
    if status != "done":
        return {"status": status, "input": len(toks), "output": 0, "hidden": 0,
                "response": ""}
    out = tokenize(y)
    if len(out) > MAX_OUT_TOKENS:            # the stop condition cuts only the visible part
        out, y = out[:MAX_OUT_TOKENS], " ".join(out[:MAX_OUT_TOKENS])
    return {"status": status, "input": len(toks), "output": len(out) + hidden,
            "hidden": hidden, "response": y}


def body(prompt):                       # the part of the prompt after the instruction
    return prompt.split("Talep: ")[-1]


def call_standard(prompt, seed=SEED, temperature=0.6):
    """STANDARD CLASS: single pass, no intermediate step."""
    COUNTER[0] += 1
    toks = tokenize(prompt)
    r = rng(seed ^ trace_id(prompt))
    status = limit_check(toks, r)
    if status != "done":
        return reply(status, toks, "")
    k = choose(score(toks, CONCEPTS), r, temperature)
    if "kod" not in toks:
        return reply(status, toks, f"etiket : {k}")
    k2 = choose(score(toks, CONCEPTS), r, temperature)      # a separate draw for the second field
    return reply(status, toks, f"etiket : {k} | kod : {CODES.get(k2, 'XX')}")


def call_reasoning(prompt, seed=SEED, temperature=0.6):
    """REASONING CLASS: produces invisible intermediate steps before writing the response.
    The text of the intermediate steps does not return to the caller but is counted as output tokens."""
    COUNTER[0] += 1
    toks = tokenize(prompt)
    r = rng(seed ^ trace_id(prompt))
    status = limit_check(toks, r)
    if status != "done":
        return reply(status, toks, "")
    g = tokenize(body(prompt))
    k = choose(score(g, TOPICS), r, temperature)                # 1st intermediate step
    hidden = len(tokenize(STEPS[0].format(k=k)))
    if "kod" not in toks:
        k = choose(score(g, TOPICS), r, temperature)            # 2nd intermediate step: verification
        hidden += len(tokenize(STEPS[0].format(k=k)))
        return reply(status, toks, f"etiket : {k}", hidden)
    code = CODES[k]                                             # 2nd intermediate step: mapping
    hidden += len(tokenize(STEPS[1].format(k=k, d=code)))
    return reply(status, toks, f"etiket : {k} | kod : {code}", hidden)


def read_fields(y):
    label = code = None
    p = y.split()
    for i, s in enumerate(p):
        if s == "etiket" and i + 2 < len(p) and p[i + 2] in TOPICS:
            label = p[i + 2]
        if s == "kod" and i + 2 < len(p):
            code = p[i + 2]
    return label, code


A = "Talebi su etiketlerden birine ata: fatura , ariza , sayac , tarife , kesinti . Talep: "
B = ("Talebi su etiketlerden birine ata: fatura , ariza , sayac , tarife , kesinti . "
     "Ayrica etiketin belge kodunu kod alanina yaz . Talep: ")

t = TICKETS[9]
print(f"FICTIONAL ticket {t['id']}: {t['text']}  (topic {t['topic']}, code {t['code']})")
for name, f in (("standard class", call_standard), ("reasoning class", call_reasoning)):
    for task, prompt in (("A one field", A), ("B two fields", B)):
        s = f(prompt + t["text"])
        print(f"{name:<16}{task:<13} output {s['output']:>3} tok "
              f"(hidden {s['hidden']:>2}) | {s['response']}")

FICTIONAL ticket 4010: hangi tarife basamagindayim ogrenmek istiyorum kayit numaram elimde yok  (topic tarife, code TR)
standard class  A one field   output   4 tok (hidden  0) | etiket : sayac
standard class  B two fields  output   9 tok (hidden  0) | etiket : tarife | kod : TR
reasoning class A one field   output  36 tok (hidden 31) | etiket : tarife
reasoning class B two fields  output  39 tok (hidden 30) | etiket : tarife | kod : TR

Four rows and a single observation on a single record: the responses have the same format, the outputs are not the same size. The reasoning class’s visible response is five tokens and its output is thirty-six tokens. The thirty-one tokens in between cannot be read, cannot be shortened, cannot be inspected. The previous lesson’s lever is not at hand here, because the text to be shortened is not inside the response.

Two Classes on the Same Task

The measurement is made on two tasks. Task A asks for one field and the prompt already lists the five labels. Task B asks for one more field: the document code of the label. Both use the same 120 fictional tickets and a single call is made for each record.

def measure(model_class, prefix, two_fields, seed=SEED):
    COUNTER[0] = 0
    tin = tout = hid = right = agree = 0
    for x in TICKETS:
        s = model_class(prefix + x["text"], seed=seed)
        tin, tout, hid = tin + s["input"], tout + s["output"], hid + s["hidden"]
        label, code = read_fields(s["response"])
        if two_fields:
            right += label == x["topic"] and code == x["code"]
            agree += code is not None and label is not None and code == CODES[label]
        else:
            right += label == x["topic"]
    n = len(TICKETS)
    return {"accuracy": right / n, "agreement": agree / n if two_fields else None,
            "input": tin, "visible": tout - hid, "hidden": hid, "total": tin + tout,
            "right": right}


JOBS = [("A standard", call_standard, A, False), ("A reasoning", call_reasoning, A, False),
        ("B standard", call_standard, B, True), ("B reasoning", call_reasoning, B, True)]
R1 = [measure(f, p, two) for _, f, p, two in JOBS]
R2 = [measure(f, p, two, seed=SEED + 1) for _, f, p, two in JOBS]
print(f"{'measure (seed 20260218)':<24}" + "".join(f"{j[0]:>12}" for j in JOBS))
for name, k, fmt in (("accuracy", "accuracy", "{:.4f}"),
                     ("two-field agreement", "agreement", "{:.4f}"),
                     ("input tokens", "input", "{:d}"),
                     ("visible output tokens", "visible", "{:d}"),
                     ("hidden output tokens", "hidden", "{:d}"),
                     ("total tokens", "total", "{:d}")):
    print(f"{name:<24}" + "".join(("-" if x[k] is None else fmt.format(x[k])).rjust(12)
                                  for x in R1))
print(f"{'per correct response':<24}" + "".join(
    f"{x['total'] / x['right']:.1f}".rjust(12) for x in R1))
print(f"\n{'second-run accuracy':<24}" + "".join(f"{x['accuracy']:.4f}".rjust(12)
                                                for x in R2))
for name, i in (("task A", 0), ("task B", 2)):
    d1 = R1[i + 1]["accuracy"] - R1[i]["accuracy"]
    d2 = R2[i + 1]["accuracy"] - R2[i]["accuracy"]
    band = max(abs(R1[i]["accuracy"] - R2[i]["accuracy"]),
               abs(R1[i + 1]["accuracy"] - R2[i + 1]["accuracy"]))
    verdict = "measured" if min(abs(d1), abs(d2)) > band else "unmeasured"
    ratio = R1[i + 1]["total"] / R1[i]["total"]
    print(f"{name}: class difference {d1:+.4f} / {d2:+.4f}, two-run difference {band:.4f} "
          f"-> {verdict}   tokens {ratio:.2f} times")

measure (seed 20260218)   A standard A reasoning  B standard B reasoning
accuracy                      0.7833      0.7500      0.6750      0.7750
two-field agreement                -           -      0.7667      0.9750
input tokens                    4340        4340        5780        5780
visible output tokens            550         547        1017        1017
hidden output tokens               0        3671           0        3438
total tokens                    4890        8558        6797       10235
per correct response            52.0        95.1        83.9       110.1

second-run accuracy           0.6667      0.7833      0.6333      0.8000
task A: class difference -0.0333 / +0.1167, two-run difference 0.1167 -> unmeasured   tokens 1.75 times
task B: class difference +0.1000 / +0.1667, two-run difference 0.0417 -> measured   tokens 1.51 times

In task A the difference could not be measured. The class difference is −0.0333 in the first run and +0.1167 in the second; its sign is not even constant and its size does not rise above the two-run difference (0.1167). The reason can be read from the table: the prompt already lists the five labels, so the job of narrowing the candidate set has been done in the prompt text. The intermediate step does the same narrowing one more time and adds no new information. In return, the invisible output tokens go from 0 to 3671 and the total reaches 1.75 times. This is the plainest form of the curriculum’s second claim: the technique multiplied the tokens and did not move the number.

In task B the situation reverses. The class difference is positive and large in both runs — +0.1000 and +0.1667 — while the two-run difference is 0.0417. The reason sits in the two-field agreement row: only 0.7667 of the single-pass class’s responses have a code field consistent with its own label. In other words, the model often finds the label right and writes the code from another topic; the two fields are not tied to each other. In the reasoning class this ratio is 0.9750 — the remaining share is the three calls that hit the rate limit and produced no response at all. Producing the second field from the result of the first closes the whole loss.

The Cost of the Invisible Token

The last row is the bill for this lesson. From 52.0 tokens to 95.1 per correct response (A) and from 83.9 to 110.1 (B). In task B the reasoning class really did raise accuracy and was still more expensive per correct response, because while accuracy rose 1.15 times, the total tokens rose 1.51 times. The number of calls is 120 in both classes; the whole increase is in output tokens, and most of those tokens were never read.

The shape of the cost matters too. The visible output tokens are almost the same in the two classes — 550 and 547 in task A. The difference is entirely in the invisible part: 3671 and 3438 tokens. This ties the prompt designer’s hands: a length constraint can be written but does not work on the invisible step, a format can be imposed but the invisible step has no format, truncated responses can be counted but the invisible step is not truncated. The only lever left is the task itself: is there a decision the intermediate step will narrow, or has the prompt already done that narrowing. The invisible intermediate step gains something only when it does a job the prompt could not do; if the prompt has already done the job, only tokens are left.

Summary

• The reasoning model class is defined not by a product but by a structural difference: it produces intermediate steps before the response, the steps do not return to the caller and are paid for as output tokens.
• The number of calls does not increase. In self-consistency the cost was in the calls and the samples could be seen; here the cost is inside a single call and it cannot be seen.
• In the single-field task the difference between the two classes is −0.0333 and +0.1167, and the two-run difference is 0.1167: the difference counts as unmeasured, yet tokens rose 1.75 times.
• In the two-field task the difference is +0.1000 and +0.1667, above the band (0.0417). Its source is two-field agreement: the single-pass class gets the code field right with its own label in only 0.7667 of the responses, the reasoning class in 0.9750.
• The cost per correct response is 52.0 → 95.1 and 83.9 → 110.1 tokens. Even in the task where it gained, the unit cost rose.

Next Step

Over six lessons a single prompt was worked over: its steps were chained, duplicated, branched, abstracted, an action field was put into it and finally it was handed over to a class whose steps are not visible. All of them shared one assumption — there is a single text looking at the question, and the good result will come from making that text even longer. Having several separate, independent prompts look at the same question, each producing its own response, and combining these was never tried. The next lesson measures this and asks the real question: do different prompts go wrong on the same examples? If the errors overlap, combining gains nothing, and this can be counted before combining.