--- title: 'Capabilities and Limits' source: 'https://academia.sh/en/courses/introduction-to-ai-engineering/capabilities-and-limits' course: 'Introduction to AI Engineering' language: en updated: '2026-10-21T05:02:30+00:00' license: 'CC BY-SA 4.0' objectives: - 'Measures five task classes on the same fictional request set with the bare call and counts in which class the hit rate collapses' - 'Shows at the token level why counting and exact arithmetic collapse: the inflected form and the carry across a piece boundary' - 'Compares the two scaffoldings added to the collapsing class side by side as hit rate, tokens, calls and latency class' --- # Capabilities and Limits Measuring five task classes with the bare call on twenty-four fictional support requests: summarization 0.7500, classification 0.8750, ranking 0.6250, counting 0.4167 and exact arithmetic 0.2917. When the seed is changed the number of the first three classes moves, the last two do not move at all, and the number of requests whose response changed comes out 0/24. The counting hit rate equals exactly the 10/24 requests that carry no inflected form, the exact-arithmetic hit rate equals exactly the 7/24 requests that carry no carry across a piece boundary. The three-run majority scaffolding changes nothing in these two classes with three times the tokens, the procedure scaffolding that takes the calculation out of the tokens carries both to 1.0000. The previous lesson measured the context window as a limit and showed that how to fill the window is a decision. The question of how many tokens the truncation decisions save and how much they lower the hit rate was answered. What was not asked comes earlier: whatever is put into the window, **what the model does well** was never measured. If a task is not going well, is the fault in the prompt, in the window, or in the task itself? This lesson runs five task classes on the same data with the same bare call and writes the hit rate per class. In two classes the number collapses, and the source of the collapse is not in the prompt, it is **at the token level**. - **LM46. The fictional request set is 24 short Turkish support requests.** The requests are generated from the combination of five main sentences and six extra sentences, and two meter readings stand at the end of each request. The texts, the classes and the readings are **fictional**. - **LM47. The model is a simulator**; no real endpoint is called. The simulator is written with the standard library, its seed is **20260218**, and it produces its response only from the token sequence. - **LM48. The tokenizer is fictional** and carries on the measure set up in the first lesson of this topic: letter pieces are split into fours, digit pieces into twos. - **LM49. Five task classes are measured:** summarization, classification, ranking, counting and exact arithmetic. The measure of each class is the **hit rate**: in how many of the twenty-four requests the response is correct. - **LM50. The cost is written in three units:** input tokens per request, number of calls and latency class. Raw time is not written. - **LM51. The uncertainty is measured with two runs.** The same task set is run with seed 20260218 and seed 20260219; in addition, the **number of requests whose response changed** between the two runs is printed. ## Bare Call on Five Task Classes ```python # COMMON -- FICTIONAL. The simulator below does not call a real endpoint; it is written with the # standard library and carries the measurable surface of a language model. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def tokenize(text): """Fictional tokenizer: letter pieces are split into fours, digit pieces into twos.""" out, buffer = [], "" for ch in text + " ": if ch.isalnum(): buffer += ch continue if buffer: n = 2 if buffer.isdigit() else 4 out += [buffer[i:i + n] for i in range(0, len(buffer), n)] buffer = "" if ch.strip(): out.append(ch) return out MAIN = [ # (main sentence, true class) ("sayac okunmadi tahmini fatura cikti", "olcum"), ("fatura tutari gecen aya gore iki kat cikti", "odeme"), ("gece boyunca su basinci dustu", "tesisat"), ("planli kesinti bildirimi ulasmadi", "kesinti"), ("abonelik devri sayaca islenmedi", "sozlesme"), ] EXTRA = ["sayacin numarasi degistirilmis olabilir", "adres bilgisi eski gorunuyor", "onceki ay ayni sorun yasanmadi", "sayaca ulasilamiyor kapi kapali", "kesinti suresi beklenenden uzun surdu", "otomatik talimat sayacta yok"] LEXICON = { # class -> pieces the simulator recognizes and fictional priority "kesinti": ({"kesi", "bild"}, 1), "olcum": ({"okun", "olcu", "tahm"}, 2), "tesisat": ({"basi", "vana", "tesi", "dust"}, 3), "odeme": ({"fatu", "odem", "tuta", "tali"}, 4), "sozlesme": ({"abon", "devr", "sozl", "adre"}, 5), } CLASSES = list(LEXICON) TASKS = {"summarization": 11, "classification": 23, "ranking": 41, "counting": 59, "exact_arithmetic": 71} SAMPLES = {} REQUESTS = [] for i in range(24): r = rng(SEED + 31 * i) a, b = 100000 + int(r() * 380000), 100000 + int(r() * 380000) REQUESTS.append({"id": 4100 + i, "label": MAIN[i % 5][1], "a": a, "b": b, "main": MAIN[i % 5][0], "extra": EXTRA[(5 * i + 2) % 6], "text": f"{MAIN[i % 5][0]}, {EXTRA[(5 * i + 2) % 6]}." f" ana sayac {a} ara sayac {b}."}) def draw(task, seed, t): """Sampling table: one stream per task and seed, one value per request.""" if (task, seed) not in SAMPLES: r = rng(seed + 1009 * TASKS[task]) SAMPLES[(task, seed)] = [r() for _ in range(24)] return SAMPLES[(task, seed)][t["id"] - 4100] def call(task, t, seed, sampling=True): """Simulator: produces the response only from the token sequence. In tasks that choose among candidates, sampling can bring the second candidate; counting and exact arithmetic produce no candidates, they follow a single path over the token sequence.""" tokens = tokenize(t["text"]) if task == "counting": return tokens.count("saya") if task == "exact_arithmetic": p = [(str(t["a"])[k:k + 2], str(t["b"])[k:k + 2]) for k in range(0, 6, 2)] return int("".join(f"{(int(x) + int(y)) % 100:02d}" for x, y in p)) r = draw(task, seed, t) if task == "summarization": # which sentence enters the summary sentence = sorted([t["main"], t["extra"]], key=lambda c: -len(tokenize(c))) return sentence[1] if sampling and r < 0.14 else sentence[0] ranked = sorted(CLASSES, key=lambda s: (-sum(b in LEXICON[s][0] for b in tokens), s)) return ranked[1] if sampling and r < 0.14 else ranked[0] def measure(seed, sampling=True): d, ordered = {}, 0 d["summarization"] = sum(call("summarization", t, seed, sampling) == t["main"] for t in REQUESTS) / 24 d["classification"] = sum(call("classification", t, seed, sampling) == t["label"] for t in REQUESTS) / 24 for g in range(24): three = [REQUESTS[g], REQUESTS[(g + 7) % 24], REQUESTS[(g + 13) % 24]] truth = [t["id"] for t in sorted(three, key=lambda t: (LEXICON[t["label"]][1], t["id"]))] predicted = [t["id"] for t in sorted(three, key=lambda t: ( LEXICON[call("ranking", t, seed, sampling)][1], t["id"]))] ordered += truth == predicted d["ranking"] = ordered / 24 d["counting"] = sum(call("counting", t, seed) == t["text"].replace(",", " ") .replace(".", " ").split().count("sayac") for t in REQUESTS) / 24 d["exact_arithmetic"] = sum(call("exact_arithmetic", t, seed) == t["a"] + t["b"] for t in REQUESTS) / 24 return d RUN_A, RUN_B = measure(SEED), measure(SEED + 1) MEAN_TOKENS = sum(len(tokenize(t["text"])) for t in REQUESTS) / 24 print(f"fictional task set: {len(REQUESTS)} support requests, five task classes") print(f"input tokens per request {MEAN_TOKENS:.1f}, calls 1, latency class single call") print(f"{'task':<22}{'seed 20260218':<16}{'seed 20260219':<16}changed requests") for g in TASKS: diff = sum(str(call(g, t, SEED)) != str(call(g, t, SEED + 1)) for t in REQUESTS) print(f"{g:<22}{RUN_A[g]:<16.4f}{RUN_B[g]:<16.4f}{diff:>2}/24") ``` ``` fictional task set: 24 support requests, five task classes input tokens per request 33.3, calls 1, latency class single call task seed 20260218 seed 20260219 changed requests summarization 0.7500 0.6667 4/24 classification 0.8750 0.8750 2/24 ranking 0.6250 0.4583 11/24 counting 0.4167 0.4167 0/24 exact_arithmetic 0.2917 0.2917 0/24 ``` The number falls from class to class: classification **0.8750**, summarization **0.7500**, ranking **0.6250**, counting **0.4167**, exact arithmetic **0.2917**. The same model, the same request, the same token count. The only thing that changes is what the task asks for. Ranking giving the lowest number among the upper three rows has its own reason. Ranking is not a single decision but the combination of three: the simulator classifies each request separately and orders the triple by the priority of the classes. Because a mistake on a single request spoils the whole order, the hit rate of 0.8750 per class falls to 0.6250 per triple. The number of composite tasks is always lower than the number of their components, and this is a rule that will continue throughout this lesson. The upper three classes also share something: all of them are a **selection** task. Summarization chooses one of two sentences, classification one of five labels, ranking the order of three requests. In selection tasks the token surface carries enough information, because the correct answer is already somewhere inside the text. The lower two classes are **generation** tasks: the correct answer is not found inside the text, it has to be computed from the text. What shows the distinction, however, is not the measure itself but how the measure is taken. The last column says this alone. In the first three classes there are requests whose response changes when the seed changes: **11/24** in ranking, **4/24** in summarization, **2/24** in classification. The hit rate of classification comes out the same in the two runs but **not on the same requests**: two requests changed their response, one improved and one got worse. In the last two classes the difference is **0/24**. Counting and exact arithmetic give **exactly the same wrong answer** in both runs. This is not sampling noise. ## The Token-Level Source of Counting and Exact Arithmetic ```python def inflected(m): return [w for w in m.replace(",", " ").replace(".", " ").split() if w.startswith("sayac") and w != "sayac"] t = REQUESTS[4] print("request:", t["main"] + ",", t["extra"]) print("tokens:", " ".join(tokenize(t["main"]))) truth = t["text"].replace(",", " ").replace(".", " ").split().count("sayac") print(f"sayac occurrences: true {truth}, simulator {call('counting', t, SEED)}," f" inflected form {inflected(t['text'])}") print(f"addition: {t['a']} + {t['b']} = {t['a'] + t['b']}," f" simulator {call('exact_arithmetic', t, SEED)}") carries = sum(any(int(str(x["a"])[k:k + 2]) + int(str(x["b"])[k:k + 2]) > 99 for k in (2, 4)) for x in REQUESTS) inflected_count = sum(bool(inflected(x["text"])) for x in REQUESTS) print(f"requests with a carry across a piece boundary {carries}/24 -> remaining {24 - carries}/24") print(f"requests with an inflected sayac form {inflected_count}/24" f" -> remaining {24 - inflected_count}/24") ``` ``` request: abonelik devri sayaca islenmedi, kesinti suresi beklenenden uzun surdu tokens: abon elik devr i saya ca isle nmed i sayac occurrences: true 2, simulator 3, inflected form ['sayaca'] addition: 139039 + 302204 = 441243, simulator 431243 requests with a carry across a piece boundary 17/24 -> remaining 7/24 requests with an inflected sayac form 14/24 -> remaining 10/24 ``` Counting first. The model does not see words in the text; it sees pieces. The word `sayaca` (to the meter) is split into the pieces `saya` and `ca`, and the word `sayac` (meter) also starts with `saya`. Because the two share the same beginning in the token sequence, the model counts the inflected form as well and writes **3** instead of **2**. The agglutinative structure of Turkish makes this a rule, not an exception: requests carrying an inflected form of sayac are **14/24**, those not carrying one **10/24**. The counting hit rate is **0.4167**, that is exactly **10/24**. The model is wrong not because it cannot count but because **it cannot see the unit it is to count**. Exact arithmetic is the number side of the same flaw. The reading `139039` is split into the pieces `13`, `90`, `39`. The model adds the pieces pairwise and keeps each piece at two digits: when `90 + 22` comes to `112`, the third digit, that is the **carry**, cannot pass to the next piece because that piece has already been written. The number that comes out is **431243**, the correct one is **441243**: the difference is exactly the lost carry. Requests carrying a carry across a piece boundary are **17/24**, those not carrying one **7/24**. The exact-arithmetic hit rate is **0.2917**, that is exactly **7/24**. Both numbers come not from the data but from **tokenization**; changing the seed does not move them. ## Adding Scaffolding to the Collapsing Class ```python TRUTH = {"classification": lambda t: t["label"], "exact_arithmetic": lambda t: t["a"] + t["b"], "counting": lambda t: t["text"].replace(",", " ").replace(".", " ") .split().count("sayac")} def majority(task): # scaffolding 1: three runs, majority response s = 0 for t in REQUESTS: y = [call(task, t, k) for k in (SEED, SEED + 1, SEED + 2)] s += max(y, key=y.count) == TRUTH[task](t) return s / 24 def with_procedure(task, t): """Scaffolding 2: the model extracts the value, a procedure outside the tokens does the calculation.""" if task == "exact_arithmetic": numbers, o, tokens = [], 0, tokenize(t["text"]) while o < len(tokens): # adjacent digit pieces form one number p = "" while o < len(tokens) and tokens[o].isdigit(): p, o = p + tokens[o], o + 1 o += 1 if not p else 0 if p: numbers.append(int(p)) return sum(numbers[:2]) return sum(w == "sayac" for w in t["text"].replace(",", " ").replace(".", " ").split()) print(f"{'task':<18}{'bare call':<11}{'majority':<10}procedure") for g in ("classification", "counting", "exact_arithmetic"): y = ("-" if g == "classification" else f"{sum(with_procedure(g, t) == TRUTH[g](t) for t in REQUESTS) / 24:.4f}") print(f"{g:<18}{measure(SEED)[g]:<11.4f}{majority(g):<10.4f}{y}") extra_tokens = sum(len(tokenize(str(TRUTH['exact_arithmetic'](t)))) for t in REQUESTS) / 24 print(f"cost -- bare call: {MEAN_TOKENS:.1f} tokens, 1 call, latency class single call") print(f"cost -- majority: {3 * MEAN_TOKENS:.1f} tokens, 3 calls, latency class parallel") print(f"cost -- procedure: {2 * MEAN_TOKENS + extra_tokens:.1f} tokens, 2 calls, latency class chain") ``` ``` task bare call majority procedure classification 0.8750 0.9167 - counting 0.4167 0.4167 1.0000 exact_arithmetic 0.2917 0.2917 1.0000 cost -- bare call: 33.3 tokens, 1 call, latency class single call cost -- majority: 99.9 tokens, 3 calls, latency class parallel cost -- procedure: 69.6 tokens, 2 calls, latency class chain ``` The majority scaffolding calls the same request with three seeds and takes the most frequent response. It works in classification: the number **0.8750** rises to **0.9167**, because the error there comes from sampling and two of the three runs hold the correct answer. In counting and exact arithmetic, however, the number **does not budge**: 0.4167 and 0.2917 stay exactly the same. The cost is real and paid up front: the number of calls goes to three, the tokens to **99.9**. A statistical scaffolding placed on top of a structural error does nothing but repeat the error three times. The second scaffolding moves the work outside the token layer: the model only extracts the numbers, and a procedure does the adding and the counting. The hit rate is **1.0000** in both classes. The cost is two calls, **69.6** tokens and the **chain** latency class — because the second call has to wait for the result of the first, while the three calls of the majority scaffolding can go in parallel. The cheaper scaffolding is the one that pays off: majority gives nothing with 99.9 tokens, the procedure corrects the class entirely with 69.6 tokens. Without writing the cost these two scaffoldings could not have been told apart. ## Summary - When five task classes are measured with the bare call on the same fictional request set, the hit rate falls from 0.8750 to 0.2917: classification 0.8750, summarization 0.7500, ranking 0.6250, counting 0.4167, exact arithmetic 0.2917. - When the seed changes, there are requests whose response changes in the first three classes (11/24, 4/24, 2/24) and 0/24 in the last two; the error of counting and exact arithmetic is not sampling noise. - The counting hit rate equals exactly the 10/24 requests that carry no inflected form, the exact-arithmetic hit rate equals exactly the 7/24 requests that carry no carry across a piece boundary; both numbers come from tokenization. - The three-run majority scaffolding carries classification to 0.9167 but changes nothing in counting and exact arithmetic; its cost is 99.9 tokens and three calls. - The procedure scaffolding that takes the calculation out of the token layer carries both classes to 1.0000; its cost is 69.6 tokens, two calls and the chain latency class. ## Next Step The wrong answers measured in this lesson were honest in a way: when the model could not count it wrote a wrong number, and the wrongness was visible in a single line. The next lesson takes up a harder kind of wrong answer. To a question, even when it holds no basis whatsoever, the model produces a **fluent and confident** response; the form of the response is the same as the correct one, and its wrongness cannot be understood by looking at the response itself. The measure of this is not the hit rate but the **fabrication rate**: how many claims in the response rest on a document. The same lesson shows the second claim of the curriculum for the first time: one of the three scaffoldings added to reduce fabrication really lowers fabrication, **and lowers the hit rate too**, and this is not hidden.