--- title: Hallucination source: 'https://academia.sh/en/courses/introduction-to-ai-engineering/hallucination' course: 'Introduction to AI Engineering' language: en updated: '2026-10-21T05:02:30+00:00' license: 'CC BY-SA 4.0' objectives: - 'Counts the fabrication rate in an ungrounded response at the claim level and reads it as a measure separate from the hit rate' - 'Compares three mitigation scaffoldings side by side as hit rate, fabrication rate, number of claims, tokens and calls' - 'Shows in the table by name the scaffolding that lowers the number and writes where the drop comes from' --- # Hallucination When twenty-four questions are asked over a fictional document pool of twenty-seven facts, the ungrounded response gets a hit rate of 0.2917 and a fabrication rate of 0.6667, and the response changes in 16/24 requests across two runs. Three mitigations are measured side by side: grounding raises the hit rate by +0.3750 and brings fabrication down to 0.3333, claim verification brings fabrication down to 0.0857 but its cost is two calls and 66.2 tokens, and the abstention option drops the hit rate 0.0417 below the baseline and leaves 10/24 requests unanswered. All of the unsupported claims left after grounding come from questions that have no counterpart in the document, and the verification scaffolding gives a difference of 0/24 across two runs. The wrong answers of the previous lesson were visible: when the model wrote a sum wrong, the result could be checked in a single line. The wrong answer of this lesson is not visible. Even when it has no basis whatsoever, the model produces a fluent and regular response; the form of a wrong response is the same as that of a correct one, it is written with the same certainty and cannot be told apart by looking at the response itself. This is called **hallucination**. Its measure is not the hit rate, because the hit rate asks only for the final value, and a response does not consist of a single value: the reasons, numbers and side information placed next to what was asked are also part of the response, and the user reads them too. The measure is therefore the **fabrication rate**: how many of the claims in the response rest on a document. This lesson puts three mitigation scaffoldings side by side on the same data, writes the cost of each as tokens and calls, and shows the curriculum's second claim for the first time — one of the three **lowers** the number, and this is not hidden in the table. - **LM52. The fictional document pool** consists of tariff descriptions, meter maintenance procedures and fault codes. Each fact is a `(topic, field, value)` triple and is **fictional**. - **LM53. Twenty-four questions are asked.** Some of the questions have a counterpart in the pool and some do not; this distinction is at the center of the measure. - **LM54. The simulator has two separate sources of knowledge:** the **document** placed inside the prompt and the **previously acquired fictional knowledge**. The second partly contradicts the pool. - **LM55. Each response carries two claims:** the field asked and a second field added next to it. A claim is **supported** if the exact same triple exists in the pool. - **LM56. Two numbers are written side by side.** The hit rate is the share of requests in which the value of the field asked agrees with the document; a request left unanswered does not count as a hit. The fabrication rate is the share of unsupported claims among all the claims produced. - **LM57. The uncertainty is measured with two runs.** A value read from the document is copied, a value coming from memory is **open to sampling**; the second run is taken with seed 20260219. ## Fabrication Rate in an Ungrounded Response ```python # COMMON -- FICTIONAL. The simulator does not call a real endpoint; the document pool, the questions # and the model's previously acquired knowledge are generated with the standard library, seed 20260218. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def tokenize(text): """Fictional tokenizer: letter pieces are split into fours, digit pieces into twos.""" out, buffer = [], "" for ch in text + " ": if ch.isalnum(): buffer += ch continue if buffer: n = 2 if buffer.isdigit() else 4 out += [buffer[i:i + n] for i in range(0, len(buffer), n)] buffer = "" if ch.strip(): out.append(ch) return out TOPICS = ["tariff home", "tariff business", "tariff garden", "meter maintenance", "meter replacement", "fault e12", "fault e31", "fault e44", "outage planned", "outage urgent", "subscription transfer", "subscription cancellation"] FIELDS = ["threshold", "duration", "step", "document"] DOCUMENT, MEMORY, CONFIDENCE, SAMPLES = {}, {}, {}, {} r = rng(SEED + 97) for k in TOPICS: # fictional document pool for a in FIELDS: p, v = r(), 2 + int(r() * 40) if p < 0.55: DOCUMENT[(k, a)] = v r = rng(SEED + 8080) for k in TOPICS: # the model's previously acquired fictional knowledge for a in FIELDS: p, v = r(), 2 + int(r() * 40) d = DOCUMENT.get((k, a)) MEMORY[(k, a)] = d if d is not None and p < 0.58 else v QUESTIONS = [(TOPICS[i % 12], FIELDS[(3 * i + 1) % 4]) for i in range(24)] r = rng(SEED + 4242) for i, (k, a) in enumerate(QUESTIONS): # fictional confidence: partly overlaps the truth CONFIDENCE[i] = (0.30 + 0.60 * r() if MEMORY[(k, a)] == DOCUMENT.get((k, a)) else 0.10 + 0.60 * r()) THRESHOLD = 0.55 def draw(i, j, seed): """Sampling table: a single draw for every value not read from the document.""" if seed not in SAMPLES: r2 = rng(seed + 5551) SAMPLES[seed] = [r2() for _ in range(48)] return SAMPLES[seed][2 * i + j] def ask(i, scaffolding, seed): """The response is a list of claims; each claim is a (topic, field, value) triple.""" k, a = QUESTIONS[i] side = FIELDS[(FIELDS.index(a) + 2) % 4] if scaffolding == "abstention" and CONFIDENCE[i] < THRESHOLD: return [] source = DOCUMENT if scaffolding in ("grounding", "verification") else {} claims = [] for j, x in enumerate((a, side)): if (k, x) in source: # read from the document in the prompt claims.append((k, x, source[(k, x)])) continue v = MEMORY[(k, x)] # a value coming from memory is open to sampling claims.append((k, x, 2 + (v + 7) % 40 if draw(i, j, seed) < 0.25 else v)) if scaffolding == "verification": # second call: each claim is looked up in the document r2 = rng(SEED + 611 * i) claims = [c for c in claims if DOCUMENT.get((c[0], c[1])) == c[2] or r2() < 0.10] return claims def measure(scaffolding, seed=SEED): claim_count = unsupported = hit = unanswered = 0 for i, (k, a) in enumerate(QUESTIONS): y = ask(i, scaffolding, seed) claim_count += len(y) unanswered += not y unsupported += sum(DOCUMENT.get((c[0], c[1])) != c[2] for c in y) hit += DOCUMENT.get((k, a)) is not None and any( c[1] == a and c[2] == DOCUMENT[(k, a)] for c in y) return hit / 24, unsupported / claim_count if claim_count else 0.0, claim_count, unanswered COVERED = sum((k, a) in DOCUMENT for k, a in QUESTIONS) QUESTION_TOKENS = sum(len(tokenize(f"what is the {a} value for {k}")) for k, a in QUESTIONS) / 24 DOC_TOKENS = sum(len(tokenize(f"{k} {a} value {v}")) for (k, a), v in DOCUMENT.items()) / len(TOPICS) hit1, fab1, n1, _ = measure("bare call") hit2, fab2, n2, _ = measure("bare call", SEED + 1) print(f"fictional document pool {len(DOCUMENT)} facts, {len(QUESTIONS)} questions," f" with a counterpart in the document {COVERED}/24") print(f"bare call, seed 20260218: hit rate {hit1:.4f}, fabrication {fab1:.4f}, {n1} claims") print(f"bare call, seed 20260219: hit rate {hit2:.4f}, fabrication {fab2:.4f}, {n2} claims") changed = sum(ask(i, "bare call", SEED) != ask(i, "bare call", SEED + 1) for i in range(24)) print(f"number of requests whose response changed in the two runs {changed}/24") print(f"cost: {QUESTION_TOKENS:.1f} tokens per request, 1 call, latency class single call") ``` ``` fictional document pool 27 facts, 24 questions, with a counterpart in the document 16/24 bare call, seed 20260218: hit rate 0.2917, fabrication 0.6667, 48 claims bare call, seed 20260219: hit rate 0.3333, fabrication 0.6667, 48 claims number of requests whose response changed in the two runs 16/24 cost: 12.1 tokens per request, 1 call, latency class single call ``` The bare call answers all twenty-four requests and produces **48 claims**. **0.6667** of them rest on no document. The hit rate is **0.2917**. What to notice is that the two numbers can move independently of each other: in the second run the hit rate rises to **0.3333** while the fabrication rate stays at **0.6667**. The hit rate measures a single field, the fabrication rate measures the whole response. There is not a single request that the model refuses to answer. Even though **8** of the twenty-four questions have no counterpart at all in the pool, all eight have a response. This shows the source of hallucination directly: the simulator does not look for what was asked in the document; it produces the most probable continuation from the **previously acquired fictional knowledge**, and that knowledge is both incomplete and partly contradictory to the pool. The fluency of the text is independent of the existence of a basis. The term itself calls for a warning here. Hallucination is **not the name of a malfunction**: the model fabricates not because it is broken but because it is doing exactly the job it was designed for. The job it does is to produce the probable continuation of a text, and probability is not the same thing as a basis. In the table the counterpart of this is the following: no scaffolding that lowers the fabrication rate touches the inside of the model; all of them put a source of basis or a checking step **around** the response. If this distinction is not held, fabrication is taken to be "a flaw that will be fixed one day" and the cost of the scaffolding is never written into the budget. ## Three Mitigation Scaffoldings ```python print(f"{'scaffolding':<14}{'hit rate':<10}{'fabrication':<13}{'claims':<8}{'unanswered':<12}" f"{'tokens':<8}{'calls':<7}latency") for d, tok, calls, lat in (("bare call", QUESTION_TOKENS, 1, "single call"), ("grounding", QUESTION_TOKENS + DOC_TOKENS, 1, "single call"), ("abstention", QUESTION_TOKENS + 9, 1, "single call"), ("verification", 2 * (QUESTION_TOKENS + DOC_TOKENS), 2, "chain")): hit, fab, n, un = measure(d) un_text = f"{un}/24" print(f"{d:<14}{hit:<10.4f}{fab:<13.4f}{n:<8}{un_text:<12}{tok:<8.1f}{calls:<7}{lat}") t1, t2 = measure("grounding")[0], measure("abstention")[0] print(f"baseline bare call {hit1:.4f}; grounding {t1 - hit1:+.4f}, abstention {t2 - hit1:+.4f}," f" verification {measure('verification')[0] - hit1:+.4f}") ``` ``` scaffolding hit rate fabrication claims unanswered tokens calls latency bare call 0.2917 0.6667 48 0/24 12.1 1 single call grounding 0.6667 0.3333 48 0/24 33.1 1 single call abstention 0.2500 0.5714 28 10/24 21.1 1 single call verification 0.6667 0.0857 35 2/24 66.2 2 chain baseline bare call 0.2917; grounding +0.3750, abstention -0.0417, verification +0.3750 ``` **Grounding** puts the relevant document inside the prompt. The hit rate rises by **+0.3750**, and the fabrication rate is cut in half. The cost is honest and is measured in tokens: **33.1** tokens per request instead of 12.1, that is, close to three times. The number of calls does not change. **Claim verification** checks, in a second call, each claim of the response against the pool one by one and drops the claim whose counterpart cannot be found. The fabrication rate goes down to **0.0857**, the lowest number in this table. Its cost is also the highest: **66.2** tokens, **two calls** and the **chain** latency class, because the second call has to wait for the output of the first. Its hit rate stays the same as grounding's, so the extra cost buys only fabrication. The **abstention option** is the third, and in the table it is **the row that lowers the number**. The hit rate goes down to **0.2500**: **0.0417 below** the baseline, the bare call. The reason is written in the unanswered column — the scaffolding leaves ten requests unanswered, and some of them were requests that would have been answered correctly. The fabrication rate goes down only slightly, to **0.5714**, because the fictional confidence measure overlaps with the existence of a basis only partly. The scaffolding takes away something definite from the hit rate, and brings something uncertain in return for fabrication. Had the number not been written, all three mitigations would have been listed as "improvements". This does not mean the scaffolding is bad; it means **the criterion was not chosen**. If the cost of stating a wrong tariff threshold is heavier than the cost of stating nothing, the abstention option wins and the loss of 0.0417 in the hit rate is cheap insurance. In the opposite case the same scaffolding is a regression that leaves ten requests unanswered. The engineer looking at the table has the job of converting these two costs into the same unit; the table's job is to make visible which number the decision rests on. The threshold itself is also a setting: a value lower than 0.55 reduces the unanswered requests and enlarges fabrication. The unanswered column stays in the table for this reason — if that column were not written, where the drop in the hit rate comes from would not be visible, and the scaffolding would be shut down as "it did not work". ## Where Fabrication Comes From ```python print(f"{'scaffolding':<14}{'hit rate A':<12}{'hit rate B':<12}{'changed requests':<18}" f"source of unsupported claims") for d in ("bare call", "grounding", "abstention", "verification"): a, b = measure(d)[0], measure(d, SEED + 1)[0] diff = sum(ask(i, d, SEED) != ask(i, d, SEED + 1) for i in range(24)) diff_text = f"{diff}/24" covered = uncovered = 0 for i in range(24): for c in ask(i, d, SEED): if DOCUMENT.get((c[0], c[1])) != c[2]: covered += (c[0], c[1]) in DOCUMENT uncovered += (c[0], c[1]) not in DOCUMENT print(f"{d:<14}{a:<12.4f}{b:<12.4f}{diff_text:<18}with document {covered:>2}," f" without document {uncovered:>2}") ``` ``` scaffolding hit rate A hit rate B changed requests source of unsupported claims bare call 0.2917 0.3333 16/24 with document 16, without document 16 grounding 0.6667 0.6667 8/24 with document 0, without document 16 abstention 0.2500 0.3333 9/24 with document 7, without document 9 verification 0.6667 0.6667 0/24 with document 0, without document 3 ``` The last column splits fabrication in two. **16** of the bare call's 32 unsupported claims are in questions that **have** a counterpart in the pool: the document was in our hands, and the model did not look at it. The other **16** are in questions that **do not have** a counterpart. Grounding **zeroes** the first half and does not touch the second half at all. This is the exact description of what grounding can and cannot do: it ends the fabrication of information that stands in the pool, it does not end the fabrication of information that does not stand in the pool. For the remaining sixteen claims the only solution is to enlarge the pool or to leave them unanswered. The middle column measures uncertainty. The bare call gives a different response in **16/24** requests across two runs. Grounding brings this down to **8/24**, because a value read from the document is copied and is not open to sampling. Claim verification gives **0/24**: exactly the same response in two runs. What narrows the uncertainty is not the setting of the model but **the response being tied to a basis**. This is the second and little-discussed gain of grounding. A design rule comes out of this, and it reaches ahead in this curriculum. The cheap way of reducing fabrication is not to force the model but **to put the basis of the response into the prompt**; the expensive way is to check the response after producing it. When the cost of the two is written in the same unit, in tokens and calls, the choice is not left to debate. But both of them leave a question unanswered: if there are hundreds of documents in the pool, which ones will be put into the prompt. In this lesson the correct document was given by hand every time, because the number of topics was twelve. When the number of documents grows, finding the correct document becomes a scaffolding that itself needs to be measured, and it brings its own hit rate and its own token cost. A later course of this curriculum builds exactly that scaffolding; this lesson only showed with numbers why it is needed. ## Summary - In the ungrounded response, 24 requests produce 48 claims, the fabrication rate is 0.6667 and the hit rate 0.2917; the model answers all eight of the questions that have no counterpart in the pool. - Grounding raises the hit rate by +0.3750 and brings fabrication down to 0.3333; its cost is going from 12.1 tokens per request to 33.1 tokens. - Claim verification pulls fabrication lowest, to 0.0857; its cost is 66.2 tokens, two calls and the chain latency class; it adds nothing to the hit rate on top of grounding. - The abstention option drops the hit rate to 0.2500, that is 0.0417 below the baseline, and leaves 10/24 requests unanswered; not every scaffolding that is added improves. - Grounding brings fabrication from 16 to 0 in claims that have a document but leaves 16 as 16 in those without one; it also pulls the requests that change from run to run from 16/24 to 8/24. ## Next Step Every scaffolding measured in this lesson was built **around** the model: a document was placed inside the prompt, a second call was added, a threshold was defined. All of them assumed that access to the model was given. Yet that access is itself a decision and it changes the numbers. Between reaching a model only through a remote endpoint and holding the weights and running them on your own hardware, which setting stays with whom, how many tokens the same job costs and where the data goes are different. The next lesson compares these two access modes on three criteria with numbers and writes in which criterion the choice changes.