--- title: 'Prompt and Context Engineering Difference' source: 'https://academia.sh/en/courses/prompt-engineering/prompt-and-context-engineering-difference' course: 'Prompt Engineering' language: en updated: '2026-10-21T05:02:33+00:00' license: 'CC BY-SA 4.0' objectives: - 'Separates whether a wrong response comes from a missing instruction or from missing information.' - 'Measures the two fixes on the same set of errors and counts which error class is fixed by which.' - 'Compares the token cost of the two fixes and their unit cost per error.' --- # Prompt and Context Engineering Difference Comparing two separate fixes on the same set of errors: on 120 fictional tickets the baseline topic hit rate is 0.8833, the fault code hit rate is 0.4083, and both correct is 0.4083. Writing a distinguishing rule into the prompt raises the topic hit rate to 0.9667 and does not touch the code hit rate at all; putting the fictional code table into the prompt raises the code hit rate to 0.9167 and does not touch the topic hit rate at all. Of the fourteen topic errors ten are fixed by the instruction alone, of the seventy-one code errors sixty-six are fixed by information alone, and one error in each class is fixed by neither. The costs diverge: the instruction adds 1680 tokens, the information adds 14520. The previous lesson recovered the rejected records by asking again, and fourteen records were not fixed by any retry. This lesson asks the cause of that residue. A wrong response can be wrong for two separate reasons, and the two do not resemble each other. The model may have misunderstood **what to do**: the instruction is ambiguous, a misleading word has pulled the decision. Or the model may have been forced to say **something it does not know**: the requested information is not in the prompt at all and no instruction can produce it. The first class is the subject of prompt design. The second is the subject of **context engineering**: not what the model is told, but what is placed next to it. This lesson only establishes the distinction and measures it. Which document is found where, how it is retrieved and how many pieces are selected is a retrieval pipeline question and is left to the Embeddings and Retrieval-Augmented Generation course; here the fictional table is put into the prompt **as it is**. - **OC33.** The code is a **simulator**; no real endpoint is called. The tickets, the fault codes and the code table are **fictional**, seed **20260218**. - **OC34. The task has two fields:** assigning a topic and a **fault code** to the ticket. The two fields carry two separate sources of error. - **OC35. The source of the topic error is misleading cues.** A word in a side clause of the ticket pulls the decision to another topic. This is an instruction problem. - **OC36. The source of the code error is missing information.** When the code table is not in the prompt, the simulator writes the **most common code** of the topic; only five of the fifteen codes can be found this way. - **OC37. The instruction fix** writes a distinguishing rule into the prompt. The rule is not always followed; the simulator applies it in a share of the responses. - **OC38. The information fix** puts the whole fictional code table into the prompt. **No retrieval is done**: which row is needed is not looked up, the table is given in full. The model matches the table against its own definition texts and reads the wrong row in a share of the responses. - **OC39.** An error is a wrong value in a field; errors are counted **per field**. The same record can carry both errors at once. - **OC40.** The cost is written as input tokens and also given as **tokens per fixed error**; the second seed for the two runs is 20260219. ## Two Errors, Two Separate Sources Two example tickets are put side by side in four scaffoldings (`SETUPS` in the code). The first carries both errors at once, the second only the code error. ```python # SIMULATOR -- FICTIONAL. No real model endpoint is called. The code table is also # FICTIONAL and is put into the prompt as it is, without any retrieval. SEED, M32 = 20260218, 0xFFFFFFFF def rng(t): x = ((t ^ (t >> 16)) * 2246822507) & M32 x = ((x ^ (x >> 13)) * 3266489909) & M32 s = [(x ^ (x >> 16)) & M32] def next_value(): s[0] = (s[0] * 1664525 + 1013904223) & M32 return s[0] / 4294967296 return next_value def pick(u, w): t = 0.0 for i, x in enumerate(w): t += x if u < t: return i return len(w) - 1 def tokenize(text): # FICTIONAL tokenizer, a piece has at most five characters out = [] for piece in text.lower().replace(",", " , ").split(): while len(piece) > 5: out.append(piece[:5]) piece = piece[5:] out.append(piece) return out def trace_id(text): h = 2166136261 for c in text: h = ((h ^ ord(c)) * 16777619) & M32 return h TOPICS = ["fatura", "ariza", "sayac", "tarife", "kesinti"] PATTERNS = [("bu ayki fatura tutari gecen aya gore cok yuksek geldi", "fatura"), ("faturami odedim ama sistemde borc gorunuyor", "fatura"), ("fatura ayrintisini nereden gorebilirim", "fatura"), ("evde su akmiyor bir ariza var galiba", "ariza"), ("baglanti bozuk oldu ve hic calismiyor", "ariza"), ("ariza kaydi actim ama ekip gelmedi", "ariza"), ("sayac okumasi yanlis girilmis endeks tutmuyor", "sayac"), ("sayac degisimi icin randevu istiyorum", "sayac"), ("sayac uzerindeki endeks ile fatura ayni degil", "sayac"), ("hangi tarife basamagindayim ogrenmek istiyorum", "tarife"), ("tarife degisikligi icin ne yapmam gerekiyor", "tarife"), ("birim ucret abone tipine gore degisiyor mu", "tarife"), ("mahallede kesinti var ne zaman gelecek", "kesinti"), ("planli kesinti duyurusu nereden takip edilir", "kesinti"), ("aksamdan beri kesinti yasiyoruz", "kesinti")] TICKETS = [] for i in range(120): # 120 FICTIONAL support tickets r = rng(SEED + i) topic = TOPICS[pick(r(), [0.24, 0.22, 0.18, 0.20, 0.16])] TICKETS.append({"id": 4001 + i, "topic": topic, "text": [x for x in PATTERNS if x[1] == topic][int(r() * 3)][0]}) CUES = {"fatur": "fatura", "tutar": "fatura", "ayrin": "fatura", "ariza": "ariza", "bozuk": "ariza", "calis": "ariza", "akmiy": "ariza", "sayac": "sayac", "endek": "sayac", "okuma": "sayac", "tarif": "tarife", "birim": "tarife", "abone": "tarife", "basam": "tarife", "kesin": "kesinti", "duyur": "kesinti", "mahal": "kesinti"} TRAPS = {"degis": "tarife", "kayit": "ariza", "yukse": "tarife", "gorun": "fatura", "ekip": "kesinti", "takip": "tarife", "giril": "fatura"} # misleading cues CODES = [("FT-21", "fatura tutari yuksek geldi"), ("FT-34", "odenen fatura borc gorunuyor"), ("FT-47", "fatura ayrinti talebi"), ("AR-12", "evde su akmiyor sikayeti"), ("AR-25", "baglanti bozuk calismiyor"), ("AR-38", "acik kayit ekip gelmedi"), ("SY-16", "endeks yanlis girilmis"), ("SY-29", "sayac degisim randevusu"), ("SY-42", "endeks ile fatura uyusmuyor"), ("TR-13", "tarife basamagi sorusu"), ("TR-26", "tarife degisiklik talebi"), ("TR-39", "birim ucret abone tipi"), ("KS-15", "mahalle kesintisi suresi"), ("KS-28", "planli duyuru takibi"), ("KS-41", "aksamdan beri kesinti")] TRUE_CODE = {t: CODES[i][0] for i, (t, _) in enumerate(PATTERNS)} TOPIC_CODE = {} for i, (t, k) in enumerate(PATTERNS): TOPIC_CODE.setdefault(k, CODES[i][0]) # first code of the topic when there is no document DOCUMENT = " ; ".join(f"{k} {a}" for k, a in CODES) COUNTER = [0] def draw(prompt, seed, salt): """One separately salted draw for each decision; decisions are not tied to each other.""" return rng((seed + salt) ^ trace_id(prompt))() def call(prompt, ticket, seed=SEED): """Topic and fault code are requested together. When the code table is not in the prompt the model writes the most common code of the topic; when it is, it chooses by overlap with the definitions.""" COUNTER[0] += 1 toks = tokenize(prompt) follows = "ayirt" in toks and draw(prompt, seed, 91) < 0.78 trap_weight = 0.9 if follows else 1.3 # distinguishing rule in the prompt p = {k: 0.15 for k in TOPICS} for t in tokenize(ticket["text"]): for table, w in ((CUES, 1.0), (TRAPS, trap_weight)): if table.get(t) in p: p[table[t]] += w topic = max(sorted(p), key=lambda a: p[a]) if "tablo" not in toks: # no code table in the prompt return topic, TOPIC_CODE[topic] tb = set(tokenize(ticket["text"])) scores = [(len(tb & set(tokenize(a))), -i) for i, (_, a) in enumerate(CODES)] i = -max(scores)[1] if draw(prompt, seed, 17) < 0.08: # chance of misreading the document i = (i + 1) % len(CODES) return topic, CODES[i][0] RULE_PROMPT = "Ayirt edici kural: yan cumledeki sozcukler etiketi belirlemez . " TABLE_PROMPT = "Ariza kodu tablosu: " + DOCUMENT + " . " print(f"FICTIONAL code table {len(CODES)} rows, {len(tokenize(DOCUMENT))} tokens") print("sample rows:", " | ".join(f"{k} {a}" for k, a in CODES[3:6])) for j in (3, 0): x = TICKETS[j] print(f"\nticket {x['id']} : {x['text']}") print(f" true topic {x['topic']:<8} code {TRUE_CODE[x['text']]}") for name, prefix in (("baseline", ""), ("instruction", RULE_PROMPT), ("information", TABLE_PROMPT), ("both", RULE_PROMPT + TABLE_PROMPT)): topic, code = call(prefix + f"Talep {x['id']} : {x['text']}", x) mark = "correct" if (topic == x["topic"] and code == TRUE_CODE[x["text"]]) else "" print(f" {name:<13} topic {topic:<8} code {code} {mark}") ``` ``` FICTIONAL code table 15 rows, 116 tokens sample rows: AR-12 evde su akmiyor sikayeti | AR-25 baglanti bozuk calismiyor | AR-38 acik kayit ekip gelmedi ticket 4004 : ariza kaydi actim ama ekip gelmedi true topic ariza code AR-38 baseline topic kesinti code KS-15 instruction topic ariza code AR-12 information topic kesinti code AR-38 both topic ariza code AR-38 correct ticket 4001 : planli kesinti duyurusu nereden takip edilir true topic kesinti code KS-28 baseline topic kesinti code KS-15 instruction topic kesinti code KS-15 information topic kesinti code KS-28 correct both topic kesinti code KS-28 correct ``` The upper ticket carries both errors at once. The topic comes out as `kesinti` (outage), because the last two words of the ticket trigger a misleading cue; when the distinguishing rule is written, the topic returns to `ariza` (fault). But on the same row the code is still wrong: `AR-12`, where the correct one is `AR-38`. The instruction fixed the topic and could not fix the code, because the correct code was written nowhere in the prompt. When the table is put in the code becomes `AR-38`, and this time the topic stays wrong. The record becomes correct only **with both together**. The topic of the lower ticket is correct from the start and only its code is wrong. The instruction row is identical to the baseline: the topic stays `kesinti`, the code stays `KS-15`. A rule of fourteen tokens was added to the prompt and no field of the output changed. The table, on the other hand, brings the code to `KS-28` in a single step. The same error form, two separate fixes — and the cost of choosing the wrong one is only tokens, with no gain at all. What the two examples say together is this: looking at a wrong response does not reveal which scaffolding is needed. The wrongness itself looks the same in both cases. The distinction appears only when **each of the two scaffoldings is tried and counted separately**. ## Which Error Is Fixed by Which The same 120 tickets are run in four scaffoldings and errors are counted **per field**. ```python SETUPS = [("baseline", ""), ("+ instruction", RULE_PROMPT), ("+ information", TABLE_PROMPT), ("+ both", RULE_PROMPT + TABLE_PROMPT)] def measure(prefix, seed=SEED): COUNTER[0] = 0 d, g = [], 0 for x in TICKETS: prompt = prefix + f"Talep {x['id']} : {x['text']}" topic, code = call(prompt, x, seed) d.append((topic == x["topic"], code == TRUE_CODE[x["text"]])) g += len(tokenize(prompt)) return d, g RESULTS = {name: measure(prefix) for name, prefix in SETUPS} print("setup topic hit rate code hit rate both correct input tokens") for name, _ in SETUPS: d, g = RESULTS[name] print(f"{name:<14} {sum(a for a, b in d) / 120:>14.4f} {sum(b for a, b in d) / 120:>13.4f}" f" {sum(a and b for a, b in d) / 120:>12.4f} {g:>12}") print("\nerror class baseline errors by instruction by information by both by neither") for name, j in (("topic error", 0), ("code error", 1)): n = [i for i in range(120) if not RESULTS["baseline"][0][i][j]] y = sum(RESULTS["+ instruction"][0][i][j] for i in n) b = sum(RESULTS["+ information"][0][i][j] for i in n) ik = sum(RESULTS["+ both"][0][i][j] for i in n) h = sum(not (RESULTS["+ instruction"][0][i][j] or RESULTS["+ information"][0][i][j] or RESULTS["+ both"][0][i][j]) for i in n) print(f"{name:<13} {len(n):>15} {y:>14} {b:>14} {ik:>7} {h:>10}") b1 = measure(RULE_PROMPT + TABLE_PROMPT, SEED + 1)[0] print(f"\nsecond run (+ both): topic {sum(a for a, _ in b1) / 120:.4f}, " f"code {sum(b for _, b in b1) / 120:.4f}, both correct " f"{sum(a and b for a, b in b1) / 120:.4f}") ``` ``` setup topic hit rate code hit rate both correct input tokens baseline 0.8833 0.4083 0.4083 1534 + instruction 0.9667 0.4083 0.4083 3214 + information 0.8833 0.9167 0.8000 16054 + both 0.9583 0.9000 0.8667 17734 error class baseline errors by instruction by information by both by neither topic error 14 10 0 9 1 code error 71 0 66 63 1 second run (+ both): topic 0.9667, code 0.8917, both correct 0.8583 ``` The second and third rows of the first table show the distinction in the plainest way. The instruction raises the topic hit rate **from 0.8833 to 0.9667** and leaves the code hit rate **at 0.4083** as it was. The information raises the code hit rate **from 0.4083 to 0.9167** and leaves the topic hit rate **at 0.8833** as it was. The two scaffoldings work on the same task but **never touch the same error**. The second table counts this per error. Of the fourteen topic errors **ten** are fixed by the instruction and **zero** by the information. Of the seventy-one code errors **sixty-six** are fixed by the information and **zero** by the instruction. One of the two columns is always the zero of the other; this is the numerical counterpart of the fact that the two fixes cannot substitute for each other. The first step in improving a system is counting which error is in which class — because every token spent on the wrong class is wasted. **One error** in each class is fixed by no scaffolding at all. On the topic side this is a call in which the distinguishing rule is not followed; on the code side it is a call in which the wrong row of the table is read. Both are the residue not of prompt design but of the uncertainty of the producer, and they do not close by writing more text. ## The Cost of the Two Fixes The cost column separates the two fixes sharply. The instruction raises the input tokens **from 1534 to 3214**: fourteen per ticket, **1680 tokens** in total. The information raises them to **16054**: one hundred twenty-one per ticket, **14520 tokens** in total and **10.5 times** the baseline. This is the cost of putting a whole code table into every request. Read per fixed error the picture softens but does not reverse: the instruction fixes ten errors with 1680 tokens, **168 tokens per error**; the information fixes sixty-six errors with 14520 tokens, **220 tokens per error**. The information is more expensive but **in return fixes more errors than can be measured against it**: the baseline carried 71 code errors and none of them would have been fixed however much text was added to the instruction. The last row also shows the cost of the two scaffoldings working together. The rate of records correct in both fields rises **from 0.4083 to 0.8667**; in the second run the same number is **0.8583**, so the two-run difference is **0.0084**. Since the gain is more than fifty times this difference, it is comfortably measurable. On the other hand the combined scaffolding is **a little worse in every field** than the single scaffoldings: topic 0.9583 instead of 0.9667, code 0.9000 instead of 0.9167. As the prompt grows, the share of calls in which the rule is followed and the table is read correctly falls. **A prompt changed without measurement is not an improvement but a guess**, and the sum of two improvements is not the sum of the two. ## Summary - A wrong response comes either from the model misunderstanding **what to do** or from its being forced to say **something it does not know**; the two are fixed by separate scaffoldings. - The instruction raises the topic hit rate from 0.8833 to 0.9667 and does not touch the code hit rate at all; the information raises the code hit rate from 0.4083 to 0.9167 and does not touch the topic hit rate at all. - Of the fourteen topic errors ten are fixed by the instruction alone, of the seventy-one code errors sixty-six are fixed by information alone; the reciprocal value of both columns is zero. - One error in each class is fixed by no scaffolding: these are the residue of the producer's uncertainty. - The instruction adds 1680 tokens and the information adds 14520; 168 and 220 tokens per fixed error. The information is expensive but fixes the errors the instruction never reaches. - The rate of records correct in both fields rises from 0.4083 to 0.8667, but the combined scaffolding is a little worse in every field than the single scaffoldings. ## Next Step In this lesson the errors were split into two sources and both were measured on single records. What was not asked is **who the errors are concentrated on**. A system can be 0.8667 correct and all of the remaining wrong answers can be concentrated in a particular set of subscribers; in that case the average is good and for that set the system is not working. The next lesson measures this: how far the subgroup gap can be narrowed with a prompt, what part of it remains, and why a prompt does not solve **data bias** is shown with numbers.