What Is a Prompt

The previous course measured the surface of the model across fourteen lessons and wrote the cost next to every number: how many tokens went in, how many tokens came out, where the limit was, what the same input gave in two runs. At its close, one thing was missing. Throughout all of those measurements, what was said to the model stayed the same. The instruction text was written once and then treated like a fixed background; how much the number would move if the same model were given a different text for the same task was never asked.

This course turns that text into an object of measurement. The model, the task and the ticket set are fixed throughout the course; the only thing that changes is the text. In every lesson three numbers sit side by side: the previous prompt, the modified prompt with a single design decision applied, and the cost of the change. The cost is written in tokens; token cost and budget were measured in the closing lesson of the previous course and are not repeated here. The rule of the course is this: a prompt is a design object; a prompt changed without measuring is not an improvement but a guess. This lesson takes that object apart.

• PD1. The code does not call a real model endpoint; it calls a simulator written with the standard library. The simulator is the same definition built in the previous course: a fictional tokenizer, a 240-token context window, a 24-token maximum output, seed-dependent nondeterminism, a rate limit and a timeout.
• PD2. The support tickets are fictional: 120 records, five topics, a misleading sentence appended to some of the records. The topic names come from the fictional document vocabulary of the field team.
• PD3. The prompt is split into three components, and the split follows the function of the text. The instruction says what to do, the context defines the world in which the decision is made, the input carries the record to which that decision will be applied.
• PD4. The criterion is the label hit rate. If no label can be read from a response, the record counts as wrong and is also shown separately in the unlabeled response share.
• PD5. The cost is tokens, and input and output are written separately. The two do not move in the same direction; this is one of the main findings of this lesson.
• PD6. The contribution of a component is measured by removing that component. The other two components, the model, the seed and the ticket set do not change.
• PD7. Every measurement is run with two seeds (20260218 and 20260219). The seed is not a setting; it is the place where nondeterminism is read.
• PD8. Token, context window and inference were defined in the language models topic of the previous course; here they are only used.

The Three Components of a Prompt

From the outside, a prompt looks like a single text. From the inside there are three separate texts doing three separate jobs, and the cost and the contribution of each are different. The instruction defines the task: assign the ticket to a label, give the response in this form. The context sets up the world in which the decision is made: what the label set is, which options exist. The input is the only part that changes in each request: the ticket to be classified itself. The instruction and the context stay the same across one hundred twenty requests; the input changes one hundred twenty times.

The first block sets up the simulator and the fiction, then writes these three components separately and counts their tokens.

# SIMULATOR. This code does not call a real model endpoint; it is a language model
# simulator written with the standard library. The support tickets are FICTIONAL too.
import math

SEED, M32 = 20260218, 0xFFFFFFFF
WINDOW, MAX_OUT = 240, 24           # context window and maximum output, in tokens


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):                    # FICTIONAL tokenizer: pieces of at most five characters
    b = []
    for s in m.lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def prompt_trace(m):                # trace that depends on the prompt content, independent of the run
    h = 2166136261
    for c in m:
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


TOPICS = ["fatura", "ariza", "sayac", "tarife", "kesinti"]
TEMPLATES = {
    "fatura": ["bu ayki fatura tutari gecen aya gore cok yuksek geldi",
               "faturami odedim ama sistemde borc gorunuyor",
               "fatura ayrintisini nereden gorebilirim"],
    "ariza": ["evde su akmiyor bir ariza var galiba",
              "baglanti bozuk oldu ve hic calismiyor",
              "ariza kaydi actim ama ekip gelmedi"],
    "sayac": ["sayac okumasi yanlis girilmis endeks tutmuyor",
              "sayac degisimi icin randevu istiyorum",
              "sayac uzerindeki endeks ile fatura ayni degil"],
    "tarife": ["hangi tarife basamagindayim ogrenmek istiyorum",
               "tarife degisikligi icin ne yapmam gerekiyor",
               "birim ucret abone tipine gore degisiyor mu"],
    "kesinti": ["mahallede kesinti var ne zaman gelecek",
                "planli kesinti duyurusu nereden takip edilir",
                "aksamdan beri kesinti yasiyoruz"],
}
EXTRAS = ["ayrica gecen ay da ayni sorun olmustu", "bu konuda memnun degilim",
          "acil bilgi almam gerekiyor", "kayit numaram elimde yok"]

TICKETS = []
for i in range(120):
    r = rng(SEED + i)
    k = TOPICS[pick(r(), [0.24, 0.22, 0.18, 0.20, 0.16])]
    m = TEMPLATES[k][int(r() * 3)]
    if r() < 0.34:
        m += " " + EXTRAS[int(r() * 4)]
    TICKETS.append({"id": 4001 + i, "text": m, "topic": k})

CONCEPTS = TOPICS + ["odeme", "sikayet", "bilgi", "genel"]
CUES = {"fatur": "fatura", "tutar": "fatura", "ayrin": "fatura",
        "ariza": "ariza", "bozuk": "ariza", "calis": "ariza", "akmiy": "ariza",
        "sayac": "sayac", "endek": "sayac", "okuma": "sayac",
        "tarif": "tarife", "birim": "tarife", "abone": "tarife", "basam": "tarife",
        "kesin": "kesinti", "duyur": "kesinti", "mahal": "kesinti",
        "borc": "odeme", "odedi": "odeme", "ucret": "odeme",
        "memnu": "sikayet", "sorun": "sikayet", "acil": "sikayet",
        "bilgi": "bilgi", "ogren": "bilgi", "nered": "bilgi"}
TRAPS = {"degis": "tarife", "kayit": "ariza", "yukse": "tarife", "gorun": "fatura",
         "ekip": "kesinti", "takip": "tarife", "giril": "fatura"}   # misleading cues
FREE_FORM = ["talebiniz {k} konusuyla ilgili gorunuyor",
             "bu kayit {k} basligina girer , ilgili ekibe yonlendirildi",
             "kaydiniz alindi , konuyu inceleyip donus yapacagiz"]
CALL_COUNT = [0]


def call(prompt, seed=SEED, temperature=0.6, max_out=MAX_OUT):
    CALL_COUNT[0] += 1
    tok = tokenize(prompt)
    if len(tok) > WINDOW:
        return {"status": "context_overflow", "tokens_in": len(tok), "tokens_out": 0, "response": ""}
    if CALL_COUNT[0] % 37 == 0:
        return {"status": "rate_limit", "tokens_in": len(tok), "tokens_out": 0, "response": ""}
    r = rng(seed ^ prompt_trace(prompt))
    if len(tok) > 90 and r() < 0.04:
        return {"status": "timeout", "tokens_in": len(tok), "tokens_out": 0, "response": ""}
    cand = [k for k in CONCEPTS if k[:5] in tok]    # concepts named in the prompt
    if len(cand) < 3:
        cand = CONCEPTS
    p = {k: 0.15 for k in cand}
    for t in tok:
        for table, w in ((CUES, 1.0), (TRAPS, 1.3)):
            if table.get(t) in p:
                p[table[t]] += w
    if temperature <= 0.0:
        k = max(sorted(p), key=lambda a: p[a])       # most likely choice
    else:
        e = [math.exp(p[a] / temperature) for a in sorted(p)]
        k = sorted(p)[pick(r(), [v / sum(e) for v in e])]
    y = k if "tek" in tok else FREE_FORM[pick(r(), [0.42, 0.26, 0.32])].format(k=k)
    ot = tokenize(y)
    if len(ot) > max_out:                            # stop condition
        ot = ot[:max_out]
        y = " ".join(ot)
    return {"status": "ok", "tokens_in": len(tok), "tokens_out": len(ot), "response": y}


def read_label(y):                                   # extract a label from the response
    for s in y.lower().split():
        if s in TOPICS:
            return s
    return None


INSTRUCTION = "Talebi asagidaki etiketlerden birine ata . Yanit olarak tek etiket yaz ."
CONTEXT = "Etiketler: fatura , ariza , sayac , tarife , kesinti ."


def build_prompt(x, instruction=True, context=True, inp=True):   # builds a prompt from chosen components
    p = []
    if instruction:
        p.append(INSTRUCTION)
    if context:
        p.append(CONTEXT)
    if inp:
        p.append("Talep: " + x["text"])
    return " ".join(p)


x0 = TICKETS[0]
print(f"FICTIONAL ticket set: {len(TICKETS)} records, {len(TOPICS)} topics")
for name, m in (("instruction", INSTRUCTION), ("context", CONTEXT),
                ("input", "Talep: " + x0["text"])):
    print(f"{name:<11} {len(tokenize(m)):>2} tokens | {m}")
p0 = build_prompt(x0)
print(f"{'full prompt':<11} {len(tokenize(p0)):>2} tokens")
for sd in (SEED, SEED + 1):
    print(f"seed {sd}: {call(p0, seed=sd)['response']}")

FICTIONAL ticket set: 120 records, 5 topics
instruction 19 tokens | Talebi asagidaki etiketlerden birine ata . Yanit olarak tek etiket yaz .
context     15 tokens | Etiketler: fatura , ariza , sayac , tarife , kesinti .
input       13 tokens | Talep: planli kesinti duyurusu nereden takip edilir
full prompt 47 tokens
seed 20260218: kesinti
seed 20260219: kesinti

Only thirteen of the forty-seven tokens of the prompt belong to the real job, the ticket to be classified. The remaining thirty-four are fixed text that is sent again in every request. The two seeds gave the same response, kesinti, to the same prompt; this does not mean that nondeterminism has disappeared, it means that two runs coincided on a single record. Throughout this course this comparison is made on the whole set.

The Token Share of Each Component

The cost of a component is not its own length but its length multiplied by the number of requests. Its contribution can be measured only by removing it. The second block does both: first it computes the share of the three components in total input tokens, then it drops one component at a time and runs the same 120 records again.

def measure(seed=SEED, **kw):
    CALL_COUNT[0] = 0
    ti = to = d = unl = 0
    for x in TICKETS:
        s = call(build_prompt(x, **kw), seed=seed)
        ti, to = ti + s["tokens_in"], to + s["tokens_out"]
        e = read_label(s["response"])
        unl += e is None
        d += e == x["topic"]
    n = len(TICKETS)
    return {"hit_rate": d / n, "unlabeled": unl / n, "tokens_in": ti, "tokens_out": to}


n_ins, n_ctx = len(tokenize(INSTRUCTION)), len(tokenize(CONTEXT))
n_inp = sum(len(tokenize("Talep: " + x["text"])) for x in TICKETS) / len(TICKETS)
total = n_ins + n_ctx + n_inp
print(f"{'component':<12} {'tokens':>9} {'share':>8}")
for name, v in (("instruction", n_ins), ("context", n_ctx), ("input", n_inp), ("total", total)):
    print(f"{name:<12} {v:>9.2f} {v / total:>8.4f}")
print()

CASES = (("full prompt", {}), ("no context", {"context": False}),
         ("no instruction", {"instruction": False}), ("no input", {"inp": False}))
print(f"{'prompt':<14} {'hit rate':>8} {'run 2':>8} {'diff':>7} {'unlabeled':>10} "
      f"{'in':>6} {'out':>6} {'total':>7}")
for name, kw in CASES:
    A, B = measure(**kw), measure(seed=SEED + 1, **kw)
    print(f"{name:<14} {A['hit_rate']:>8.4f} {B['hit_rate']:>8.4f} "
          f"{abs(A['hit_rate'] - B['hit_rate']):>7.4f} {A['unlabeled']:>10.4f} {A['tokens_in']:>6d} "
          f"{A['tokens_out']:>6d} {A['tokens_in'] + A['tokens_out']:>7d}")

component       tokens    share
instruction      19.00   0.3945
context          15.00   0.3114
input            14.17   0.2941
total            48.17   1.0000

prompt         hit rate    run 2    diff  unlabeled     in    out   total
full prompt      0.7833   0.7250  0.0583     0.0250   5780    194    5974
no context       0.7083   0.5083  0.2000     0.0667   3980    197    4177
no instruction   0.7167   0.4583  0.2583     0.2167   3500   1251    4751
no input         0.1417   0.2167  0.0750     0.0250   4080    117    4197

The first table gives the proportion to remember throughout the course. The real data, the ticket to be classified, holds 0.2941 of the prompt. The instruction and the context together hold 0.7059, and these two texts are identical letter for letter across one hundred twenty requests. The average ticket is 14.17 tokens, the fixed text is 34 tokens: in every request, 2.4 times as much fixed text as data is sent again.

When a Component Is Removed

The second table gives the contribution of the components and shows that the three do not do the same job.

When the context is removed, the hit rate drops from 0.7833 to 0.7083 and total tokens drop from 5974 to 4177, that is, by 1797 tokens. At first glance the gain and the loss look balanced. But the unlabeled response share has risen from 0.0250 to 0.0667: when the label set is not written in the prompt, the model also answers with words outside the five topics, and no label can be read from those responses. The job of the context is not to make the model smarter but to close the answer space.

When the instruction is removed, the hit rate drops to 0.7167 and input tokens fall from 5780 to 3500 — and output tokens jump from 194 to 1251. This is the harshest row of the table. The instruction was the longest component at nineteen tokens; dropping it gained 2280 tokens on input, but took back 1057 tokens on output, and the net gain fell to 1223 tokens. Because the responses turned into sentences, the unlabeled share rose from 0.0250 to 0.2167: one in every five responses became unreadable by machine. In this prompt the instruction is not the most expensive component per unit length but the one with the best return.

When the input is removed, the number that comes out is the most instructive row of this lesson. The prompt is still valid, it still spends 4197 tokens, the unlabeled response share is still 0.0250 — that is, the output looks trouble-free. The hit rate, however, collapses to 0.1417; because the same prompt goes to all one hundred twenty records, the model produces the same guess every time, and that guess is right only as often as the share of that label in the set. A prompt produces a well-formed and confident response even without carrying data. This is why judging a prompt by its form does not count as measurement.

A Prompt Changed Without Measuring Is a Guess

The diff column of the table runs the third rule of the course for the first time. The full prompt gave 0.7833 and 0.7250 on the two seeds; the two-run difference is 0.0583. The effect of removing the context on the hit rate is 0.0750, and the effect of removing the instruction is 0.0667. Both numbers sit just above the two-run difference and are not large enough to trust on their own. In this course such a number counts as unmeasured; the rule is applied by name in the following lessons.

The measured side of the same rows is elsewhere. The unlabeled response share rose from 0.0250 to 0.2167, output tokens grew sixfold, and the own two-run difference of the prompts missing the context or the instruction rose to between 0.2000 and 0.2583. The three components stabilize the hit rate as much as they add to it: while the full prompt moves 0.0583 from run to run, these two prompts move three to four times as much. The components of a prompt do more to narrow the range of freedom of the model than to give it information; the gain comes from there.

The cost is also written apart from this reading. The full prompt spends 5974 / 0.7833 / 120, that is, 63.6 tokens per correct label. The prompt without context spends 49.1 tokens, the prompt without instruction 55.2 tokens. So the most accurate prompt is also the most expensive prompt, and these two must stand on separate rows. Removing a component lowers the cost; what kind of loss that removal is becomes visible only when the hit rate and the unlabeled share are read together.

Summary

• The prompt is split into three components: instruction (what to do), context (in which world), input (which record). On 120 fictional tickets the shares are 0.3945 / 0.3114 / 0.2941; the real data is about a third of the prompt.
• The context closes the answer space: when removed, the hit rate goes from 0.7833 to 0.7083, the unlabeled response share from 0.0250 to 0.0667, and 1797 tokens are gained.
• The instruction is the longest and the best-return component: when removed, 2280 tokens are gained on input but 1057 tokens are paid back on output, and the unlabeled share rises to 0.2167.
• A prompt without input also produces a well-formed response and spends 4197 tokens; its hit rate is 0.1417. A well-formed output does not mean that the prompt has been measured.
• The two-run difference of the full prompt is 0.0583; of the prompts missing the context or the instruction, between 0.2000 and 0.2583. The measured contribution of the components is as much narrowing the movement between runs as raising the hit rate.

Next Step

In this lesson the instruction and the context were rewritten into each of one hundred twenty requests and held 0.7059 of the total input. Because the code did this in a single line, the cost stayed invisible: the same thirty-four tokens were sent one hundred twenty times. A behavior definition is written once but paid for in every request, and as the number of requests grows this repetition becomes the dominant line item of the total cost. The next lesson measures this: how the total tokens of a persistent system instruction grow with the number of requests, how many tokens shortening the instruction saves, and how much of the hit rate that shortening gives back.