System Instruction

The previous lesson split the prompt into three components and measured that the instruction and the context held 0.7059 of the total input. That measurement was made inside a single request. When the code sent the same two texts one hundred twenty times, the cost stayed hidden in a single line, because the repetition was inside the loop and was not counted anywhere.

This lesson counts that repetition. A system instruction is the persistent behavior definition written so that the model behaves the same way in every request: what the task is, which options exist, in which form the response is given, what to do under ambiguity. Its name is persistent but the place where it is stored is the prompt itself; the model remembers nothing between two calls. The instruction is therefore written once and paid for in every request. That is what will be measured: the total of the repetition, the share of the repetition, and how much of the hit rate shortening the instruction gives back.

• PD9. The code calls a simulator; no real model endpoint is called. The support tickets are fictional and are the same 120 records as in the previous lesson.
• PD10. The system instruction and the input are inside the same prompt. The simulator has no separate instruction field, because what is measured is the token cost and the name of the field does not change the cost.
• PD11. Four instruction lengths are built incrementally: none, short (the label set only), medium (the label set plus the task and the response form), long (that, with ambiguity rules added).
• PD12. The cost of the instruction is length times number of requests. In the table the repeat column is this product and it is included in the total input tokens.
• PD13. The criterion is the label hit rate; the unlabeled response share is also printed. Tokens per correct label is found by dividing total tokens by the hit rate and the record count.
• PD14. Every row is run with two seeds (20260218 and 20260219). A hit rate difference that stays smaller than the two-run difference counts as unmeasured in this course.
• PD15. The rate limit fires on every thirty-seventh call and returns an empty response. In one hundred twenty calls three records are left unlabeled because of it; this share belongs not to the prompt but to the surface.
• PD16. Token cost and budget were measured in the closing lesson of the previous course; here only the token unit is used.

The Instruction Does Not Stay in the Model

The first block sets up the simulator and the fiction, defines the four instruction lengths incrementally and writes the total cost of each over one hundred twenty requests. At the end a single fictional ticket is called with the short and the medium instruction, with two separate seeds.

# SIMULATOR. This code does not call a real model endpoint; it is a language model
# simulator written with the standard library. The support tickets are FICTIONAL too.
import math

SEED, M32 = 20260218, 0xFFFFFFFF
WINDOW, MAX_OUT = 240, 24           # context window and maximum output, in tokens


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):                    # FICTIONAL tokenizer: pieces of at most five characters
    b = []
    for s in m.lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


def prompt_trace(m):                # trace that depends on the prompt content, independent of the run
    h = 2166136261
    for c in m:
        h = ((h ^ ord(c)) * 16777619) & M32
    return h


TOPICS = ["fatura", "ariza", "sayac", "tarife", "kesinti"]
TEMPLATES = {
    "fatura": ["bu ayki fatura tutari gecen aya gore cok yuksek geldi",
               "faturami odedim ama sistemde borc gorunuyor",
               "fatura ayrintisini nereden gorebilirim"],
    "ariza": ["evde su akmiyor bir ariza var galiba",
              "baglanti bozuk oldu ve hic calismiyor",
              "ariza kaydi actim ama ekip gelmedi"],
    "sayac": ["sayac okumasi yanlis girilmis endeks tutmuyor",
              "sayac degisimi icin randevu istiyorum",
              "sayac uzerindeki endeks ile fatura ayni degil"],
    "tarife": ["hangi tarife basamagindayim ogrenmek istiyorum",
               "tarife degisikligi icin ne yapmam gerekiyor",
               "birim ucret abone tipine gore degisiyor mu"],
    "kesinti": ["mahallede kesinti var ne zaman gelecek",
                "planli kesinti duyurusu nereden takip edilir",
                "aksamdan beri kesinti yasiyoruz"],
}
EXTRAS = ["ayrica gecen ay da ayni sorun olmustu", "bu konuda memnun degilim",
          "acil bilgi almam gerekiyor", "kayit numaram elimde yok"]

TICKETS = []
for i in range(120):
    r = rng(SEED + i)
    k = TOPICS[pick(r(), [0.24, 0.22, 0.18, 0.20, 0.16])]
    m = TEMPLATES[k][int(r() * 3)]
    if r() < 0.34:
        m += " " + EXTRAS[int(r() * 4)]
    TICKETS.append({"id": 4001 + i, "text": m, "topic": k})

CONCEPTS = TOPICS + ["odeme", "sikayet", "bilgi", "genel"]
CUES = {"fatur": "fatura", "tutar": "fatura", "ayrin": "fatura",
        "ariza": "ariza", "bozuk": "ariza", "calis": "ariza", "akmiy": "ariza",
        "sayac": "sayac", "endek": "sayac", "okuma": "sayac",
        "tarif": "tarife", "birim": "tarife", "abone": "tarife", "basam": "tarife",
        "kesin": "kesinti", "duyur": "kesinti", "mahal": "kesinti",
        "borc": "odeme", "odedi": "odeme", "ucret": "odeme",
        "memnu": "sikayet", "sorun": "sikayet", "acil": "sikayet",
        "bilgi": "bilgi", "ogren": "bilgi", "nered": "bilgi"}
TRAPS = {"degis": "tarife", "kayit": "ariza", "yukse": "tarife", "gorun": "fatura",
         "ekip": "kesinti", "takip": "tarife", "giril": "fatura"}   # misleading cues
FREE_FORM = ["talebiniz {k} konusuyla ilgili gorunuyor",
             "bu kayit {k} basligina girer , ilgili ekibe yonlendirildi",
             "kaydiniz alindi , konuyu inceleyip donus yapacagiz"]
CALL_COUNT = [0]


def call(prompt, seed=SEED, temperature=0.6, max_out=MAX_OUT):
    CALL_COUNT[0] += 1
    tok = tokenize(prompt)
    if len(tok) > WINDOW:
        return {"status": "context_overflow", "tokens_in": len(tok), "tokens_out": 0, "response": ""}
    if CALL_COUNT[0] % 37 == 0:
        return {"status": "rate_limit", "tokens_in": len(tok), "tokens_out": 0, "response": ""}
    r = rng(seed ^ prompt_trace(prompt))
    if len(tok) > 90 and r() < 0.04:
        return {"status": "timeout", "tokens_in": len(tok), "tokens_out": 0, "response": ""}
    cand = [k for k in CONCEPTS if k[:5] in tok]    # concepts named in the prompt
    if len(cand) < 3:
        cand = CONCEPTS
    p = {k: 0.15 for k in cand}
    for t in tok:
        for table, w in ((CUES, 1.0), (TRAPS, 1.3)):
            if table.get(t) in p:
                p[table[t]] += w
    if temperature <= 0.0:
        k = max(sorted(p), key=lambda a: p[a])       # most likely choice
    else:
        e = [math.exp(p[a] / temperature) for a in sorted(p)]
        k = sorted(p)[pick(r(), [v / sum(e) for v in e])]
    y = k if "tek" in tok else FREE_FORM[pick(r(), [0.42, 0.26, 0.32])].format(k=k)
    ot = tokenize(y)
    if len(ot) > max_out:                            # stop condition
        ot = ot[:max_out]
        y = " ".join(ot)
    return {"status": "ok", "tokens_in": len(tok), "tokens_out": len(ot), "response": y}


def read_label(y):                                   # extract a label from the response
    for s in y.lower().split():
        if s in TOPICS:
            return s
    return None


E1 = "Etiketler: fatura , ariza , sayac , tarife , kesinti ."
E2 = " Talebi bu etiketlerden birine ata . Yanit olarak tek etiket yaz ."
E3 = (" Birden cok etiket uyuyorsa talebin ana konusunu sec ."
      " Hicbiri uymuyorsa en yakin etiketi sec . Aciklama ekleme .")
INSTRUCTIONS = (("none", ""), ("short", E1), ("medium", E1 + E2), ("long", E1 + E2 + E3))


def build_prompt(s, x):                     # persistent instruction + input that changes per request
    return (s + " " if s else "") + "Talep: " + x["text"]


print(f"{'instruction':<11} {'tokens':>9} {'in 120 requests':>15}")
for name, s in INSTRUCTIONS:
    b = len(tokenize(s))
    print(f"{name:<11} {b:>9d} {b * len(TICKETS):>15d}")
x0 = TICKETS[1]
gb = len(tokenize("Talep: " + x0["text"]))
ub = len(tokenize(INSTRUCTIONS[3][1]))
print(f"long prompt: instruction {ub} + input {gb} tokens, instruction share "
      f"{ub / (ub + gb):.4f}")
print(f"FICTIONAL ticket {x0['id']} ({x0['topic']}): {x0['text']}")
for name, s in INSTRUCTIONS[1:3]:
    CALL_COUNT[0] = 0
    for sd in (SEED, SEED + 1):
        print(f"{name} / seed {sd}: {call(build_prompt(s, x0), seed=sd)['response']}")

instruction    tokens in 120 requests
none                0               0
short              15            1800
medium             33            3960
long               62            7440
long prompt: instruction 62 + input 12 tokens, instruction share 0.8378
FICTIONAL ticket 4002 (fatura): fatura ayrintisini nereden gorebilirim
short / seed 20260218: talebiniz fatura konusuyla ilgili gorunuyor
short / seed 20260219: bu kayit fatura basligina girer , ilgili ekibe yonlendirildi
medium / seed 20260218: fatura
medium / seed 20260219: fatura

The last four lines show what the medium instruction does. With the short instruction the two runs give two different sentences, and the second one contains no label at all; with the medium instruction both runs give a single word, and the label is correct in both. What changes is not the hit rate but the readability of the response, and its cost is eighteen tokens.

The lines above are the actual proportion to be measured. The long instruction is 62 tokens; the ticket to be classified is 12 tokens. In a single prompt the instruction holds a share of 0.8378. Across one hundred twenty requests the same 62 tokens are sent one hundred twenty times and the total rises to 7440 tokens. Dropping one token from the instruction saves one hundred twenty tokens over one hundred twenty requests; this means that every decision taken while writing the instruction is multiplied by the number of requests.

The Total of the Repetition

The second block runs the four instructions on the same 120 records, then repeats the long instruction with three different request counts, and finally sets the tokens that each shortening gains next to the hit rate it drops.

def measure(s, seed=SEED, n=None):
    CALL_COUNT[0] = 0
    dataset = TICKETS if n is None else TICKETS[:n]
    ti = to = d = unl = 0
    for x in dataset:
        c = call(build_prompt(s, x), seed=seed)
        ti, to = ti + c["tokens_in"], to + c["tokens_out"]
        e = read_label(c["response"])
        unl += e is None
        d += e == x["topic"]
    m = len(dataset)
    return {"hit_rate": d / m, "unlabeled": unl / m, "tokens_in": ti, "tokens_out": to,
            "instruction": len(tokenize(s)) * m, "total": ti + to}


R = {name: measure(s) for name, s in INSTRUCTIONS}
print(f"{'instruction':<11} {'repeat':>7} {'in':>6} {'share':>7} {'out':>6} {'hit rate':>8} "
      f"{'run 2':>8} {'unlabeled':>10} {'per correct':>13}")
for name, s in INSTRUCTIONS:
    v, k2 = R[name], measure(s, seed=SEED + 1)["hit_rate"]
    print(f"{name:<11} {v['instruction']:>7d} {v['tokens_in']:>6d} "
          f"{v['instruction'] / v['tokens_in']:>7.4f} "
          f"{v['tokens_out']:>6d} {v['hit_rate']:>8.4f} {k2:>8.4f} {v['unlabeled']:>10.4f} "
          f"{v['total'] / v['hit_rate'] / len(TICKETS):>13.1f}")
print()
print(f"{'requests':>8} {'instruction tokens':>18} {'total input':>13} {'share':>7}")
for n in (30, 60, 120):
    v = measure(INSTRUCTIONS[3][1], n=n)
    print(f"{n:>8d} {v['instruction']:>18d} {v['tokens_in']:>13d} "
          f"{v['instruction'] / v['tokens_in']:>7.4f}")
print()
print(f"{'shortening':<16} {'tokens gained':>19} {'hit rate diff':>13} "
      f"{'per 0.01 hit rate':>19}")
for (a1, _), (a2, _) in zip(INSTRUCTIONS[1:], INSTRUCTIONS[:-1]):
    kb = R[a1]["total"] - R[a2]["total"]
    fi = R[a2]["hit_rate"] - R[a1]["hit_rate"]
    o = f"{kb * 0.01 / abs(fi):.0f} tokens" if fi < 0 else "lossless"
    print(f"{a1 + ' -> ' + a2:<16} {kb:>19d} {fi:>+13.4f} {o:>19}")

instruction  repeat     in   share    out hit rate    run 2  unlabeled   per correct
none              0   1700  0.0000   1285   0.4583   0.6333     0.2917          54.3
short          1800   3500  0.5143   1251   0.7167   0.4583     0.2167          55.2
medium         3960   5660  0.6996    196   0.7500   0.7500     0.0250          65.1
long           7440   9140  0.8140    203   0.7417   0.7583     0.0250         105.0

requests instruction tokens   total input   share
      30               1860          2295  0.8105
      60               3720          4547  0.8181
     120               7440          9140  0.8140

shortening             tokens gained hit rate diff   per 0.01 hit rate
short -> none                   1766       -0.2583           68 tokens
medium -> short                 1105       -0.0333          332 tokens
long -> medium                  3487       +0.0083            lossless

The second table shows how the repetition behaves. At thirty requests the long instruction comes to 1860, at sixty requests to 3720, at one hundred twenty requests to 7440 tokens: the cost grows linearly with the number of requests. The share of the instruction in total input stays steady between 0.8105 and 0.8181. There is no such thing as the share of the repetition thinning out as scale grows; whatever the number of requests, more than three quarters of the input is the instruction. However well an instruction is written, its cost is not its own length but its length multiplied by traffic.

The Cost of Shortening

The first table says where the hit rate saturates. The prompt without an instruction gives 0.4583. The short instruction raises it to 0.7167. The medium instruction raises it to 0.7500. The long instruction gives 0.7417, that is, below the medium one. The twenty-nine-token addition — ambiguity rules, main topic selection, the ban on explanation — added a total of 3487 tokens and took 0.0083 away from the hit rate. The cost per correct label went from 65.1 to 105.0 tokens. In this course such a row is written by name: the cost of the added text is certain, its gain is uncertain.

The third table reads the shortening from the opposite direction. Going from the long instruction to the medium one saves 3487 tokens and takes nothing from the hit rate; this is a lossless shortening. Going from medium to short saves 1105 tokens and takes 0.0333 of hit rate, that is, 332 tokens for every 0.01 of hit rate. Going from short to none saves 1766 tokens and takes 0.2583 of hit rate, that is, 68 tokens for every 0.01 of hit rate. The return on shortening is not constant: the shorter the instruction gets, the more expensive each remaining token becomes.

The unlabeled column of the table shows the real loss of shortening. With no instruction, no label can be read from 0.2917 of the responses, and with the short instruction none can be read from 0.2167 of them. With the medium instruction this share falls to 0.0250 and stays there. The remaining three records belong not to the prompt but to the surface: the rate limit returns three of the one hundred twenty calls empty. So the medium instruction has closed the gap that can be closed with a prompt. Since the long instruction has nothing left to add, it adds only tokens.

Finally the run 2 column is read. The prompt without an instruction gave 0.4583 and 0.6333 in two runs, the short instruction 0.7167 and 0.4583: in the second run the prompt without an instruction beat the prompt with the short instruction. The gain of the short instruction shows in one run and does not carry over to two. The medium instruction gave 0.7500 in both runs. The long instruction gave 0.7417 and 0.7583; its own two runs differ by 0.0167. The 0.0083 difference between the long and the medium instruction stays below that and counts as unmeasured in this course. The rule is applied by name in the next lesson.

Summary

• The system instruction is a persistent behavior definition, but its persistence is in the prompt, not in the model: it is sent again in every request and paid for again in every request.
• The 62-token long instruction holds 0.8378 of a single prompt and 7440 tokens across 120 requests. The share is steady between 0.8105 and 0.8181 at thirty, sixty and one hundred twenty requests; the total grows linearly.
• The hit rate saturates at the medium instruction: none 0.4583, short 0.7167, medium 0.7500, long 0.7417. The long instruction added 3487 tokens, took 0.0083 from the hit rate and raised the cost per correct label from 65.1 to 105.0.
• The return on shortening is not constant: going from long to medium is lossless, going from medium to short gains 332 tokens per 0.01 of hit rate, going from short to none gains 68 tokens.
• The real gap the instruction closes is the unlabeled response share: it falls from 0.2917 to 0.0250 and stays there. The remaining share comes from the rate limit, so it cannot be closed with a prompt.

Next Step

The difference between the medium and the long instruction stayed below the own run difference of the long one: the twenty-nine added tokens produced nothing measurable. This does not close the question of what to add to an instruction. A frequent answer is to tell the model who it is: an expertise, a job title, a style. That this addition raises the hit rate is a common belief, and just like the ambiguity rules it is paid for in every request. The next lesson measures this: what difference specifying a role adds, how many tokens it costs, and whether that difference is larger than the difference between two runs of the same prompt.