Fine-Tuning

In the previous two lessons the weights were always taken as given: a scale was chosen, the number of digits was lowered, and memory and accuracy were written side by side. The weights themselves were never touched. Yet this is the real possibility of a local setup that a distant endpoint does not have; because the weights are in your hands, they can be changed.

The table at the end of the second lesson imposed a choice: when the budget forces the small scale, accuracy on this task stays low. There are two ways to raise it, and both need the same thing, labeled examples. The first way is to put the examples in the prompt; with every request the model is shown a few solved tickets. The second way is to write the examples into the weights; the model is trained once and the prompt stays short. This lesson measures both at the same example count and shows the point where fine-tuning does not win. The two-stage structure of pretraining and fine-tuning was covered in the architectures topic of the Introduction to Machine Learning course and in the language models topic of the Introduction to AI Engineering course; that distinction is not repeated here, only the numbers of the two paths are compared.

• LA21. The code is a simulator; the tickets are fictional, the seed is 20260218. The scale is the small scale of the previous lessons, because the budget forced it.
• LA22. The first 60 of the 120 fictional tickets are the training pool, the last 60 are the test set. No scaffolding touches the test set.
• LA23. Prompt scaffolding: the chosen examples are put into the prompt. An example in the context gives its own pieces a temporary support for its own topic; the support is not carried from request to request.
• LA24. Fine-tuning: an error-driven update on the same examples. Weights move only on examples the model knows wrongly, and the weights that move are permanent.
• LA25. The number of epochs is how many times the training set is passed through from start to end; the number of update steps is the example count times the number of epochs.
• LA26. The two paths use the same examples. The comparison is at equal labeled example counts; the only thing that changes is where the example is put.
• LA27. The two runs are two separate draws of the examples (seed 20260218 and 20260219). At sixty examples the two draws give the same set, so the difference there is zero.
• LA28. Gain rule: the superiority of a path does not count as measured unless it points the same way in both draws.
• LA29. The cost is in two separate units: prompt scaffolding is tokens per request, fine-tuning is a one-time example count and epoch count.
• LA30. The weight table is 121 rows times five topics, that is 605 weights; the number of records fine-tuning moves is counted out of this.

The Same Examples, Two Different Places

The simulator is the same as in the previous lessons; the only change is that a correction table can be added on top of the score. If the correction comes from the prompt it is temporary and goes away with the context of that request; if it comes from training it is permanent and the prompt stays short. The input of both is the same labeled examples.

# SIMULATOR -- FICTIONAL. The simulator of the previous lessons, built in shortened form.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def pick(u, w):
    t = 0.0
    for i, x in enumerate(w):
        t += x
        if u < t:
            return i
    return len(w) - 1


def tokenize(m):                        # FICTIONAL tokenizer, a piece is at most five characters
    b = []
    for s in m.lower().split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


TOPICS = ["invoice", "breakdown", "reading", "tariff", "outage"]
TEMPLATES = {"invoice": ["bu ayki tutar gecen aya gore cok yuksek",
                         "odedim ama sistemde borc gorunuyor",
                         "ayrintili dokumu nereden gorebilirim"],
             "breakdown": ["evde su akmiyor bir sorun var",
                           "baglanti bozuk oldu ve hic calismiyor",
                           "ariza kaydi actim ama ekip gelmedi"],
             "reading": ["okuma yanlis girilmis endeks tutmuyor",
                         "cihaz degisimi icin randevu istiyorum",
                         "endeksi ile fatura ayni degil"],
             "tariff": ["hangi basamaktayim ogrenmek istiyorum",
                        "tarife degisikligi icin ne yapmaliyim",
                        "birim ucret abone tipine gore degisir mi"],
             "outage": ["mahallede elektrik yok ne zaman gelecek",
                        "planli duyuru nereden takip edilir",
                        "aksamdan beri kesinti yasiyoruz"]}
EXTRAS = ["ayrica gecen hafta da aradim", "kayit numarasi verildi",
          "ekip bilgi verecekti", "degisiklik talebim de var"]
TICKETS = []                            # 120 FICTIONAL support tickets
for i in range(120):
    r = rng(SEED + i)
    k = TOPICS[pick(r(), [0.24, 0.22, 0.18, 0.20, 0.16])]
    m = TEMPLATES[k][int(r() * 3)]
    if r() < 0.34:
        m += " " + EXTRAS[int(r() * 4)]
    TICKETS.append({"id": 4001 + i, "topic": k, "text": m})

PIECE_TOPICS = {}
for k, items in TEMPLATES.items():
    for m in items:
        for t in set(tokenize(m)):
            PIECE_TOPICS.setdefault(t, {})[k] = 1
for e in EXTRAS:
    for t in set(tokenize(e)):
        PIECE_TOPICS.setdefault(t, {})

BASE, WEIGHTS = 0.50, {}
for i, t in enumerate(sorted(PIECE_TOPICS)):
    r = rng(SEED + 7919 * (i + 1))
    row = {k: BASE for k in TOPICS}
    if PIECE_TOPICS[t]:
        for k in PIECE_TOPICS[t]:
            row[k] = BASE + 0.05 + 0.40 * r()
    else:
        row[TOPICS[int(r() * 5)]] = BASE + 0.02 + 0.10 * r()
    WEIGHTS[t] = row

ROWS = 12                               # small scale: the scale the budget chose
LEARNED = set(sorted(WEIGHTS,
                     key=lambda t: (BASE - max(WEIGHTS[t].values()), t))[:ROWS])
TRAIN, TEST = TICKETS[:60], TICKETS[60:]
INSTRUCTION = "Label this ticket as one of: invoice breakdown reading tariff outage . "


def score(text, correction=None):
    """The base score is the highest value of the learned rows; the correction is added on top."""
    pieces = tokenize(text)
    p = {k: BASE for k in TOPICS}
    for t in pieces:
        if t in LEARNED:
            for k in TOPICS:
                p[k] = max(p[k], WEIGHTS[t][k])
    for t in pieces:
        for k in TOPICS:
            p[k] += (correction or {}).get((t, k), 0.0)
    return max(sorted(p), key=lambda a: p[a])


def measure(correction=None):
    return sum(score(x["text"], correction) == x["topic"] for x in TEST) / 60


print("training pool", len(TRAIN), "records ; test", len(TEST), "records")
print("baseline accuracy (small scale without scaffolding):", f"{measure():.4f}")
print("instruction tokens:", len(tokenize(INSTRUCTION)),
      "; mean ticket tokens:",
      round(sum(len(tokenize(x["text"])) for x in TEST) / 60, 2))

training pool 60 records ; test 60 records
baseline accuracy (small scale without scaffolding): 0.6000
instruction tokens: 18 ; mean ticket tokens: 10.48

The baseline is 0.6000. This is the state, on this task, of the scale the budget imposed at the end of the second lesson, and it is the number both paths have to beat. An average ticket is 10.48 tokens and the instruction is 18 tokens; that is, a request without scaffolding is about 28 tokens. This number will be the yardstick when reading the cost of prompt scaffolding.

As the Example Count Grows

The code below puts the same example set in both places and measures only on the test set. The examples are chosen from the training pool with two separate draws; the second draw is there to decide whether a gain counts as measured.

EXAMPLE, STEP = 0.30, 0.05              # support from the context, and the fine-tuning step


def pick_examples(n, seed):
    r, h = rng(seed), list(range(60))
    for i in range(59, 0, -1):
        j = int(r() * (i + 1))
        h[i], h[j] = h[j], h[i]
    return [TRAIN[i] for i in sorted(h[:n])]


def from_context(examples):             # examples put into the prompt: temporary support
    return {(t, o["topic"]): EXAMPLE for o in examples for t in tokenize(o["text"])}


def train_model(examples, epochs):      # error-driven update; the weights are permanent
    D = {}
    for _ in range(epochs):
        for o in examples:
            s = score(o["text"], D)
            if s != o["topic"]:
                for t in tokenize(o["text"]):
                    D[(t, o["topic"])] = D.get((t, o["topic"]), 0.0) + STEP
                    D[(t, s)] = D.get((t, s), 0.0) - STEP
    return D


print(" n  prompt A  prompt B  tune k=1  tune k=3  tuneB k=3  prompt tok  updated")
for n in (4, 8, 16, 32, 60):
    a, b = pick_examples(n, SEED), pick_examples(n, SEED + 1)
    ia, ib = measure(from_context(a)), measure(from_context(b))
    y1, y3, yb = measure(train_model(a, 1)), measure(train_model(a, 3)), measure(train_model(b, 3))
    tok = sum(len(tokenize(o["text"])) for o in a)
    print(f"{n:>2}  {ia:>8.4f}  {ib:>8.4f}  {y1:>8.4f}  {y3:>8.4f}  {yb:>9.4f}"
          f"  {tok:>10}  {len(train_model(a, 3)):>7}")

print("\nn=60 fixed , number of epochs varies:")
print("  k  accuracy  update steps  updated records")
for k in (1, 2, 3, 6, 10):
    D = train_model(pick_examples(60, SEED), k)
    print(f" {k:>2}  {measure(D):>8.4f}  {60 * k:>12}  {len(D):>15}")

mean = sum(len(tokenize(x["text"])) for x in TEST) / 60
tok60 = sum(len(tokenize(o["text"])) for o in pick_examples(60, SEED))
with_scaffolding = len(tokenize(INSTRUCTION)) + tok60 + mean
tuned = len(tokenize(INSTRUCTION)) + mean
D60 = train_model(pick_examples(60, SEED), 3)
print(f"\ncontext per request: prompt scaffolding {with_scaffolding:.2f} , fine-tuning {tuned:.2f} tokens"
      f" ({with_scaffolding / tuned:.1f} times)")
print(f"cost of fine-tuning n=60 , k=3: {60 * 3} update steps , {len(D60)} records ,"
      f" {len(D60) / 605:.4f} share of the 605 weights")

 n  prompt A  prompt B  tune k=1  tune k=3  tuneB k=3  prompt tok  updated
 4    0.5667    0.6500    0.6000    0.6000     0.7333          38        0
 8    0.6500    0.6667    0.7333    0.7333     0.8000          73       35
16    0.8333    0.7500    0.8833    0.8833     0.8667         138       49
32    0.9000    0.9500    0.8500    0.9167     0.9000         292      101
60    0.9667    0.9667    0.9333    0.9500     0.9500         595      181

n=60 fixed , number of epochs varies:
  k  accuracy  update steps  updated records
  1    0.9333            60              121
  2    0.9333           120              162
  3    0.9500           180              181
  6    0.9500           360              181
 10    0.9500           600              181

context per request: prompt scaffolding 623.48 , fine-tuning 28.48 tokens (21.9 times)
cost of fine-tuning n=60 , k=3: 180 update steps , 181 records , 0.2992 share of the 605 weights

The top row is where neither path works. With four examples, fine-tuning moves no weight: the number of updated records is 0, because the model already knew those four examples correctly and an error is needed to trigger an error-driven update. Accuracy stays at the baseline, at 0.6000. When the same four examples are put into the prompt, on the other hand, accuracy falls to 0.5667; the four examples in the context pull a part of the test records toward their own topics. In the second draw the same two numbers are 0.6500 and 0.7333, that is, not even the direction is the same. With four examples nothing has been measured.

At eight and sixteen examples fine-tuning is clearly ahead. At eight it is 0.7333 and 0.8000, above 0.6667, the best of prompt scaffolding in the two draws; at sixteen it is 0.8833 and 0.8667, above 0.8333 and 0.7500 of prompt scaffolding. Because it points the same way in both draws, this superiority counts as measured. The reason is structural: an error-driven update uses an example several times and corrects only the places that are known wrongly, while an example put into the context is read once and also moves the places that were already known correctly.

The Point Where Fine-Tuning Does Not Win

At thirty-two examples the table turns. Fine-tuning with three epochs gives 0.9167 and 0.9000, prompt scaffolding gives 0.9000 and 0.9500. The bands overlap and the superiority does not point the same way in the two draws; according to LA28 there is no measured gain here. In the same row, fine-tuning with a single epoch is 0.8500, that is, below both values of prompt scaffolding. Raising the epochs to three gains 0.0667, and this is a gain that comes not from adding examples but from passing over the same examples again.

At sixty examples no uncertainty is left. Because both draws select the whole pool, the numbers are the same: prompt scaffolding 0.9667, fine-tuning 0.9500. The difference is 0.0167 and the run-to-run variation is zero. This is the point where fine-tuning does not win, and it is not chance. If all sixty examples sit in the context, the model reads every request together with a solved set; fine-tuning, on the other hand, has used from the same examples only the ones that gave errors, has moved 181 of the 605 weights and has left the rest untouched. Squeezing all the labeled data you have into a single training run is a loss on this task.

More epochs cannot close this gap. When sixty examples are held fixed and the epochs are increased, accuracy is 0.9333 at one epoch, 0.9500 at three, and stays there; six and ten epochs give the same 0.9500. Because after the third epoch no errors are left to correct in the training set, the number of updated records also settles at 181. The update steps go from 180 to 600 and nothing is received in return. The ceiling of fine-tuning on this task is 0.9500, and that ceiling rises not by adding epochs but only by another way.

What decides is not the accuracy column but the last two rows. With sixty examples, prompt scaffolding raises the context of the request from 28.48 tokens to 623.48 tokens, that is, to 21.9 times, and this cost is paid again on every request. According to the computation of the first lesson, context length is proportional to the intermediate state per request; twenty-one times the context means twenty-one times fewer concurrent requests in the same budget. The cost of fine-tuning, on the other hand, is one-time: sixty labeled examples, three epochs, 180 update steps and a 0.2992 share of weights that move. Every later request is still 28.48 tokens.

The comparison therefore does not close with a single number. When labeled examples are few, fine-tuning is both ahead and cheaper. As examples multiply, prompt scaffolding catches up and passes in accuracy, but it does so by enlarging the context; if how many requests are processed at the same time matters, an accuracy difference of 0.0167 is not worth twenty-one times the capacity. What has to be measured is both, and these two numbers are not in the same unit.

Summary

• Both paths ask for the same input: labeled examples. The only thing that changes is whether the example is put into the prompt or into the weights; so the comparison is made at equal example counts.
• With four examples fine-tuning moves no weight and the baseline stays at 0.6000; when the same four examples are put into the prompt, accuracy falls to 0.5667. The direction of the two draws is not even the same.
• At eight and sixteen examples fine-tuning is ahead in both draws: 0.7333 and 0.8833 against 0.6500 and 0.8333. This superiority counts as measured.
• At thirty-two examples the bands overlap (0.9000 and 0.9500 against 0.9000 and 0.9167); there is no measured gain, and single-epoch fine-tuning falls behind with 0.8500.
• At sixty examples prompt scaffolding is 0.9667 and fine-tuning 0.9500; because the run-to-run variation is zero, the loss of fine-tuning is real.
• The costs are not in the same unit: prompt scaffolding raises the context 21.9 times per request, fine-tuning moves a one-time 180 update steps and a 0.2992 share of the 605 weights.

Next Step

The 0.2992 share in the last row was passed over without being asked about. Fine-tuning did not decide which weights would move; the error-driven update touched whichever piece passed through an example known wrongly, and 181 records came out that way. This number is a by-product, not a design decision. Yet the number of weights that move is directly a cost: it is the thing that will be stored, carried, rolled back and kept separately for every task. Could the same accuracy be obtained by moving fewer weights, what would be lost if it were decided beforehand which weights to freeze, and how many numbers would be stored if the update itself were written onto the table as an add-on: none of these was tried. The next lesson measures side by side the full update, layer freezing and a low-rank correction added to the table, and prints two numbers: the updated-parameter share and accuracy.