Unit 03 of 03
Output Control
Measuring how the output of an uncertain generator is constrained in form and in content: the combined behavior of temperature, top-p (nucleus) and top-k, and which of them narrows the difference between two runs; the repetition rate and the number of prematurely cut responses when a repetition penalty and a stop sequence are added; the schema match rate and token cost of free text, a format description and an imposed schema; the schema and business rule violations that validation catches, the responses that retrying recovers and the extra calls it costs; a wrong response that is sometimes fixed by a better instruction and sometimes only by missing information; how much balancing through the prompt narrows the subgroup gap and that it does not solve data bias.
Published Updated
Lessons
- 01 Sampling Parameters Measuring temperature, top-p (nucleus) and top-k together on the same prompt: label accuracy 0.6417 and two-run difference 0.5667 on the base setting, 0.8750 and 0.0417 at temperature 0.2, 0.8417 and 0.1333 at top-p 0.50, 0.7833 and 0.3583 at top-k 3, 0.8667 and 0.0083 when all three narrow together. The fixed cut leaves three candidates on average while the top-p cut leaves 1.20 candidates. Input tokens stay constant at 5300 and the call count stops at 120, so the cost of the change is thirteen output tokens. 8 min read
- 02 Penalties and Stop Sequences Measuring the repetition penalty and the stop sequence on one response set: on the base scaffolding the repetition rate is 0.3431 and 107 of 120 responses are truncated at the length limit; a negative constraint written into the prompt lowers the rate to 0.2167 while raising the input tokens from 2734 to 3574, and a 0.45 repetition penalty does the same job with zero extra tokens, down to 0.1972. The stop sequence lowers the number of truncated responses to 55, and to 24 together with the penalty, and brings the output tokens from 4073 to 2653, but it cuts twenty-five responses prematurely, before the label sentence is written, and lowers usability from 0.8833 to 0.7417. The two-run difference on the stop sequence row is 0.0543, and the 0.0746 gain over the base is marginal. 14 min read
- 03 Structured Output Measuring free text, a format description written into the prompt and an imposed schema side by side on the same 120 fictional tickets: the rate of schema-conforming responses is 0.0000 in free text, 0.8000 with the format description, 0.8833 when an example is added and 1.0000 with the imposed schema. The two-run difference of the format description row is 0.0833 and, since the gain from adding an example is also 0.0833, that gain counts as unmeasured, and it adds 2040 tokens. Topic hit rate stays fixed at 0.8833 at all four levels. Input tokens rise from 2254 to 5614 with the format description and to 7654 with the example, while the imposed schema stays at 2974; that is, the cheapest prompt gives the highest conformance. 7 min read
- 04 Output Validation Counting schema validation and business rule checking separately and building a retry loop on top: 24 of 120 fictional responses do not conform to the schema, 24 of the 96 records that pass the schema get caught on a business rule, and with no validation the faulty record rate is 0.4000. The two checks together bring the acceptance rate down to 0.6000, one retry recovers 26 records and raises it to 0.8167 at a cost of 48 extra calls. The second retry recovers eight more records but its gain of 0.0666 stays below the two-run difference of 0.0667. The retry that writes the error into the prompt recovers 23 records and adds 514 tokens. 14 min read
- 05 Prompt and Context Engineering Difference Comparing two separate fixes on the same set of errors: on 120 fictional tickets the baseline topic hit rate is 0.8833, the fault code hit rate is 0.4083, and both correct is 0.4083. Writing a distinguishing rule into the prompt raises the topic hit rate to 0.9667 and does not touch the code hit rate at all; putting the fictional code table into the prompt raises the code hit rate to 0.9167 and does not touch the topic hit rate at all. Of the fourteen topic errors ten are fixed by the instruction alone, of the seventy-one code errors sixty-six are fixed by information alone, and one error in each class is fixed by neither. The costs diverge: the instruction adds 1680 tokens, the information adds 14520. 11 min read
- 06 Bias-Reducing Prompts Measuring how much of a subgroup gap prompt balancing closes and what it leaves: on 200 fictional tickets the central hit rate is 0.9016, the rural hit rate 0.5513 and the gap 0.3504. A bias-reducing notice in the prompt moves the gap to 0.3586 and adds 3800 tokens; a balanced example brings it down to 0.2888 and adds 14400 tokens. The hit rate of the sampled local spelling rises from 0.4848 to 0.8788, while the unsampled one stays fixed at 0.3023. The prompt closes only a 0.1758 share of the gap. The course wrap-up gathers the previous prompt, changed prompt and cost rows of twenty-two lessons. 11 min read