Unit 03 of 03
Evaluation and Monitoring
Measuring how we know the defense we built works and how it is monitored in production: the same defense giving three separate numbers on a representative, a harm-weighted and a borderline-example set, with the resolution of the set written down; the agreement of a model-as-judge with the human label read side by side with the judge's own difference between two runs under a criteria-based approach; inter-rater agreement changing with the scale and the guideline, and the class in which the disagreement concentrates; the number moved by prompt and model changes compared with the run band, and which change stays unmeasurable; how many records later each of three production indicators shows a degradation; the user feedback loop making the false alarm look smaller than it is because a blocked user cannot give a signal; and the measures a system card carries and does not carry, with every measure it does not carry remaining a claim.
Published Updated
Lessons
- 01 Evaluation Set Design Reading the number of a defense together with the composition of the set: the same rule-based filter, unchanged, gives three separate numbers on three sets. Precision comes out at 0.7629 on the representative set, 0.9136 on the harm-weighted set and 0.4444 on the borderline example set; recall stays exactly the same on the first two sets because the harmful subset did not change at all. On the borderline example set recall falls from 0.7957 to 0.6400 and the false block rate rises to 0.7143. The resolution of the set is written down too: the step of recall is 0.0108 on the representative set and 0.0400 on the borderline example set, and no difference smaller than that step can be measured. 13 min read
- 02 Automated Evaluation Measuring two ways of saying that a record is harmful: writing structural criteria and asking a language model. The criteria-based approach gives 0.7629 precision, 0.7957 recall and 0.8250 agreement with the human label, makes no calls and prints exactly the same number in two runs. The model-as-judge gives 0.7876 precision, 0.9570 recall and 0.8833 agreement; the entire gain comes from the corrupted input class, which the criteria never look at (1.0000 against 0.2632). The cost is 246 calls, 10779 input tokens and the judge's own uncertainty: between two runs the decision changes on 16 records, recall moves three steps, and the difference in false block rate between the two approaches stays far below this band. 14 min read
- 03 Human Evaluation Measuring the human label the previous lesson rested on: three fictional raters looking at the same flag list give 0.8417 agreement on a binary scale, 0.7583 on a three-valued scale and 0.9306 with an instruction with examples. The instruction raises agreement by twenty-one steps, but the overlap of the majority label with the actual class stays at 0.8417 and the accuracy of unanimity falls from 0.9126 to 0.8744. Disagreement gathers in the harmful classes and in records with exactly one flag: 0.7368 in corrupted input, 0.0000 without flags. Measured against the majority label, the same rule-based filter reaches 1.0000 recall, because the label column shares its blind spot. 13 min read
- 04 Regression Testing Measuring four changes against a fixed evaluation set and comparing every difference with the run band. Rewriting the instruction with the same meaning stays below the band; removing the format constraint drops recall to 0.7742 but does not move the false block rate; losing one trace record shows only in the per-class breakdown, where corrupted input recall falls from 1.0000 to 0.7895. 14 min read
- 05 Production Monitoring Measuring how many records later three indicators show that a deployed defense has degraded: over a fictional stream of six hundred records the undefended baseline gives 0 blocks, 0 false alarms and 120 escaped harmful records, while the threshold 0.50 setup blocks 95 records, 11 of them false alarms, and lets 36 harmful ones through. After record three hundred and one two separate degradations begin: a traffic shift raises escaped harm from 36 to 84, a silent threshold change raises false alarms from 11 to 27. The escaped-harm proxy reports the wave after 20 records but never reports the threshold shift, the block rate sees the threshold after 29 records, and the label-dependent false alarm share gives 45 and 220 records over two runs. No indicator catches both degradations. 8 min read
- 06 The Feedback Loop Measuring why the user signal comes back biased when it returns to evaluation: the threshold 0.50 setup blocks 74 records, 8 of them false alarms, and 27 harmful ones escape; 0.6296 of the escaped harm produces a complaint, but the covered share of the false alarms is 0.0000 with the appeal channel closed and 0.3750 with a narrow one, and only three of the six appeals are real false alarms. A threshold tuned for five rounds by the two numbers read from the signal drops from 0.50 to 0.30: false alarms rise from 8 to 27, the false block rate from 0.0544 to 0.1837, recall from 0.7097 to 0.9032, and the loop sees only the second. Total errors go from 35 to 36: the loop does not reduce errors, it changes who pays for them. 12 min read
- 07 Responsible Release Measuring what the published number covers: the threshold 0.50 setup publishes a single recall number (0.7097), but per class the same defense spreads between 0.9583 and 0.2105, and the published number is 0.4992 above the weakest. A narrow system card ties 2 of nine commitments to a number, a wide card 6; the other 3 stay claims. This lesson closes this course and the AI Engineering curriculum, gathering twenty-one lessons in one table. 6 min read