Unit 02 of 03
Defense Layers
Counting the cost of every defense layer together with the harmful it stops: stacking the rule-based filter and the model-based filter catches no new harmful record while it lowers precision and raises the false block rate; per-class precision and recall expose what a single average hides, and in at least one class the defense falls below the baseline; the records masking cleans against the tasks lost because it deletes a field the task needs; the harmful stopped against the tasks left incomplete as the permission set narrows; the number of records untrusted content marking neutralizes without blocking anything or producing a false alarm, and the harmful that remains; how many of the records sent to the human approval gate are really harmful, and how the harmful caught shrinks as the approval rate falls.
Published Updated
Lessons
- 01 Input and Output Filters Counting the harmful records that rule-based and model-based checks block on the same record set together with the clean records they block by mistake: when the rule-based filter is split into an input side and an output side, one blocks 55 harmful records and the other 44, and their union rises to 74; the model-based filter at a 0.50 threshold blocks fewer harmful records but blocks far fewer clean ones; stacking the two is worse than the rule-based filter alone — recall stays at 0.7957, precision drops from 0.7629 to 0.7551, the false block rate rises from 0.1565 to 0.1633, and the layer blocks one extra clean record without catching a single new harmful one. 14 min read
- 02 Content Moderation Counting that blocking harmful content and giving it a class name are two separate jobs: a moderation step that produces a class name gives 0.9355 recall and 0.7311 precision in the binary reading, but per class the precision spreads between 0.2917 and 0.6364 and the recall of the class label between 0.3333 and 0.8235; in the indirect instruction class the moderation step puts the label on 21 records and knows 8 of them correctly, and it marks 9 entirely clean records with the same label, so when the class label is tied to an action the action fires wrongly far more often than rightly and the defense falls below the baseline in that class; the recall of the corrupted input class rises from 0.2632 to 0.8421 while the false block rate rises from 0.1565 to 0.2177. 14 min read
- 03 Personal Data Masking Reading the two numbers of reduction in the data that goes to the model together: flag-based masking removes a field from 32 of 240 records, and 14 of them belong to the hidden field request class and 9 are entirely clean records; the same policy makes 23 tasks that require the field unsolvable, and 7 of those 23 tasks belong to clean subscribers; 3 records of the hidden field request class do not carry the flag and so escape masking, the narrow policy raises the escaped count to 14 while lowering the broken tasks to 8, and the policy that masks every record misses no request but breaks 169 tasks, 104 of which belong to clean subscribers. Because masking blocks no record it has no false alarm, and reading it like a filter is misleading. 14 min read
- 04 Permissions and Sandboxing Counting tool permission as a defense layer: the same fictional set of 240 records runs under five permission sets, and the harmful records stopped, the false alarms and the completed tasks are printed side by side. On the undefended baseline the harmful stopped is 0 and the false alarm is 0; closing the execution capability stops 12 harmful records, blocks no clean record and gives a precision of 1.0000; closing the side-effect capability gives 24 harmful against 3 false alarms; the allowlisted set stops 34 harmful and lowers the completed tasks to 0.8844, read-only stops 62 harmful and lowers them to 0.6667. The marginal precision of narrowing falls from 1.0000 to 0.4167: the last two steps block more than one clean record for every harmful record they stop. The class distribution shows the blind spot of the gate: 1.0000 of the privilege escalation class is stopped, only 0.0588 of the hidden field request class. 13 min read
- 05 Untrusted Content Marking Counting a defense that blocks no record: text coming from a source other than the user is marked as data and is not counted as an instruction even if it carries the instruction mode. On the undefended baseline the harmful stopped is 0 and the false alarm is 0; marking neutralizes 42 of the 93 harmful records without blocking any record, 51 remain, and the false alarm is zero. Neutralization gathers in two classes: 0.9167 of indirect instruction, 0.8696 of source spoofing; 0.0000 in the hidden field request, privilege escalation and corrupted input classes. The real value of marking is that it lets the filter be tightened: the rule filter alone covers 0.7957 and blocks 23 clean records, marking with the 0.50 threshold of the model filter covers 0.7634 and blocks only 8 clean records. The cost is not a false alarm: 47 of the 103 marked records are clean and only 1 of them carries the instruction mode, so the path lost is a single record and exactly one step of the resolution. 12 min read
- 06 Actions Requiring Human Approval Counting the precision and cost of a human approval gate on irreversible actions: the narrow gate sends 27 records to a human, 24 of them really harmful, precision 0.8889, recall 0.2581, 3 unnecessary interruptions. Widening raises recall to 0.7957 under unlimited attention, but under a fixed attention budget of 27 prompts the harmful caught falls to 23 and 17 in two runs and never passes the 24 of the narrow gate. As the review share goes from 1.00 to 0.20 the harmful caught falls from 24 to 4 and recall from 0.2581 to 0.0430. The cost is not a blocked task but an interruption: blocking the same 27 records would lose three clean tasks; asking loses none. 13 min read